Annotating Exploder

I have tried to find a way to make my web annotation work in Internet Explorer. I wrote this to illustrate how painful and expensive it is to support outdated or poorly-designed systems, such as Internet Explorer, and to solicit advice. I have tried to avoid technical details, but I hope I’ve included enough that this can also help programmers who may run in to some of the same problems I did.

When the user selects a passage of text, the major browsers all provide a way to find out what that text is. For annotation though, this isn’t enough: I also need to know where it is so that I can highlight the passage when a user creates an annotation. Five years ago, the W3C released the Range standard to address this. To my knowledge, this has only been adopted by the Mozilla family of browsers.

Microsoft, however, is famous for coming up with their own non-standard mechanisms for doing things, and text selection is no different. Internet Explorer does provide a way to find out more about what text the user has selected. It even provides the location — but it doesn’t indicate what paragraph or character in the document: instead it returns the pixel location of the selection on the screen. This is useless, because that location could vary according to font size, margins, window shape, etc., and there’s no way to convert from a pixel location into something more useful.

But it is possible to modify the text the user has selected. So, in theory, I could insert an invisible marker in the text, then search for it and — presto! — find out where it is in the document. There is a small difficulty, because there’s no way to insert a marker — I can only replace the entire passage of selected text. But that’s OK. I can find out what was selected, add the marker, and then replace the passage with the modified version.

But this is Microsoft. In typical fashion, they have tried to be smart only to end up stupid. When IE copies the text, it checks to see whether there’s anything missing. For example, there are markers that indicate where a paragraph ends and begins. If you copy the last part of a paragraph, the marker indicating the start of the paragraph will be missing, so IE helpfully adds it back. In other words, if you copy and paste the text — even without making changes — IE may insert paragraphs and mangle the web page.

It’s a Catch-22, like some sort of perversion of the Uncertainty Principle. I can know what text the user selected, or I can know where it is on the page, but I can’t know both!

I didn’t give up, though. If Microsoft was adding something to the selection, I could remove it again. I inserted hidden location numbers in the text before it was copied, which indicated where in the document each paragraph (or other element) started. When Microsoft copied the text, I could find the original location numbers, compare the copied versions with the originals, and remove any garbage added by IE.

And you know what? It worked! Now I could paste my text back in to the document, with my new marker added, search for it, and find the selected region. I tried it. And just about cried. Guess what Internet Explorer did?

It added markers.

If I copied the end of one paragraph and the start of another, it added a new blank paragraph in between. Again, Microsoft’s big brain was causing trouble. Only this time, the changes were in the middle of the text I was pasting, and I had no idea what the rule for adding them was. I gave up, but the problem stayed in my mind.

You have probably noticed the flaw in my reasoning. When I compared the hidden location numbers in the selection with the original document, I was figuring out the location of the selection in the original. So why didn’t I just use that location and forget about pasting in my hidden marker? The reason is an exception: there may not be any markers in the selected text. In that case, I need to paste something in in order to find the selection.

But there’s the the answer. Case 1: Internet Explorer only adds garbage if the selection crosses a paragraph boundary or the like. In this case, I can use the hidden location numbers and never paste anything. Case 2: The selection does not cross a paragraph boundary etc., so I can freely paste in my hidden marker and IE won’t mess things up.

As soon as I realized this, I hurried to make it work. It did! The problem was solved. But I realized there was one rare exception. Under certain conditions, if the document contains a lot of repetition, the solution for Case 2 might become confused and select the end of a paragraph when it should select the beginning. The user would select one passage of text, only to see the software move the selection to a different (although identical) passage. This is very unlikely, but it bothered me. I slept on it.

When I woke, I remembered something about Internet Explorer. In addition to providing the (useless) pixel location of the selected text, and allowing the selection to be replaced, it provides a way for the programmer to move the start and end points of the selection. So, for example, I could extend the selection by a word or a sentence. But what if I reduced it instead? What if I made the selection only one character long? Such a short selection could not cross any paragraph boundaries, so Internet Explorer wouldn’t add any gobbledygook. Then I could always safely apply the solution to Case 2. The end of the selection wouldn’t be a problem — I would just count how long the selection was before I changed it. And, best of all, this implementation would never suffer from the exception that afflicts Case 1.

So that’s what I did. It works. And it’s embarrassing, because after all of that, it really only takes nine lines of Javascript code to do something I thought couldn’t be done. I can produce the same location information that the W3C standard specifies.

The next step was to add IE support to my annotation code. I had already added some support; now I made it possible to create annotations. It works. But it also crashes, frequently, somewhat randomly, and with no apparent connection to my selection solution. It crashes so hard it offers to send a bug report to Microsoft and recommends rebooting Windows.

Now maybe there’s a bug in my code — not that that would be an excuse for Exploder, but at least I might be able to fix it1. But, unlike Firefox, Explorer provides virtually no facilities for debugging. So, for now, although I’ve struggled to avoid it, annotation is Firefox only.

The experience emphasizes three lessons:

  1. Focus is essential to programming, but it makes it hard to see if you’re on the wrong path.
  2. Writing about the problem, like talking about it, helps to gain perspective.
  3. Giving up can also help. Several times I said “it’s just not possible” and walked away. Each time, when I came back it was because I realized I had been too focused on a particular aspect on the solution and was ignoring something important.
  4. Microsoft’s selection API is very poorly designed. It seems to simply expose the information they need to provide the features they implement (copy, paste, keyboard selection), with no thought to how it might be used by anyone else.
  5. Speaking of Microsoft, don’t try to be smart until you can handle being simple.
  6. Supporting poorly-designed or obsolete systems is expensive — partly because it takes a long time, but even more so because the amount of time required is impossible to predict. Until the very end, there is no certainty the problem can even be solved. Even now, I don’t know whether my time was wasted or whether a solution will appear.

Notes

1 Apparently IE’s Javascript garbage collector is buggy. It’s highly likely, therefore, that I am leaking memory. Yet it seems unlikely this would be enough to make it crash as fast and as hard as it does.

2005-07-16

Comments