Died on Christmas Day, shortly before 5:30 p.m. PST, 2006.
We'll miss you, Grandpa. Another one of the Greatest Generation, for whom we now play Taps.
The Seattle Seahawks, by the Arizona Cardinals defeating the San Francisco 49ers, have clinched the NFC West division and are returning to the playoffs.
Woo hoo!
Congratulations to the San Diego Chargers for pulling off that last-minute fluke touchdown.
In developing Verbosio, I'm making localized patches to my mozilla.org source code (as Daniel Veditz correctly points out, fixing mozilla.org bugs that Verbosio needs is Verbosio development). However, these patches haven't yet been checked into the mozilla.org code, and may need further work before they do. To account for this, I hope to have an addition to the Verbosio bugs page listing the status of certain Verbosio-required bugs with patches people can apply. It needs some minor PHP work I don't know how to do yet, so stay tuned.
I've also updated the make-project code to fix a couple conditions where the Verbosio objdir just wasn't usable. No more manual hacks there.
I've also filed a tracking bug for bugs blocking Verbosio 0.1. Of these, implementing a DOM-Inspector-like viewer for documents is close to completion (the only one so far). It can let you edit elements, attributes, text nodes, character data nodes, processing instructions and comments. Deletion works, and I'm about to start work on insertion, creation and clipboard operations. What to display in the Inspector view for document nodes, I haven't decided yet.
Tomorrow, I turn 29. Maybe it's time for another article from me ruminating on technology. It's really shocking how much I've changed, learned and grown as an engineer in three years. Certainly I'm not as depressed as I was then.
I knew the day was going to be bad when I started commenting out JavaScript assertions in my own code. I just didn't know how bad. The extended entry has all the gossip.
First off, I'm playing in Verbosio with attribute nodes, XUL text boxes, XUL menu items, and the DOM user data interfaces. New stuff, good stuff... except the attribute node that I got back from the user data entries didn't match the node I was trying to match. Equal, but not the same, inside a DOMNodeRemoved event listener -- which I thought was pretty strange.
Since they were equal, I thought, "Oh, just use isEqualNode() instead of isSameNode()." In other words, fake it. Bad practice, but this is 0.1, not 1.0, right? I'd implemented isEqualNode earlier this year, so why not? That made deleting the attribute work, and undoing the deletion worked too. But redoing the deletion didn't work, because the attribute value from user data is null for some reason.
Several dump statements (and probably two hours) later, I'm no closer to figuring out the cause than when I started. I need Venkman. But I'm sick and tired of patching my code to make Venkman work. Instead, I thought, just grab a snapshot of Venkman and hack it into the Verbosio build process.
That didn't work. It was always phoning home to addons.mozilla.org, and every time it did, addons said, "We don't support Verbosio. Go away." To which Verbosio responded by disabling Venkman. Try as I might, I couldn't make it work. I thought about forcibly inserting an update.rdf reference into Venkman's install.rdf (that would've solved the problem, but was too dirty a thing to do).
Next I tried to figure out just how XULRunner was talking to addons and override it. I thought, "Well, this is Verbosio. Of course addons.mozilla.org won't support Verbosio. Verbosio's not a mozilla.org product. So I'll just run my own mini-update service from the Verbosio website." I discovered there's a preference, extensions.update.url which refers to the addons website. I replaced it with a custom update.rdf location. No good; it's still pointing to amo. This didn't make any sense. Finally I dove into the C++ debugger and found out the preference service ignored the preference completely and relied on a extensions.properties
file in a localization. This file, which had easily over 100 properties in it, included the one property I cared about. I could replace that file too, but it seems like a whole lot of bloat for one little damn detail. Why a localization file has stuff that can't be localized is beyond me...
At this point, I'm pretty steamed from having battled it almost the entire day and making no progress on Verbosio. The best fix really is to make Venkman not depend on any one XULRunner application, but installable for all of them, as an extension to XULRunner's toolkit. Unfortunately, our extension manager code can't do this (bug 299716). It can support one product - whatever the application.ini says for the ID - and that's it. Even if it's based on another product - say, XULRunner toolkit - you're out of luck.
Bug 299716 looks like a bug that, while important, has been shelved by the primary developers in favor of other, higher-importance bugs. I can't fault the community or the developers watching the bug for that. At the same time, if I need Venkman in Verbosio "the right way", this bug must be fixed immediately. I can't do anything more than request the bug block Gecko 1.9 in order to convince others to work on it. I am reluctantly forced to concede that the only way this bug will get fixed in a timely manner is if I write at least a first-round fix.
I really hate diving into unfamiliar code. Even if it's in a language I know very, very well (like JavaScript). Ultimately, though, it's the best of a bad set of options. My frustrations with not being able to easily apply Venkman, Inspector, etc. to Verbosio without repetitive work or scripting by shell or Perl have reached a boiling point, where I just can't take it anymore. The bug would still be there if I hacked around my problems and forced things to work, and I'd just bump into the problem again for some other project (like XUL Widgets, JSLib, which I've already done some evil hacking for in Verbosio). Sooner or later, the hacks have to stop and good code fixing the core problem must be written. Now is that time, at least for this bug, and I dare not wait for someone else.
I'm going to work on the bug, but I won't assign it to myself until and unless I generate a patch. Worst-case scenario, I don't get anywhere. But I doubt that; I think I'll make some progress and post at least a partial patch. Very likely, because I'm so blocked, I'm going to post a complete patch and get reviews.
Needless to say, Verbosio development has been shelved. Again. That makes me madder than anything else: that there's quite literally nothing I can do until I fix a couple bugs that are critical to me, and merely irritating to almost everyone else.
Now Verbosio can delete a range of nodes and undo that deletion.
I actually was scratching my head last night trying to figure out how I'd break this thorny problem down into manageable pieces. This morning, it occurred to me mozilla.org's editor code had already solved this problem in the old Composer application in SeaMonkey, with a DeleteRangeTxn class. If I could create instances of that class from chrome or JS-based component code, I'd be in business.
Unfortunately, nobody seems to have done that before, either. So I filed bug 363290 for that. The patch is in the review pipeline.
I also found out, from reading the code behind this old transaction class, that I needed a nsIEditor object... something I've been avoiding until now in Verbosio. So I wrote up a quick & dirty nsIEditor component with most of the methods and properties not implemented. I expect, right now, to use it only as a placeholder for other calls into native transaction code.
If ETNA and Verbosio merge, hopefully ETNA will have a more complete nsIEditor implementation, and use it better than I do. For now, it'll work, though.
I estimate that it would have taken me at least a week to implement deleting a selection from the document before I thought of Composer. By reusing that code, I reduced my implementation time down to about four hours (nine to ten if you count the long breaks I took today.) That sort of time savings makes me a happy coder.
Verbosio can now edit text nodes.
That may sound like very little ("sixteen months of work on this, and the best you can do is edit a little string of text?"), but I've put a lot of effort into building infrastructure. Just editing text nodes itself turns into a little hassle. Text boxes work differently with keystrokes and clipboard pastes, so I had to handle that carefully. I also had to give people the impression that their changes in the textbox were live (that is, the text node updated as they typed) while letting undo work. My work this week has been centered around handling that case in a somewhat predictable manner.
On top of that, it's not sixteen months of work on just Verbosio... I've had to set it aside often, for weeks at a time.
Really, this is just the tip of the iceberg. Now I can work on more interesting abilities, like deleting a range of nodes and undoing that, or changing attributes on an element. Later on, well, things will get very sweet indeed.
New in this release:
A couple years ago, I was handed a task I did not know how to do: profile a corporate application and see where the code is spending most of its time. I had a professional tool (Quantify) available to help me do it. It took me weeks to figure out how to use it at a basic level, just enough to confirm what we already suspected: that the bottleneck wasn't in Mozilla code.
Yesterday, while working with some experimental code, I hit a new perf issue that absolutely should not be happening. (Bug 362694 for anyone interested.) Profiling and performance analysis is something I do so infrequently that I never took the time to really learn how to do it.
Ultimately, I found that there are a lot of free tools available to help me gather data... but this is worse than useless for me because I can't figure out what the data is saying. PEBKAC. Someone suggested Shark to me for the Mac platform, but I found Shark didn't capture anything from my XULRunner app. I'm not blaming Shark; I just don't know how to use it.
So I wouldn't mind a good written tutorial in profiling analysis from start to finish for Mozilla code. Particularly under the constraint of free or inexpensive tools. I'd write it if I had a clue what I was doing.
I had hoped to have a certain list of "immediate to-do" items finished by now, per my previous blog entry. Instead, I got sidetracked. Royally.
First, I ran into an as-yet-undiagnosed layout bug where the DOM had all the nodes in the right place, but certain XBL bindings were not applying. This caused me so much grief that I set the problem aside after a couple days banging on it alone. Then I had a few Bright Ideas.
To spare people who don't care (because this is a long blog entry), the rest of this article is in the extended section.
First, I could use the SAX API's to trace a document's source code from beginning to end, and with a DOM TreeWalker, walk the equivalent DOM document from root node to end. Why would I want to do that? Well, if I also have the source code of the document (which I need for SAX), then I can match DOM nodes to source code. Almost completely (there are some minor nits that Mozilla's SAX implementation doesn't support yet, but not unsolvable ones).
The result is a new interface for Verbosio (xeINodePositionService.idl) and a new component (nodePositionService.js). Don't worry if the JS file looks extremely cluttered and sub-optimal. It's only a first draft, and can easily go through a couple rounds of code clean-up. Later.
This also ties into my Basic XBL validation blog post a few weeks ago. The code, as it was written, was pretty bloated, and of course untested. That didn't stop me from having another Bright Idea at just about the same time as the previous one.
Because now I can identify specific character nodes and attribute values which get fed into JavaScript, and know exactly where they are in the source code, I can check each one for syntax errors. But having a XBL language pack generate a huge string matching JavaScript in source to JavaScript in the string, and whitespace everywhere else, was inefficient, and more bloat.
Instead, I created another interface (xeISourceMap), for taking these JS strings with known locations and contents, and placing them into a much compacted string internally. Then I evaluate that smaller string in a sandbox, and look for a syntax error. If the source map finds such an error, it translates the error location in the smaller string back to its corresponding location in the original source document. The resulting JS component is jsSourceMap.js.
Now, I should make it clear that I have tested the node position service a bit, but the JavaScript Source Map component has not been tested, and probably doesn't work quite right. Also, the names for these components and interfaces isn't final; basically, I picked them out of thin air, and I couldn't come up with a better name. I'll take suggestions.
With the above components, though, XBL validation reduces to four steps:
Nice, neat, simple. Three adjectives every programmer tries to obtain for their code.
Of these, the second step may also be partially reduced if I write a "validation-purposes-only" DTD and a DTD-parsing component. I say partially because DTD's can't do the detailed levels of validation that XML schemas and custom code can, but DTD + a little extra JavaScript hook function (where the DTD reader component calls back into the XBL language pack for additional checking) should suffice for most validation purposes. If you really want rigorous validation, write a XML schema document and build XULRunner with the schema-validation extension. (Or, wait for Verbosio 0.1 and write a XML Schemas language pack and user interface!)
Sure, this sounds like a lot of work just for a XBL validation process. However, thanks to XPCOM, these components are reusable in other contexts, as I pointed out in the "Basic XBL Validation" blog entry.
Another idea I have on the to-do list is preferences-controlled validation. For example, in the XBL scenario above, a user might not care much about whether his tags aren't in the correct order, but a JavaScript syntax error would stop him dead. So he sets a preference that makes Verbosio treat an inline JS syntax error as seriously as it would a fatal error in XML parsing ("stop parsing and force me to fix this"). Implementing the control is easy; figuring out how to name the preferences (in a given format) is more difficult - assuming that I want to use the preferences service.
Incidentally, that layout bug I opened the article with? It was for a DOM Inspector-like viewer for XML documents. The right-hand panel (where you get nice things like attributes, text content, etc.) just wasn't working. I could move nodes in to where they should show, and I could manipulate them, but they didn't have any renderings or XBL bindings. Major headache, especially since I confirmed repeatedly that my code was working correctly.
I planned on creating a reduced testcase by gutting a copy of Verbosio's code. I was not looking forward to that at all. Verbosio has a lot of little pieces that fit together, and removing a piece in the wrong order without putting an appropriate bypass patch in would have meant some other bustage. I understand the Verbosio code very well (to date I've written all of it), and by and large I think it's good, solid code that combines to make an application. Ripping Verbosio apart to isolate a bug feels like ripping apart a trunk engine to replace one broken bolt: arduous, time-consuming, and not very pleasant... but ultimately necessary.
I may still gut a Verbosio copy to isolate the bug, but suddenly, I'm not under pressure to do it. What I realized is that I was locked into a particular mental model ("you must use a XUL deck element here"), and that the model wasn't necessarily true. I was having problems with the deck, but not with ordinary XUL boxes. When I visualized the panels of the deck side-by-side horizontally instead of one-at-a-time, I saw intuitively that it could work, and work much better than a deck. It means the user can see more and do more at a glance than they would with the deck. Win-win situation.
It's another case of common sense in hindsight in application interface design. So there's an obscure layout bug that I hit. I have a workaround that is actually easier for the end-user to... well, use in the end. I'm perfectly happy with that.
Now, the funny thing is I wrote most of this code in the last 72 hours. I'm not entirely sure why, but I probably spent a good 40 hours coding and maybe 12 hours (including only four last night) sleeping. I literally went to bed at 7 am this morning, and I've only eaten once in 24 hours (dinner about, oh, three hours ago). It's one of those things I don't do very often. :-)