One of Abacus's goals is to define and implement a XML language of analytic geometry. So I'm working on JS code to implement a subset of the language.
I'm also using the JS code as an example I'd like to write an article on, contrasting XBL and XTF for implementing XML languages. One emphasis of this article is on including more than one element in the language implementation.
There's only one problem with picking a practical example. Namely, to make it work, you have to write a lot of code. Probably more than the article's readers would want to wade through... I'll cut a part of it for the article.
But, when I get it finished, I'll have a nice long demo of how you can create XPI's to install XML languages into Mozilla.
Now, where will I find a technical journal willing to buy the article... :)
First, install the newest JSLib (thank you very, very much, Mr. Collins -- and sorry for lambasting your crew earlier). Then if you're in the mood for something that works but will have a good deal of behind-the-scenes revamping soon, try out Abacus again.
I'll probably do some work on it over the next few days, and if I'm lucky, Abacus 0.1.3 will come out mid-month.
Next on the hit list for Abacus? Just a nice XHTML editor.
Abacus 0.1.1 is broken in recent Mozilla builds. Sometime in the last six months, there was an upgrade to Mozilla's XML parsing. The upgrade, thanks to some not-quite-sane sanity checking I did for 0.1.1, caused Abacus to hang Mozilla in an endless JavaScript loop.
Oops.
I've fixed that in my local Abacus, and with another fix to JSLib I've posted a patch for, Abacus 0.1.2 should be available soon. No new features, just making it work again. (Really, it's ready now, but I want to upload it directly to mozdev, not here.)
Then I'll start work on Abacus 0.1.3, which will be a bit streamlined and support much more MathML.
After much thought, I've realized that the internationalization efforts of editing MathML (not the application, but the generated markup) is really a waste of time. Not that many people in France will care to have markup in documents they edit to have, say, Mandarin presentations of the mathematics.
This means that a lot of the code I struggled to write will be rewritten and simplified. The end-user won't notice, really, and the code will be easier to maintain. It's also a big enough change that it's a challenge, and I like challenges. They drive me.
So, Abacus is going to get a sizable rewrite in the next few weeks, and you'll see a version 0.1.2 based on it, most likely.
No, not really. I just need a good incentive to work on it again.
My incentive for working on version 0.1 (and 0.1.1, which I released when I realized 0.1 was broken) was the O'Reilly Open Source Convention 2004. Since then I've received e-mails about it from about a dozen people. That's over the past six months.
It's a little disheartening. :)
I've had other things take up my time, and not a lot of motivation to get Abacus going forward again. No one has filed bugs or offered patches to it, and no one's indicated they're actively testing it. (I've had a couple people, including one today, who said they want to see it working in N|vu, but there's a N|Vu bug with XPInstall that stops Abacus in its tracks.) All the work I did on Abacus was on my dime. Nobody's really reviewed the code.
Abacus is stagnating because it's a community of one.
In a way, I suppose that's my fault. Editing mathematics isn't as "sexy" as IRC chat, user-friendly browsers and mail clients, Gmail notifiers, etc. I picked a niche, and I got a niche community. :)
(I should note my serverpost widget, which seems a lot more immediately useful to the Mozilla app developers crowd, is also a community of one...)
Oh, well. At least I haven't gotten any real complaints about Abacus, either.
There are a number of (unfiled) bugs in Abacus, and there are some bugs in Mozilla which Abacus tries to work around. I'd really, really appreciate having some developers hammer with it a bit, file bugs as appropriate (in the right bug database), and start fixing them.
I really don't want to be the only developer on Abacus. I've spent too much time on this alone, and there's a lot to do. I really, really need help at this point.
On another note, one feature I would like to implement for Bugzilla would be a "transfer between installations" feature. It would probably be something like setting dependencies between one database and another. The easy part would be writing code and MySQL tables to store the dependencies. The hard part would be writing protocols for Bugzilla installations to talk to each other... including failure cases for when a cross-database dependency doesn't take effect. I wouldn't mind hearing opinions on how we could do this.
Incidentally, this is my 100th blog entry.
The website is up!!!
As of this moment, the XPI file for Abacus isn't quite available yet (I don't know why, it got checked in with everything else). I assume it is propagating to the download servers as we speak.
I am once again a happy man.
UPDATE: Yes, the XPI file is available. I have just tested the screenshots, and the files are horked on my local drive. I'm checking it now.
Incidentally, there is a Windows crash bug 258365 with Abacus right now. It appears to affect only Windows on 1.7+ builds. Be warned.
UPDATE 2: Screenshots have been fixed.
After a little floppy hell (I need a new computer badly), I managed to put this out for your consumption. The documentation and launch of the abacus.mozdev.org site should be underway (assuming pete@mozdev can grab the .tar.gz I put up for him too).
It doesn't yet work in N|Vu (I don't know why), and there is a Windows crash bug which this package reveals, but that's the best I could do.
A few weeks ago, I griped about a crasher bug involving DOM appending a node, a bunch of XBL bindings being constructed (descendant nodes of the appending node), and JS not being directly responsible. Fortunately, it was a side case that I could do without for Abacus 0.1.
I didn't want to, but I didn't have a whole lot of choice.
So, in releasing Abacus 0.1, I managed to foul up the directory structure for the .tar.gz and zip releases (which is why it didn't work). It was an ancient bug I'd forgotten about and manually hacked to make work. I put together an XPI for Abacus, and decided I'd better rerelease it as Abacus 0.1.1 -- acknowledging that I goofed on the 0.1 version.
That XPI is finished, and I'm working on the abacus.mozdev.org website for a simultaneous release and documentation of the project.
Except that nagging little crash is back... and this time, it affects me every time I click the "Apply Template" button. Such is a critical piece of the Abacus user-interface, and failure is not (much of) an option here.
Now, here's the weird part: On my Mandrake Linux 9.1 o.s., it never crashes at the same part in my code. So my attempts to get a stack trace detailing the problem are somewhat useless.
Needless to say, I am even less happy than before.
Abacus 0.1.1 will be released concurrently with the launch of abacus.mozdev.org in a few days.
Download file as tar.gz archive
Installation instructions (if these don't work, let me know what needs to be done):
* Requires Mozilla 1.7, Mozilla Firefox (untested!) 0.9, Nvu 0.3 (untested!)
* Requires JSLib installed from jslib.mozdev.org
* Edit /chrome/installed-chrome.txt to include the following lines:
content,install,url,resource:/chrome/abacus/content/
skin,install,url,resource:/chrome/abacus/skin/modern/
skin,install,url,resource:/chrome/abacus/skin/classic/
locale,install,url,resource:/chrome/abacus/locale/en-US/
* Unzip the file into your chrome directory. This should create an abacus directory in your chrome.
* Shut down Mozilla or your application entirely.
* Delete /chrome/chrome.rdf and /chrome/overlayinfo .
* Restart Mozilla, open Composer Application
* If everything worked, there should be a MathML button to the right of the spell check button.
If you're in Mozilla Firefox, the only URL you will be able to use right away is chrome://abacus/content/templates/editor.xul . (This is a MathML Template Editor.)
If you're in Nvu 0.3, you should be able to work with it right away.
Documentation is forthcoming.
XPI Request: I've not had the chance to take this code and make a cross-platform installer for Mozilla. If you are willing to contribute one, PLEASE have the XPI output flat-file chrome! Because of the MathML Template Editor requirements, an abacus.jar file is unacceptable at this time.
I am not quite launching abacus.mozdev.org yet; it took a little too long to prepare my code for release. So, the mozdev group will not be happy with me in a few days... Also, if there is something seriously horked with the installation (I doubt it), I will silently replace the code and post a notice here.
O'Reilly & Associates is hosting the slideshow I used at the Open Source Convention 2004. It's an OpenOffice document.
You wouldn't think Abacus could introduce a potential security concern, but...
At one point in my pre-release software, I mandated that a mEdit:execute attribute's value would conditionally run through an eval() statement. I tried to restrict the possible activities by making sure the statement to be run (the value was split by semicolons) contained a particular string of code representing a value:
<foo mEdit:execute="mEdit:loopVariable('x') += 2;"/>
Unfortunately, that wasn't in my opinion good enough:
<foo mEdit:execute="alert(hiddenVar) //mEdit:loopVariable('x');"/>
It bothered me. I wanted really to only allow expressions of the first type, not the second. So, after a little thinking, I came up with this:
function test(untrusted) {
var re=/x\s*[\+\-\*\/\%]?=\s*\d*/;
// For now we are deliberately forcing the first character to be x.
// This illustrates for the example how closely we are watching the expression.
var matches = untrusted.match(re);
if ((!matches)||(matches.length != 1)) {
return false;
}
var match = matches[0];
if (match != untrusted) {
return false;
}
return true; // this means we would execute the code
}
The "x" variable in the regular expression will have something else there instead. I just used it to simplify my testcase.
Thanks to Justin Wood for consultation on #mozilla when I was trying to figure this out.
There is still some work that needs to be done to this code to make it an 0.1 release. I have not yet gotten it to work in Mozilla Composer (more on that in a moment), and one of the more important features, adding a new annotation-xml with encoding and xml:lang attributes, is not developed fully yet.
Of course, there are some other issues, notably localization and MPL'ing the whole thing; my entire source code at this point is available upon request under the MPL 1.1/GPL 2.0/LGPL 2.1, even if it doesn't explicitly say so. I do not anticipate writing any Abacus code that is not offered under this scheme at the present time.
I'll be writing documentation for the 0.1 release as well; I'm scrambling to release Abacus 0.1 for the Open Source Convention (which starts Monday, so...)
On Mozilla Composer: it's going to be easy once I figure out how to write a Range.prototype.replaceContents() function. Seems the DOM-2 specification missed that... I need something like this in order to correctly replace a selection in the document being edited (which is rather important, in order to give Composer the same feel in replacing text and hypertext being edited as it has now).
Also, Nvu developers (Daniel Glazman, I hope you're reading this!) should probably take note of the way I apply stylesheets in the overlays. That is how the Document Object Model says they should be applied, and with DOM-2 Events, I can control very precisely when they apply and when they don't. When the user calls on a "save" or "save as" command, DOM-2 Events will let me remove the stylesheets before that command executes. This is very important, particularly for Composer extensions such as Abacus.
I have some strong opinions about Mozilla Composer; there are some aspects of the design I decidedly do not like, for the simple reason that to extend the current design in any significant way, you have to do your homework. Overlays like mine have a very tough time of following Composer's rules. I'd love to have a shot at stating some design goals for a new set of rules...
Of course, I'd also love for someone to ask me to work for them at OSCON... :)
(In case you're wondering why I would release a set of files prematurely, there's two reasons. One, I need a backup set just in case all hell breaks loose between here and OSCON. Two, it's about time I showed something to Mozilla enthusiasts everywhere for analysis and tinkering. It's open-source for a reason!)
Once again, a quick reminder that the Open Source Convention from July 26-30 in Portland, Oregon will feature several speakers, including me and my session on the Abacus MathML Editor project. With luck (probably without it, too, come to think of it), I'll be releasing an Abacus 0.1 and launching the official abacus.mozdev.org website before the convention.
My speaking session is on Thursday.
The MathML expression after applying the template
Moving the selection up to a sub-expression (useful for replacing larger parts at once)
You may not see the content MathML, but I assure you it is there... :)
I've just found a crasher bug in Mozilla 1.7 RC1. It's bad enough to block what I had called a required feature for Abacus 0.1.
It is the worst kind of crash I can imagine:
* A single untested line of JS makes a DOM call to insert a node.
* That single line executes several XBL bindings (<constructor/>).
* These bindings have been repeatedly tested and work perfectly.
* A run through the JavaScript Debugger shows that these lines do not cause the crash.
* Immediately after all these tested lines run, JSD returns to this one untested line... and THEN Mozilla crashes.
In other words, though it's definitely 100% reproducible, a "minimal testcase" from this code is virtually impossible. Just to run the current testcase, you have to have Abacus installed... and of course, I'm the only one in the entire world who has Abacus installed. Under these circumstances, filing a bug at Bugzilla is a waste of time, more so because I cannot connect to the Internet from my home computer, where Abacus is being built. So I can't even use Talkback to give a stack trace to mozilla.org.
What was this blocked feature?
Well, one goal of Abacus is to allow you to create new MathML semantics "branches" (I don't know what else to call them), which (aside from the first branch) are children of annotation-xml elements that the reader can choose from. The first branch is the content MathML encoding (which is never rendered), and the others are XML-based annotations (for instance, a MathML-Presentation annotation in the en-US language).
To create them, Abacus has to have some idea of how the already-existing branches could be created using Abacus. So I've spent a couple weeks figuring out just how the user can teach Abacus to do that. (The assumption is that Abacus could be used to edit MathML generated by another application.)
One key portion of this special user-interface just went thermonuclear.
Needless to say, I am not happy.
I'm glad I haven't released an 0.1 of Abacus yet. Because I had to go back and damage what I've already got in order to get another feature working.
To explain: I'm working on figuring out a mechanism for creating a detailed presentation "branch" (as opposed to simple ones you create in the Template Editor), based on a content MathML "branch" of the semantics element. The first step is of course to figure out which content MathML templates from the template editor could be used to recreate the current MathML fragment. (In English, that means figuring out how the user wrote it before.) From there, I can then directly translate to the appropriate templates in presentation MathML, XHTML, what-have-you.
I think I solved the first half of it, the search portion. The only complication I see on the second half is from the fact that I permit and recommend a "fitb-set" element frequently. It's basically a for-loop in XML, so I need to code for that appropriately.
... when you finish basic testing of a feature that lets your pet project write good markup, and then turn around and read the markup back in.
I know I'm good, but I can't help thinking that there was some serious divine intervention in making it work so quickly. Gervase Markham isn't the only one hacking for the good Lord. 8-)
I contemplated putting in an easter egg about that sort of thing, but I will probably not include it. Given that I intend this project to eventually be used in public education, and this country's First Amendment guarantees, it would mean a scandal when schools figured out the easter egg.
Incidentally, this happens to be the 75th entry in my weblog (not that it matters).
Over the weekend, I wrote out code for exporting MathML content from the Abacus MathML Editor into the application that calls on it. It was remarkably painless.
Just as important, though, is figuring out how to import MathML content from the caller app into the editor.
That has so far meant two days of straight JavaScript code writing, and will probably take another day of testing and debugging to get it right (hence the title of this entry).
Already for the 0.1 release, I'm forced to make some sacrifices which probably will not endear me to those who want a MathML editor in Mozilla.
For one, I'm currently working on making sure Abacus can read what it writes. This is one aspect of "dogfood" as I see it. (Remember, Abacus is oriented towards content MathML, with presentation MathML as annotations.) Once that's done, I will work on making it read MathML markup which is exclusively content MathML.
Yes, that means presentation MathML gets left in the dust.
It's too bad, really. Amaya (which as I've said before was a big inspiration for Abacus) only supports presentation MathML. But the problem for Abacus lies in the fact that presentation MathML does not convey exact definitions. That's why we have content MathML. Trying to deduce content MathML from presentation MathML is as bad as trying to deduce what the browser should do from a "tag soup" of non-standard HTML. It's a classic apples and oranges situation.
This breakage -- the emphasis on content MathML at the expense of not supporting pure presentation MathML -- is a very undesirable side effect. How many MathML editors output presentation MathML only? How many of them output content MathML as well?
I'm trying to figure out how to create a UI that will let the user guide the MathML Editor through the process, given the templates that Abacus makes available. But for the moment, I'm completely stumped.
By the way, a lot of code can be written in two days... for this one feature, preparing a MathML fragment for Abacus, I'm already up to at least 600 lines of pure JavaScript...
Specifically, the key feature, overlaying one MathML template onto another, works perfectly.
I'm working on some other features for the editor, some of which are pretty important (being able to import a MathML <math> element that Abacus didn't create, for one big example). But all in all, what I've got now looks pretty slick, I have to say...
By the way, I'm looking for ideas on bug 247849.
I'm a little surprised by how much JS code it took to handle editing XML files directly in a specific manner. But I can now claim that my template editor for the Abacus project is now at a point where I don't need to edit the files with a text editor.
This means that one-third of the Abacus MathML editor project is stable (if not quite complete). The next part, the actual MathML editor, I don't anticipate a lot of problems with. Though when it comes to reading in a MathML fragment that's already there and not Abacus-compliant yet, I may have some trouble. I haven't figured out yet how to take in a MathML fragment that's all presentation markup or all content markup yet, and sooner or later I'll have to...
I really wish I could release an 0.1 of Abacus based on what I have right now, but without a working MathML editor to demonstrate just what I'm doing, it would be counterproductive in the extreme.
For most XML documents, one or two namespaces is enough. XUL applications often require three or four (XUL, XBL, RDF, XHTML on occasion). For Abacus, I had enough namespaces where a common DTD actually made a little bit of sense...
The best part about it is I also have a script in there which I can include by &namespaces.script; entity.
The template editor is nearly done! I finally took the time for the editor to save its own templates. So now, things are looking very sharp indeed.
Just as a quick sample, I'm including current XML and DTD files for how Abacus template files should look. (This is not frozen, however! Depending on my experience in editing, these files may change.)
templates.dtd -- templates XML language
summary.xml -- pseudo-documentation of how things should be organized in a templates file
A sample template file under construction
A base templates file (wrappers for each language)
Yes, I know they have Windows newline characters in them (\r\n). They will go away.
