This evening, I realized there's something wrong with my virtual: protocol handler. It degrades gracefully to real protocols if a virtual file isn't in the file service, but it doesn't edit the loaded real file's links to other files. I can't point from a real file to a virtual one.
There's really only two simple solutions to that problem: either have everything a virtual file points to become virtual, or have nothing it points to become virtual. A little thinking makes it obvious you can't have everything virtual. If I'm editing foo.com, and I link to www.w3.org, the latter is content I don't have any control over, so that mustn't be virtual. On the other hand, if nothing's virtual, you lose a lot of the benefits of having a virtual protocol in the first place; you can't see the impact of one file on another.
After a little more thinking, I realized a deeper issue, a conceptual one. Specifically, some XML documents link to other files regularly (XUL, XHTML, XBL). But other XML files usually don't (MathML, SVG, RDF). It's a big consideration, one that HTML editors never had to think about. HTML is all about hypertext, which fundamentally is about linking.
This morning, I also hit on the thought that most files in a web environment are organized in a hierarchy of directories. This is something that's obvious to web developers, but it also means an editor should consider hierarchy. You usually have a common point of reference for all files and directories within the hierarchy. This common point is often called the "base URI".
When I edit web documents or files, I have to deal with one hierarchy in two or more places. There's a server-side base URI, where the files really live, and a client-side base URI, where I edit my local copies before uploading changes. Plus, if I'm doing any sort of server-side processing of XML, I probably have a staging area where I do tests away from the production website.
So I figure content I control belongs in a repository, with its top being a base URI. That lets me maintain multiple repositories if I want: a virtual one for on-the-fly editing, a local one where the files are directly accessible, a staging repository for testing, and a production repository for the actual files. At the same time, I don't have to use a repository structure if I'm editing individual files.
When you add it all up, it becomes ultimately an issue of user capabilities and user interface. Users will want the ability to edit stand-alone XML documents, and users will want the ability to edit interdependent XML documents. So the Verbosio back-end core code must be scalable to the repository model, while the front-end user-interface must be able to force a singleton model on the back-end. It doesn't work any other way.
No, none of this is talking about FTP uploads, CVS or SVN repositories, or any publishing methods. This is strictly about how Verbosio will refer to documents and files it edits. Publishing, while very important, comes later.
Creating XPCOM interfaces to store the needed information is easy. The hard part is in coming up with an intuitive user interface for matching local files (or directories) to remote locations!
With the completion of the virtual: protocol handler and virtual file service, I decided to begin releasing bits and pieces to the community at large. With that, I created a "grab bag" directory, purposefully not organized according to the final Verbosio source, and launched the Verbosio project with it.
Over the next few days, I'll be adding more goodies to the grab bag, pieces which will become part of Verbosio, but are not there yet. The grab bag is just temporary housing while I figure out the structure of things to come. There is no prototype Verbosio editor available yet, but I'm working on it, and may have something soon.
Feedback is welcome!
After two very long days of coding and one bug patch, I've almost succeeded in the third approach to the string contents as files problem.
My tests prove I can feed JS strings through a XPCOM service, a special protocol handler and a corresponding nsIChannel implementation into the browser. Most of it was heavily inspired by the data: protocol handler and channel implementation.
The only detail left unresolved is the handling of hash targets for XBL... a crucial weakness. (I've gotten HTML hash targets working.) The hash targets are part of the URL, but they're not getting recognized. This makes the protocol slightly better than data:.
There has to be some way to make this work. I just have no idea what it is. I've not seen any evidence of it in the protocol / channel pairs.
UPDATE: As far as I can tell, the problem is actually upstream of my code. My onStartRequest() and onDataAvailable methods get called normally, but my onStopRequest() doesn't get called until I decide to move on to another page or close the window. The HTML page through my virtual: protocol does call onStopRequest with a status of 0. So this is probably something nutty in XBL.
UPDATE 2 Nope, it landed squarely back in my face. I'd filed bug 306217, and debugging led bz, biesi and I through a long series of hoops. Eventually, biesi guessed (correctly) that I should've closed the output stream.
But with that fixed, I can declare my virtual: protocol handler operational. Not optimal, but operational.
I'm hitting a conceptual problem in Verbosio's development. Say the user is editing a XUL document, one sprinkled throughout with URL's: scripts, overlays, DTD's, stylesheets, you name it. Most of these point to files which presumably don't exist yet (for example, chrome://package/content/file.xml). So how do you load all these files from JavaScript strings, so the user can preview the changes he's making without saving?
I have at the moment four ideas, none of them very nice:
data: URI's
Basically, I feed it the content-type, then a comma, then the contents of the string as the second argument. The two disadvantages I know of are (a) URI's are supposed to be short, and data: might be abusing that, and (b) data: URI's don't support hash targets (#foo). That means they're useless for pointing within the document to a specific ID attribute (XBL depends on that pretty heavily, and (X)HTML likes it).
file:///userHomeDir/cache/path/to/file
This means saving the file in a cached form to the local file system. This is probably the simplest, but it involves reading and writing to the file system repeatedly. (Every time the user changes to the document or a file depending on it, most likely.) It also doesn't allow for granting chrome privileges that easily (though I suppose I could set CAPS security policies as needed). In case you're wondering, this is %APPDATA%/(appName) in Windows, not .../dist/bin/anything.
virtual:protocol:///path/to/file
This would mean creating a bunch of XPCOM components, starting with protocol handlers and channels. This is unfamiliar ground for me, but it would run fast and not abuse the system.
Something like this is very likely to destabilize the GRE running this chrome application, and would probably involve a lot of the same work as the previous idea. It's just not feasible.
I really, really need some help here. If left to my own devices, I will go with the second route. I would welcome implementations, except for the data: system. I'd also welcome other ideas to solving this problem.
Assuming jslib gets a fix for a perceived bustage in fileUtils.js soon, here's some code I whipped up for grabbing files from chrome, either jarred or flat:
const fUtils = new FileUtils();
var localPath = fUtils.chromeToPath(sourceURI);
var isJarRE = /^jar:(file:\/\/\/.*\.jar)!\/(.*)/.exec(localPath);
var fileAsString = "";
if (isJarRE) {
include(jslib_zip);
var jarPath = fUtils.urlToPath(isJarRE[1]);
var jarFile = new File(jarPath);
var jarContents = new Zip(jarPath);
jarContents.open();
fileAsString = jarContents.readEntry(isJarRE[2]);
jarContents.close();
} else {
var fileObj = new File(localPath);
fileObj.open("r");
fileAsString = fileObj.read();
fileObj.close();
}
This assumes you've loaded chrome://jslib/content/jslib.js and chrome://jslib/content/io/io.js already.
On a side note, I've finished the first step of the 34-step Verbosio smoketest. This code above was needed to make sure I did things right.
I really like XULRunner's elegant simplicity. No clutter around your environment, and it starts with a clean slate every time you call on it to run a XUL app. For those who develop applications with flat chrome, this is very nice.
One thing I've noticed missing, though. Creating your own component IDL's requires something that doesn't come with the nightlies: the XPIDL compiler.
I should have seen it coming. Actually, it makes a lot of sense. Your average XULRunner user won't need it -- it's strictly a development tool. Plus, anyone who's really developing entirely new applications should probably have compiled their own build.
If only my laptop could compile XULRunner... Microsoft VC6 has crashed at least three times trying to complete a XULRunner compile. So, I've decided to compile a standalone XPCOM application from Mozilla 1.8 branch, and hope that compiles IDL files well enough for XULRunner. That, and I have to teach myself how to use XPIDL... I only hope Chapter 8 of Creating Applications With Mozilla is still up-to-date, and that XPIDL-generated code in one Mozilla executable works just as well in others.
I haven't touched Verbosio in a little while, and this evening I realized what I needed to do was to create a demo / smoketest that Verbosio must pass to have rudimentary functionality. The final document would render identically to this sample document.
The HTML style element would be missing, replaced by appropriate <?xml-stylesheet ?> processing instructions. Still, creating this without having access to source code could get interesting.
This demo has 34 steps:
* Create root document. * Select root node. * Insert text node: This whole sentence is red, while the word "sentence" is always larger. * Select the text node. * Colors UI, select red, choose "Surround", hit OK. * Select the first word "sentence". * Size UI, choose set. * Precondition, select "Bigger", choose "Surround", hit OK. * Select the second word "sentence". * Size UI, choose set. * Precondition, select "Bigger", choose "Surround", hit OK. * Select root node. * Colors UI, choose green, choose "As Last Child", hit OK. * Select green node. * Insert text node: This whole sentence is green, except for the word "sentence", which is black. The word "sentence" is also a hyperlink to "about:mozilla". * XLink UI, type is "simple", href="about:mozilla", choose "Surround". * Select the word "sentence" in green text node. * Precondition, hit OK. * Lather, rinse, repeat for the other two "sentence" mentions. * Colors UI, choose black, choose "Surround". * Select the word "sentence" in green text node. * Precondition, hit OK. * Lather, rinse, repeat for the other two "sentence" mentions. * Colors UI, select blue, choose "Insert before" * Select green node. * Precondition, hit OK. * Select blue node. * Insert text node: This whole sentence is blue, while the word "sentence" is always smaller. * Select the word "sentence" in blue text node. * Split text node. * Select the text node for "sentence". * Size UI, choose set. * Precondition, select "Smaller", choose "Surround", hit OK. * Lather, rinse, repeat for the other sentence mention.
Basically, the goal is to create a specific XML document from scratch. The document would have three distinct XML languages: one for colors, one for sizes, and the root document language (which would be just a wrapper). That means five distinct user-interfaces working together: Verbosio's core UI, one for colors, one for sizes, one for XLink, and one for the root document creation. This doesn't count the user-interfaces for actually viewing the XML document in different ways...
Implementing this will give me an excellent idea what works in my design, what is too cumbersome and should be redone, and what is just plain evil. One thing I know is evil right off the bat is denying the user the ability to edit the source code of the XML document directly, but that's a key factor in this smoketest. If the user can create this document without editing source code, then I've done my job right.
Doing this will be hard, but it will not be impossible. It will very much be worth it.
Javadoc-style comments are wonderful. Doxygen uses them to generate on-the-fly documentation. They describe properties, methods, arguments of methods, return values, etc. To compilers and interpreters, they're just comments. So why can't we do that for XBL?
For most XML languages, it wouldn't make any sense. But XBL-implemented properties, methods, and event handlers are perfect for javadoc'ing, with XML-style comments.
Instead of:
/** * Function description. * * @param firstArg The first argument. * * @return Boolean true if the first argument is a number, false otherwise. */
We could have:
<!-- - Function description. - - @param firstArg The first argument. - - @return Boolean true if the first argument is a number, false otherwise. -->
I would love to see Doxygen modified to provide support for this. I would love to see Mozilla's toolkit(s) use this style of commenting to document the widgets. I would love to see this style used in whatever xulwidgets project I'll launch.
I'd also like your opinions. A little XML commenting in bindings never hurt (except in bindings' content, where it hurts a little). Is this the right style, and is it worth doing?
UPDATE: Okay, how about variation 1:
<!-- /** * Function description. * * @param firstArg The first argument. * * @return Boolean true if the first argument is a number, false otherwise. */ -->
This follows the old commenting style, and embeds it in XML comments. Or, variation 2, suggested by Nico:
<!--* - Function description. - - @param firstArg The first argument. - - @return Boolean true if the first argument is a number, false otherwise. -->
This one takes less characters, and still has a special format to identify the comment to a system like doxygen. I personally prefer #2, as the javadoc comment then remains adjacent to the method, instead of wrapped in another comment.
I am well and truly overwhelmed by the responses I received. 35 in about 24 hours. I think we can all learn a little about community efforts from this example.
I gave a very simple, harmless test, without explaining why. Community response was huge. This is the essence of quality assurance work.
I went wading through a list of mail/news bugs today. The sheer number of bugs there which are not properly triaged is staggering: over five thousand from a simple search (excluding any summary searches). I'd bet 60% or more are UNCO. The mail/news developers simply cannot use Bugzilla under those circumstances. I know, because one of them said so to me earlier today.
We're coming up on a tree closure soon. I think it's the perfect time for a Mozilla Development Stand Down, to help out with QA & triage.
I'm serious: we need to schedule such an event. A time when we just take a breather from active dev work, and contribute an all-out effort to getting our affairs in order. One to three work days (excluding weekends) where triage is the priority. The hard-core hackers of our community, who know the codebase well, would be available to answer questions.
I know a fair number of those hackers will not heed my request. Certainly none should who are working on the Gecko 1.8 release! But if we really take even one full day for bug triage exclusively, I'm sure our developer community will appreciate it immensely. It will help them get a clear picture on the bugs that drive most of us crazy.
What if there's not enough time for a Stand Down during the freeze? That's always a distinct possibility. I would suggest someone write a community petition to mozilla.org's drivers for a Stand Down in that event.
Yes, it's a big leap from a simple test to a massive QA effort. But consider: Mozilla development has continued uninterrupted for about seven years now. Some of the problems, including QA and documentation, have grown so large that many of us feel we can't do anything about them. That attitude has to stop. It will only stop if large numbers of people unite to make it stop. We can do this. We just need to pick times and goals, and strongly suggest to the community as a whole that we do this.
Thank you for your time.
P.S. I decided to close the comments on the original thread because my goal had been accomplished, and I didn't want more comment spam to approve one-by-one. :)
http://www.cnn.com/2005/WORLD/asiapcf/08/09/game.death.reut/index.html
:|
You guys rock, you know that? I posted the test request an hour ago, and already I have enough conclusive evidence to file a bug.
Here's the situation.
I'm developing a XBL binding to do what Venkman (ye handy dandy JavaScript Debugger) does with floating panes, except it won't create new windows every time. So, I want to use global toolkit icons for the various pieces of the widget. I would use the Close.gif image I asked you about to hide a pane from the user.
In SeaMonkey for Windows, chrome://global/skin/icons/close.gif works. It doesn't work for Mozilla Firefox 1.0.6 (WinXP). The URL I gave you, with a capital C, works for both SeaMonkey and Firefox.
This sort of discrepancy is not good for developers. I'm actually surprised both worked in SeaMonkey. But I needed to know if it would work across the board... and you've proven that it won't.
I don't want to spend my whole time creating theme-specific stylesheets. Global is global is global. I'm filing a bug very shortly on this. I'm not filing because Close.gif doesn't work... I'm filing because I don't have a consistent choice to rely on.
Thanks, everyone!
UPDATE: I've filed Bug 303974 based on your feedback. Anyone want a "good first bug" to create a patch for?
Please load chrome://global/skin/icons/Close.gif into your browser, exactly as typed.
I just need a quick test on the following platforms to see if the X renders:
| Mozilla 1.7.x | SeaMonkey | Firefox 1.0.x | Deer Park | |
|---|---|---|---|---|
| Windows | - | + | ? | + |
| Mac | - | - | - | - |
| Linux | ? | - | ? | ? |
Legend:
Please comment. I am hoping the community can help me fill in the blanks, so I know whether I need to file a bug or not.
(Note: I got the icon on WinXP, SeaMonkey and Firefox 1.0.6.)
UPDATE: Enough already. Comments closed.
I remarked yesterday about the session I spoke at. Now let me comment about how the OSCON went overall for me.
First off, I walked out of there with good introductions to Eclipse, Bacula and Subversion. (Subversion I'd been exposed to at ManyOne, but I now know I was doing a lot of things wrong.) Good stuff, though I was a little underwhelmed at Eclipse. Its XML capabilities leave a bit to be desired, and I'll be looking into Axel Hecht's Solid extension for it shortly. As for Bacula: open-source backup system for Windows. I need that.
I also got introduced to XSLT in such a way that I started to understand it. The last time I attempted to learn XSLT, I was buried under it because I had to learn XPath as well. I've got a good grip on XPath now, so the XSLT tutorial made a little sense to me.
I attended a couple Mozilla talks and learned a few things. For instance, I'd been opposed to Greasemonkey on the grounds of editorial control -- but I learned that wasn't what it was about at all. I even walked out of that talk with a bit more knowledge: a little known method of XPCComponentsUtils called "evalInSandbox"... I need to play with that one!
At another Mozilla talk, I was introduced to xpistubs. Being a big fan of the makexpi.pl toolkit, I think this might be something to look into as well.
On Wednesday, I attended a session on "Writing, Reviewing, and Instigating O'Reilly Books". Having written one book before, I wouldn't mind another opportunity. With the Firefox 1.5 release coming up, I'd say books.mozdev.org is due for a second edition... Ian? MozDevGroup? What do you think? :)
On Thursday morning I was introduced to a really cool idea, the JavaScript Archive Network. The t-shirt on the back says something like "CPAN".replace(/CP/, "JS"). This should've been done years ago, and I'll try to send them a couple gems I've drawn up.
Asa Dotzler gave his keynote this morning on "Linux In Search Of The Desktop", and I thought it went very well. He was right on target, and echoed a few reasons why I still develop on Windows myself.
That's all I can think of right now. Last year when I went to OSCON to speak, I attended every Mozilla session I could think of and was bored out of my mind. Not that it was boring, just that there was nothing I wasn't really familiar with. This year, I diversified, I picked up a lot, and I had a blast. Strangely enough, I passed up on the Ruby on Rails sessions entirely. I just didn't have any interest in that this year, a decision I may come to regret in the next several months for some odd reason.
Thank you, O'Reilly & Associates!
I just finished a panel, with Daniel Veditz and Nitesh Dhanjani, on issues that confront open-source developers when it comes to security. My deepest thanks to the two of them. This session could not have been successful without them.
I think we should do this panel again next year.
I'm thinking we really should have a mozdev project for useful XUL widgets, written in XBL. Somewhat like JSLib, but for XUL.
I've put out several widgets before, and I continue to work on new ideas.
Who else wants a bigger toolkit? I've got so many ideas, easy to do in XBL, for additional bindings. Who has their own custom widgets to contribute?
P.S. The Open Source Convention is a blast.