« December 2005 | Main | April 2006 »

February 19, 2006

Python and JavaScript

Mark Hammond's work to support Python in XUL is nearly done. The DOM_AGNOSTIC2_BRANCH should land in the next few weeks. Already I see many on the PyXPCOM list testing Mark's fine work, chomping at the bit to use Python in XULRunner.

This brings to mind a hot topic in my recent hacking: infusing JS with Pythonic generators and iterators, including array comprehensions. Brief taste: 

js> function count(n) {
    for (var i = 0; i < n; i++)
        yield i;
}
js> g = count(10)
[object Generator]
js> g.next()
0
js> g.next()
1
js> two_to_nine = [i for i in g]
2,3,4,5,6,7,8,9
js> squares_to_20 = [i * i for i in count(20)]
0,1,4,9,16,25,36,49,64,81,100,121,144,169,196,225,256,289,324,361

Why borrow from Python for iteration, generators, and comprehensions? I typed this into a wiki recently:

Given the years of development in Python and similarities to ECMAScript in application domains and programmer communities, we would rather follow than lead. By standing on Python’s shoulders we reuse developer knowledge as well as design and implementation experience. The trick then becomes not borrowing too much from Python, just enough to gain the essential benefits: structured value-generating continuations and a general iteration protocol.
The specific context was iterators and generators, not general Python imitation. There is no point in trying to turn JS into Python, but I see little to gain from differing gratuitously where the two languages are already quite similar (arrays/lists, for/in loops).

JS, unlike Python, was a rushed little hack for Netscape 2 that was then frozen prematurely during the browser wars, and evolved significantly only once by ECMA. So its early flaws were never fixed, and worse, no virtuous cycle of fine-grained community feedback in the form of standard library code construction, and revision of the foundational parts of the language to better support the standard library and its common use-cases, ever occurred.

This is changing, finally. Great toolkits such as MochiKit and Dojo have been developed and disseminated (with docs and tests, even). I should also mention OpenJSAN.org as a welcome development. We can finally see the outlines of a standard JS library system. Of course, this space is young and necessarily fragmented. I will blog some concrete thoughts soon on how browsers can expedite the evolution of a proper standard library.

But what about the core JS language? It's not going to go away, or turn into Python. It needs to evolve, and to co-evolve with libraries that become standard. How can this all happen in relatively short order, and on the web?

As noted here before, I am working in ECMA TG1 with colleagues from Mozilla, Adobe, and Microsoft. Adobe's interest lies not only in ActionScript, but also in SpiderMonkey, which has been embedded in Acrobat for years.

I'm happy to announce that we are now working with Dave Herman, a fourth year graduate student at Northeastern, of PLT and lambda-the-ultimate renown, whom I invited as an expert to help ECMA TG1 develop sound specifications for critical parts of ECMAScript Edition 4 (ES4), also known as JavaScript 2 (JS2).

We seek a coherent mix of static type checking if you want it and dynamism if you don't, but with better dynamic checks and blame reports. We aim for completeness defined as helping programmers reliably program in the large as well as the more typical (for JS, until version 1 of your web app ships ;-) prototyping at a smaller scale. With JS2 and (I hope) ES4, you should be able to extend the core language without being forced by a too-near cliff into writing C++, Java, or another "host language".

The wiki in which I typed that brief "let's be Pythonic" argument quoted above is a docuwiki instance set up for ECMA TG1 by Graydon Hoare, who (I'm thrilled to report) has joined us at Mozilla recently. We hope to make this wiki world-readable at some point not too far off. In the mean time, I will blog with more details about the emerging ES4 language design, and about what features from it will show up when in JS1.7 and higher numbered versions to be shipped in Firefox 2 and 3.

My gut says that we will successfully evolve JS into a much stronger language, with a better and more standard library ecosystem, for the next ten years of its life. I will wager that other browsers will follow the ES4 standard, possibly even in 2007. If necessary, I'll do the work to extend KJS (or KJS2 or whatever it's called) for Safari ;-).

Posted by brendan at 4:18 PM | Comments (20)

February 9, 2006

Fresh XPCOM thinking

Everyone who gets far enough into Mozilla code has that "wow, this is chatty... verbose... inefficient even" reaction to XPCOM -- or so I thought. Having played Cassandra once in the dark days before Netscape 6, lived to witness deCOMtamination, and watched the next generation of core hackers grow up wise from birth, I foolishly believed that we were well past worrying about XPCOM abuse.

But we aren't. Perhaps SVG-the-standard is to blame, but not exclusively. We have to do better, and not just with peer pressure, but I'll spend a paragraph on exhortation too:

Please don't imitate COMtaminated code. Don't use XPCOM in core rendering and layout data structures. Do use XPCOM where it wins, as high-level inter-library and inter-programming-language glue, where the QueryInterface and virtual method call costs are noise, and the benefits for programming in the large are obvious.

More than warning again, I believe it is time to change XPCOM in deeper ways, to fix several problems:

  1. The inability to free cyclic structures automatically.
  2. The code bloat and cycle costs of nsresult return type.
  3. The lack of nsIClassInfo implementation.

Probem 1 means memory leaks, forever, because we'll never cover all cases, and new cases are being added constantly (sometimes just with JS completing the cycle, no C++ changes -- so via a Firefox extension, for example).

Problem 2 makes our source code overlarge and hard to read, never mind the terrible effects on register pressure, instruction cache utilization, memory bandwidth wasted on the out param that wants to be the return value, etc.

Problem 3 means we can't readily reflect on instance-of relationships in JS except for DOM objects. Having DOM class information (what interfaces does this object implement? is this object thread-safe?) is cool, but we still don't take advantage of nsIClassInfo (e.g. for the solution to problem 1 suggested below) for want of ubiquity. (Another example: we believe we could use it to avoid all locking costs in XPConnect.)

These problems can be solved:

  1. I pointed dbaron and graydon at work by David Bacon et al. on reference-count cycle collecting that we could infuse into XPCOM, curing all cycle-induced leaks. We may hope to have fixed the last such leak, but Murphy was an optimist. XPCOM without some kind of cycle collector is like a car without a brake.
  2. I'm looking for volunteers to help run some experiments testing the state of C++ exception support on the compilers we care about (mainly GCC and MSVC). What is the code size increase from turning on exceptions (rtti too if required)? That's the first experiment to run. Actually converting our mega-lines of C++ code to use exception handling instead of nsresult returning and testing will be real work, and lots of it. I have some thoughts for how to semi- or fully-automate that work that I'll blog about soon.
  3. Right now the way you implement nsIClassInfo involves painful C macros, so no one does it. With some C++ sugar, and with the right defaults, defining class information for each concrete XPCOM class should be simple, so we can actually count on it being there.
These problems should be solved, and I'm committed to working to solve them. Who is with me?

Posted by brendan at 2:57 PM | Comments (27)