Burning Chrome

What the bail-out really means

The bailout in simple terms

Back to Basics: Judo with nsIVariant

With XPIDL, XPCOM components can be very strict about what they allow. Simply put, if you try to pass in an argument to a XPCOM component, and the component requires a type that your argument doesn't support, it won't work. In C++, your program won't compile; in JavaScript, XPConnect throws exceptions. This is a good thing.

Many of the basic data structures - nsIArray, nsIPropertyBag, etc. - take a nsISupports argument. This covers most objects you construct and work with. This is all well and good... unless you want to pass in a raw type such as a number, a true or false value, a string, etc. Then you're stuck.

Fortunately, all is not lost. There is an interface named nsIWritableVariant, which XPCOM provides for wrapping native types in an nsISupports object. (There are type-specific interfaces like nsISupportsPRBool as well, but nsIWritableVariant is an one-size-fits-all solution.) Even more interesting, its read-only companion, nsIVariant, is "magical" to XPConnect: JavaScript receives its values as native types, not as nsIVariant objects.

Thus, nsIVariant turns the strengths of XPIDL barriers so that they are no longer fighting you, but working with you. This is what I am calling judo with nsIVariant. Read on in the extended entry for more details.

Creating a variant

In C++, there's a number of methods for setting types. If you have a PRUint8, you can call setAsPRUint8(val), for example. The rest of the methods are pretty self-explanatory, but two methods are also interesting: setAsISupports() and setAsInterface(). These two allow you to wrap any nsISupports object in a variant.

In JavaScript, almost all the methods are available, but there's a nice shortcut: the setAsVariant() method. The beauty of this is that you can pass in any JavaScript value to this - XPConnect converts the argument into a nsIVariant for you. This has special, "magical" implications.

Retrieving a variant

In C++, you have to work a little bit. First, you have to get the data type (call GetDataType() on the variant). This will tell you which (if any) methods of nsIVariant to call.

In JavaScript, it's even easier. If the interface you're accessing specifies the argument is a nsIVariant, then you don't have to do anything - XPConnect will convert it for you. This is more of the "magical" nature of these variants. If the interface specifies nsISupports, then you'll get back a nsISupports object which implements nsIVariant. You can get the native value by querying the object for nsIVariant, as shown below:

js> var variant = Components.classes["@mozilla.org/variant;1"].createInstance(Components.interfaces.nsIWritableVariant);
js> variant.setFromVariant(2)
js> variant
[xpconnect wrapped nsIWritableVariant @ 0xe27280 (native @ 0xe0e6c8)]
js> variant.QueryInterface(Components.interfaces.nsIVariant)
2
js> typeof variant
object
js> typeof variant.QueryInterface(Components.interfaces.nsIVariant)
number

It's worth noting that, technically, XPConnect is violating the rules of nsISupports here. The QueryInterface() method should technically always return an object that you can call QueryInterface() on again. This is not the case for nsIVariant:

js> variant.QueryInterface(Components.interfaces.nsIVariant).QueryInterface(Components.interfaces.nsISupports)
typein:8: TypeError: variant.QueryInterface(Components.interfaces.nsIVariant).QueryInterface is not a function

On the other hand, it is consistent with JavaScript object identities:

js> var variant = Components.classes["@mozilla.org/variant;1"].createInstance(Components.interfaces.nsIWritableVariant);
js> var func = function() { dump("Hello World!\n"); }
js> variant.setFromVariant(func)
js> variant
[xpconnect wrapped nsIWritableVariant @ 0xe27280 (native @ 0xe0e6c8)]
js> variant.QueryInterface(Components.interfaces.nsIVariant)
function () {
    dump("Hello World!\n");
}
js> variant.QueryInterface(Components.interfaces.nsIVariant)()
Hello World!
js> variant.QueryInterface(Components.interfaces.nsIVariant) == func
true
js> variant.QueryInterface(Components.interfaces.nsIVariant) === func
true

I had thought about introducing a number of nsIVariant-based interfaces to parallel the basic data structures, but while working on this article, I realized... why? The whole point of nsIVariant is to wrap changeable values in a nsISupports structure. Since these data structures work with nsISupports naturally, it doesn't make any sense to add an extra layer of abstraction.

Back to Basics: XPCOM Arrays and Memory

A couple weeks ago, I started reading "Learning Python", by Mark Lutz. It's an interesting feeling, going back to the basics of programming - numbers, strings, arrays, etc. In reading this book, I kept thinking there were some things missing from the basic XPCOM tool set. Therefore, I decided to attempt implementing these missing pieces.

Now, I could be totally wrong about any or all of these being missing... but it's also a good exercise for me to push my boundaries and strengthen my C++ skill set. The result is not only code that I can use for proof-of-concept, but code that may be useful elsewhere.

This is the first article in a multipart series titled "Back to Basics", exploring these concepts. This first article is about native C++ arrays through XPCOM - and making sure I don't leak memory. Read on in the extended entry for details.

The problem: Verbose code

In XPIDL, you define an interface for an array like this:

void getValues(out PRUint32 count,
               [array, size_is(count), retval] nsIFoo values);

In C++, this looks a little like:

NS_IMETHODIMP
nsBar::GetValues(PRUint32 *count,
                 nsIFoo ***values)
{
  PRUint itemCount = mValues.Length();
  nsIFoo** retval = nsMemory::Alloc(itemCount * sizeof(nsIFoo*));
  NS_ENSURE_TRUE(retval, NS_ERROR_OUT_OF_MEMORY);
  for (PRUint32 i = 0; i < itemCount; i++)
  {
    retval[i] = mValues[i];
    NS_IF_ADDREF(retval[i]);
  }

  /* Other operations may be here which return early, without freeing
     the above memory.  In other words, these other operations may
     inadvertently cause several leaks - retval, plus the members of 
     mValues (because their reference count has been increased).
  */

  *count = itemCount;
  *values = retval;
  return NS_OK;
}

There are several steps that happen this approach:

• We allocate the memory ourselves
• We set the return values as two separate lines, ourselves
• We addref the return values ourselves
• Oh, and if we return early, we either must free the allocated memory and release the reference counts ourselves... or we leak.
• NS_ENSURE_SUCCESS(rv, rv) and NS_ENSURE_TRUE(foo, NS_ERROR_WHATEVER) are great ways to cause leaks here.
• By the time we're done handling all the early returns, the code has gotten much, much larger and harder to read.

I step back and ask, "Is that all necessary?"

My solution: A local memory-handling class

It'd be a lot easier to just write something like:

NS_IMETHODIMP
nsBar::GetValues(PRUint32 *count,
                 nsIFoo ***values)
{
  PRUint itemCount = mValues.Length();
  nsresult rv;
  // XXX This is pseudo-code, and will not necessarily compile!!
  MemoryManager mem(nsIFoo*, itemCount, rv);
  NS_ENSURE_SUCCESS(rv, rv);
  for (PRUint32 i = 0; i < itemCount; i++)
    mem.setIndex(i, mValues[i]);

  mem.Finalize(count, values);
  return NS_OK;
}

This has several advantages:

• The mem object is responsible for memory allocation and de-allocation.
• The mem object provides a simple API for setting members (and presumably takes care of reference counting)
• The mem object provides a simple API for setting the return values from nsBar::GetValues - specifically, the number of items, and the pointer to the item array.
• Best of all, when GetValues exits, the mem object destructor executes.
• This is a key difference between C++ and JavaScript - JavaScript objects don't have the concept of destructors (and thus, code that runs before they disappear into garbage collection).

Fortunately, all this is possible with C++. I implemented a pair of template classes, nsMemoryArray and nsMemoryRefArray. With these template classes, the above code looks like:

NS_IMETHODIMP
nsBar::GetValues(PRUint32 *count,
                 nsIFoo ***values)
{
  PRUint itemCount = mValues.Length();
  nsresult rv;
  nsMemoryRefArray<nsIFoo*> mem(itemCount, rv);
  NS_ENSURE_SUCCESS(rv, rv);
  for (PRUint32 i = 0; i < itemCount; i++)
    mem.setIndex(i, mValues[i]);

  mem.Finalize(count, values);
  return NS_OK;
}

How it works

1. The nsMemoryRefArray<nsIFoo*> declaration defines the class via a template.
2. The mem constructor takes the itemCount and allocates a private nsIFoo** pointer with size itemCount * sizeof(nsIFoo*). That means it can be an array holding itemCount number of nsIFoo* values.
3. Through mem.setIndex(), members of the private nsIFoo pointer array are set - and "addref'ed" (their reference counts go up by one).
4. Should GetValues() exit early, the mem destructor executes, "releasing" the addref'ed items and freeing the memory.
5. Otherwise, in mem.Finalize(), the count and values pointers are appropriately set - and a private boolean indicates the mem object no longer needs to worry about memory.
6. GetValues() exits.
7. In the mem destructor, the private boolean from Finalize() means do not free the memory or release any addref'ed items.

What if your array of objects doesn't have reference counting? No problem!

NS_IMETHODIMP
nsBar::GetValues(PRUint32 *count,
                 PRUint32 **values)
{
  PRUint itemCount = mValues.Length();
  nsresult rv;
  nsMemoryArray<PRUint32> mem(itemCount, rv);
  NS_ENSURE_SUCCESS(rv, rv);
  for (PRUint32 i = 0; i < itemCount; i++)
    mem.setIndex(i, mValues[i]);

  mem.Finalize(count, values);
  return NS_OK;
}

Here, I made one significant change - I declared mem as a different type. XPIDL required a different argument set in the GetValues() definition, but other than that, the code is essentially identical.

Practical applications

A quick search on mxr.mozilla.org turns up several promising candidates. In particular, I notice nsConsoleService::GetMessageArray and several implementations of nsIClassInfo::GetInterfaces. If my nsMemoryArray.h file were to pass reviews and the "sniff test", it'd probably be worth using to clean these up.

I also notice there's very few C++-based components which implement native arrays (other than the nsIClassInfo::GetInterfaces one). The nsMemoryArray.h helpers would probably lower the barrier for such components. (Yes, I know, C++ is falling out of favor as a language for XPCOM components... but I'll deal with that in a later article.)

Thanks for reading!

"Dancing Dude at Skyfire" - Hello World

Original video and YouTube

Yeah, that's me. You'd think I'd burned through my fifteen minutes of fame already...

Yes, I wore the Mozilla shirt deliberately. My role at Skyfire is to basically customize Mozilla Firefox for our needs. So it seemed fitting to combine the shirt and the little Skyfire badge.

As for the dancing itself... I think I can hold my own. :-) I'll say this much - at that level, it is a workout in addition to a whole lot of fun. I don't care about being alone out there either. I basically make it up on the spot (or I used to, anyway...)

Replacing JSLib in Verbosio

I came to an unpleasant decision last week. While working on reorganizing Verbosio's source code, I realized that JSLib should not survive the transition.

I didn't really like this very much. JSLib has been useful to me in the past. But when I thought about it, the only thing I used JSLib for was its file I/O utilities. These utilities should be used from chrome (and I abused that in my file search components).

Plus, I'm a much more experienced developer than when I first started with Abacus over four years ago. The Mozilla code base has similarly improved, and I just felt I could do the same job I was asking JSLib to do, with less (and cleaner) code.

I'm not saying JSLib is a bad library or a bad idea - far from it. I'm saying what I needed from JSLib is too small to justify having it around. Libraries for Mozilla are a popular idea - FUEL, STEEL, and JavaScript modules (.jsm) come to mind.

Besides, if you're going to replace a library with one of your own, you should add value (not just cut costs).

I decided to write a library that, in theory, would be able to read and write to flat files, zip archives... and zip archives within zip archives. Think JAR files inside a XPI. Then, to make sure that I did it right, I decided to write a xpcshell test case for the module. (Yes, you can do that. The test harness doesn't care whether you're testing components or a .jsm file.) After a couple of days, I came up with FileCommon.jsm and test_FileCommon.js.

(The key is in FileCommon.getFile(), near the end. It takes multiple arguments, beginning with the path to a zip archive in the operating system, then an entry inside the zip (which is also a zip), then an entry inside the second zip archive, and so on.)

I'm sure there are bugs in this implementation. Right now no code in Verbosio uses it yet, and as I work on replacing JSLib with FileCommon.jsm, I'll continue to add to the module. Eventually, I may make it a standard components module with new interfaces. Like so much else in Verbosio, it's right now a proof-of-concept.

With JSLib, the time has come to move on. XUL Widgets will also fade, as I absorb its efforts into the Verbosio project directly. XUL Widgets (and Verbosio) doesn't have any other customers, anyway, so centralizing under one roof makes sense.

Thanks for getting me started, Pete.