The Inside Track on Firefox Development.
« January 2006 | Main | March 2006 »
February 24, 2006
Bye Bye Blackberry?
I cannot believe people are discussing life without these things. It's like this: I have a patent on television. I don't plan on doing anything with it, but I'm going to shut TV down for all of you, and you're going to sit about and think about life without TV? What's wrong with people?! Is this the world we all want to live in, where people without the interest or capability to pursue technology can hold everyone else captive? That's not the world I want to live in.
Posted by ben at 10:57 AM | Comments (9)
February 17, 2006
More on Memory
Firefox's caching behavior is just one area of memory usage. I'm really glad that there's been such a lot of discussion in the previous post I made, since many people have raised specific issues, bugs have been filed, and people are looking at the things people are reporting. This sort of feedback system is one of the things that makes the open development model great. Firefox 2 will be much better because of your help!
Posted by ben at 11:44 PM | Comments (19)
February 14, 2006
About the Firefox "memory leak"
A lot of people complain about the Firefox "memory leak(s)". All versions of Firefox no doubt leak memory - it is a common problem with software this complicated. We look to fix the issues where we can. David Baron and others have done a huge amount of excellent work in this area.
What I think many people are talking about however with Firefox 1.5 is not really a memory leak at all. It is in fact a feature.
To improve performance when navigating (studies show that 39% of all page navigations are renavigations to pages visited < 10 pages ago, usually using the back button), Firefox 1.5 implements a Back-Forward cache that retains the rendered document for the last few session history entries. This can be a lot of data. It's a trade-off. What you get out of it is faster performance as you navigate the web.
For those who remain concerned, here's how the feature works. Firefox has a preference browser.sessionhistory.max_total_viewers which by default is set to -1. When set to this value, Firefox calculates the amount of memory in the system, according to this breakdown:
| RAM | Number of Cached Pages |
| 32MB | 0 |
| 64MB | 1 |
| 128MB | 2 |
| 256MB | 3 |
| 512MB | 5 |
| 1GB | 8 |
| 2GB | 8 |
| 4GB | 8 |
(reference: nsSHistory.cpp)
No more than 8 pages are ever cached in this fashion, by default. If you set this preference to another value, e.g. 25, 25 pages will be cached. You can set it to 0 to disable the feature, but your page load performance will suffer.
Edit: In the comments, Boris and David pointed out that I misread the code, and that this is a global preference so that there are no more than 8 cached pages for the entire session, not per tab. My initial posting had claimed that it was per-tab. Oops!
Posted by ben at 9:34 AM | Comments (317)
Camino 1.0
Congrats to the Camino team for shipping a Camino 1.0 at last! Camino has existed for a standalone browser project for longer than Firefox, so years of effort have been expended on producing this great software. If you're a Mac user, take it for a spin.
Posted by ben at 8:58 AM | Comments (2)
February 10, 2006
No Firefox2 a1 Today
Despite much reporting, and earlier schedules indicating that Firefox2 a1 would be released today, the reality is a little different. Read Mike Beltzner's note from the Firefox2 dev meeting on Tuesday. We're pushing it off a little so that we can actually have some new features for you to play with.
Dogfood
Dogfood is a term in computer jargon that means software has reached a point that it's basically usable, even if some functions are missing, and others are clumsy. It means it's something you can use as a day-to-day browser. The goal for Firefox2a1 is to reach dogfood on several of our key features.
Posted by ben at 10:09 AM | Comments (26)
February 6, 2006
Where Did Firefox Come From?
The story of Mozilla is long and rich in detail. There are many perspectives. This is mine.
Getting Involved
I got involved with Mozilla because I loved the idea of working on something that had the potential to make an impact on millions of people. My friends and I lived in our browsers, so there was also a tangible payoff for contributions that made it into a shipping Netscape release. After switching gears on the layout engine, it looked like Netscape needed all the help it could get. In early 1999 only the most basic elements of the old Communicator suite were in place in the new browser; you could barely browse or read mail as Netscape's engineers worked furiously to erect the framework of the application.
I thought about what I could offer. It was difficult — through my website I had taught myself JavaScript and HTML, but I did not know C++. Perhaps more importantly my AMD K6/166 was not really capable of compiling a codebase as large as Mozilla (even when it could almost fit on a floppy!). What I did notice though was that the user interface was being developed using something similar to HTML and JavaScript. Realizing that my fussiness and attention to detail could be useful, I began working on improving the UI. I became involved both polishing the developing browser front end and in the design of the XUL toolkit that rendered it. After a few months I was offered a job on the Browser team.
It was mid January, 2000, and I stood in the dimly lit international arrival lounge at SFO waiting for my ride. After a few moments a smiling man with a burst of reddish hair approached. It was gramps, and I had arrived.
An Awkward Alliance
The relationship between Netscape and the Mozilla open source project was uneasy. Mozilla wanted an independent identity, to be known as the community hub in which contributors could make investments of code and trust, while companies like Netscape productized the output. Netscape was not satisfied to let Mozilla turn the crank however; building and shipping a product with as many constraints as the Netscape browser was — and remains — a mighty challenge. Netscape was convinced it was the only one that knew what needed to be done. At the time, I think it was true.
While there were many community volunteers, there were more missing essential features than were present. There were more unfixed critical bugs than fixed. The only organization that had a strategy to drive the ominously lengthy bug lists to zero was Netscape.
Netscape's Communication Problem
Netscape made two mistakes. They did not publish enough product management information to enable the community to help them achieve their goals. They did not even consistently communicate what these goals were. The cohabitation of engineers and PMs did not contribute to open dissemination of information as it was easier to communicate with your immediate team than those on the outside. Without understanding the importance of publicly available documentation to effective community development, the extra steps to publish the results of internal discussions were not always taken.
The other mistake was not having a clear vision for product development. For the folks working in Client Product Development (CPD — the browser/mail software division at Netscape), the idea was to rebuild and improve upon the Communicator suite using the newly selected layout engine. The goal was to make a better browser and mail reader.[1]
Netcenter's Agenda
More importantly however, Netscape had to make money to survive. Across campus, the Netcenter division management had browser ideas of its own. Once the most popular site on the web, Netcenter's fortunes were waning as users switched to other portals like Yahoo and MSN. Netscape had to replace revenue from declining traffic and it looked at what it could do to differentiate itself. The one thing Netcenter had was close affiliation with a browser. Netcenter sought to monetize the browser in two ways: by linking the browser very closely to the portal itself in design and content and by linking the browser to features supplied by business deals.
Netcenter passed various requirements on to CPD. Over time these included things like a browser “theme” that was supposed to merge seamlessly into a particular iteration of the Netcenter web site. Users could be forgiven for thinking, when using the freshly themed browser, that their monitor was broken. Adding to the injury was an overload of links in the browser UI to Netcenter and partner properties, and a large eye-catching but mostly useless sidebar panel with additional partner content vying with web content for the user's attention.
Eventually, Mozilla.org forced Netscape to push most of its business specific functionality into its own private branding cvs server, where features like Netscape Instant Messenger were also developed. The controversial “Modern” theme, panned in the Netscape 6 Preview Release 1, was eventually removed and replaced by a more stylish but similarly non-native theme.
Mozilla's UI Dysfunction
Since most of the user interface design for the Netscape products was done by Netscape staff working to Netcenter requirements, the Mozilla user interface suffered. Instead of being a clean core upon which Netscape could build a product to suit its needs, the Mozilla suite never felt quite right; it was replete with awkward UI constructs that existed only to be filled in by overlays in the Netscape's private source repository — the “commercial tree.”
Compounding this dysfunction, at the time the project was being developed by over a hundred engineers in different, sometimes poorly connected departments within CPD. Netscape had grown rapidly in previous years, and with an uneven hiring bar engineers with abilities that would suggest they needed more assistance from others had far too much autonomy in feature design and implementation. User experience assistance was sparse, and as a result the application quickly bloated.
Fighting For Change
Engineers in the Browser and Toolkit groups were not happy with the way things were going. For my part, I began working on what would become known as the “Classic theme” — a visual appearance that respected the system configuration and would later form the foundation of what would be Firefox's theme. Since I had little graphic design talent I used icons from Netscape 4. This became the default theme for Mozilla distributed releases, Netscape ultimately choosing to use its “New Modern” theme for Netscape 6. Several of us argued strenuously against this decision but were unsuccessful. In the eyes of reviewers, the alien appearance of Netscape 6 would be the straw that broke the camel's back. Suck.com, an early pop-culture review, wrote an article decrying the themed interface, using Netscape 6 as its prime example of the failure of theming.
Despite the failure of Netscape 6, engineers were still enthusiastic about continuing development. Now that Netscape could include its partner customizations[2] — engineers called them “whore bars” — in its commercial tree, the engineering teams focused on improving the Open Source releases instead, hoping Mozilla would be the suite they had dreamt of building. Netscape branded products were largely ignored.
Many contributors, myself included, pushed for further improvements to the user interface. We got extensive pushback from people within the company. On more than one occasion I tried to flex my muscle as “user interface module owner” (Mozilla parlance, then a something of a novelty — Mozilla had granted me this role in an attempt to show autonomy in project development after the disaster that was the original Modern theme). It did not go well. Weak management stressed the importance of seniority over logic when it came to feature design. I was told I could not expect to use Open Source tricks against folk who were employed by the Company (all hail!). I held true to my beliefs and refused to review low quality patches. I was almost fired. Others weren't so lucky. It became a source of great frustration and disillusionment for me. I lost motivation. I realized Netscape's Byzantine stranglehold permeated the design of the Mozilla product still, and that now as a Netscape employee I was expected to use my “module ownership” to support its whims. I was to be a puppet.
I disengaged. I no longer treated every patch I thought was low quality as a battle to be fought and won at all costs. Netscape and Mozilla continued to ship releases. The application improved in terms of stability and performance, but the user interface remained baroque.
New Beginnings
One night in mid 2001 I went to Denny's late at night with David Hyatt. Dave is the sort of person with an enthusiasm that is magnetic. You cannot help but become caught up in the excitement of the ideas generated during a discussion with him. We discussed the rot within Mozilla, which we blamed on Netscape and Mozilla's inability to assert independence. He suggested it'd be perhaps preferable to start again on the user interface – much of the code in the front end was so bloated and bad that it was better off starting from a fresh perspective. We talked about using C# and .NET, and Manticore was born. As is often the case with ideas and prototypes, the fun quickly deteriorates into tedium as the magnitude of the task becomes clearer. A couple of weeks after it was begun, Manticore died. Dave tried again though, first with Camino, and finally with Firefox.
These browser efforts were reactions to the rot we had seen in the Mozilla application suite. As Netscape began to lose favor within AOL, Netcenter's grip on CPD loosened, and the strength of the community grew. However even after Netscape cleaned up a lot of its engineering operations the UI did not improve. Rapid improvement to the user interface was now restrained by the same processes established to preserve code quality when weak engineers had free reign to check in.
There was no organized vision for the browser UI, and the ability to make changes was too widely distributed: anyone could make any change or addition to the UI as long as they had two other reviews. There was no clear plan for improving the resulting clutter. There was no vision.
Firefox was different. After 0.6 I laid out a plan for reaching 1.0. After a few cuts and sanity checks, a year and a half of engineering work by a motivated core of engineers on the front end and the continuing development of Gecko beneath, Firefox 1.0 shipped.
During this time Mozilla finally gained the independence it had long sought, establishing a non-profit foundation on the same day many of the remaining client engineers at Netscape were laid off in a mass termination that can only be described as a bloodbath.
There was and remains much resentment towards Firefox and its development model. At its creation, there was much shouting about how the many were not always smarter than the few, the merits of small development teams with strong centralized direction, the need to adhere strictly to Mozilla's module ownership policy[3]. In practice, these statements resulted in effectively locking everyone but the Firefox team out of the Firefox source code. We railed against the inefficiencies of past UIs. We were unnecessarily harsh, and polarized opinions. We had been badly wounded by the Netscape experience and the disorganization that had followed. I don't think a lot of people understood that. It wasn't something we could easily communicate.
To many, it looked like we were breaking ranks. We were claiming their work had no value. It was said that what we were doing went against the principles of community development. That wasn't true — as most open source projects are centrally managed by a small few. Many have well defined release plans and maintain tight control over what contributions make it in. We had hurt our case though by being so dogmatic up front. We did not do a good job of PR.
Vindication
We were determined, and we brought the product through to a 1.0 release. It was a long, difficult, all consuming road. I may eventually write more about it. But for now I'll just say that despite the rocky start, and the criticisms faced along the way, the model worked. It worked better than it ever had before for Mozilla or Netscape. Firefox 1.0 shipped to over a million downloads on the first day alone, 10 million downloads in ten days, and over a hundred million downloads before it was replaced by Firefox 1.5 just over a year later.
Today Firefox is one of the brightest stars in upcoming tech brands. It was voted by BrandChannel readers as the number 7 global brand across all segments — ahead of eBay and Sony. It is the first browser to stem the decline of market share against Internet Explorer and has reclaimed a healthy 10-25% share depending on country. It is still growing. Firefox's mail counterpart, Thunderbird, is a successful and comprehensively featured mail application. The dream that team of engineers had back in the darkest days of Netscape 6 product hell had been delivered at last.
Here & Now
I am writing this because I want to provide a historical perspective on where we are today with Firefox. A lot has been told about the development of the Firefox browser since Firefox 1.0. The reality is that the story is bigger than just Firefox 1.0. It goes back years, spans continents, and includes a cast of thousands. It's a fantastic story, with all of your standard themes — greed, rage, turmoil, love lost. But mostly it's a story of dedicated people laboring to create something they truly believe in. That's something I think everyone should be able to relate to - no matter what their walk of life. That's why Mozilla is so powerful and extends well beyond just Firefox.
For me, the story included the realization that I had never believed in something this much before, and discovering how easily and arbitrarily your dreams could be snatched away. Ultimately though I realized that with some patience and good old-fashioned hard work, anything is possible.
Over the years, Mozilla finally gained the ability to be that crossroads where people could come together and share their thoughts on the internet and where it is going. Different people have different ideas, and these are borne out in the different projects that exist: Camino, SeaMonkey, Thunderbird, Sunbird, Chatzilla, Bugzilla and so on. These projects create the ecosystem that is Mozilla. While related projects may not always agree on approach, the work that is done is inevitably beneficial — one project feeding off the ideas of another, and vice versa. Whatever project scratches the itch of any particular person, having their contributions and ideas around is beneficial for all projects. Generic tools to support many instances provide a backbone to support today's demands and tomorrow's as well.
Firefox is so successful today that it is gaining attention from many quarters. Many new contributors are finding the project and new ways to help out. This sort of thing is essential to keep the project vibrant and maintain the flow of innovation. It is important that those of us who've been round the block a few times share what came before, what did and did not work. The struggles that were fought, the price that was paid. This project has not been successful by accident. Its success represents the sum total of the energy expended by thousands people around the world for more than half a decade.
No contributor, no matter how new they are or what their motivation should let the story of Mozilla stray too far from their mind.
February 4, 2006.
Footnotes
- Some people claimed building a browser and a mail client at the same time was muddying the waters. I don't think the two are necessarily mutually exclusive however, and many of the needs of one helped influence improvements which benefited he other.
- “How to Monetize™ your browser”, “The IE Advantage”
- “Module Owners”
- “Moving Target”
Posted by ben at 10:19 AM | Comments (82)
©1997-2006 Ben Goodger. All Rights Reserved.
Opinions expressed here are my own, and not those of any organization that I may be affiliated with.
Reload icon is © Stephen Horlander;
Firefox logo is by
Jon Hicks, and is a
trademark of The Mozilla Foundation.
GetFirefox buttons are from rakaz
