The Inside Track on Firefox Development.
« Racing Towards 2 | Main | Feed Reader Questions »
January 22, 2006
Battling Firefox Bloat
This is the first in a series of essays about Firefox design and development.
One of the core goals of Firefox has always been to provide you, to our best approximation, with the browser you need. The way we handle features and configuration follows the following approximate formula:
| Feature used by majority of people | Included, on by default |
| Feature used by significant minority | Feature included, perhaps disabled by default |
| Internal feature configuration and finetuning | Configurable via about:config |
| Feature used by minority | Feature not included, use an extension |
It is by maintaining Firefox's extension system that we allow people to experiment with new and interesting ideas, or create esoterically useful features. It is not however the role of the core browser to support minority preferences when clear defaults and probable parameter ranges are well understood.
Over the years, Firefox has grown in size, due to an increasing level of polish and number of required features (the Software Update System that shipped with Firefox 1.5 is an excellent example). At the same time it has grown in unfortunate ways. Some necessary systems (like the Software Update System) are too complex, this is in part due to our (read: my) lack of understanding of how the system would work over time from a user perspective. In future releases, we will simplify it. But this is not happening in other parts of the browser.
All too often, code in the core browser has been extended in unnecessary directions in the interest of power user geek-dom. Such changes are usually made as hidden options, configurable through about:config.
about:config is not a back door for old-world style feature and user interface design
But this is what is happening. As engineers, adding hidden "toy" features to the browser under the secret protection of about:config (or other suitably obscure invocation paths) is problematic for at least the following reasons:
- You increase the complexity of all the surrounding code by adding to its bulk, thus increasing the mental bandwidth required by other engineers on understanding what's going on.
- Your feature won't be as well tested since it's not on by default or exposed through the user interface, nor will it likely figure prominently enough to feature on any test plan.
- Even if your feature is documented so that engineers making changes nearby know about it, you increase their testing burden and the difficulty that they have at making more legitimate changes.
- Since it's not likely to be well tested, your feature may be broken by API changes, and when it ships in the product be useless, so that if someone does stumble across it it may (at best in this circumstance) not work, at worst do something destructive.
- You add another site that people changing APIs have to update and maintain.
- You add to the ever-increasing download size of the product. One of Firefox's advantages is its trim size, and every line of code you add, even well-compressed JavaScript robs a small amount of download size. It may not add much to the bottom line - it's more of a principle thing, a way of operating. Playing fast and loose with bloat in one area indicates to others that it is acceptable overall.
Firefox is not a playground where every patch has its day. It is professional software and working on it requires discipline, not just discipline in code quality but also in feature quantity. Look at the list above. It all adds up. The more features you add, even subtly hidden ones, the more fragile the house of cards becomes.
What I'm Doing
I'm being ruthless (did you expect anything less from me? ;-) I'm evaluating all code I touch and figuring out if it can be simplified. Good engineering practice tells us functions should be succinct and well documented. Remove whatever you can. As you do so, the maintainability of the surrounding code will rise implicitly. If you remove a feature (features aren't just big ticket items on PRDs - anything controlled by a pref, no matter how tiny, can be considered a feature), write it up somewhere on the Mozilla Wiki and if you like that function encourage extension developers to build an add on. Or do it yourself.
If you are writing major new features, do make them configurable to test the best possible workflows, but once these become clear, try and simplify them. If you want to add an esoteric backroom workaround to an existing feature stop and think - is this so useful to the world at large that it requires a UI pref? If it's not, then reconsider. You're doing no one favors - especially not the Firefox project and its success at large - by adding more bloat.
Posted by ben at January 22, 2006 12:06 AM
Comments
> You add to the ever-increasing download size of the product. One of Firefox's advantages is its trim size, and every line of code you add, even well-compressed JavaScript robs a small amount of download size.
Replace that with "and every line of code you add, even well-compressed JavaScript robs a small amount of download size and performance."
Posted by: Anonymous at January 22, 2006 2:50 AM
I can't help it, but this sounds if someone wants to remove View->Source again :-P
Posted by: Anonymous Coward at January 22, 2006 4:26 AM
I know exactly what you mean. IE View is almost 50k and I was able to rewrite it (as IE View Lite), without taking out any features, to be under 5K, just by simplifying the sloppy code. I wish that engineers would write good, clean, fast, small code from the beginning. It sure makes things better for the end-user.
Posted by: Grayson Mixon at January 22, 2006 4:30 AM
You're right on what users want. They really need less bloat and more performance. Good luck!
Posted by: Payalnik at January 22, 2006 5:17 AM
Could you elaborate on where the numbers have come from when you determine whether something is used/not-used by a majority? Have users voted somewhere, or did you use statistical sampling surveys? Who got to vote, or who got surveyed?
(I understand you may use different procedures in the future, so I'm not asking you to give us a fixed policy, just getting an idea of how you have applied it.)
(For example, if you surveyed by keeping counters in the product to see how much each feature is getting used, and collected the counters as part of the update process, then the statistics would be skewed toward individuals who update directly, and against schools and enterprises and overseas organizations which don't use that update mechanism. In that hypotheical case, we would know that we should use update if we want to be included in the survey.)
Could you elaborate on how you have decided when a 'minority' is significant or is not significant?
(I understand the threshold will depend on how big a change is needed to support it.)
(An obvious example is that developers are a significant minority because they influence whether sites are built compatibly with firefox. Most non-developers probably never use 'view source' or even 'page-info', at least not more than a couple times to see what it is, but those have small impact on the UI and download size, whereas developer tools like DOM inspector or Venkman are large, so omitted.)
(An example of a minority that didn't meet threshold is Cambodian TB users: since they mostly have less expensive small 800x600 screens and their tall super/subscripting writing needs more space, several dialogs would have had to be redesigned to fit, or allowed overflow:auto scrollbars, but even that was too large a UI change.)
Thanks in advance.
Posted by: Gc at January 22, 2006 6:09 AM
Best post i have read on Firefox development in a long time!
Most security bugs i found in Firefox were related to rarely used features just to satisfy a minority of the user base aka "power users". Most of those issues (for example the "favicon" or "set as backroung image" bug) were not really a bug, they were just the result of sloppy programming and bad QA.
I prefer to not have the "fancy feature" to see a faked preview of the desktop background with the image instead of seeing a shell starting deleting my harddisk. And if we add those features we need enough capacity for QA.
As Firefox get's used by a larger base we need to focus on getting the core functions right and build a stable platform. Bad habits and concepts we allow now will follow us for years.
Posted by: mikx at January 22, 2006 6:23 AM
Really, great post makes me want to jump in
and start trimming code down.
Posted by: Todd at January 22, 2006 6:42 AM
As a user it would be nice to have some way of assesing the quality of an extension.
Couldn't there be a person or group of people that know what they're doing that assess extensions and certify them? Atleast popular ones.
It could improve extension quality atleast, and let the user know how good the extension is performance-wise, the assessing could even be done through users on the website. With every new release of an extension allow the user to seperately rate if it has issues or not.
It would be great customer service.
Posted by: Eric Irvine at January 22, 2006 7:20 AM
I'm going to disagree with the size argument a bit.
Over time Firefox should grow, but at an established rate. Computers get faster, hard drives bigger, internet connections get faster.
4MB 2 years ago is what 8MB is now.
I don't think the "my 9600bps modem would take along time to download firefox" is really a rational argument.
Several home broadband ISP's (including Comcast of all companies) are expected to offer 10Mbps by the end of the year (see this thread among others). 2 years ago it was a mere 4Mbps. Many ISP's have done the same in competition.
A few years ago a 120GB HD cost me $150. I could get the same or better for
Same for processors, etc.
I think it's essential to scale with the time. Or we become lynx.
I don't think Firefox 1.5 is bloated, or that software update is to big. It's fine. It's 1 full year after Firefox 1.0. Things got better in the industy. Millions of new PC's were purchased. Millions of old computers were decommissioned.
btw: The new cairo work doesn't seem like it would work on my Dad's old IBM 5150. ;-)
Posted by: Robert Accettura at January 22, 2006 8:13 AM
Anonymous -
No one wants to remove view source! View source is one of those "significant minority" features - it's exposed in the UI and used by a lot of web developers. It's not obscured as much as a hidden preference.
Gc -
The features that are and are not useful are evaluated based on use, our determination as to what the market needs, usability studies, etc. It would be extremely unlikely that any such determination be driven by a vote - since the people participating in such a vote are likely to be skewed towards the "likely to contribute to open source browser" demographic, which is significantly less than our total target audience.
Robert -
Download speeds are not so good everywhere, even in places you might not expect - for example in New Zealand (a modern, developed country) the average connection rate is not much faster than 256kbps. In other parts of the world net connectivity can be significantly worse. Or even in places like the Bay Area, but just off the beaten track (e.g. in unincorporated areas where DSL and digital cable do not reach). Even if you do have the best connection available, I think that you're still better served by smaller size since you get your new browser update faster and we as humans crave instant gratification. The model of discipline in code size and feature quality that has followed Firefox so far is, I believe, a good one - it expects excellence of everyone contributing and has had great positive effects. I don't think it's something that should just be abandoned because once you do you set a precedent that anything goes.
Posted by: Ben at January 22, 2006 10:07 AM
Even if processing power increases, increasing code size negates those gains rather than taking advantage of them. I'd love to have Firefox be as responsive as Opera is...
Posted by: David Bruhn at January 22, 2006 10:54 AM
Where do experimental features like the ping feature fit into this conceptual framework?
Posted by: guanxi at January 22, 2006 11:49 AM
Are you guys measuring the software size with function points or something like this? :-)
Posted by: Christian at January 22, 2006 12:51 PM
View Source should be an option in the installer, off by default. RSS should be an option, on by default.
Clear Private Data should be an extension.
Stuff like "View Background Image" and "View Image" should be merged.
Stuff like "Save Page As" and "Page Info" shouldn't be present in context menu.
I still see room for bloat improvement. :p
Posted by: Nanaki at January 22, 2006 1:05 PM
I guess the argument could be extended not only in the size of the code, but also on its efficiency. All the recent talking about the memory leaks in Firefox as well as the fairly large footprint of memory used speaks for itself. One thing is to consider bloatware the installer (which you use only once during the installation, everyhting else comes to the system updater). I consider bloatware when a program uses 100MB of ram. I don't care how big the executable is, I prefer having an installer of 10MB but efficient than a 5MB installer of memory-hungry software. I have the impression that too much development is dedicated towards reducing code size, not its efficiency. Is it only my impression?
Posted by: feranick at January 22, 2006 1:15 PM
> If you want to add an esoteric backroom workaround to an existing feature stop and think - is this so useful to the world at large that it requires a UI pref? If it's not, then reconsider.
I would flip this around too. Is this really useful to the world at large and an improvement on the way things were before? If it is, then for Pete's sake just do it and don't add another silly undocumented pref no one will ever find.
I see a lot of unnecessary tentativeness on Bugzilla when it comes to adding a new feature or fixing some behavior that's obviously broken - "oh, someone somewhere might not like this, so let's add a pref or even make it turned off by default."
Of course on the other hand if you haven't even committed it yet and people already hate it enough that they're posting in the bug demanding a pref to turn it off, maybe you shouldn't be adding it in the first place.
Posted by: Justin Kerk at January 22, 2006 1:45 PM
Ben, keep up the great work -- you brought up some good points and I hope the other developrs will follow your example.
Posted by: morgamic at January 22, 2006 5:05 PM
The smaller FF is, the faster it runs.
Make it small and stop too many extensions being integrated into the system!
Posted by: Greg S at January 23, 2006 12:36 AM
All of this talk about downsizing and reducing features makes a good argument for using Lynx. Let's not get too gung-ho about cutting things out at the risk of losing functionality. Fast efficient code will make the difference, not just doing away with things indiscriminately.
Posted by: By-Tor at January 23, 2006 8:58 AM
Ben,
I think you neglected to cover a pretty important point of discussion with respect to features that are used by a minority, but which are important for pushing forward an agenda. Perhaps that is intended for a follow-up essay?
For instance, the bfcache feature is not going to be important to the majority of people but is important in reaching feature-parity with Opera.
The drag-and-drop tab reordering is not used by a majority of people, and in fact the majority of people aren't even concerned with tabs at all, and yet Mozilla sees this as an important part of their identity and so code which extends the usability and discoverability of the tab-system would be seen as positive enhancements to the code even though it isn't important to the majority of people.
The "clear private data" feature reinforces the fact that Firefox is a security-conscious browser, but can hardly be considered an important feature to most users. I've observed many, many users who depend more on the URL-bar's auto-complete to navigate to their most commonly visited sites than they do on bookmarks. So for them, "clear private data" is a destructive feature they would most like to steer clear of.
The new, experimental, <a ping> feature that recently landed on the trunk is great for web-developers but it is clearly debatable whether it provides anything for the people using Firefox. And I don't forsee that feature getting a spot in the preferences dialog, so it will most likely end up a hidden-pref, on by default.
I'm not arguing that I disagree with any of the decisions that led to these features becoming part of Firefox. I just think that they highlight a disconnect with your essay's claim that the driving force for feature inclusion should be based on an assessment of current and future users primary needs.
Posted by: Kevin Hamilton at January 23, 2006 9:57 AM
Kevin,
How is bfcache not important to the majority? Let me explain: how is improved browsing performance not important to the majority? It seems to me that most browser UI features are less important than the performance of the core rendering, which _everyone_ uses.
Posted by: Ben at January 23, 2006 10:00 AM
Ben, how about (not necessarily you) taking a day or two to review the dozens of patches requesting review on bugs that would reduce code? There are dozens of open bugs with patches to do this and yet I've seen them sit open requesting review for past months even some upto almost a year.
Posted by: Kurt (supernova_00) at January 23, 2006 11:25 AM
Well, you were discussing issues of bulk and complexity. It is my understanding bfcache adds considerably to both.
Weighing that against the number of times I've sat looking at my screen wondering when the Back operation was going to complete ( almost nil ), and the number of times I can imagine my parents or grandparents thinking the back or forward operation were too slow, I don't see it as a feature that people need or want. It is a fancy feature, don't get me wrong. But faster "back" operation isn't something I see the average Jane clamoring for. Perhaps you have data to the contrary. I can't comment to that, because the decision making aspect of Mozilla - that which drives the drivers - is not as "open" as the rest.
Posted by: Kevin Hamilton at January 23, 2006 12:28 PM
Hrm.. thought I posted something yesterday on here, but I probably didn't hit post.
It's already kind of been raised now, but a drive to ensure that less wanted features aren't in the core code is going to drive more people to using extensions. My impression (and dbaron's post makes a far better case than I can) is that extensions suffer more from code quality issues and lack of efficiency, and are certainly subject to frequently being broken by changing APIs.
For the significant minority of people that use extensions, their experience isn't going to be improved if the core code gets better but they then turn to more extensions and end up with problems...
Posted by: Michael Lefevre at January 23, 2006 4:27 PM
I’m going to play some stupid number games by pulling figures from my ass (a la Steve Jobs).
Let’s assume Firefox has 10 million unique users (I'm shooting way under real estimates to make a point).
Let’s assume a quarter of these users are on dialup. Lots of the regional broadband penetration numbers I have seen reported seem to be at about the 50% mark and I'll just claim that Firefox is 3 times as popular with high-bandwidth users compared to dial-up users. Again, this is based on nothing but the aforementioned ass-numbers.
Decent dialup can usually pull compressed binary files at about 3-5 KB/s, so a 3-5 KB increase in download size (one tenth to one eighteenth of one percent of the current 1.5 release) means these users each spent about one second more downloading the product.
It is true that cutting 3-5 KB of "bloat" would hardly be noticed by those individuals, but it would have saved over 4 whole person-weeks in aggregate.
Also, with 137+ million downloads, each 5 KB turns into more than 600 GB of data transmitted.
Every well-placed cut may only be boosting performance by some infinitesimal fraction of a percent or may be reducing each download by less than the size of one refresh of Slashdot, but this all adds up when you are talking about millions of users. One crasher caused by some obscure code that wasn't or can't be well tested which only ever affects 5,000 users is still 5,000 lost browsing sessions and 5,000 muttered curses.
We all have pet peeves and "must-have" features, but the tough part for all of us is remembering that "my" want isn't "your" want and things that benefit "me" may in fact cost "you". This is as true in mass-market software design as it is in politics and socio-economics, though we have been conditioned by culture and marketing to ignore all but "me and mine".
Ben is on my heroes list because he is looking at everything and thinking "this is going out to millions of people".
Now that I am done rambling, I have no idea how I am going to fit these numbers back in my ass, so I'll just leave them lying here.
Posted by: Kommet at January 23, 2006 4:50 PM
Ben: even if people don't have OC-48 lines in their living rooms yet, broadband has been improving. How long can we hold out for a small percentage of people?
I know someone who still has a 386 as his primary computer. Dialup connection, I believe NN3. Do we need to support him too?
My only point is that cut off line needs to move. We're quickly moving on 2 years after 1.0 (only 10 months away). Will we still target the same minimum specs? Apple will have changed their entire platform to x86, Microsoft will have a new OS (possibly, maybe) out the door.
The line needs to move. Perhaps not as quick as the bleeding edge. But it can't be static, or you become lynx. And last I checked, the spreadlynx campaign didn't do to well.
Posted by: Robert Accettura at January 24, 2006 12:47 PM
I agree that features should not only be judged on their usage but also on their impact and PR-value.
Even though most people don't use clear private data, still we had a feature that appealed to users that surf for porn nd other stuff even though they will not use it much in practice (impact). Also, the improved security was useful to establish our safer image (PR).
Also, I think some features should be added for innovation and to flex the muscles of the open source community (leadership).
Ben, we are looking for your critique on our views! ;)
Cheers,
Cheski
Posted by: Cheski Frank at January 24, 2006 1:54 PM
Robert, my point is that most of the world _isn't_ on 1.5Mbps+ broadband.
Posted by: Ben at January 24, 2006 11:04 PM
Gosh darn it, just keep View Source in
Posted by: Alan at January 25, 2006 12:40 PM
Oh, and leave in Clear Private Data and tab reordering; both very useful, not just for webdevs, even... i shouldn't have to install 25 extensions to get Firefox on par with Opera's usability and features. Or believe me, I will use Opera!
Posted by: Alan at January 25, 2006 12:42 PM
Good post. What I need the Firefox community need is:
1) A stable, quick and easy to use browser for the masses.
2) Some officially distributed kick-ass "extension packs" for important minority groups like web developers and such.
Someone also mentioned that Firefox need to reach feature-parity with Opera - I think it's better to listen to common people than other browser developers.
Posted by: Jonas at January 26, 2006 1:37 AM
"2) Some officially distributed kick-ass "extension packs" for important minority groups like web developers and such."
Seems to me that doing that would defeats much of the point of "battling the bloat" in the first place (depending what you mean by "officially distributed").
Moving stuff out into officially supported extensions helps with primary download size and might help a little with the code complexity issue. But in terms of having code that is less well tested, breakage from API changes, and testing burden, you are, if anything, making the problems worse than having hidden features in the main code.
Posted by: Michael Lefevre at January 26, 2006 11:14 AM
Yes,a great post!
I agree that feature bloat is bad, worse than size bloat. I'd guess that 90%+ "software hooks/logic for the future" never get used. When you finish writing the code the first time you can always see a better way to rewrite it next time. Meanwhile the extra hooks/logic gets broken - the (non)test cycle you mention.
What I don't know is how to allow features to be built somewhat fat the first time, and then get them trimmed in a second re-implementation without causing problems in-between. It is all lots of work, but the rework produces great quality and allows us to remove the unneccesary bells and whistles.
Posted by: Jed Cooper at January 27, 2006 9:59 PM
All that's required is for someone to make a new fork: Firefox without the bloat. Eventually that will become bloated, and the whole cycle can start all over again.
-dj (who remembers when Phoenix was called "mozilla:browser")
Posted by: damaged justice at January 30, 2006 6:55 AM
The word "bloat" is overused, conveying attitude in the same way that "right to life" and "right to choose" phrases do.
The extensions system is the Achilles heel of Firefox, IMAO. I think Opera is on the right track in making sure that many desirable enhancements, notably for tabs and GUI enhancements, are built in and available without some outside coder's intervention. There's no guarantee that that programmer from outside Moz really is competent and alert to security (recalling Greasemonkey...).
To say that the majority don't want these enhacements is naive -- when people see what Firefox can be when fitted out with two or three dozen extensions, then they do want the souped up version.
Posted by: ckjnigel at February 1, 2006 10:14 AM
Ben has it exactly right. Firefox currently strikes a really nice balance between configurability and the need to do configuration. Hopefully that balance can be maintained in a world where web browsers are expected to do more things year by year.
Also, as one of those people who live where I can only get a dialup connection, I really appreciate the effort the developers have expended in keeping the download size under control.
Posted by: Stephen Thomas at February 2, 2006 5:41 PM
I'm pretty sure MNG support (which is desired by a prominent minority) would actually cut down on bloat:
http://forums.mozillazine.org/viewtopic.php?t=115619
Posted by: Block Sheep at February 4, 2006 10:18 PM
A little late to the party, but does this mean that "Smooth Scroll" will eventually be turned on by default? I think it is a huge usability win for end users and great UI benefit. I have even tested quite a few non-technical users that would never know to turn it on themselves and they have all loved it. It makes the UI so much more natural and fluid. Anyone know the rationale for it not being on by default? I remember the bugzilla thread discussions back when the code was first landed and even when the UI for it was first exposed in the prefs in Mozilla, but I think that it's been well-tested enough to ensure stability by this point.
Posted by: Christopher at February 9, 2006 5:37 AM
Oh, and I forgot to mention the point that ties my post into the discussion at hand: the feature already in place and is "so useful to the world at large" as Ben put it. For compatibility with older hardware, I can certainly see leaving a UI pref to turn if off (even though removing it would cut code! ;-), but it seems beneficial enough to justify on-by-default status in the above table.
Posted by: Christopher at February 9, 2006 5:56 AM
Here what should firefox be able to offer:
1) URL bar
2) back button
3) forward button
4) stop button
5) reload button
6) an open file entry in the menu
7) a save to file entry in the menu
7) a print/preview print entry in the menu
8) a minimal config entry, with default secure settings
9) a quit entry in the menu
And that's it. Gimme a secure reliable browser implementing only those features and I will use it, I will recommend it to everybody I know. But firefox might be the least bloated, it remains a bloated browser.
BTW, you want to know how to "hack" into email boxes. Go to some public internet access / internet cafe where they installed firefox because some dolt told them it was secure. Generally they will leave it with default settings (ya know, cause it's secure), and regular users often panic/don't know what to do with all those dialog boxes. Go to preferences and harvest the email/password's. That's just one of the crap feature - add sounds, various autocompletions, memory bloat - I am avoiding the tabs now, the Toolbar Folder, the middle-click paste into I'm feeling lucky google-fuck, RSS, drag-and-drop (to nowhere - I am using Windowmaker), and other shit. Anyway, for local documents (like O'Reilly bookshelves and other docs), I am not using firefox anymore (w3m works fine). Too big, too slow.
Posted by: Jean at February 15, 2006 8:38 PM
Firefox takes-up(uses)15.3Mb of disk space and is installed all over the place. IE uses less space. Opera uses 4.06Mb and is installed in only one location.
Posted by: hott at February 22, 2006 6:04 AM
©1997-2006 Ben Goodger. All Rights Reserved.
Opinions expressed here are my own, and not those of any organization that I may be affiliated with.
Reload icon is © Stephen Horlander;
Firefox logo is by
Jon Hicks, and is a
trademark of The Mozilla Foundation.
GetFirefox buttons are from rakaz
