Which market research organization would you trust more, the one that puts Netscape at 0.7% or the one that puts Netscape at 11.85%?
The answer should probably be "none of the above" since all of these metrics have their strong points and their weak points and comparing them to each other is not an apples to apples comparison. Geography makes a big difference, as you can see by Xiti's measurement of Firefox at nearly 25% in Europe overall, and as high as 44.5% in Slovenia and 41.3% in Finland. The type of user surveyed makes a big difference too. The latest report from FreshBooks puts Firefox at 38.95% share among small to medium-sized businesses and BoinBoing.net has Firefox at nearly 50%. The time of the survey also impacts the results with the total web using population dropping some in the summer and on or right around holidays like Christmas.
So how should we measure Firefox growth? Well, first, we shouldn't be looking at any single-sourced metric. Second, we probably shouldn't be looking at month to month data and instead look to longer trends that cross seasons or even years. Maybe we shouldn't be looking at usage stats at all and move to some other metric. That's a lot on what we shouldn't do. What should we do?
What do you all think? The trends are clearly going well for Firefox over the last three years and we're seeing consistent increases in browser downloads and trials with each of our new releases. As I've said for quite some time, though, downloads don't all turn into users and as the paragraphs above suggest, usage patterns are quite different depending on geography, the kind of user, and the time of year one measures. So, what and how should we measure so that we can make the most informed decisions regarding our efforts to increase the pace of adoption?
Posted by: Anders | April 1, 2007 12:44 PM
The actual values of market share don't really matter. What matters is the trends. I think you should be comparing on a month by month basis, the same stat against the same stat. Stick it on a graph but don't number the Y axis. If it's going up then great, if not then you need to work in that area and the particular stat will tell you what area to work in (Europeans, non-techies etc.). Jumps in the graph should hopefully relate to releases ad marketing drives.
Posted by: Mossop | April 1, 2007 1:44 PM
One way for the Foundation/Corporation to have another source of stats and to see the growth across (and for a specific locale too) would be to use the data "provided by" the RSS feed in the 2.0-series. As the RSS feed is redirected through Mozilla servers, normal http-request data of the requester is collected.
I did a similar thing for the Finnish Firefox when the feed was redirecting through our servers in the 1.5-series. A blurb about that: http://www.mozilla.fi/blogi/arkisto/firefoxin-paivitykset-ja-tietoturva-firefoxs-updates-and-security/
Posted by: Ville Pohjanheimo | April 2, 2007 8:45 AM
The problem I have with most of these browser stats reports is that they only come from one company that has the ability to track a section of users. Most of them come from a company that has a stats package which reports back to them. So these stats are going to be skewed, because they only come from sites that use their stats package. Same thing with BoingBoing, which you usually correctly to identify as just a slice of the blogosphere, so you are only seeing visitors related to BoingBoing.
I think a possible solution is to go back to the tried and true survey, where you manually survey a few thousand people. But these cost a lot of money if done properly, and they have other problems. You can't rely on phones anymore, because of the "do not call" lists. If you use snail mail, you will only get people that don't throw away 'junk' mail (which most of us do). And if you go to a public location like a mall, you also get skewed results (only people that go to the mall). Not to mention, if you ask "What web browser do you use?", the average person will answer, "I don't know."
So it's a very difficult question to answer.
Posted by: Jacob Munson | April 2, 2007 10:55 AM
I think that at the very least a celebration would be in order. I’ve sugested it on spread firefox. But I don’t think one forum post by one user will be enough to get the process going. Lend a hand please.
http://www.spreadfirefox.com/node/28308
lp
U5K0
Posted by: U5K0 | September 12, 2007 8:25 AM
Kind of a dull answer, but shouldn't you first find out what question you are trying to answer (or why you are trying to get it answered) and then find the best data sources for that? And you might be able to trust both market research organizations, you just have to find out what questions they are answering.
If developing a new website for some organization, I would look at the browser shares on their current site to find out what kinds of browsers their visitors use. In one site I worked on, it was rather obvious that people used IE at work but firefox at home (causing a larger firefox share in the vacations), and a rather large part of the IE share was automated bots. YMWV.
If you are just going to use the statistics to write press releases, just find the statistics that show the larges firefox number and use that.