« Bug Day Nov. 21st | Main | Test Tool Spotlight »
November 22, 2006
Performance Testing
For the last couple of months I've been working on reexamining how we collect performance metrics for Firefox. For those who monitor tinderbox results (in this case for the 1.8 branch, also known as Firefox 2) you may have noticed the mysterious Tp and Ts results. Tp is a test that cycles through a series of web pages collecting page load times while Ts tests the time taken for Firefox to startup. The data sets collected by Tp and Ts are each reduced down to a single number which is then compared to previous runs of the same tests. From that we can determine if performance is getting better or worse with each code check-in.
Performance is always at the forefront of any major Firefox release. It is the first thing that a new user will notice when they load up the browser and we want to ensure that we are as quick as we can be. While the old Tp and Ts tests have helped us greatly in the past they are showing their age.
I've been working to essentially recreate Tp and Ts to use more modern web pages and collect a greater variety of process information - and then send this data to new, prettier graphing software. So far the new performance tests are only for Windows machines, but it is definitely on the radar to get these ported to other operating systems.
Check out the current status (and pretty, pretty graph pictures) of the new performance testing effort here: QA performance testing wiki. For now the tests are only being run on my own machines as the code changes that I've made need to be cleaned up and checked in.
Posted by alice at November 22, 2006 2:21 PM
Comments
A few questions that I couldn't find answers to:
1) Will the new setup allow finding the pageload time for an individual page in the test set?
2) How easy will it be to run the test on multiple page sets? See my recent post to .planning about intl issues.
3) Will there still be _something_ reported to tinderbox that will allow one to easily notice if there's a performance problem with a checkin?
Posted by: Boris at November 24, 2006 12:34 PM
In response to Boris -
1) Yes - the reporting allows you to view the page load time for an individual page.
2) Right now you have to drop a list of urls to cycle through into an array in a Javascript file. This is obviously not the easiest thing to do. I'd like to make it easier to switch in and out page sets so that people could do the types of pages that they are interested in (be they international, flash heavy, commerce sites, etc). This also dovetails with the effort to make it easier to grab and clean new page sets; the ideal situation would be to be able to grab a new set of pages and integrate them into the test in as few operational as possible.
3) Currently, no - there is no reporting to tinderbox. A lot of what I've put together is still in the demo stage of development. I wanted to put together something that would show what could be done and open up the discussion to how we want things to be done. Reporting is a huge issue in all of the automated tests currently being run. So far I've only looked at it by working to better graph collected results, but coming up with ways of reporting to multiple locations in a streamlined way is also important.
I'm glad to get comments on this project to help me find a reasonable direction to go in. From the QA standpoint I may end up building something that is great for me, but which may not cover the actual needs of other groups. It would definitely be a failure to put a lot of effort into this and not have it do any better than the old tests and the old reporting methods.
Posted by: alice at November 26, 2006 2:01 PM
For purposes of tinderbox, dropping the URIs to test into an array is just fine. We could just have different "start pages" for the different versions of the test -- we don't need to quickly on-the-fly switch what we're testing, just need the ability to set up multiple pageload test sets.
Thanks for working on this!
Posted by: Boris at November 28, 2006 4:38 PM
Dear friend: 09.Dec.2006
I know your company from Internet .Now I write this letter to approach you in order to establish the business relationship with you .
WUXI YUEDI CONFETTI CO.,LTD is one of the largest professional manufactures of exporting confetti/streamer paper and confetti launcher for entertainment industry in China.Our products have been sold to over 50 countries in the world .Our products offer a great ways to create warm atmosphere at certain occasion,Joyful effects in the air leave an unforgetable impression on people .such as wedding,public performance,opening ceremony,parties and shows. Confetti/Streamer paper is available in many varieties and styles ,sizes and colors .If our products fall within the scope of your business activity,I hope to trade with you .
We will provide top quality products and the most competitive price for you .We glad to supply you quotation upon your specific requests (size,color,shape and materials).Our material is available of both colorfast and flameproof.
I look forward to receiving your reply quickly!
Yours faithfully
Amy
ADD: Hudai Wuxi China
Mobile:86-013961897583
TEL:86-510-85591333 86-510-85599222
FAX:86-510-85593188
E-mail:Amy@confetticannon.cn
MSN: Amy1983_8@hotmail.com
HTTP://www.chinalibinhua.com
www.confetticannon.cn.com
Posted by: amy at December 8, 2006 5:46 PM
I've just been hanging out not getting anything done. What can I say? I've basically been doing nothing worth mentioning, but pfft. Not that it matters. Pretty much nothing exciting happening to speak of. I haven't been up to much these days.
Posted by: Sten91315 at December 26, 2006 6:06 AM
I haven't gotten anything done today. I feel like a fog, but what can I say? I've just been letting everything wash over me lately, not that it matters. Shrug.
Posted by: Sten99439 at May 16, 2007 6:43 PM
thanks
Posted by: ticaret at May 19, 2007 7:59 AM
Nike1
Posted by: abrir mi e mail spanish at June 18, 2007 3:42 PM
Lion
Posted by: www xxl com at June 20, 2007 4:52 PM
Lion
Posted by: www xxl com at June 20, 2007 5:01 PM
myfreepaysite
Posted by: Michael at June 28, 2007 2:37 PM
dodge
Posted by: John at July 4, 2007 6:48 PM
dodge
Posted by: Unison at July 4, 2007 7:18 PM
dodge
Posted by: Michael at July 4, 2007 7:33 PM
dodge
Posted by: John at July 4, 2007 7:48 PM
dodge
Posted by: Guruchel at July 4, 2007 8:04 PM
dodge
Posted by: Rick at July 4, 2007 8:34 PM
dodge
Posted by: Rick at July 4, 2007 9:15 PM
dodge
Posted by: Mister at July 4, 2007 9:59 PM
dodge
Posted by: Luk at July 4, 2007 10:13 PM
dodge
Posted by: John at July 4, 2007 10:27 PM
dodge
Posted by: Bobi at July 4, 2007 10:43 PM
dodge
Posted by: Rick at July 4, 2007 10:58 PM
dodge
Posted by: Bob at July 4, 2007 11:22 PM
dodge
Posted by: Vanessa at July 5, 2007 12:10 AM
dodge
Posted by: Anne at July 5, 2007 1:02 AM
dodge
Posted by: Rick at July 5, 2007 1:48 AM
dodge
Posted by: Peter at July 5, 2007 2:03 AM
dodge
Posted by: Brytney at July 5, 2007 2:33 AM
dodge
Posted by: Lohness at July 5, 2007 3:03 AM
dodge
Posted by: Victor at July 5, 2007 3:18 AM
dodge
Posted by: Guruchel at July 5, 2007 3:33 AM
Nice...
Posted by: Anonymous at July 8, 2007 7:01 AM
Sorry :(
Posted by: Anonymous at July 8, 2007 7:01 AM
Interesting...
Posted by: Anonymous at July 9, 2007 1:04 PM
Cool!
Posted by: Anonymous at July 9, 2007 9:33 PM
Cool!
Posted by: Anonymous at July 9, 2007 9:56 PM
Nice!
Posted by: Anonymous at July 10, 2007 3:55 AM
phentermin
Posted by: Luk at July 10, 2007 2:14 PM
Sorry :(
Posted by: Anonymous at July 10, 2007 8:11 PM
Sorry :(
Posted by: Anonymous at July 10, 2007 9:18 PM
Managed Hosting, Colocation and Data Center Services by victoryushchenkonashpresudent ...
Posted by: Robert at July 26, 2007 11:20 PM
tramqadol
Posted by: Jak at August 8, 2007 11:49 AM
tramaxdol
Posted by: Bob at August 8, 2007 11:56 AM
tramxadol
Posted by: Lohness at August 8, 2007 12:04 PM
trmadol
Posted by: Unison at August 8, 2007 12:19 PM
tramaodl
Posted by: Bobi at August 8, 2007 12:27 PM
Fed!!!!!!!!
Posted by: Dertert at August 30, 2007 8:51 PM
If you want do delete your site from our spam bases - just email us with domain of your site:
abuse-here@inbox.ru
thank you!
Posted by: Alexxiy at October 2, 2007 3:30 PM
If you want do delete your site from our spam bases - just email us with domain of your site:
abuse-here@inbox.ru
thank you!
Posted by: Alexxiy at October 2, 2007 3:30 PM
If you want do delete your site from our spam bases - just email us with domain of your site:
abuse-here@inbox.ru
thank you!
Posted by: Alexxiy at October 2, 2007 3:30 PM
Nice!
Posted by: Anonymous at November 8, 2007 3:28 PM
Sorry :(
Posted by: Anonymous at November 13, 2007 4:58 AM
Nice!
Posted by: Anonymous at November 13, 2007 5:35 AM
Cool...
Posted by: Anonymous at November 13, 2007 10:47 AM
Interesting...
Posted by: Anonymous at November 13, 2007 1:06 PM
Cool...
Posted by: Anonymous at November 13, 2007 1:13 PM
Cool...
Posted by: Anonymous at November 13, 2007 3:59 PM
Nice!
Posted by: Anonymous at November 13, 2007 6:44 PM
Cool!
Posted by: Anonymous at November 13, 2007 7:54 PM
Nice!
Posted by: Anonymous at November 13, 2007 9:17 PM
Interesting...
Posted by: Anonymous at November 13, 2007 11:01 PM
Sorry :(
Posted by: Anonymous at November 14, 2007 12:10 AM
interesting
Posted by: Anonymous at November 15, 2007 2:01 PM
buy tamadol
Posted by: Guruchel at November 16, 2007 2:29 AM
tablets buy tamadol
Posted by: Mister at November 16, 2007 2:40 AM
tramadol
Posted by: Victor at November 19, 2007 9:00 AM
tramaodl
Posted by: Bobi at November 19, 2007 9:05 AM
amateur pussy forums
Posted by: Jak at November 20, 2007 11:25 AM
tramwadol
Posted by: Bobi at November 21, 2007 8:56 AM
traadol
Posted by: Unison at November 21, 2007 12:08 PM
propec9ia
Posted by: Bobi at November 22, 2007 11:09 AM
szoma
Posted by: Lohness at November 22, 2007 12:03 PM
ultrxam
Posted by: Ron at November 22, 2007 12:31 PM
Nice
Posted by: Anonymous at November 27, 2007 6:23 AM
levitrra
Posted by: Jak at December 1, 2007 12:07 PM
levit5ra
Posted by: Luk at December 1, 2007 12:25 PM
levitrs
Posted by: Lohness at December 1, 2007 12:39 PM
levbitra
Posted by: Luk at December 1, 2007 12:46 PM
kevitra
Posted by: Medical at December 1, 2007 12:55 PM
medical-25
Posted by: Mister at December 3, 2007 3:59 AM
pharmacy
Posted by: Unison at December 3, 2007 4:08 AM
pharmacy
Posted by: Victor at December 3, 2007 4:19 AM
Pharmacy
Posted by: Tablets at December 3, 2007 4:29 AM
tablets
Posted by: Unison at December 3, 2007 4:39 AM
Golrtok
Posted by: Luk at December 3, 2007 1:49 PM
Qwest
Posted by: Niger at December 3, 2007 2:08 PM
Larisa
Posted by: Vladimir at December 3, 2007 2:28 PM
Tolik
Posted by: Xlop at December 3, 2007 2:42 PM
Eger
Posted by: Muter at December 3, 2007 2:56 PM
Golrtok
Posted by: Niger at December 3, 2007 3:20 PM
Oktavian
Posted by: West at December 4, 2007 10:39 AM
Lohness
Posted by: Bob at December 4, 2007 10:51 AM
Nikolay
Posted by: Xolms at December 4, 2007 11:07 AM
Sorry :(
Posted by: Anonymous at December 4, 2007 11:24 PM
Kola
Posted by: Nolkour at December 5, 2007 11:33 AM
Levon
Posted by: Natalia at December 5, 2007 11:57 AM
Nice
Posted by: Anonymous at December 9, 2007 12:00 PM