« More on distributed VCS performance. | Main | Mozilla cvs file "moves" considered harmful. »
February 16, 2007
bzr and different network protocols.
As "D" kindly pointed out to me in a comment in my previous post, the comparison I did was unfair in that I used sftp for bzr when pulling from a server, but http for hg.
While I realized there's crypto overhead etc involved in sftp, I didn't imagine that the overhead would be noticeable, but to be fair I re-ran the tests. The results are interesting to say the least. I did two pulls over http with bzr for the same repository I used for my earlier tests (still on localhost, served through apache), and I've yet to see a number that shows a significant speedup with http rather than sftp. One run took 28 minutes (vs 33 with sftp), the other took almost 44. Go figure. I also tested the bzr protocol, and that is faster, a pull using the bzr protocol, with bzr itself serving it, took 15 minutes, about half the time of my initial tests with sftp. So that's definitely a good improvement, but why http is as slow, or even slower, than sftp is beyond me.
Posted by jst at February 16, 2007 4:27 PM
Comments
http is not very desirable for serving files, even with keep-alive.
I don't know about bzr, but if it does not use keep-alive, it would have to set up the communication for each individual file.
ftp, on the other hand, always reuses the existing connection.
Posted by: Asrail at February 16, 2007 9:01 PM
That was actually John Meinel who pointed that out, not me, heh. Comment attribution is confusing that way. I mean, there's a big divider separating the comment text from the commenter-information and no divider between that and the next comment.
Posted by: D at February 16, 2007 10:33 PM
Wasn't that comment by John Meinel, not D?
Posted by: voracity at February 18, 2007 3:05 AM
Wasn’t it John Meinel that pointed it out?
Posted by: Laurens Holst at April 12, 2007 11:12 PM
Also, this is on Linux, right? Given bzr�s non-usage of hard links because they�re not available on all systems, I wonder how the performance compares on Windows and Mac OS X? And can Mercurial deal with FAT32 and HFS+ file systems? I recognise John Meinel�s words about hard links on NTFS, I�m pretty sure I don�t have any of them on my entire system :), it�s definitely not a commonly used feature. I actually wouldn�t really know what to expect... If you delete one file, the other is still there, right? But if you modify it, both files change?
But anyway, I suppose the right thing to do for bzr here is, if the lack of hard links thing is the main performance problem, to implement hard linking if it�s supported by the OS, and otherwise emulate it.
~Grauw
Posted by: Laurens Holst at April 12, 2007 11:19 PM
Yes, that was of course John Meinel who pointed that out, not "D" as I mistakingly said in the blog. Thanks for pointing that out!
Posted by: jst at April 13, 2007 1:43 PM