« And nothing else compares | Main | Disgusting Little Urchins »

Subversive Subversion Conversion?

Bug 347069, "Setup a production SVN server," was recently RESOLVED as FIXED. This has spurred some confusion, so I wanted to answer some of the common questions we've received:

Is the Mozilla Project switching to Subversion?

There have been many discussions in the past few months about the version control system that the Mozilla project entrusts its code to. It's safe to say there's a desire from most of the community to thank CVS for taking good care of our source code—for the most part—and move into the 21st century.

Obviously, such a move is a big deal, and impacts the very core of the Mozilla Project: our source code. It's not a decision to be made lightly, or by a limited subset of people. It's a project-wide discussion.

The first part of these discussions has already taken place, and a set of Project's requirements for a version control system has emerged. As they wiki page notes, some of these requirements are in conflict, so they represent a utopian ideal of version control systems.

But no decision has been made on which version control system to switch to, nor have any concrete plans (schedules, etc.) even been considered.

Why has a "production" Subversion server been setup?

As part of evaluating various options, we would want to test various systems out.

Subversion is an obvious contender and various other open source projects (Gnome, KDE, Apache) use it. The goal in filing the bug was to set up a production quality server to provide a place to experiment with setting up a robust infrastructure for not just Subversion, but another production-quality revision-control-system-that-is-not-CVS. Additionally, it allows us to gain some experience and insight into our (often implicitly expressed) requirements around things like authentication and clustering.

Please don't get hung up on the word "production." (More on that below.)

Its use merely means that we wanted this on a machine that wasn't someone's desktop and wanted to ensure that IT was included in its setup and evaluation, so their requirements for providing A-level service and support for a CVS-replacement could be (experimentally) gathered and added to the requirements list for a new revision control system.

It is important to note:

  • This server has a significantly lower SLA than most of the other IT services; basically, 9-5, M-F support. It is not considered required infrastructure. There is no oncall@ support for Subversion.
  • This server instance may get blown away and re-created multiple times. We will be playing with hook scripts, repository layout, and other options. We may decide to "start over" possibly multiple times, while exploring these options.
  • This server does not have any support services, such as Bonsai, LXR, Tinderbox coverage, fine-grained access control setup, or even backups!

Why Subversion first?

While there is a healthy debate about which system to switch to, I think everyone can agree that Subversion is one of the viable options.

Subversion was chosen as the first test system because some small portions of code that we rely on were in Subversion repositories (at OSL, for instance).

Additionally, members of the Subversion project came forward to offer assistance with deploying an initial setup, and we've successfully performed a test import of the Mozilla CVS tree into Subversion, thus meeting a (pretty important) initial requirement listed in our requirements.

What is this "production" Server going to be used for, then?

The server will be used for projects that have no dependencies on code in the CVS repository.

Currently, the guinea pig project is AMO3. Again, as they work on the project, we'll undoubtedly play with the server's options and possibly layout. Recreating the server is a likely outcome.

As this trial installation matures, we may solicit help in the form of testing from additional "segmented" projects to ensure that the infrastructure we've created/migrated meets the needs of other Mozilla projects/initiatives.

Can I get an account on the Subversion server?

If you're working on a project that is on the SVN server (currently only AMO), testing it out, file an IT ticket indicating your username on the CVS server, and which project you're working on.

If you'd like to help out by testing, please email which project you'd like to volunteer, where it is in the CVS repo, and what specific goals/requirements you're trying to test out to build@mozilla.org.

Note that we'll want to get AMO settled and other parts of the support infrastructure setup before we let more people into the sandbox, and that is likely to take at least a couple of months, especially given other priorities like Firefox 2.

Also, we're likely to tend towards projects that allow us to get the most coverage of disparate requirements, to maximize the utility of this pilot testing project.

Hope that clears everything up! Feel free to post any other questions you may have.

Comments

Hey! Flock is using SVN for many months now.

We did a lot of work to make SVN play nice for Mozilla codebase. We even tried another product (Mercurial) because SVN was too slow. We fixed all the issues and we can help you probably if you'll decide to switch to SVN with Mozilla sources.

I'm all for it. I've been using Subverison for personal proejcts since May, and have found it to be great. My favorite feature is actually not in SVN directly...

TortoiseSVN is sweet. Unlike the CVS version it has much more polish, and working diff, and browsing abilities. Just awesome.

We switched from CVS to SVN at Open Source Applications Foundation (OSAF) some time ago. So far we've been pretty happy.

We had lots of problems with (the default?) SVN db backend, but none after switching to fsfs (? I should check with our build engineer:).

We were using Bonsai as well with CVS, and did some investigation. There is one "svn bonsai" that was written using Plone that we considered, but eventually hacked Bonsai to support SVN. The diffs might not have made it to bugzilla yet. There are some parts that we didn't do yet, like blame.

Blame can be gotten with other tools, though (we have several web frontends: viecvs, websvn, ...). Have to say I liked Bonsai's blame best, though (mouse over checkin comments are cool).

You can see our stuff working:

http://bonsai.osafoundation.org/svnqueryform.cgi

http://websvn.osafoundation.org/

http://viewcvs.osafoundation.org/

Well, I have to say, we do have SVN working for Flock but it fills me with hate. It feels way slower than it should be. On the other hand, I could never get the hang of branches in CVS...

Nice post, Paul. In my experience Subversion has been much easier to work with (again, with the fs).

I remember the days when I would randomly corrupt the db and hope that svnadmin recover would save the day. I'm glad that stuff is history.

I think we can gain a lot through use of externals, especially if we can form some shared libraries for webtools, etc., that could be maintained externally from their main applications. Externals have been one of my favorite SVN features.

An interesting product I thought I'd mention is FishEye, which I used a while back and seemed pretty damn cool.

I like ViewVC just fine for our needs, though.

Advantages of svn over cvs should be huge in a project like mozilla. Essentially things like merging, branching and other release management activities are much easier (and more dependable & reliable) in subversion.

Also subversion offers some nice opportunities for integration into bugtracking tools. A cool trick is to store the bugid as a property on commit messages. Also the current practice of storing patches in the bug system could be reconsidered by creating (multiple) temporary branches for bugreports and merging & deleting them when the bug is closed. You could even tag such patch branches as reviewed, peerreviewed, etc.

We switched to svn some while at work, and despite all the positive points it does show that performance is an important criteria and svn might have some problem there.
Maybe we're not expert enough to find how to solve that but lengthly commit/update and apparently memory requirements proportional to the size of the tree/source is just a pain in the ass and might be a no-go for a tree the size of the one of mozilla.

For all those saying that Subversion is slow, I'd like to point out that significant performance improvements have been made in the up-and-coming version 1.4 (out in a few weeks). See also the release notes:

http://subversion.tigris.org/svn_1.4_releasenotes.html

Git does have extremely good performance (and it's a distributed SCM, unlike cvs and svn), and by now it should work on all platforms - paper: htt://members.cox.net/junkio/200607-ols.pdf

(Don't know about the rest of requeriments, just a suggestion. svn is a good CVS replacement, but then almost any SCM is a good CVS replacement these days)

Git seems to generate very large downloads for the initial checkout. But I haven't read the docs, I was just doing what someone told me to... cg-export http://git.example.com/something.git/ , IIRC. Does that download the entire repository, or just the current version?

Git itself also has low level commands, AIUI, and I think that has led to repos getting broken. I don't know how much the higher-level scripts like cogito prevent that sort of thing.