I took another stab at directly cross-compiling (not via distcc) using toolwhip. My previous attempt failed, but not due to libffi at all. The problem there was that I had the Mac ld in my path before the host ld. After fixing that and a few mozilla build system issues and making sure the build system can find the right SDK and the right
ar and such I now have a cross-compile setup working. No
make package yet, since that requires the dmg-creation utility which I don't have on the Linux machine. No
--enable-breakpad support yet due to bug 543111.
--enable-shark needs headers I don't have on that machine yet, so I'd need to sort that out. But for basic bisecting, this works.
Just so I can find it later, the working mozconfig file I'm using is something along these lines:
mk_add_options MOZ_MAKE_FLAGS="-s -j10" export MOZ_MAKE_FLAGS="-s -j10" mk_add_options CC="ccache /Developer/usr/bin/gcc-4.2" export CC="ccache /Developer/usr/bin/gcc-4.2" mk_add_options CXX="ccache /Developer/usr/bin/g++-4.2" export CXX="ccache /Developer/usr/bin/g++-4.2" mk_add_options CROSS_LIB_PATH="/Developer/usr/lib:/Developer/SDKs/MacOSX10.5.sdk/usr/lib" export CROSS_LIB_PATH="/Developer/usr/lib:/Developer/SDKs/MacOSX10.5.sdk/usr/lib" mk_add_options CROSS_COMPILE="1" export CROSS_COMPILE="1" export LD=/Developer/usr/bin/ld export AR=/Developer/usr/bin/ar mk_add_options PATH="/home/bzbarsky/bin:/usr/lib64/qt-3.3/bin:/usr/kerberos/sbin:/usr/kerberos/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/Developer/usr/bin:/Developer/SDKs/MacOSX10.5.sdk/usr/bin" export PATH="/home/bzbarsky/bin:/usr/lib64/qt-3.3/bin:/usr/kerberos/sbin:/usr/kerberos/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/Developer/usr/bin:/Developer/SDKs/MacOSX10.5.sdk/usr/bin" ac_add_options --target=i686-apple-darwin9 ac_add_options --enable-application=browser ac_add_options --with-macos-sdk=/Developer/SDKs/MacOSX10.5.sdk ac_add_options --disable-crashreporter ac_add_options --disable-optimize ac_add_options --enable-debug ac_add_options --disable-tests
Various other --enable-whatever are hanging out in there too, but the above are sort of the minimal set one needs at the moment.
I've seen a number of comments recently along the lines of "I'm in Europe, so the H.264 patents don't apply to me; why are you not letting me have a browser that plays my Youtube videos?" So I took a look at the list of patents on H.264. Or rather, the first 6 pages of the 43 page list. Excluding the US, there are relevant patents granted in at least the following countries:
I wasn't reading very carefully; I'd be suprised if I didn't miss a few, or if a few more don't come later in the list somewhere.
Today Arlan happened to look at my screen when part of the desktop was actually visible. That led to approximately the following conversation:
Little did he know that the fourth one (3.5 to be exact) was hiding under the iTunes window....
A few days ago I mentioned that I've been experimenting with distcc. I finally took the time to do some timings. All times below, unless stated otherwise, are for a completely clean debug non-libxul build on my Mac. In particular, none of the files are in ccache, but ccache _is_ being used (so the compile time includes the cost of caching the new object files). distcc is being invoked via
CCACHE_PREFIX. The distcc builds were done with -j10 (I also tried higher -j numbers and saw things between no effect and a slowdown as I got up close to -j20) and allowing 10 distcc connections to the server. The non-distcc builds were done with -j3, which is my normal build setup (since the mac in question only has 2 cores). distcc over ssh did not use ssh connection sharing, and was doing X forwarding on each ssh connection; I should retest with those two things changed at some point, if I decide I don't like doing things over tcp. The network is a wireless 802.11g on the Mac laptop end and gigabit ethernet to the wireless router on the Linux desktop. I found that ,lzo for the distcc builds didn't materially affect things (may have made the build 1-2% faster when used).
|local build on the Mac||38 minutes|
|distcc over ssh||50 minutes|
|distcc over tcp||32 minutes|
This was obviously distressing. I didn't test distcc over ssh after this, but particularly distressing was the fact that the compilation seemed largely diskbound and therefore distcc didn't help much. This got me started looking into hard drive performance as a function of utilization, since I could swear that the Mac's hard drive has been getting slower as it fills up. Turns out, this is actually the case. So I moved my rarely-used Linux VM to an external hard drive, which took free space on my internal disk from 4GB to 65GB or so. With that change done:
|local build on the Mac||32 minutes|
|distcc over tcp||22.5 minutes|
Progress! Next, I tried dropping ccache, which let me measure the impact of pump mode. Since I do generally see a benefit from ccache, and wanted to measure it, I also tried a build without distcc but with a primed ccache (so build a tree, delete the objdir, rebuild that tree). Results:
|no-ccache distcc in pump mode||16 minutes|
|no-ccache distcc without pump mode||17 minutes|
|Local build coming entirely from ccache||15 minutes|
So if I were pretty sure I'd almost always be near this Linux box, just not using ccache and using pump mode might be the way to go. As it is, using distcc + ccache seems like a good approach for now.
Now all I need is a way to figure out a way to get this sort of data without it taking several hours, a way to pick good values for -j, and a faster internal hard drive in this laptop.
Ted recently pointed me to the toolwhip project, which is a way to run Apple's gcc on Linux, cross-compiling to produce Mac binaries. Clearly desirable, since my Linux box is a lot faster than my Mac one. Here are the steps to get it working, as far as I can tell, at least if you Mac has OS X 10.5:
cctools.READMEto compile and install cctools. Note that the toolwhip tree comes with an existing
usr_includeso you can ignore the REQUIREMENTS section in this readme.
ld64.READMEto compile and install ld64. Once again, you can ignore the REQUIREMENTS section.
gcc_42-5566in your toolwhip source tree. If you plan to use this compiler with distcc, make sure it's the same exact version as that installed on your Mac. If you just plan to cross-compile, I'm not sure that it matters quite as much.
gcc_42-5566.patch) to the gcc source tree.
gcc_42-5566.READMEto compile and install gcc. This time, follow the REQUIREMENTS section as needed; in particular you will need to copy whatever SDKs you want to your Linux box.
ln -s Developer/usr/bin/gcc-4.2 /usr/bin/ ln -s /Developer/usr/bin/g++-4.2 /usr/bin/for all those makefiles that call the compiler with an absolute path.
At this point you should have a working cross-compiler. The remaining instructions are for setting up distcc with it.
distcc.READMEto build distcc.
distcc.READMEto build distcc on your Linux box.
What you do after this point probably depends on exactly how you plan to run distcc and distccd, whether you plan to use ccache, whether you want pump mode, etc. There are lots of directions in
distcc.README. So far I have a setup that does distcc over ssh to my Linux box. I'm hoping to sort out the rest of the distcc stuff (not using ssh, and maybe using pump mode) sometime in the next week or so. To get what I have working so far going:
PermitUserEnvironmentto "yes" in
/etc/ssh/sshd_configon the Linux box.
~/.ssh/environmenton the Linux box.
Now you run a test. In my case, I use ccache, so this worked:
export DISTCC_HOSTS=bzbarsky@linux-box:/path/to/toolwhip/distccd/on/linux,lzo export CCACHE_PREFIX=/path/to/toolwhip/distcc/on/mac export DISTCC_VERBOSE=1
and then compiling Firefox normally, using
make -f client.mk. If you don't use ccache, you would need to skip the
CCACHE_PREFIX and make sure that symlinks called
g++-4.2 came in your
PATH on the Mac before the real compilers and pointed to the toolwhip distcc. I think that should do the trick, but haven't tried it. The ",lzo" in the
DISTCC_HOSTS above just says to gzip-compress the data; it can be left out if desired.
Question for those with some distcc experience: is pump mode worth forgoing ccache?
One last note: I have not yet verified that this gives me faster compiles! Linking has to happen on the distcc client, and that tends to be a pretty slow part... Perhaps I would be better off just compiling entirely on the Linux box then copying the resulting .dmg to the Mac.
UPDATE: While the above distcc setup works for compiling an already-configured Mozilla tree, it seems that trying to configure from scratch fails because it thinks it failed to find Core Text, or more precisely because it fails to find ANSI C headers. But only if you use DISTCC_VERBOSE=1, so don't do that. It also seems that actually cross-compiling from Linux doesn't work because libffi insists on passing gnu ld options to an ld it detected as not being gnu ld. That's very sad.