Test on multiple platforms

January 10, 2009

approximately 2 minutes to read

Just testing on one platform is not enough. Over the last two days, I’ve been porting a fairly large project over to Linux, and today I fixed the last problems. So far, all 400 unit tests that work on x86 and x64 Windows work also on x64 Linux, which gives me 6 (!) different versions of my project to test (OS x Architecture x Debug/Release). During the porting, I caught a few non-standard C++ constructs, which could create problems (some ambiguous stuff), but which were trivial to fix.

Much more important was however a misinitialisation, in which I forgot to init a member variable. The class has more than ten members, which are all initialized in an initializer list. Well, all except two, and those two worked fine in every configuration except for Linux/x64/Release. I was pretty puzzled at first, suspecting it might be some compiler bug, as the code that was executed was rather complex (lots of template metaprogramming). Some digging around didn’t bring up a good solution, so I thought I’d give valgrind a try. Valgrind is an excellent tool for capturing memory leaks and unitialized variables, and it immediately gave the right hint. Half an hour later, the bug was fixed and everything works fine again now (remember, that’s 6x400 = 2400 tests running!).

The lesson to learn here is: Try actively to port as much of your application as you can. Even if everything works fine on your current development platform, it can very well contain hidden bugs, which may pop up later on when the compiler changes on your platform, or if the memory block you place the stuff gets different, or when you ship it to your customer. Moreover, porting highlights other problems as well:

lower/upper-case names: Linux is case-sensitive, and captures mistakes in misspelled header names. Better check often than having to update hundreds of headers when you eventually have to port to a case-sensitive platform. Or if you try to access a case-sensitive web-server.
build dependencies: You see immediately on which libraries your application depends, and you are forced to think twice about using precompiled ones (for example, the distro libraries) or compiling and linking yourself. Also spots problems like “I’m expecting version 1.2 of library X, but my Linux comes with 1.3, which breaks Y, and I never noticed cause Windows didn’t come with it at all”.
Path-handling problems: A funny one, but some of my build tools were expecting “app.exe”, while on Linux, it’s of course just “app”, and hence didn’t work. Same goes for ‘' instead of ‘/’. Takes rather long to fix if not done right from the beginning.

Another interesting possibility that I have now is to run a few of the tools in the project on our Linux server, which is a pretty beefy machine. Previously, it would have required a developer to keep his machine running for hours (or maybe even days), but now we can use the server for this long running tasks. This, and the fact that I could iron out lots of smaller problems, was surely worth two days of porting. I’m also hoping I’ll be able to profile the tools on Linux using callgrind, which saves quite some money for VTune and friends.