Changes

It's been a while since my last post, sorry for that, I've been quite busy at university. Read on for some news about me as a Linux-based developer, stack machines, consoles and more.

Linux

After my long journey during which I tried Debian, Gentoo, OpenSUSE and Archlinux I eventually installed Ubuntu Linux (Dapper Drake 6.06) on my notebook which is also the first Linux that worked nearly right out of the box (except it didn't recognize my notebook monitor). There are still some minor things left to fix, notably:

  • WLAN is not working because my WLAN button is software only and acer_acpi refuses to compile (args!)
  • tetex is broken (Link)

Hopefully both things will be sorted out during time. All in all, I'm quite pleased with Linux for my development work. Some things could be of course better (GCC's error messages ... I like Microsoft's VC++ more with its Cxxxx error for which you can usually find an example case on the MSDN) but all in all I'm already quite productive under Linux. I've written three university homeworks during the last two weeks, and didn't hit any larger problem during that time. What I really like is callgrind/valgrind, I'd love to see something like that bundled with the Visual Studio Professional. Currently, I'm forced to use make for building, but I plan to start using SCons soon. Although make is a powerful tool, I like the concepts behind SCons more. The latest university homework was (actually, it still is) a simple Linux shell. It's quite similar to what I want to use in niven, so I decided to implement a bit more stuff than required. Anyway, implementing stuff like a shell in plain, C89 C is really a pain. I'm really looking forward to porting it to C++ which should also give me an extra bit of flexibility. The shell comes also with a complete stack/register-machine similar to what I plan for niven's script VM. It turned out to be rather simple to write the stack/register-machine backend. During the next few weeks, I'll try to write a front-end compiler for it that takes some kind of high-level code and generates stack/register-machine opcodes. As I don't know yet how this is supposed to work, I think it'll be quite interesting :).

Music

Took me quite a long time, but I finally decided to add a new "music" section. Here I'm going to recommend from time to time some great music. Read on for the first recommendation.

A-ha - The swing of things

If you get a chance, listen to the song The swing of things from the album How can I sleep with your voice in my head - a-ha LIVE. It's surely one of the best songs I've ever heard, unfortunately I don't remember that it ever ran on air here. The song has very interesting lyrics, really worth thinking about and a truly addictive chorus. All in all, the whole album is full of excellent songs, but this one really shines apart. Go and get it!

A bit about me

I've got a really big passion for music. I have a rather large collection of music albums at home, most of which contain songs that you probably never heard about. My primary music system is a Creative Soundblaster Audigy 2 ZS bundled with a Teufel Concept E surround set. This bundle offers extremly good sound for it's not-too high priece. The Concept E comes with a truly impressive 10 kg wood subwoofer which has no real rival in its range, it beats even three times more expensive ones from JBL easily. The subwoofer is strong enough for any room up to 25m². It's really worth a try :) The soundcard itself is very nice because it hardware accelerates music playback so your CPU is free for other things.

Packages update

I finally finished all stuff needed to store packages - since today, niven is able to store any package to disk. I hope to finish the multi-threaded package loader this week, and then I'm finally done with most of niven core and I can get busy on the renderer again - w00t!

File I/O performance

Over the weekend I've been implementing some more low-level classes for the package system. The new archives which work directly on the filesystem are around 20 times slower than the memory based ones, so it's rather very likely I'll stick with the memory archive approach - storing an object to memory first and streaming it to file later.

Write performance

My first test consists of writing 100.000 3-float vectors to an archive. This results in 300.000 raw write calls à 4 byte on the underlying stream.

MemoryStream: 0.22 s
FileStream: 3.11 s

The FileStream timings get much worse when some other disk I/O is going on, sometimes by as much as 50%! All this timings are from a fully optimized release build running on my machine with 2GiB RAM and a S-ATA2 RAID1 disk array. Note that the resulting file is smaller than the HDD cache let alone the system cache, so the OS should be able to buffer the complete file instead of writing 4-byte chunks.

Read performance

Reading in the same file, again 300.000 raw reads à 4 byte.

MemoryStream: 0.09 s
FileStream: 1.22 s
StreamView (FileStream):  1.52s

Again, the memory stream is outperforming the filesystem by a factor of 10 - although the file was probably put into the system cache already. Anyway, as bad as this sounds, 1 sec for loading 100.000 micro-objects is not that bad, and most objects are going to be rather huge (textures, models, etc.). Reading is also not that sensitive on other processes, it becomes at most around 10% slower. The StreamView code is performing additional checks on every low-level read/write which slows down the whole process by another 20%. This is not much if you take a look what it really does - in every single read call it checks for a valid stream, checks the stream mode, gets the current position in the stream, checks if the next read is valid and passes the read call over to it's contained stream. All these functions throw exceptions on error, and this only adds 20% overhead!

Raw write performance

Something rather odd is the raw write performance. Here the file system wins because it does not need to copy the 1.2 MiB of data into a temporary buffer - no matter how often I run this test, the memcpy code seems to take the 0.02 secs, although there is enough memory available.

MemoryStream: 0.02 s
FileStream: 0.00 s

Conclusion

What does this mean for niven? I'll probably stick with the memory archives for storing data but use file streams when loading stuff. After all, the file system gets better the larger the chunks get, so I expect that it won't matter any longer as soon as I'm loading megabyte-sized textures.