Getting yourself into the "zone", and some notes on programming as a craft

Do you remember the last time you were in the "zone", coding away like 5 people, and feeling good? I remember it very well. It was a few weeks ago, and it's the reason why I'm posting. In three hours, I've written 1500 LoC, and I got insights how to continue from here on, and if I would have gotten more sleep the day before, I would have probably continued hacking too long into the night.

But let's look at this "zone", also known as the "flow", which is a nebulous thing. There is a great diagram describing the conditions when the flow sets it. Basically, if you have high skills, and an appropriate challenge, then you are right in the sweet spot. There's one catch: First of all, most of the time you don't have the right skill level, and second, not every task is a high challenge.

I remember a famous quote of -- as far as I know -- a Russian coach, saying "talent is what allows us to survive the training". I think this also applies to programming, and getting into the zone is surely easier for people who have the programming talent than for the rest. But even if you are not gifted with it, there is one sure thing which helps me to get into the zone, and might help you also:

Stop thinking about your problem. Start coding.

I remember each time I start thinking through details, designing large pieces of code in my mind, thinking of many algorithms, I stop being productive. It's so simple. Most probably, the task is large, and if you start thinking through it in detail, you immediately see problems en masse. Which makes you think more, how to avoid these problems, making the whole task even more complicated. It's a vicious circle.

There is only one cure: Stop worrying. Start working on it, by writing code. Often enough, you'll be able to write some working solution, just to get going. This is of course not an excuse to start without any design at all: Think up front how to divide your task, even make an UML diagram, but don't try to solve all problems up front. Rather, after having some working code, iterate over it! You'll be impressed how much faster you get the work done if you either prototype something and rewrite, or simply try as good as you can, and refactor. After all, even the worst code can be rewritten or refactored, but it still requires at least some crappy code to start with.

The same applies to these stupid tasks which simply take time, and where you hesitate to start them because you know you'll have to do a lot of work. Point in case: In a large personal project, I've been replacing a STL class with a custom one. Well, I immediately estimated that I would have to touch a few hundred places, and it will take me one day at least. The same rules as for profiling apply here as well: Assume nothing, but measure! A quick search told me 150 locations have to be touched. At this moment, the task was no longer a mysterious beast, it's simply a marathon, and you can train yourself for it. Grinding through code requires just this, training, and a clear progress indicator. After less than half an hour, I was done with what I previously assumed would take me 16 times longer!

Learning by failing

A different aspect of our craft: How do you improve your skills? If you do as described above, you're using a very effective method -- you start working on problems which are too complicated, and often enough, you'll fail. This might sound counter-intuitive at first (how the hell does this make me better?), but the learning effect is much higher than in all other cases: First of all, in your first try, you probably attack the problem from a totally wrong side, and second, you understand why your particular approach is not good (side note: This post has been prewritten several weeks before it has been posted).

I probably use this approach too often, but so far, I'm still pleased with the results. Funnily enough, I guess I'm getting better at failing -- during this week, I think 3 algorithms I implemented turned out to be not sufficient, and required a lot of additional work. So if you're out to learn just like I do: The next time you hit a problem where you are about to look up the answer, just try on your own, and if you fail, come here and comment ;).

Pixel aligned rendering in OpenGL, and direct state access

Two main topics here: The first one is how to render pixel-aligned objects in OpenGL, the other one is a great extension for OpenGL.

Pixel perfect rendering

If you want to render a 2D overlay, you usually want to have the vertices snapped exactly to pixels, so a line from 3,4 to 3,6 is exactly 1 pixel high and two pixels large. With OpenGL, this requires you to move place your vertices exactly at half pixel offsets. Of course you can also translate the whole scene by 0.5 to the bottom right.

The other thing you have to set up is a projection matrix which maps the vertices directly onto pixels. In the easiest case, you set the projection to be an orthogonal matrix, and the model view matrix to identity. For the orthogonal matrix there is a neat trick to move the origin to the top left, and this can be done by setting the matrix like this:

glMatrixLoadIdentityEXT(GL_PROJECTION);
// -1 mirrors in Y
glMatrixOrthoEXT (GL_PROJECTION, 0, screenWidth, screenHeight, 0, -1, 1);
glMatrixLoadIdentityEXT (GL_MODELVIEW);

In case the calls look a bit unfamiliar to you, welcome to the second topic! We're finished with the first one actually, because after this few lines, you are ready to work in 2D. Just remember to position your vertices at half-offsets, and everything will match up properly. You'll probably also want to disable depth tests at this stage, and rely on proper overdrawing.

Direct state access

This is the most practical OpenGL extensions of the last few years: EXT_direct_state_access. It allows you to set a variable without having to query the state, change it, and set it back to what it was before. A real time saver, and if you look at the extension author list, you know why it's as slim as it is. Right now, it is supported by the latest nVidia drivers, and works just as expected.

Small rant, can't resist. Call me a fanboy, but in my opinion, we should be all happy for the support OpenGL is getting from nvidia. If it would be AMD only, I believe OpenGL would be even more dead than it is. I guess it's the SGI heritage and the strong workstation market which keeps nvidia dedicated to OpenGL, and I hope they won't give up. Even though I'm not using OpenGL right now, I consider it important that there is always a viable alternative. Who knows, I might end up doing more OpenGL in the future again, and then I'll be glad if I won't have to endure to much pain and suffering.

Carmack's keynotes

If you are even remotely interested in the game industry, and you haven't seen Carmack's yearly QuakeCon keynotes, you should go and do it, right now. In the whole games industry, his keynotes are absolutely unique: He speaks about the industry, his work and other things as he sees fit. Some highlights from his keynotes:

  • He explains how he was arguing with Steve Jobs about the iPhone. It's really funny, especially as he tells how he was going like "buuuh" when Jobs announced that you can write web applications for the iPhone. Who else would have the nerve to go to Steve Jobs and argue with him, if not Carmack?
  • His random rants on SDKs, developer productivity and stuff. This is a real gold mine, when he goes out and says the Sony PS3 SDK is crap, or how he was reading books about writing better code, and considered none of them useful. The important thing is that he does not simply make things down, but always explains the reasons behind. It's really a difference if someone who has ported games to 20 different consoles personally says that console X has a bad SDK! Plus, the others are extremely careful with their statements, and would never say something negative on one independent vender. Carmack on the other hand does not seem to care too much, knowing that the companies will rather try to fix stuff than to punish him.
  • Graphics thoughts. Another cave full of treasures. Carmack speaking about virtual texture mapping and multi-threading is even more interesting now, in retrospective, than it was when I heard it the first time. Now, we know what came out, and the early keynotes give us an idea how the idea started actually.
  • War stories: Carmack jumping in and fixing the audio code in Doom3? Going off to a hotel to make a Nintendo DS engine in 4 days? This is practically stuff you would only hear otherwise if you work directly at id.
  • Random anecdotes: Carmack grinning when explaining how he is looking forward to start a rocket using his mobile phone is simply hilarious.
  • Carmack and academia: From early failures when nobody was interested in Doom, until he was peer reviewing SIGGRAPH papers, it's interesting to see his relationship with the formal research side. Always keep in mind that he said "above all, I'm pragmatic".

You should probably read the excellent book "Masters of Doom", about the beginning of id software, and -- if you are more interested in the technical aspects and a good amount of lore -- Michael Abrash's black book in the extended edition. Personally, I think it's great for the game industry to have someone like Carmack, who is willing to share his experiences with us, and not isolate himself.

Enjoy the ride

Sounds like the new slogan of a car maker, but this post is about the beautiful song by Morcheeba named "Enjoy the ride", from the album "Deep Dive". While I have mixed feelings about the album, I'm absolutely astonished by the song "Enjoy the ride", which is also the opener of the album.

First and foremost, the lyrics are incredibly good. Even if you just read them, you can feel the emotions and the hopeful message in them. It's not a funny song, nor one with a happy background, but it's surely about hope, and making the best of the life.

Second, it's perfectly executed. The instruments and the voice fit perfectly into the mood, and together with the lyrics you can close your eyes and go on a real deep dive while listening to the song. The instruments are never intrusive, and there is nothing which interrupts the flow of this song.

A great moment is halfway through it, when the vocals set out for a moment, and you can hear the pure melody. After that, you get caught by the melancholic voice again, exactly at the right moment.

You should try to listen to the music first, and watch the movie clip afterwards, as it's equally superb and you'll have a hard time not to think about it when listening to the music again. So for the best experience, try the music alone first, and watch the clip later, and you'll be rewarded twice for it.

Regarding the rest of the album, it also has a few highlights: "Run honey run" is a really great, especially the lyrics, "Gained the world" is close to "Enjoy the ride", and beautifully sung, and finally there is "Au-delà". Compared to these, the other songs seem to have gotten less polishing. They are all very different, and they all manage to set up a mood, by for me they lack this special kick that makes "Enjoy the ride" not merely good, but an amazing piece of music.

So, stop searching for good music, just enjoy the ride!

.NET 4.0/Visual Studio 2010 Beta 1 is out

Already two days old, but I didn't have time so far to try on my own. Today, I gave the new Visual Studio 2010 Beta1 a try. For a much longer list of features, see Jason Zander's announcement.

Installing is painless, you get a web-downloader or an ISO, and need to fetch around 1 GiB only, which is not too much, considering that it's VS Team Suite and the .NET Framework 4. One reboot required, due to Windows Installer 4.5. Tested on Vista x64 with SP1 installed.

The new GUI is okay-ish, but really not a dramatic difference to VS 2008, even though it is built on WPF. The C++ editor is now a bit closer to the C# editor, you get errors immediately in the editor window with red underlines, which is a real time saver. Overall, I'm not happy with the new Visual Studio though, as it looks like a merge of 3-4 different versions of VS into one. For some things, they use still the wizards from VS 2003, in other places, you have VS 2005 looks, some are 2010 obviously, and all in one package (for example, they still have two different properties widgets, one in WPF, one in Windows Forms ...). So this is not really good. Another small, but nevertheless annoying issue is that if you add a new item to a C++ solution, the list starts with a "Windows Form", then other unrelated items, before "C++ file (.cpp)" shows up. I mean, if I work on a C++ solution, most likely, I'll want to add a C++ source or header file, especially if I'm not using the unnative extensions.

I've already reported my share of bugs again (goddamn, the SSE code generation is as bad as it always was. I'm so close to buy a copy of Intel C++ ...), maybe they will fix some of those before the release. Ahh, and before I forget it, I could already observe a problem where IntelliSense could not lock the .sdf file, so symbol information was lacking (remember, they use SQL Server for symbol storage, which uses .sdf files, instead of the old .ncb ones). On the other hand, the C++ compiler gained a few C++0x features in the front end, which are always good news. First of all, auto support is in, and this one is a major time saver alone :)

We'll have to wait and see how they continue the work on VS, but for me it's a bit of good progress on one side (C++ frontend), but a suprisingly large lack of effort on other sides (consistency is worse than it is in VS 2008, and the C++ compiler backend is not improved either, at least not in the areas I care about). I guess it will take 6 more months until the release, so go ahead and report every problem you find now ;)