Skip to main content

Graphics APIs done wrong

This is going to be partly a rant post, but I think we're really moving into the wrong direction with current graphics APIs. Especially as the GPGPU revolution is waiting for us, current graphics APIs put a huge unnecessary burden on the developers, and if they are meant to survive, changes will have to come.

Current problems

So where are the current problems, and where do they come from? Back in the old days, when the first graphics hardware was designed, everything was very low-level. No validation nowhere, no usage hints, but little flexibility. Later, as hardware started to become programmable, the APIs added new functions and abstractions for the new functionality. We got vertex buffers, index buffers, textures, and shaders.

The problems became obvious when things started to mismatch. If a shader would ask for a different format than the vertex buffer provided, the DX9 driver would do a fix-up to make them match. More and more validation was put into the runtime.

This turned out to be a large performance problem, so DX10 requires everything to be set in stone at creation time. If you create a buffer, you have to specify whether you want to read or write to it, whether you want it to be immutable, and if you want read/write access, you have to say how often so the driver can decide where to put the texture.

The same happened for shaders, now you have to create them along with the vertex format they are going to be used with, or you get an error. However, you can now reinterpret buffers, so you can for example render into a vertex buffer.

With DX11, you can also bind most buffers as sources in the compute shaders, and you get "unstructured data" while you still have to specify the usage etc. up front.

So even while we are getting more and more abstractions, we have to specify more and more data, and the whole pipeline is very rigid, despite the flexibility. If you want to take an index buffer, and simply treat it as a texture, you have to specify lots of stuff before this is allowed. If you map a write-only texture for reading, you will get an error, even if you specified that the texture was to be placed on the host. These and other things start to sum up. With DX11, we can create a display list now, which translates all the commands we issue into the "real low-level driver commands which are ready to be run on the GPU, without any validation whatsoever.

The road to the future

So where are the problems? Basically, all this added validation and abstractions have moved us so far away from the hardware that people are willing to write a renderer from scratch, for example for Larrabee native. This leads us to the main problem: We actually need two new graphics APIs.

At the high-level, we need an API which makes it easy to do stuff. If you need a texture, it gives you a texture, which you can access from GPU or CPU, for both read and write. However, it logs whether you have actually written to, or read from it (ideally with callstack if wanted). Later on, you can query this information to optimize the usage. You should be also allowed to treat this texture as a vertex buffer, or whatever you want to, and freely pass it to compute shaders. The performance of such an API might be horrible, but it will increase developer productivity enormously. If you think about it, it might even do run-time optimization based on the observed usage (as long as the texture is not modified on the CPU side for example, it does not have to be copied to the GPU)

The other one requires a fresh look at what we are actually doing: If I create an index buffer, or a vertex buffer, or a texture, all I do is to allocate some memory. How I am going to use this memory should be of no concern for the graphics card" whether it's going to be a 32-bit float shadow map, or a 2x 16 bit integer input to the compute shader does not make a difference how it is stored. What's important is where it is stored, whether it's GPU or CPU memory. Ideally, each time I allocate GPU memory, I can pass along a structure which I can query later on. For instance, if I want a vertex buffer, I get a pointer to GPU memory, and I store the type etc. on my own in a custom structure. No need for the GPU/driver to know this as well, all the GPU knows is the size of the buffer. By default, the GPU treats all data as opaque blobs.

Moreover, I can specify where to put the buffer (GPU, CPU, both), and optionally some hints if the GPU guys really need them. When rendering, I'm free to do what I want with this. If I bind it as a vertex buffer, I have to take care that the data can be interpreted correctly by the shader. All the GPU checks is that the number of requested shader invocations times the size of the shader input is less or equal the buffer size.

There are some special cases I can think of where a GPU will require some type hints to provide good performance" especially when creating textures, due to padding issues. But these should be made explicit to the users, just like CUDA makes padded allocations explicit. This low-level API allows me to get top performance (as I can tune everything) and gives me the possibility to write parts of the pipeline using compute shaders -- as the GPU does not know what's in the buffers anyway.


Unfortunately, what we currently have is the worst of both worlds. We have to do low-level specification of how we are going to use our data, while not gaining really low-level access, on the other hand, we don't get an abstraction which frees us from all this trouble. So why is this becoming a problem with GPGPU? Mainly, because GPGPUs shows us that internally graphics cards are really general. There is no special "index buffer memory" for index buffers. In CUDA, they are just memory. There is also no problem to reinterpret data as something different. It's really all up to us, the developers.

I do hope that in the future we will stop doing highly constrained graphics on GPUs, but treat them as a library of tools which we can use to do graphics (among other things). A GPU is not something built with index buffers as primitive data types, but a large graphics toolbox, which can do several tasks very well" like creating primitives from a buffer containing integers, and another one containing weird blobs of random data. We should get more or less direct access to the GPU command buffer, and issue what we need to get the work done, harnessing the full flexibility of the GPU. After all, we are used to this on the CPU side! We have general memory, we have pointers to data, so please hardware vendors, put us graphics developer on equal footing on the GPU.

If you're doing graphics, what's your opinion? I'd be really interested to hear whether I'm the only one who has these problems, or whether this is really common.

Building Qt from source with Visual Studio

In case you do UI development, Qt 4.5 and later is now a very interesting alternative since it is licensed under the LGPL. However, if you use Visual Studio, you'll find that there are no precompiled binaries, so you have to build them on your own.

Qt is a rather large library, and comes with a custom build system -- qmake. Building Qt is quite straightforward. I assume you are using Visual Studio 2005 or 2008, with both I had no trouble at all.

Before you start, you should be aware of one important thing: qmake contains all the paths hard-coded in it, so you cannot move your Qt installation. I simply compile it directly at the target location, but it should be possible to compile to a different target by setting the prefix and running an installation. For the remainder of this post, I'll be not setting the prefix, but installing at the target location. If you had luck with using prefix and installing, please comment, as I couldn't get this working.


Grab the Windows source package, and extract it to where you want to have Qt. Let's assume you want to install to Q:\\Qt-4.5.2, then simply move the extracted data into that folder (that is, Q:\\Qt-4.5.2 should contain the file configure.exe). You cannot move this folder afterwards, so be careful. Side note: You should have several GiB free, as the compilation requires lots of disk space.

The compilation itself is straightforward. Open your Visual Studio command prompt (x86 or x64), optionally run the dx_setenv command from your DirectX SDK (it's in $DXSDK/Utilities/bin/dx_setenv.cmd), go into your Qt folder, and run configure followed by nmake. To skip the examples and the demos, you should pass -nomake demos -nomake examples -- highly recommended, as you don't need them usually. This will usually take two or more hours (with demos and examples, prepare for several hours!), so far, I didn't find a way to enable the /MP switch (comment if you know how!). Using configure -fast doesn't make a noticeable difference, as the build time is not dominated by makefile generation. After the compile finishes, you should run nmake clean which removes temporary files. This is not equivalent to what make clean does with real makefiles! It this case, it really deletes only the *.obj files and leaves all your compiled binaries in place.

A short note on compiling: Before Qt 4.5.2, link-time code generation is enabled by default on Windows. This may be a problem for WebKit, which requires approx. 1.5 GiB of memory for linking (and it takes a long time). So be sure to compile the latest version if you development machine does not have too much memory.

Using from CMake

Well, that's it. To use Qt for example from CMake, you would point at your newly built qmake in the /bindirectory. All the folder paths are hardcoded in it, so you don't have to set up anything else.

[Update] Added how to skip the demo/example creation, thanks to reader Michael.

New MechWarrior game incoming

I had seriously lost all hope that we would ever see a MechWarrior game again, but it seems I have been proven wrong: MechWarrior (5) is in the making! And it really looks to be a real MechWarrior title, with heat management, proper cockpit view, and not an arcade style shooter like MechWarrior 4. Really looking forward to this one! Especially as there are two more serious games for infantry combat (Arma2, Flashpoint2), it seems that the market is ready for a simulation-heavy MechWarrior.

My only worry is how close to the universe this will be, 3015 seems a bit early for seeing an Atlas in battle, isn't it? Plus I'd really like to fight with a good ol' Timberwolf (Mad Cat), even though the reliability was always bad ;)

[Update] Seems the Atlas is indeed used way before 3015. According to Gamestar, the player will fight House Kurita, which is a nice change after all this clan bashing in MW 2-4 ;).

Antibes, Sophia-Antipolis, Cannes

As promised, a post about the area. Specifically, I'll cover three towns (Cannes, Antibes, Sophia-Antipolis) located in the south-east of France.

A general note on the area: When money was invented, it was probably not far away from here. It's unlikely that there is any other area with so many people who are extremely rich. In the area I come from, you would see cars like a Bentley Continental GT maybe once a year, usually on the highway -- here, it's common for me to spot one or two while driving to the beach. Of course, the houses and especially the yachts are much more impressive, but I'll come to this later. Nevertheless, it's obvious that quite a large percentage of the people here has basically unlimited funding.


First, let's start with Antibes. Antibes is a small town right at the coast, which has a quite large yacht harbour (according to Wikipedia, it's the biggest one in Europe). Antibes has a small old town, but the nicest part of Antibes is not the town. Basically, there are two places to visit, if you ever come here.

As noted, the harbour is large. However, large is only one thing, as a large empty harbour would be not too interesting. In Antibes though, the harbour is basically full up to the last place with yachts ranging from normal over extremely expensive to the highest end. Most of the time, you have around 100 yachts or so with ~ 30 m (50M€) in the harbour, with the corresponding cars (mainly Porsche) standing around. Most of the owners arrive in the evening, so if you want to see them being used, you should come late. However, I'd say at most one quarter of them is used every day, most of them stay dark.

Another category of yachts is always lit up in the evenings: The real high-end (100M€ and more), parked in the far end of the harbour. Last time I was in the harbour, I counted 8 of them, plus a "transport" one which contains only jet skis and other stuff to keep it off the main one. In the evenings, you can be pretty sure that each and every one of them is lit, and it's an impressive view. To give you an idea, most of these yachts are like 12 m wide, have 3 decks above the hull, and are 80 m or longer. These yachts have a helipad, several boats, and usually a really large crew which starts to prepare the yacht at around 8 in the evening.

Well, enough of the yachts, let's look at the other place to visit: The cap d'Antibes. Depending on how much money you have, there are two things to see there. If you are rich (that is, you'd be considered rich in this area!), there's the Hotel du Cap "Eden Roc", which is well hidden in a huge park. The room costs scale from 600 to 10k € (per night!), but it's supposed to be one of the best hotels in the world.

If you are not that rich, you can (and should) take the walk around the cap itself. Basically, the state has taken the seafront of a few estates (3? 4?), and built a path right into the rocky coast. You walk along the outer wall of some really huge estates with some extremely large houses. If I say wall, I should be slightly more precise: Seen from outside, it's one large wall, 3-4 m high, half a meter thick, with 360° cameras, lamps, normal cameras, laser light barriers every few meters, and dangerous looking security guards patrolling on the other side of it. You'll be rewarded though by the magnificent path with a constantly impressive view on the seam, and even though it takes around one hour to walk around, it's never boring. I can recommend to come in the morning, at least I tend to go there quite early (in case you are currently located in Sophia-Antipolis or nearby, feel free to contact me, maybe we can go together in the morning).


Sophia-Antipolis is a rather weird town: It consists of companies, labs and universities only. Until recently, there was not a single shop here, and only one bar. To put this in context: Over 20.000 people are working in Sophia-Antipolis! Every major IT company has a branch office here, next to my place, you can find Cisco and HP for example.

Due to the fact that there are no shops nor bars in the city, it's basically dead in the evenings. Plus you get insane traffic jams in the direction of Antibes and from there on to Cannes.


Even though Cannes is very famous, there's nothing that impressive actually. During the day, you usually get stuck in the traffic to the beaches. The beaches themselves are long, but just 10 m wide. The most interesting part is probably the "Croisette", the road from the film palest along the coast. The road is wide, with palms on it, and hotels like the Ritz-Carlton, but that's it. The yacht harbour is also nice, but compared to Antibes, it's rather tiny. On the other hand, there is much more traffic, basically the whole quay along the film palest is busy every evening, and you can see Escalades, Rolls-Royce and Ferraris standing around.

Things are a bit different if you are looking for some night-life, but there's something else which I'd like to mention first: The lack of good cinemas. You would assume that Cannes has a 32-cinema multiplex, with THX certificates everywhere, and playing the movies with the original voices regularly (as there is even an English radio-station here). In this regard, you'll be severely disappointed.

Of course, there's also Nice and Monaco here, but I was only once in Nice, and I saw no reason to go again there. I haven't been to Monaco in this summer yet (I plan to go there for the Super-Yacht-Show in September.) I hope this post gives you a good idea of the region, as usual, feel free to comment if you have questions. I didn't upload pictures here, as you can find pretty nice ones in the web (for example, there are loads of impressive pictures of the harbour in Antibes at night.)

Short status update

Since a few months, I'm an intern at INRIA -- this is a French research institute. I'm in the REVES group, which is located in Sophia-Antipolis, next to the famous city of Antibes. In case this does not ring a bell, the next city to Antibes is Cannes to the south, and Nice to the north.

I'll write a bit more about the location, especially Antibes, as I personally never heard about this town before.

Another short announcement: I'm going to SIGGRAPH this year as well, so I might be difficult to reach from 3-8 August.