fishsoup

Now that we know that we want to draw in frames, the question arises: how do we decide when to draw a new frame?

Some criteria:

• smoothness – how high a frame rate can we sustain?
• latency – when we get an event, how quickly can we display that on the screen?
• reliability – if we push latency to a minimum, will we occasionally skip a frame?
• responsiveness – can we still respond to other things going on while rendering?

Vertical blanking is important concept in frame scheduling, The contents of video memory (the “front buffer”) are constantly being read out and used to refresh the screen display. The point at which we finish at the bottom of the screen and return to the top is the ideal point to update the contents of the screen from one frame to the next. There is actually a gap (”vertical blanking interval”) where no screen read-out is done. By updating the contents during that interval we avoid tearing.

One way to do frame updates is a “buffer swap”: during the vertical blanking interval we change the address that the graphics card is scanning out from to point to the buffer we were rendering into (the “back buffer”) and then reuse the old front buffer as a new back buffer.

The simplest timing algorithm is to start handling the next frame as soon as possible – as soon as the buffer swaps completes. Ideally, we have a way of doing the buffer swap asynchronously. We tell the X server to swap buffers, then go off and do our other business (queue up input, handle D-BUS requests, map application windows, etc.) while we are waiting for the swap to complete. This looks like:

What’s marked as “Render Scene” in the diagram includes all the steps of the frame processing described in my last post: processing queued input, updating animations, doing layout, and then rendering.

Unfortunately, standard GL API’s don’t give us an easy way of doing the asynchronous vblank-synced swap shown above. If we only have a blocking buffer swap operation, then the picture looks more like:

So we never actually have any “idle time” – we are either processing a frame or blocking for the swap to complete. This isn’t the end of the world (we still process pending events before starting frame rendering), but it will reduce the overall responsiveness of the system. So, one thing we can do is to try to predict the correct time to start frame rendering so we complete just before vblank:

This also has a secondary benefit of reducing latency a bit. The downside is that we have to guess how long the frame rendering will take (a good guess is as long as the previous frame), and if we guess wrong, we miss the vblank, producing a stutter in the output. Using a safety factor and allowing for, say, a 50% longer rendering time than the last frame reduces the chance of this.

Finally, what should we do if we don’t have any support for VBlank synchronization at all. Then the only real approach is to simply pick a frame rate, and try to render frames at that rate:

Picking the frame rate to use in this case is a little tricky. You probably don’t want to use the exact frame rate of the display, since that will result in tearing at a consistent point on the screen, which is quite obvious. Using a slightly slower frame rate may produce visually superior results.

To be continued… the next (and I think last) part of this series will cover how to extend frame timing from the compositing manager to the application. Things get more complex when we have three entities involved.

Note: although I’ve shown the X server doing the buffer swaps, the considerations in this post are exactly the same if the compositing manager is talking directly to the kernel graphics driver to do buffer swaps; there’s nothing actually X-specific about this.

Just wanted to quickly mention the three Google Summer of Code proposals that were accepted related to GNOME Shell.

David Jordan is going to work on Application-aware Window Management . This is a really interesting proposal to allow applications to expose more content to the desktop window switching than just a single toplevel; for example, to make it possible to look at all of the tabs of your web browser and switch directly to one of them. Marina will be the mentor the project.

Nathan Lo is going to work on Multiple monitor support for workspaces. The combination of multiple monitors and multiple workspaces has always been uncomfortable in GNOME. Switching all your monitors at once just isn’t what you normally want. It’s more likely that you want to keep one monitor fixed and switch another. This problem becomes more pressing in GNOME Shell where we’ve tried to make workspaces easy to use and intuitive. So the project will explore how it works to have separate workspaces on the different monitors and what the appropriate way is to present that to the user. I’ll mentor this one.

And finally, Siegfried Gevatter is going going to work on Zeitgeist Integration. Currently GNOME Shell has a very simple flat list of recent documents in the overview. We’ve wanted to move to a more time-base “journal” approach, and possibly add extra features like tagging. That is very similar to what the Zeitgeist project is about, so rather than duplicating effort, it would be very nice to just take the data that Zeitgeist is collecting and show it in the GNOME Shell user interface. This will be mentored by Seif from the Zeitgeist project.