I’ve been working recently on figuring out application to compositor synchronization. One aspect of that is the timing information does the compositor need to send back to the application and how should the application use it. In the case where everything is lightly loaded and we can hit 60fps, it’s pretty obvious what we want – we just output constantly spaced frames:
But what if we can’t do that? Say we only have the CPU and GPU resources to draw 40fps. To keep things simple and frame timing consistent, do we drop every other frame and draw the animation at 30fps?
(Gray frames are frames where we don’t do an update and reuse the previous image. The dotted circles show the theoretical position of the moving ball at the time of the frame.)
Or maybe it would be better to show more frames, to drop only one out of every three frames?
Or maybe we need to do something more sophisticated than to just drop frames – maybe when rendering a frame we need to take into account how long the frame will be displayed for and calculate positions at the center of the frame display period?
The answers to what looked better wasn’t at all obvious to me, even after a few years of playing with drawing animations for GNOME Shell, so I wrote a demo application to try out various things. If you want to test it out, note that it needs to be run uncomposited, so under GNOME 3, run metacity --replace & from a terminal and then use Alt-F4 to close the “something has gone wrong” screen. (gnome-shell --replace & to get back to your desktop.)
So, what conclusions have I drawn from looking at my demo? The first conclusion is that 60fps is visually way better than anything else. This wasn’t completely obvious to me going in – after all, movies run at 24fps. But movies have motion blur from the exposure time, which we don’t have here. (Adding motion blur to desktop animations would increase computational work considerably, and it seems unlikely that 30fps + motion blur looks better than 60fps without motion blur.)
The second conclusion is that how we time things matters a lot. Of the two methods above for dropping every third frame, the second method is obviously much better than the first one.
The third conclusion, is that if we can get frame timing right, then running at 40fps looks better than running at 30fps, but if we don’t get frame timing right, then the visual appearance is about the same, or possibly even worse.
What does this mean for an application to compositor synchronization protocol? I don’t have the final answer to that yet, but in very general terms we need to support applications that want to draw at frame rates like 40fps, because it can potentially look better – but we have to be careful that we support doing it with algorithms that actually look better.
Update: BTW, if anybody knows useful literature references about this area, I’d be interested.