git bz push

September 5, 2009 – 2:45 pm

A while ago I blogged about my git-bz tool for Git and Bugzilla integration. If you recall, the original idea was:

git bz attach http://bugzilla.example.com/show_bug.cgi?id=43215 HEAD

But that left a gap – once you were happy with the patch, how did you close the bug? You had to go back to your web brower, and twiddle a bunch of form controls. So, I’ve now added:

git bz push

Which looks at the URL that ‘git bz attach’ added to your commit message, finds the bug from there, and pops up a prefilled editor. It looks something like:

# Bug 43215 - Split HTTP communication out of Bug class - UNCONFIRMED
# http://bugzilla.example.com/show_bug.cgi?id=43215
# Enter comment on following lines; delete everything to abort

Attachment 892 pushed as f319ea4 - Split HTTP communication out of Bug class

# Comment to keep bug open
# possible resolutions: [F]IXED, [I]NVALID, [WON]TFIX, [WOR]KSFORME
Resolution: FIXED

# Lines below change patch status, unless commented out
# possible statuses: [n]one, [rev]iewed, [rej]ected, [c]ommitted, [o]bsolete
committed @892 - Split HTTP communication out of Bug class - none

You can just accept it, or you can make changes and add additional comment text. Once you exit the editor, the bug is closed and the attachments are marked committed.

Even more recently (just today) I implemented a feature request from Benjamin Otte for:

git bz push --fix 43215

This combines ‘git bz attach’ and ‘git bz push’ into one. So, to review, what it does is:

  1. Rewrites the commit to include the bug URL
  2. Attaches the commit to the bug, and marks it ‘committed’
  3. Pushes the commit
  4. Adds a comment to the bug and mark it resolved

The only thing it doesn’t do is mail your boss and ask for a raise.

Some of the other recent activity for git-bz:

  • Fixed to work with new GNOME bugzilla
  • ‘git bz edit’ – like ‘git bz push’, but not tied to a push. (So, you can just use it instead of the web when convenient.)
  • Customizable “URL addition” -  you can have “Bug nnnn – ” on the subject line instead of an URL appended to the commit body.
  • “URL addition” defaults to on, so you don’t have to remember to pass -u/–add-url
  • No need to specify the bug # when re-attaching an updated version of a commit
  • XML-RPC used for bug filing when available (improves error handling a bit)
  • Chromium cookie-reading support (from Colin)
  • Docs split out into a man page

As always, installation is trivial – just grab the script and put it somewhere in your path. (Or get the git repo URL from cgit, clone it, and symlink.)

By Owen | Posted in Coding, Git | Comments (19)

Hacking local defaults into GConf

June 7, 2009 – 10:19 pm

It’s very useful to create a limited JHBuild moduleset that builds just a tiny portion of GNOME. We do this for GNOME Shell and it allows people to get GNOME Shell running in just a few minutes without debugging build problems in a gigantic pile of modules. However a persistent problem with doing this is GConf defaults. Because people are testing within their current session, instead of running a completely new session, the gconfd is the system installed gconfd and is looking in the system defaults database for schemas. But the install rules for the jhbuilt modules can’t install schemas there. So the applications don’t get the right default values.

We’ve known for a long time that the right thing to do is to have the application read the defaults information directly; as well as fixing this problem with multiple installlation prefixes, it increase robustness. dconf and GSettings will work this way. But I wanted a quick fix for the gnome-shell JHbuild, not an entire new subsystem. So, today I sat down and wrote gconf-defaults-hack.

gconf-defaults-hack is a small LD_PRELOAD library (yep, yuck), that intercepts two core calls from libgconf: gconf_engine_get_fuller() and gconf_engine_get_default_from_schema(). First these functions are executed as normally, but then if they fail and no default at all was found, the hack library goes and looks in $sysconfdir/etc/gconf/schemas to see if it can find the defaults itself. Obviously, parsing even through a small set of schema XML files on every lookup would be prohibitive, so the first time a lookup is triggered, all the schema files are read at once, and the extracted defaults written to a SQLite database. As long as the mtime of the schema files don’t change, subsequent lookups will continue to use the same database.

You can clone it from git://git.fishsoup.net/gconf-defaults-hack (browse; README)

Is there some better way of making a non-system install of an application work without modifying or hacking GConf? If you know of such, speak up. Would this be better as a modification to libgconf? Conceptually, probably, yes, and making such a modification could be an interesting project for someone. But I was most interested in getting this working without thinking too much about what was long-term supportable.

By Owen | Posted in Coding | Comments (4)

Timing frame display

June 2, 2009 – 7:01 pm

Now that we know that we want to draw in frames, the question arises: how do we decide when to draw a new frame?

Some criteria:

  • smoothness – how high a frame rate can we sustain?
  • latency – when we get an event, how quickly can we display that on the screen?
  • reliability – if we push latency to a minimum, will we occasionally skip a frame?
  • responsiveness – can we still respond to other things going on while rendering?

Vertical blanking is important concept in frame scheduling, The contents of video memory (the “front buffer”) are constantly being read out and used to refresh the screen display. The point at which we finish at the bottom of the screen and return to the top is the ideal point to update the contents of the screen from one frame to the next. There is actually a gap (“vertical blanking interval”) where no screen read-out is done. By updating the contents during that interval we avoid tearing.

Vertical Blanking

One way to do frame updates is a “buffer swap”: during the vertical blanking interval we change the address that the graphics card is scanning out from to point to the buffer we were rendering into (the “back buffer”) and then reuse the old front buffer as a new back buffer.

The simplest timing algorithm is to start handling the next frame as soon as possible – as soon as the buffer swaps completes. Ideally, we have a way of doing the buffer swap asynchronously. We tell the X server to swap buffers, then go off and do our other business (queue up input, handle D-BUS requests, map application windows, etc.) while we are waiting for the swap to complete. This looks like:

Drawing as soon as possible

What’s marked as “Render Scene” in the diagram includes all the steps of the frame processing described in my last post: processing queued input, updating animations, doing layout, and then rendering.

Unfortunately, standard GL API’s don’t give us an easy way of doing the asynchronous vblank-synced swap shown above. If we only have a blocking buffer swap operation, then the picture looks more like:

Blocking SwapBuffers

So we never actually have any “idle time” – we are either processing a frame or blocking for the swap to complete. This isn’t the end of the world (we still process pending events before starting frame rendering), but it will reduce the overall responsiveness of the system. So, one thing we can do is to try to predict the correct time to start frame rendering so we complete just before vblank:

Delaying Frame Rendering

This also has a secondary benefit of reducing latency a bit. The downside is that we have to guess how long the frame rendering will take (a good guess is as long as the previous frame), and if we guess wrong, we miss the vblank, producing a stutter in the output. Using a safety factor and allowing for, say, a 50% longer rendering time than the last frame reduces the chance of this.

Finally, what should we do if we don’t have any support for VBlank synchronization at all. Then the only real approach is to simply pick a frame rate, and try to render frames at that rate:

No VBlank

Picking the frame rate to use in this case is a little tricky. You probably don’t want to use the exact frame rate of the display, since that will result in tearing at a consistent point on the screen, which is quite obvious. Using a slightly slower frame rate may produce visually superior results.

To be continued… the next (and I think last) part of this series will cover how to extend frame timing from the compositing manager to the application. Things get more complex when we have three entities involved.

Note: although I’ve shown the X server doing the buffer swaps, the considerations in this post are exactly the same if the compositing manager is talking directly to the kernel graphics driver to do buffer swaps; there’s nothing actually X-specific about this.

By Owen | Posted in Coding | Comments (9)

Frames not Idles

May 28, 2009 – 4:07 pm

The classic update cycle in GTK+ has very little overall coordination. If the widget hierarchy needs layout, we add an “idle function” to do that at the next opportunity. If a widget needs a redraw, we add an idle for that. If we want to animate something, we add a timer function to update the animation at 20 frames per second. When we receive a mouse event we go ahead and process it. The end result looks something like:

Timeline of updates without frame synchronization

There’s all sorts of redundant work going on that the user never sees; we lay out several times before redrawing. We update the animation or the mouse position several times in a row. And then we finally draw something. A better way to handle things is to organize around frames; we want to synchronize things so that we do each type of update exactly once before we redraw:

Timeline of updates synchronized to frames

So, how close are we to that ideal with Clutter? I lied a bit above: GTK+ was fixed a few years ago to bind together relayout and redraw: every time GTK+ does a layout, it does a redraw immediately after. This makes sure that, when you are resizing a window, GTK+ doesn’t spend all its time laying out the contents and never get around to redrawing them. Clutter copied that approach and also binds relayout and redraw together. Beyond that, the Clutter 1.0-integration branch now has work by Emmanuele Bassi to bind animation into the frame cycle as well.

Motion compression. The less obvious thing that a frame based approach provides is a theoretically sound basis for motion compression. Motion events have always been a bit of a problem; if you are doing anything at all expensive in response to a motion event, you can build up a huge backlog of pending motion events. The user drags an object around, the user releases the mouse button, and the object keeps going on its own for a few seconds.

Various approaches have been taken to this: PointerMotionHintMask is an X mechanism where only a single motion is sent until the application acknowledges it; this had advantages for not flooding 1980′s era networks, but introduces latency and complexity. Clutter currently rate-limits motion events by time; with this approach either too many or too few events may be dropped, and it’s possible to drop the last event and never catch up to the current mouse position when the user stops moving the mouse.

With frames it all becomes easy: you just queue up input events until you are ready to draw a frame. Then you run through that queue and drop any motion event where the next event is also a motion event.

I’ve tried implementing this technique in gnome-shell (by queueing and processing events before sending them on to Clutter) and it works well. But it should be generally applicable to all Clutter applications.

To be continued… So how should the frame drawing be scheduled? when do we start drawing a new frame? How does it relate VBlank synchronization? I’ll cover that in a follow-up post.

By Owen | Posted in Coding | Comments (2)

GNOME Shell SOC Projects

April 20, 2009 – 7:59 pm

Just wanted to quickly mention the three Google Summer of Code proposals that were accepted related to GNOME Shell.

David Jordan is going to work on Application-aware Window Management . This is a really interesting proposal to allow applications to expose more content to the desktop window switching than just a single toplevel; for example, to make it possible to look at all of the tabs of your web browser and switch directly to one of them. Marina will be the mentor the project.

Nathan Lo is going to work on Multiple monitor support for workspaces. The combination of multiple monitors and multiple workspaces has always been uncomfortable in GNOME. Switching all your monitors at once just isn’t what you normally want. It’s more likely that you want to keep one monitor fixed and switch another. This problem becomes more pressing in GNOME Shell where we’ve tried to make workspaces easy to use and intuitive. So the project will explore how it works to have separate workspaces on the different monitors and what the appropriate way is to present that to the user. I’ll mentor this one.

And finally, Siegfried Gevatter is going going to work on Zeitgeist Integration. Currently GNOME Shell has a very simple flat list of recent documents in the overview. We’ve wanted to move to a more time-base “journal” approach, and possibly add extra features like tagging. That is very similar to what the Zeitgeist project is about, so rather than duplicating effort, it would be very nice to just take the data that Zeitgeist is collecting and show it in the GNOME Shell user interface. This will be mentored by Seif from the Zeitgeist project.

By Owen | Posted in Coding | Comments (4)