GNOME Shell 2.28.0 – A Preview

October 7, 2009 – 8:29 pm

Going from releasing GNOME 2.27.x unstable releases of GNOME Shell to GNOME 2.29.x unstable releases of GNOME Shell naturally involves crossing 2.28 in the middle, so we thought we’d take the opportunity to make a set of tarballs that are little more stable than the typical Friday snapshots. A good place for people to try things out. I’ve just uploaded mutter-2.28.0 and gnome-shell-2.28.0 tarballs to ftp.gnome.org.

If you want to building things yourself, you still should use the JHBuild setup described on our wiki page. It’s the easiest way to do it. The tarballs are really designed for people creating distribution packages.

Looking back, I haven’t blogged about GNOME Shell changes since February. That’s a very long time so I’m not going to make any attempt to describe here what all has changed since then. (693 commits just to the gnome-shell Git module.)

To find all the nooks and crannys of the GNOME Shell user interface you can use the neat cheat sheet that Marina created. It’s a good way to learn about the secret commands supported by the Alt-F2 dialog without going to the source code, as well as to get an overview of the more user-oriented parts of the shell.

Considering only at the changes in the last few weeks, the biggest change recently has been to the Alt-Tab dialog; Jon McCann and Jeremy Perry (one of the designers here at Red Hat) came up with some ideas about how the Alt-Tab could better integrate with the way we group windows by application in the Activities Overview, and Dan Winship did a lot of work to make those ideas into something we could try out. Steve Frécinaux also jumped in more recently to help out with this. Jon’s going to describe the design ideas involved in more detail soon, so save your ideas for improvement for his post, but here’s a quick screenshot to whet your appetite:

New GNOME Shell Alt-Tab Selector

The other big recent change isn’t so screenshot-able, but Colin and I took some interesting parts of the NBTK library from Moblin and imported it into GNOME Shell as ST (the Shell Toolkit). (We’re still cooperating closely with Moblin on the work; the rename is just to keep things non-confusing.) We get several nice things from this: scrolling, better layout capabilities, but the big change is the ability to use CSS for styling. NBTK already had a good start on this, and I then merged in some work I did last year for hippo-canvas to get extra capabilities like CSS styling of borders and fonts. This is really intended to make development easier – so we don’t have to have all sorts of hardcoded fonts, colors, and pixel sizes in the code, but also should provide a good entry point for artists to work on alternate appearances for the shell.

Oh, and I quickly used ST to hack up a drop-down calendar when you click on the clock. So you can finally find out what day of the month it is in GNOME Shell without having to run ‘cal’ in a terminal.

We hope people will try it out, and see how it works for them in practice; packages will be in Fedora rawhide shortly and the JHBuild I mentioned above really is easy to get going on any recent Linux distribution if you aren’t running something quite so up-to-date. If you are in the Boston area, come to the GNOME Summit this weekend. All the developers working at Red Hat on the shell will be there, and hopefully a few other shell developers as well. You can see the shell in action, talk to us, and just in case you are having trouble getting it compiling, we’ll help you with that too.

By Owen | Posted in Coding | Comments (41)

Splinter – patch review for Bugzilla

September 23, 2009 – 7:19 pm

OK, time to make good on my promise to write about Splinter some more and show it off. The basic idea of Splinter (hinted at in my last post) is that instead of doing patch review as a completely new system and building all the infrastructure from scratch, it should be possible to do the kind of interactive patch review featured by systems like Review Board or Rietveld directly on patches attached to Bugzilla.

When you visit Splinter’s display of patch attachment, you see a standard two-column patch display like:

Splinter review with comment

Any coments that people have previously made will show up inline. If you have comments to make yourself, you can double-click on a line and an editor will open where you can type the new comment:

Adding a comment with Splinter

I put up a (read-only) demo instance of Splinter where you can try this out live.

When you publish your review, it will be published as a comment in Bugzilla that looks (in part) something like:

::: js/ui/lookingGlass.js
@@ -355,3 +355,3 @@
             inspector.connect('closed', Lang.bind(this, function() {
                 this.actor.show();
-                global.stage.set_key_focus(this._entry);
+                Main.acquireKeyboardFocusStack(this._entry);

I'm not sure why we are setting the keyboard focus (either way) when the
inspector is closed, but using a stack-based acquire() seems
definitely wrong.

The idea of the format is to be readable if someone is just reading the comments in the review or in emails from Bugzilla, while still being parseable by Splinter for inline display.

A neat feature of Splinter is it uses the new DOM Storage mechanism to save in-progress reviews. You can leave the page for a review and come back to it any time and your review will be as you left it. (This requires a brand new browser, like Firefox 3.5.)

Using it yourself

Assuming that there’s interest, I’d like to make this available on bugzilla.gnome.org. The first step would be just having it there behind the scenes if you know the URL. The second step would be to write a small Bugzilla extension to link to add links to Splinter from review comments and from the attachments section.

But you don’t need to wait for this. If the above is something that you want now, then you can clone it from git://git.fishsoup.net/splinter (browse, README). In the proxy/ subdirectory, there’s a README file that describes how to run a local proxy server pointed to a Bugzilla instance. It’s really easy. Then connect to:

http://127.0.0.1:23080/index.html

in your web browser. Some aspects are a bit obscure as of yet; for example if posting a review hits “a mid-air conflict” then it will fail and you need to reload the page (safe because of the use of DOM Storage!) and try again. But it’s definitely usable.

Behind the scenes

The most interesting thing about Splinter is that the server referred to above is just a proxy server. The only reason that it is there at all is to safely get around cross-site-scripting restrictions; there’s no server side component at all. Instead, the Javascript code of Splinter uses the existing exposed web interfaces of Bugzilla (and your existing Bugzilla login credentials) to get information about the bug and post reviews back. Everything – parsing the patch files, building the display – is done in Javascript in your web browser.

One of the main reasons for doing it this way was language choice. I’m doing a lot of Javascript coding for gnome-shell, so its very much at my fingertips. And there was going to be Javascript and DOM manipulation in any case – I wanted a modern interactive interface for editing reviews. If I was going to write a server side component, I’d need to do it in Perl to integrate nicely with the Bugzilla environment, but my Perl is distinctly rusty. Everything in Javascript side-stepped that issue.

The Javascript approach also is easy for development – you don’t need a test Bugzilla instance to work on, you can just hack on the Javascript and HTML files, and use the proxy server to run your changes against a public Bugzilla instance.

The Javascript code uses JQuery extensively. It’s my second project using it, and I’m liking it a lot. In some ways, it’s really a meta-language on top of Javascript, but it gives a lot of power compactly and readably. (Readably if you don’t get too clever with it.) The source code also contains a SpiderMonkey-based test harness so that I could write tests for the non-interactive parts like parsing patches and run ‘make check’.

Future

Splinter is never going to be as nice for the single task of patch review as Review Board. But there’s quite a bit of low-hanging fruit to make it better. In addition to the small Bugzilla extension mentioned above, other things I’m thinking of include: better control of how much context is shown with a comment. A collapsed view of a review that just shows the review and the sections being commented on. A threaded display of the attachments of a bug showing their relationship to each other and the reviews on each version. The ability to add extra context around a particular section of a patch.

By Owen | Posted in Coding | Comments (5)

Review Board vs. git-bz

September 15, 2009 – 6:52 pm

I’ve recently been spending some time working on a light-weight web-based patch review tool, called Splinter, to go along with Bugzilla. I’m not quite ready to reveal it to the world, but I wanted to deal in advance with a question that is certainly going to come up – why not use Review Board? This is a very reasonable question to ask, especially since Splinter was directly inspired by seeing Review Board and saying to myself “I want one of those!”

In thinking about this, I realized that the comparison wasn’t really between Review Board and Splinter, but between Review Board and attaching patches in Bugzilla. You really have to look at the overall process. How do you go from a enhancement request or reported crash, to a first candidate code change, to reviews on that change, to a final version of your change, to pushing those changes to the official repository, to closing the original report? git-bz automates and makes convenient large sections of this. Splinter is just a solution for a gap in the process – how do you review a patch once it’s attached to Bugzilla? But Review Board replaces the entire thing: you don’t attach patches in Bugzilla, you just link to a review request that you’ve filed (with your patch) in Review Board.

First I should list some of the advantages that Review Board would bring to the overall process: first, it has a very slick and well developed web interface for reviewing and commenting on patches. The tool I’ve been working on is pretty nice for a couple of weekends of work, but there’s only so far you can go in a couple of weekends. Second, Review Broad has a very well-defined workflow. Each review request is submitted to a person or group for review. You are presented with your outbox: your patches waiting for review, and your inbox: patches that are waiting on review. With our Bugzilla, the process is buried in a large and complex user interface and is much less explicit. (Attachment flags can be used to do explicit review requests in Bugzilla, but we don’t use them on bugzilla.gnome.org.) Finally if we used Review Board for patch reviews in GNOME, we’d reduce the amount of noise for bug reporters. While I imagine it’s exciting to see two developers discussing obscure details of a patch to fix your reported crash, I also can imagine that it can be confusing and intimidating, and maybe people would prefer to just get a high-level summary when a patch was comimtted rather than all the details in their mailbox.

So, Review Board stacks up pretty well for web user interface and developer workflow. Where things become more problematical are the integration with Git and the integration with Bugzilla.

The normal way to interact with Git is from the command line, so any integration of a review tool with Git will start there. While the web portions of Review Board are highly advanced, the Review Board post-review command-line leaves a lot to be desired when compared to git-bz. It doesn’t follow the git command line conventions. It can’t take patches from Review Board and apply them locally. It can’t insert references to the review into the commit messages. And so forth. That’s partly a question of attention, and probably even more a question of being cross-version-control-system. There’s only so much you can do while handling Subversion, and CVS, and Git, and Mercurial, and …. Certainly a specific “git-rb” tool is possible that does many of the things that git-bz does is possible – the lack of such a tool isn’t an inherent defect of Review Board, just a current lack.

There is a more fundamental problem here – a review in Review Board is of a single diff (potentially with multiple revisions), but that’s not very natural to how people work with Git. More complex enhancements are almost always done as patchsets, with each patch in the set being kept as standalone as possible. In the Bugzilla/git-bz workflow, the entire set of patches attached to a bug logically makes up a patch set. git-bz makes it easy to attach a series of commits to a bug as patches or apply all the patches from a bug to a local branch. Individual patches can be commented upon, marked as accepted-commit_now, replaced with newer versions, but it’s only when everything has been agreed upon and pushed that the bug can is closed. Trying to handle this explicitly in the Review Board user interface would require some substantial changes: you’d want a Review Request to cover an entire patch set, but have Reviews apply to individual patches within it.

The issues with using Bugzilla and Review Board together lie both on the user side and on the infrastructure-development side. On the user side, there’s a simple but non-trivial problem that we’d suddenly have two different identifiers and URLs for the same issue – there would be the Bugzilla bug URL but also the review board review request URL. Which one should you reference in in your commit message? Which one do you search for? There would also be two separate accounts to deal with.

On the infrastructure side, there’s the standard work of installing, maintaining, upgrading, etc. Beyond that, we might want to try and figure out some sort of single-sign-on across Bugzilla and Review Board to help with the multiple accounts problem. We’d have to have some automated process to keep the repository list in Review Board in sync with the 600+ GNOME repositories and auto-create maintainers groups for the repositories. We’d need a bugzilla extension to link to reviews. We’d need a Review Board extension to update bugs in Bugzilla when a review was created.

My summary here is that moving GNOME to Review Board for patch review would be quite a bit of work and a substantial change in our current work habits. There would be significant advantages, but also some definite downsides in having to juggle two systems. If it’s possible to, with a small addition on top of Bugzilla, get a reasonable approximation to the web-based patch review aspects of Review Board without adding an entire second system, I think that’s pretty interesting. Once I put a demo of Splinter out there (days, not weeks), you can judge that “if” for yourself.

By Owen | Posted in Coding, Git | Comments (7)

git bz push

September 5, 2009 – 2:45 pm

A while ago I blogged about my git-bz tool for Git and Bugzilla integration. If you recall, the original idea was:

git bz attach http://bugzilla.example.com/show_bug.cgi?id=43215 HEAD

But that left a gap – once you were happy with the patch, how did you close the bug? You had to go back to your web brower, and twiddle a bunch of form controls. So, I’ve now added:

git bz push

Which looks at the URL that ‘git bz attach’ added to your commit message, finds the bug from there, and pops up a prefilled editor. It looks something like:

# Bug 43215 - Split HTTP communication out of Bug class - UNCONFIRMED
# http://bugzilla.example.com/show_bug.cgi?id=43215
# Enter comment on following lines; delete everything to abort

Attachment 892 pushed as f319ea4 - Split HTTP communication out of Bug class

# Comment to keep bug open
# possible resolutions: [F]IXED, [I]NVALID, [WON]TFIX, [WOR]KSFORME
Resolution: FIXED

# Lines below change patch status, unless commented out
# possible statuses: [n]one, [rev]iewed, [rej]ected, [c]ommitted, [o]bsolete
committed @892 - Split HTTP communication out of Bug class - none

You can just accept it, or you can make changes and add additional comment text. Once you exit the editor, the bug is closed and the attachments are marked committed.

Even more recently (just today) I implemented a feature request from Benjamin Otte for:

git bz push --fix 43215

This combines ‘git bz attach’ and ‘git bz push’ into one. So, to review, what it does is:

  1. Rewrites the commit to include the bug URL
  2. Attaches the commit to the bug, and marks it ‘committed’
  3. Pushes the commit
  4. Adds a comment to the bug and mark it resolved

The only thing it doesn’t do is mail your boss and ask for a raise.

Some of the other recent activity for git-bz:

  • Fixed to work with new GNOME bugzilla
  • ‘git bz edit’ – like ‘git bz push’, but not tied to a push. (So, you can just use it instead of the web when convenient.)
  • Customizable “URL addition” -  you can have “Bug nnnn – ” on the subject line instead of an URL appended to the commit body.
  • “URL addition” defaults to on, so you don’t have to remember to pass -u/–add-url
  • No need to specify the bug # when re-attaching an updated version of a commit
  • XML-RPC used for bug filing when available (improves error handling a bit)
  • Chromium cookie-reading support (from Colin)
  • Docs split out into a man page

As always, installation is trivial – just grab the script and put it somewhere in your path. (Or get the git repo URL from cgit, clone it, and symlink.)

By Owen | Posted in Coding, Git | Comments (19)

Hacking local defaults into GConf

June 7, 2009 – 10:19 pm

It’s very useful to create a limited JHBuild moduleset that builds just a tiny portion of GNOME. We do this for GNOME Shell and it allows people to get GNOME Shell running in just a few minutes without debugging build problems in a gigantic pile of modules. However a persistent problem with doing this is GConf defaults. Because people are testing within their current session, instead of running a completely new session, the gconfd is the system installed gconfd and is looking in the system defaults database for schemas. But the install rules for the jhbuilt modules can’t install schemas there. So the applications don’t get the right default values.

We’ve known for a long time that the right thing to do is to have the application read the defaults information directly; as well as fixing this problem with multiple installlation prefixes, it increase robustness. dconf and GSettings will work this way. But I wanted a quick fix for the gnome-shell JHbuild, not an entire new subsystem. So, today I sat down and wrote gconf-defaults-hack.

gconf-defaults-hack is a small LD_PRELOAD library (yep, yuck), that intercepts two core calls from libgconf: gconf_engine_get_fuller() and gconf_engine_get_default_from_schema(). First these functions are executed as normally, but then if they fail and no default at all was found, the hack library goes and looks in $sysconfdir/etc/gconf/schemas to see if it can find the defaults itself. Obviously, parsing even through a small set of schema XML files on every lookup would be prohibitive, so the first time a lookup is triggered, all the schema files are read at once, and the extracted defaults written to a SQLite database. As long as the mtime of the schema files don’t change, subsequent lookups will continue to use the same database.

You can clone it from git://git.fishsoup.net/gconf-defaults-hack (browse; README)

Is there some better way of making a non-system install of an application work without modifying or hacking GConf? If you know of such, speak up. Would this be better as a modification to libgconf? Conceptually, probably, yes, and making such a modification could be an interesting project for someone. But I was most interested in getting this working without thinking too much about what was long-term supportable.

By Owen | Posted in Coding | Comments (4)