Reinteract – Better interactive Python

Recently, I’ve been doing quite a bit of reading about computer analysis of musical sound and some experimentation. But there wasn’t really anything I knew of that fit my desires for an experimental platform. Python + numpy provided a good combination of a pleasant language and signal processing ability. But beyond that I wanted an interactive environment that created a persistent record of what I was doing. And that means the ability to edit: to go back and fix up mistakes and add comments. Because a literal transcript of a shell session is worthless… it’s full of typos and dead ends. Also, I wanted inline plots and images. In essence, I wanted something like the Mathematica notebook user interface, but for Python.

The inline plot part can be found in Nicolas Rougier’s Pylab GTK Console, but other than that, it’s just the standard Python interactive mode in a GTK+ window. IPython has a lot of features but it’s still a shell in a terminal. So, I decided to create something myself. A couple of months of occasional evenings and 4000 lines of Python later, Reinteract is beginning to shape up. A screencast of it in operation can be found below:


Screencast (5 minutes)

I’m not ready to make a tarball release yet, but you can get the code out of git. (Instructions here.) The chances of it working out-of-the-box on a recent Linux system are pretty good. In theory, it should be completely portable to Windows and even to gtk-quartz on OS/X, but figuring out the details would take some work.

P.S. – Python shell envy within the Online Desktop / Mugshot team? It’s really a pretty different sort of thing from Colin’s project.

36 Comments

  1. Martin
    Posted November 10, 2007 at 10:12 pm | Permalink

    Cool!

    Noticed a typo in trying it out section (the git clone fails if you just copy&paste). s/fishoup/fishsoup/ :)

  2. Owen
    Posted November 10, 2007 at 10:18 pm | Permalink

    Typo fixed. Thanks Martin!

  3. Posted November 10, 2007 at 11:38 pm | Permalink

    Wow, this is insanely sweet. In the past I really wanted a tool like this, and I think now I need to find an excuse to want one again.

    Awesome work, Owen!

  4. woot
    Posted November 11, 2007 at 12:47 am | Permalink

    Looks pretty awesome. I somewhat amazed something like this didn’t already exist, what with the great mountain of stuff the scipy people seem to have produced.

  5. Posted November 11, 2007 at 3:58 am | Permalink

    Wow. That looks totally awesome. Defnititely something I’ll find use for in my studies.

  6. Posted November 11, 2007 at 5:01 am | Permalink

    Very impressive !

    I hope it will be package one day for most distributions.

    Nice work, Thanks a lot !

  7. Andy
    Posted November 11, 2007 at 6:19 am | Permalink

    Looks amazing :D I will definitely be keeping my eye on this software.

  8. Posted November 11, 2007 at 7:40 am | Permalink

    Awesome. Really awesome. Wow.

    What’s needed to get custom painting for a Python class? Which interface has to be implemented?

    Wow.

  9. Chapi
    Posted November 11, 2007 at 8:30 am | Permalink

    Really cool, neat, clear & consise démo.

    Thank you.

  10. 42
    Posted November 11, 2007 at 8:35 am | Permalink

    This is pretty similar to the “notebooks” of SAGE (http://www.sagemath.org/), but obviously much more lightweight, and doesn’t use a Web browser.

  11. Posted November 11, 2007 at 8:39 am | Permalink

    It seems there is a bug with special characters for me. Hitting a § (for example) immediately crashes the application with this traceback:
    /home/tnorth/../reinteract/reinteract/lib/reinteract/shell_buffer.py:1012: GtkWarning: /tmp/buildd/gtk+2.0-2.10.13/gtk/gtktextiter.c:3859: Incorrect byte offset 1 falls in the middle of a UTF-8 character; this will crash the text buffer. Byte indexes must refer to the start of a character.
    end.set_line_index(end_index)

    Gtk-ERROR **: file /tmp/buildd/gtk+2.0-2.10.13/gtk/gtktextsegment.c: line 196 (_gtk_char_segment_new): assertion failed: (gtk_text_byte_begins_utf8_char (text))
    aborting…

  12. Posted November 11, 2007 at 8:45 am | Permalink

    That’s excellent, I’m very impressed :) I’m sure I’ll find use for something like this in the future…

  13. Trond Danielsen
    Posted November 11, 2007 at 8:48 am | Permalink

    This is just awesome! I definitively will keep an eye on this project.

  14. Posted November 11, 2007 at 9:22 am | Permalink

    Is the screencast available in an open format?

  15. Jon Smith
    Posted November 11, 2007 at 11:10 am | Permalink

    Very nice work. If it’s not already, I recommend making this compatible with matplotlib (i.e., adding the ability to set matplotlib as the plotting back-end), since it seems to be the default plotting library for scientific and engineering Python applications.

  16. Owen
    Posted November 11, 2007 at 11:19 am | Permalink

    Matthias: Custom results are pretty simple. The CustomResult base class has a single method create_widget() that you override. Then the widget is inserted into the buffer. I’ve some thoughts of extending it to have a two-way link with the statement that created the result, so you could interactively adjust the output and have the statement updated to match.

    TNorth: Yep… I had some indecision about whether to work internally with byte strings in UTF-8 or Python Unicode strings, and I need to go back and clean that up. I’ll put it at the top of my list.

    Yannick: Sorry, no. I didn’t know of a solution that would easily give me both swf and and Theora, and swf was the priority for quality and wide-viewability. I’ll definitely try to figure out how to do both next time. I would be interested to know if the screencast works with free swf players like swfdec or gnash.

  17. Niels L Ellegaard
    Posted November 11, 2007 at 11:25 am | Permalink

    Have a look at the Symbide project. That is another gui frontend to python. These two projects may gain from working together.
    http://code.google.com/p/symbide/

  18. Posted November 11, 2007 at 11:32 am | Permalink

    When I was doing image processing / generation with PIL I thought it’d be a neat idea to somehow display images in the interpreter inline (to quickly check results). To do this in Reinteract all you need to do is have your object inherit from reinteract.CustomResult? What about in the case of a class that’s already defined elsewhere? (in PIL’s case the Image object)

  19. Ben
    Posted November 11, 2007 at 12:49 pm | Permalink

    What’s the best way to stay up to date on how well Reinteract is coming along? Or are you finished with it for now?

  20. hugues
    Posted November 11, 2007 at 12:55 pm | Permalink

    Very nice !

    Just an idea : add the ability to detach the custom result from the editor (say : put it in a handlebox)

  21. Posted November 11, 2007 at 1:54 pm | Permalink

    Yay!!! I’ve hacked a small patch to make it run on Maemo. Please take a look:

    http://pastebin.ca/769791

    It would be great if we had this running in a Nokia Internet tabled device, specially the new N810 with keyboard.

    Excelent work! Thank you very much!

  22. Joonatan Kaartinen
    Posted November 11, 2007 at 2:07 pm | Permalink

    I’ve done a preliminary fix to the issue with utf-8 characters. It fixes the crashing but still has some warts.
    git-repository here:

    git clone http://ril.endoftheinternet.org/~jojkaart/git/reinteract.git

  23. Posted November 11, 2007 at 2:20 pm | Permalink

    Displays fine for me in Gnash on Ubuntu gutsy. The audio kept cutting out, but I think that’s because of my net connection being heavily used rather than anything to do with you :)

  24. frey
    Posted November 11, 2007 at 3:15 pm | Permalink

    Looks great. I think could be in very useful for my physics classes/research.

    Keep up the good work!

  25. Alex
    Posted November 11, 2007 at 3:29 pm | Permalink

    Thanks this is great! – I have been wishing python had something like this for ages – just didn’t have the skill to do it myself..

  26. Owen
    Posted November 11, 2007 at 4:16 pm | Permalink

    Steve: I’ve definitely thought about having a plugin system where you could register special handling for different types of result object. For now, as Mathias demonstrated, you can create a function that just wraps up an object in a CustomResult.

    Ben: That’s a good question. For now, I’ve created a special category in my blog, so you can subscribe to http://blog.fishsoup.net/category/reinteract/feed
    in a feed reader. If I later create a mailing list, I’ll blog about it.

    Joonatan: Many thanks for the patch! I decided, after thinking about it a bit, to go the opposite route and keep the internals as byte strings. I’ve pushed a patch fixing up the tokenization for that. I’ve also fixed the encoding on parsing, so it’s possible to use non-ASCII characters within strings and not have them mangled.

  27. Posted November 12, 2007 at 3:54 am | Permalink

    I’m new to Python, but this looks great. I’ll definitely give it a try. Thanks!

  28. Jeremy Dunck
    Posted November 12, 2007 at 10:51 am | Permalink

    @Owen:
    You may like this howto for going from swf to other formats:
    http://hublog.hubmed.org/archives/001557.html

  29. Posted November 12, 2007 at 12:54 pm | Permalink

    Awesome thing! What do you think about adding some usual shell shortcuts (like Alt-b for going word backward, Alt-f – word forward, C-w – delete word backward, Alt-d – delete word forward and so on)? This will make reinteract more usual for any familiar with usual shortcuts. :)

  30. Posted November 12, 2007 at 2:00 pm | Permalink

    Very cool! Is there a way to change the prompts so I can past code directly into doctests? (Ie, get >>> and … instead of your colored boxes)

  31. Posted November 12, 2007 at 9:24 pm | Permalink

    Wow, this is pretty cool.

  32. Posted November 12, 2007 at 10:02 pm | Permalink

    Awesome. I just hacked a simple “rehtml”:
    http://advogato.org/person/eopadoan/diary/7.html

  33. Posted November 15, 2007 at 2:59 pm | Permalink

    Very nice indeed. Do you realize that you’ve in a very real sense created a data-flow language from a scripting language.

    Many data flow languages are visual and include pd (pure-data) and Max/MSP. They all have this always-on-ness that is presented here.

    The always-on-ness makes a language very useful for gestural programming. By this i mean, that one can change them and see/feel/hear the result immediately. Something of the feedback and artist uses in see their hand gesture on the canvas. This is exceptionally important for programming as composition, for example in generative sound composition.

    Once again brilliant.

    Another advantage of visual data flow languages is that they have many tops (main loops if you want to thing of it like that)

    This means that creating parallel action/processing is very easy. This is also very import in creating a multi-layered work.

    Can reinterpret have multiple instances that live in the same sand box?

  34. Owen
    Posted November 15, 2007 at 6:41 pm | Permalink

    Brady: I think you are on to something with the idea that programming in an environment like Reinteract is fundamentally different than just writing a static script. We all know that the faster the cycle time is, the more productive the programmer is. Reinteract can reduce the cycle time to basically zero.

    Because I see Reinteract as a tool for creating records of computational experiments, I’m not necessarily so interested in going completely dynamic … you move a slider, the playing sound changes. It’s related, but a different emphasis. That also reduces the interest in multithreading and parallelism… to me the main reason for adding threading would simply be to not block the users during a long-running computation.

    I should point out that there is no real data-flow analysis going on in Reinteract currently. Each statement is considered to depend on all the statements above it. Better dependency analysis could be done and probably eventually will be done, but I haven’t found a huge need for it yet.

  35. Posted November 22, 2007 at 3:56 pm | Permalink

    Owen: i don’t think data flow languages do any analysis. And theoretically single threaded. Their trick, ask far as i know from reading documentation of pure data ( but not it’s code), is an event scheduler that runs at the top, of the data flow chains, each of which is deterministic.

    Timers or UI actions seed this scheduler, otherwise nothing happens. And nothing interrupts an execution chain until it’s stack is done.


2 Trackbacks/Pingbacks

  1. [...] | 0 comments | 0 pingbacks | python, shell Симон Виллисон дал ссылку на reinteract, интерактивный интерпретатор для питона, можно даже [...]

  2. [...] who have been reading my blog for a while may remember Reinteract. I haven’t written anything about it here for a while, but I’ve still been working on [...]

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: