Now for some of my cows

Vandaag, om 19:08, ben ik voor de tweede keer vader geworden. En Sandra mama en Sverre grote broer!

Rune is zijn naam.

Hij ligt nu heerlijk met mama op bed te kroelen. En zijn eerste luier te produceren. Die is dan weer voor papa...

We zijn gelukkig!

(kijk: http://jw.n--tree.net/photos/view/rune)

We use Grok at work to (re)build our publishing platform. One of the things though that Grok does that we definitely do not want, is to have all views "open" by default. So, only when you explicitly tell Zope (through Grok) to require a permission for a particular view, Zope will trigger authentication in order to authorize the request.

There're some vague plans to have this optional. To have a switch somewhere that you can flip and have all views closed by default.

In meantime we thought of this quick and elegant hack: use a custom grokker that looks for view (-like) components and raises an error if the view component does not require any permission. Since this error will be raised "at grok time" it will effectively prevent the application from starting, if it has unprotected views:

class CheckRequireGrokker(martian.ClassGrokker):
  component_class = grok.View

  def grok(self, name, factory, module_info, config, **kw):
      if not grok.util.get_default_permission(factory):
          raise GrokError(
              'This application requires %r to use the grok.require '
              'directive!' % factory, factory)
      return True

Basically, on an application level you can check all kinds of stuff on your components!

Update: of course, to be sure all view like components are protected, you need to check on the XMLRPC, REST, JSON, and *Form components as well.

I didn't finish my report on the Neanderthal sprint. Let's see if I can wrap it up now:

On Thursday and Friday Luciano and I continued on pinning down versions for the dependencies of Grok. Thursday existed mostly of testing and trying to get as much "properly" released packaged included in the list as possible. Even if this meant we had to downgrade some of these.

One of the more restrictive criteria was to maintain compatibility with the Windows platform. Some of the eggs contain platform specific dependencies but these binaries are not always up-to-date with the most recent source releases.

At the end of Thursday we had a more or less working versions list and ideas on how to make buildout use an "official" versions.cfg file, that is published as part of a Grok release.

The solutions are described here: http://grok.zope.org/releaseinfo/readme.html

Friday consisted mostly of writing that report. But of course we had to properly wrap up the sprint as well. We gathered the groups of people that were still there (some had to leave on Wednesday and Thursday already) and reviewed the checkins that were done, had a first look at the new Grok website, and made a list of post sprint follow ups.

Last week I got postive feedback on the changes on grokproject and merged the changes to the trunk. Martijn and I got Grok-0.10.1 released and pushed grokproject-0.6 out the door as well..!

And by now, Grok turned 1 year old! And we're heading for a 1.0 release...

other people about the Neanderthal-sprint: Martijn's blog entry , Luciano's photos , and two entries by Lennart Regebro. When I find more, I'll add them.

Yesterday we had a cultural break. Aroldo arranged a trip to Neanderthal, with the Neanderthal museum in particular. Neaderthal is one of the first places where remains of prehistoric mankind have been found.

The museum is very well made and very interesting. We were guided by a lady that had to cope with this group of geeks now that happily accepted her invitation to ask questions and discuss the things she explained :-)

When we retured at the hotel in the evening we decided to order pizza's, get some beers in and basically continue the sprint just there.

I meant to jot something down after the first day of sprinting, but I was basically too tired. I'm again tired but I felt I needed to write down something now, because else it wouldn't happen anymore.

The trip to Koln by motorcycle went just fine. It is less than 300km, so you should be able to do it about 3 hours. If you don't forget to take your own wallet with you and find out only when you're about to refuel that is...

At the hotel we - I happened to bump into Lennart when I arrived - quickly met up with Martijn and Jasper and Guido and Aroldo in some restaurant. Later that night Uli and Luciano arrived - it is great to be able to meet the people in real life that you only know from the mailinglist!

Monday morning started off with identifying sprint topics and tasks and to try and do some planning.

Besides lots of other stuff, Luciano and I expressed interest to continue to work on the Grok reference documentation, because it is severly lacking. Together with Jasper, Guido and Uli we decided to continue Uli's work and to use the RestructuredText version of the original LaTex source files that Theuni started a while ago and on which I tried to work as well.

Jasper and Uli wrote some code and gathered pieces of the puzzle to render nice HTML from the ReST files, esp. in regard to the specific tags and directives that we use. We are looking at the current Python (2.5) documentation there too, because that documentation effort has apparently switched to ReST as well.

Luciano and I concentrated more on the content-level of the reference documentation: we decided on the structure of the documents, cleaned up whatever content that was already there and continued writing new sections. One of the interesting conclusions was, that, although for in Grok itself the theme of ICavemen, ICaves, IFireplaces and IClubs for writing test classes and interfaces and so on is nice and cute, for the reference documentation it might be worthwhile to come up with more realistic and pragmatic examples of how to use the code. That way we think to give better examples of how to solve actual use cases.

Today (well, actually, yesterday by now) we knew we would continue on the reference documentation, but wanted to some coding as well. First we finally concluded the seemingly simple task of making Grok skip tests and ftests modules and packages in Grok-based application when Zope is started.

If these packages are not skipped, the registrations done in the test code (for testing the application) show up in the running Zope which is usually not what you want. I had a Martian branch (where this skipping is actually done) sitting there that needed a merge and Uli did some work too on this. We reviewed Uli's work and my branch, improved it here and there, released martian-0.9 and then updated Grok to make use of the skipping-modules-during-the-scan-process- API.

Then we headed for another important task: try to see if we could come up with a list of versions of dependencies of Grok that are known to work. Currently we find ourselves in a quite messy situation, where the toolset we use to make predictable and reproducible buildouts of our code bases and its dependencies does not deliver and appears to make life more difficult instead of easier. I think most of this is not so much due to the tools, but due to the learning process the people doing releases of the software we depend on (and remember, where would we be without these people and the things they have built!!) are going through. But looking at the Zope3-dev list, I think I can see clear signs that this is another complex situation the community is sorting out.

But more on that later, now I really need to go to sleep, because tomorrow we will go to Neanderthal and maybe even "meet" the great-great-great-grandfather of Grok...

There'll be another Grok sprint from Monday on. And I'm attending! :-)

I feel quite lucky to have an employer that allows its people to attends things like sprints and conferences.

Since I had some stuff to do at home today, I decided to go by motor bike instead of taking the train with Martijn, Jasper and Guido. But its not to far away anyway, so it'll be a nice trip going there. Everything is packed now and ready to go.

moved

Two easy_install tricks I should remember:

setup(
     ...
     install_requires=[
         ...
         'PIL'
         ]
 )

setup(
    ...
    install_requires=[
        ...
        'Imaging'
        ]
)

<a href="http://www.effbot.org/downloads/Imaging-1.1.6.tar.gz#egg=PIL-1.1.6">
  PIL-1.1.6
</a>

easy_install -f http://username:secret@domain.tld/path/to/index

My computer at work is a Mac Mini. Its Intel based, so I figured installing Ubuntu on it should be no big deal. However, downloading the Feisty Fawn beta ISO image took a while, so in the meantime I installed OS X just for the heck of it anyway.

Initially I thought to completely reformat the disk when installing Ubuntu, but since I had setup OS X already it seemed like a waste of time not to try and make it a dualboot machine. You never know when it comes in handy.

Setting it up like that was fairly painless. There were three hiccups, but they proofed not too difficult to solve:

I had the OS X install use the complete disk, so the only option left for installing Ubuntu seemed to be to reformat the disk completely after all. But parted to the rescue! It allowed me to dynamically resize the HFS+ partition back to 20GB, and emtpying up the remaining 60GB for the Ubuntu install. Great!

After having installed Ubuntu on the new partition, the time had come to reboot. That's always exciting after a partition resize and having a second OS installed... And... it just booted OS X. Darn.

A quick Google and I learned about rEFIt which is an opensource bootmenu for EFI based system and thus Intel Macs. After installing rEFIt and a reboot I suddenly had an interactive boot menu including the option to boot from the freshly installed Ubuntu partition. Yeah!

The last hiccup was supporting the very very very nice Apple Cinema HD 23" display. For some reason, the Xorg installation actually detected and configured the display's native resolution of 1920x1200, but X would only start in 1600x1200.

This Mac Mini model has a onboard 945GM/GMS/940GML graphics controller and using 915resolution I could overwrite on of the video BIOS's settings to persuade into actually knowing about this wide screen resolution:

sudo 915resolution 5c 1920x1600

And tadaa! Works!

Its a nice little machine and quite workable as well, especially for a 1.66GHz CPU and "only" a 5400RPM disk. And the wide-screen... Yummy!

some keywords: apple cinema hd 23" 1920x1600 mac mini refit dualboot ubuntu feisty fawn 915resolution parted

Big dilemma. Ditch J for Emacs?

J's been my text and code editor for like, more than seven years now. That's a long time. You can say J's editting features are engrained in my muscle memory by now. Quite literaly actually.

However, J is an obscure editor. Noone knows it. Well, except for a handful of adept users, sure, but nowhere near being surounded by a "community". The last official release was somewhere in 2004. And although there's still some development going on and I am working with a more or less up-to-date version from CVS, the main developer spends most of his attention to an Lisp implementation in Java.

And that's ok! I mean, there's no obligation to me for keeping the editor up-to-date of course! But the fact there's no real development anymore makes me feel a bit uneasy every now and then.

Then there's the Java issue. Well, it's not really Java issues per sé of course.

First of all, starting J is not particularly fast. Whenever an JVM instance has been started not too long ago (like, within the last couple of minutes), a new J instance launches roughly within 1 to 2 seconds. But whenever the filesystem cache lets go of the files that are in play, it can take as much as 10 seconds.

Second point is more of usability. It could very well be that I'll be working more in a Mac OS X environment, when I start at my new job. And J-on-OS-X is not very elegant. I remember from using J on my old Powerbook I was constantly fighting with fonts and menubar-locations and the Ctrl key versus the Command key (for cut/copy/paste/undo etc.)

I'm sure Java-based applications can be made to behave more elegantly on OS X, but it'll need some work. And then there's some other usability points as well: I'd like to have a better CSS mode, a better Javascript mode. Will the Python mode see an update now that Python-2.5 is released? I'd like to see improvements if it comes to buffer management (the buffer list on the left hand side doesn't cut it, and it takes up screen real estate, that I think can be used more efficiently)

And yet, there's the things I just do naturally: inserting elements in XML mode, it's just so easy. The great automagic indentation of code and markup works almost perfectly. The little things like Shift+<arrow down|up> to select the complete line, even if the cursor is not at the beginning of the line, Shift+<delete> to delete the complete line, regardless of cursor position, Alt+<arrow left|right> to move from buffer to buffer, Alt-x to easily invoke commands on the buffer, or just on a selected region in the buffer. Things I do and use without thinking about it anymore. And there's probably a bucket load more of these features - Ah, right! I remember one more now: the smart Home key: it does not just go to the beginning of a line, but it goes to the beginning of the text on the line. A subsequent Home keypress then brings you to the absolute beginning of the line. That's so convenient, especially in Python mode...

It is these things that make it so very difficult to change "my editor of choice".

But I'm trying anyway.

I'm typing this in Emacs now.

(let the Editor War commence :-)

To convince me (myself) to use Emacs from now one for all my text editting and programming work, I need to see whether:

• The fact that Emacs is usuaslly either already installed or easily installable on most OSs
• The fact that Emacs can be reconfigured ad infinitum to make it behave and work like I want it to
• There're a lot of Emacs resources, like editting modes (this is being authored in ReStructedText mode, what editor can say it has a ReST mode? Well, yes sure, the editor from hell can probably too), documentation, tutorials, scripts to make Emacs work in different ways, etc. etc.

...is enough to overcome having to:

• Learn a new command and key binding set
• Learn a new sets of editting modes, especially for XML
• Make Emacs have some of the key bindings that I definitely want to keep, like the Shift-<delete> and Shift-<arrow up|down> and common shortcuts like Ctrl+X, Ctrl+C, Ctrl+V. etc. etc.
• Get to know the insanely complex key mapping system
• Get over sentiments for J

But, it's the first time since ever that I'm actually seriously trying something else. All other attempts to try a different editor have miserably failed just after a couples of minutes.

Time will tell, if and when I'll be running back screaming to my beloved J.

p.s. I have filed some resources that proofed useful in setting up Emacs to my needs.

Last weekend I had the opportunity to attend the second Grok sprint. Martijn invited me to join in, as we did work on some grok related issues and testcases already the last couple of weeks.

Grok makes is easier to make use of the great toolset Zope-3 has to offer. It essentially replaces the default configuration and composition system of Zope-3 (based on zcml, an XML language) with another one.

Martijn and I left for Halle, Germany on thursday morning. In Halle we would sprint at Gocept's "headquarters" for one day on friday. We decided to go by train, which would be a seven and a half hour long trip. Time passed by quite quickly actually!

We had diner in Halle and after diner we got back at Theuni's place and I got introduced to the highly addictive game called "Guitar Hero"...

On friday, Christian Theuni, Wolfgang Schnerring, Martijn and I started on making plans for the rest of the sprint weekend and identified two feasible and interesting goals to aim for:

• Make it easy to register local utilities, with being able to easily install and use the Zope catalog as pratical use case.
• Get view-based security to work.

Most of the friday was spent on this planning and figuring out how we would the security and local utility registrations to work. Since my experience with hacking on Grok was still somewhat limited, I took the role of a potential developer of applications with Grok. From that perspective I tried to give feedback on how I would perceive the API that was being designed.

This process of first designing the grok API and how the configuration directives would look like, together with the strong attention to detail in the error reporting of Grok, resembled how user interfaces can be designed to quite some extent. In that sense, Grok is actually an user interface for the application developer to Zope-3's libraries and functionality.

On saturday morning we left for Philipp von Weitershausen's place in Dresden. There we would continue the sprint. Theuni of course brought Guitar Hero there too :-) After a two hour drive to Dresden, we discussed the plans for the rest of the sprint weekend with Philipp.

We then split up in two teams, where Philipp and Theuni would work on the security topic. Martijn, Wolfgang and I would work on the local utility registration with being able to use the catalog as a clear and concrete target to reach.

Since local utilities need to registered for concrete instances of objects in Zope (unlike global utilities that get registered as soon as Zope is started), quite some infrastructure work needed to be done in Grok where core Grok code was refactored for the second, third or even fourth time.

After doing this refactoring-groundwork, Theuni and Philipp reporting succes on getting the Zope-3 security mechanisms to do what we wanted it to do, and enjoying Philipp's lasagna for diner, we played lots of Guitar Hero doing hilarious imitations of how Grok, the caveman, would play rockstar guitar. Grokstar, the dude man! :-)

On sunday we continued working on the same topics in the same teams, until Wolfgang had to leave at the end of the afternoon. We presented the teams' achievements to eachother and concluded that getting the catalog to work, using the local utility infrastructure that now was built, should be fairly easy.

And it is! With just a few lines you can now setup one or more catalog instances, including what indexes to use, for you application.

(It was a bit sad to realise Wolfgang had worked so hard on all the different test cases to cover for the local utility registrations, but had to leave just before we enjoyed actually using it and see it in action)

I think I can say that we went to diner with quite a satisfied feeling, having achieved quite something in this three days! We enjoyed the food, wine and beer and played more Guitar Hero. And then some more :-)

The trip back home yesterday, going from Dresden now, took more than nine hours... This was a bit tiring, but I'm glad I went to this sprint!

The next sprint is already being planned for somewhere in April. The goal then is to get Grok in a polished, documented and releasable state...

update: the next sprint should be somewhere around April, not March and Martijn wrote a blog entry about the sprint too!

zc.buildout is a system for managing development buildouts. Or in other words (mine): for deploying applications, for production usage or development work.

zc.buildout works by way of recipes where one or more recipes define what should be done and how. E.g. "install Zope-3, create an instance and make sure this and this egg is available on the python path". Usually a recipe needs some more information from the user to know where to install things or where to look for dependencies. This is specified in the buildout.cfg file.

I learnt using zc.buildout mainly by looking at other people's work (e.g. grok).

There you can see the specification for the buildout (in the buildout.cfg file) are part of project's hierarchy. So, you check out the project from, in this case, the Subversion repository and you have the project's sources and the specification to make a buildout for this project. You run the buildout command and voila, you have a running project set up.

This makes sense while developing on this particular project, but when you're deploying an application for production use, you do not necessarily want to checkout the sources for the project, since you probably already tagged the release version, uploaded the eggs for you project to some package index (e.g. the cheeseshop or some custom index) and created a buildout.cfg specifically for the deployment of this application.

For these situations I'm trying the following pattern "in the field":

In my project's hierarchy I add a sub directory called "deploy". This directory contains basically only a buildout.cfg. This buildout.cfg knows where to go and look for all the eggs it needs. Now you only have to get this "deployment" buildout.cfg onto the end user's machine and run it to set up the application.

todo: I need to have a look at http://svn.zope.org/zc.recipe.deployment/

Yesterday I installed and played with gallery2. Gallery is web photo album software with a large and thriving developer (and user) community.

Installation was a breeze and although the initial look&feel of the thing is, uhm, sub-optimal, there's so much built-in functionality I felt compelled to give it a thorough test to see if it would fit my needs.

And it does!

So, Pilaster is dead now?

Well, yes, realistically speaking I think it is. After tons of attempts with lots of interesting ideas, I think I now wanted to use something that Just Works (even though I spent a day tweaking the look&feel and it will take some more time) and has lots of cool features already builtin.

So, I have finally overcome my dose of NIH - Not Invented Here - !!

(and I'll just completely ignore the fact it's written in PHP ;)

Yesterday I upgraded my home machine to Edgy Eft. This brings me, beside all kinds of nice goodies, Firefox 2.0 which is cool too of course.

FF 2.0 however changed the UI for closing tab - instead of one close button at the right side of the tab bar, it now has, by default, close buttons on each individual tab.

You can think of this whatever you like (apparently actual research has been conducted for the usability of closing tabs), but I was very much used to closing multiple tabs without having to move my mouse around.

Luckily there's user preference option that brings back my right hand side close button (and there're more possibilities BTW):

browser.tabs.closeButtons  3

(if this option is not yet available, you can add a new integer-type option in the about:config page.)

A quick first one hour try to create a panorama resulted in an already quite impressive result...

I consider myself lucky to have a job where I can use my prefered OS and browser. Actually, all people at the institute are strongly urged to use Firefox, so that's cool I'd say!

Anyway, for some reason page loading in my Firefox on Linux was way slower than page loading in Firefox on Windows. Even on Windows running in a VMWare session on my Linux box page loaded more quickly... It turned out to be that the DNS lookups were somehow slow (I mean, with a direct Surfnet uplink, bandwidth should not be problem, right :).

First I thought our the DNS server was to blame, but we couldn't find a plausible explanation. Then, some more Googling hinted toward IPv6 support to be to blame.

Most of the suggestions I found were related to disabling IPv6 support on a kernel module level, but I also found a hint where in the Firefox configuration you can disbale IPv6 DNS lookups.

So I tried that first by setting the following key/value:

network.dns.disableIPv6   user set   boolean   true

And lo and behold, I think that did the trick already!

Yesterday Walco and I found a great tool, called exiftool for both reading and writing exif metadata from and to images (e.g. taken with a digital camera).

Its actually both a command line utility and a library. But unfortunately its written in Perl - so I see no easy way to integrate it in other tools written in Python.

However, the command line utility is already very very useful!

The command I currently use to extract images from the compact flash card and to store these images in a directory structure based on the date the images were taken:

exiftool "-Directory<${DateTimeOriginal}" -d %Y/%m/%d/ -r -o /tmp /path/to/cf

After many experiments, I final got Xorg configured in such a way that I could use both the optimized (but closed source) NVidia driver and the Nec 20wgx2's native resolution of 1680x1050!

I assume its the Option "ModeValidation" "NoMaxPClkCheck, NoEdidMaxPClkCheck" in the config file below that actually did the trick. It basically tells the driver to ignore whatever the videocard claims its pixel clock limitations are.

Obviously the videocard is not aware of its own power...

Technical info and keywords:

GeFroce FX 5200, AGP 8x, 128MB
NVidia driver version 1.0 build 8762
Xorg version 7.0.0 (7.0.0-0ubuntu45)
Ubuntu "Dapper Drake" 6.06
Nec 20wgx2 1680x1050 pixels

Some references that got me to try this:

http://www.jwdt.com/~paysan/dell2405fpw.html http://www.nvnews.net/vbulletin/showthread.php?t=52052#4 http://www.nvnews.net/vbulletin/showthread.php?p=633393

And see my current xorg.conf file...

from the 'why-didn't-I-think-of-this-before' department:

To trace the resolution of a domain name, use:

dig +trace <domain name>

Thanks to my del.icio.us I found dnsreport.com and dnsstuff.com and immediately started testing n--tree.net, lentiform.org.

The show was great!

Like I predicted last time: Immensely powerful. And at a very high level of musicality.

Highlights to me were: Forty Six & 2, Sober(!) and Æema.

Reading some more opinions some people think the band puts some distance between them and "us". I didn't mind at all. I came to see them play, to listen to their music (and watch the visuals). And I think they did great!

moved