After a long haul, and diversions into other more important projects — including starting a family — OWL BASIC today produced its first executable. Its not much. In fact its hardly anything. Just 2048 bytes of Windows PE executable containing the global variable declarations from Acornsoft’s 1982 Sphinx Adventure. Each file of BASIC source code will be converted to a single .NET static class, with the global variables as private static fields.

The first executable produced from OWL BASIC.

Above you can see the executable loaded up into .NET Reflector, which can be used to introspect the executable, and in this case attempt to disassemble it into C#. Now we see what makes .NET such a great platform for compiler construction; below is the IronPython source code for the embryonic assembly generation function. It clocks in at fewer than ten lines of code to create an assembly, create a module, create a class, add one private static field to it for each global variable, and save the result as an .exe.

def generateAssembly(name, global_symbols):
    domain = Thread.GetDomain()
    assembly_name = AssemblyName(name)
    assembly_builder = domain.DefineDynamicAssembly(assembly_name, AssemblyBuilderAccess.RunAndSave)
    module_builder = assembly_builder.DefineDynamicModule(name + ".exe")
    type_builder = module_builder.DefineType(name, TypeAttributes.Class | TypeAttributes.Public, object().GetType())

    # Add global variables to the class
    for symbol in global_symbols.symbols.values():
        field_builder = type_builder.DefineField(symbol.name, ctsType(symbol),
                                                 FieldAttributes.Private | FieldAttributes.Static)

    result = type_builder.CreateType()
    assembly_builder.Save(name + ".exe")

where global_symbols is the global symbol table constructed during traversal of the Abstract Syntax Tree and the Control Flow Graph and the ctsType function maps OWL BASIC types to their equivalent Common Type System types for .NET. Everything else is provided by Reflection.Emit and other parts of .NET.

Its interesting that no validation was applied to the variable names supplied to Reflection.Emit. As you can see, the variable names still include the sigil suffixes for variable typing (e.g. $ for string) and Reflector happily dissassembles these into invalid C# identifiers. For the final version these names will need to be mangled (Hungarian notation?), or merely de-sigiled if no conflicts result, for compatibility with other .NET languages and tools.

Europython 2009 was my first Python conference, this being the first year that I’ve been able to use Python professionally for application development. We’d made a decision that if we were to be using Python commercially, we should be active within the Python community, so two of our team travelled from Norway to Birmingham, UK. Prior to Europython 2009 my use of Python had been confined hobby projects, occasional scripts and substantial commercial use with a several industrial-scale SCons builds.

I’ll start out by stating that I came away from the conference a little dissappointed, although I must emphasise in no way cheated, by the experience. I’ve attended a great many professionally and community organized conferences over the years, and I felt in many regards Europython was somewhat further down the quality scale than it should be, given the rising importance of Python. I’m concious that this posting may come across as a litany of complaint, but I feel that some changes could significantly improve the experience for everybody involved. Organisers of such events are forever soliciting feedback, so here goes!

Europython 2009 was a multi-tracked conference with up to six parallel sessions, so its not possible for me to report on even a representative sample of the sessions. Nonetheless, the overview of the not-so-random sample presented below may give you a flavour of the experience. I’m not involved in web development currently so I tended to shy away from the plethora of sessions covering web frameworks. What does interest me are the alternative Python implementatons, IronPython and Jython and uses of Python for scientific computation. Unfortunately, I couldn’t make it to the SCons talk, another interest of mine.

Size, venue and organization

Europython had around 500 delegates in attendance, which is I believe a record. The chosen venue was something of a disappointment. Although the main lecture theatre (The Adrian Bolt Hall) was effective, the other rooms in use were without exception compromised in significant ways. One was quite small with insufficient capacity and little or no air conditioning – although it should be noted that Birmingham was experiencing something of a heatwave at the time. Another featured a concrete pillar between the speaker and his audience – although much better air conditioning in here – and a third had the audience seated on a raked seating arrangement that creaked and groaned with the slightest movement. I wonder how the musicians who usually inhabit the placed manage?

The lunch arrangements were wholly inadequate. Lunch involved collecting a polystyrene plate, bowl and a drink from a refectory upstairs, loading these with food canteen style and then precariously balancing these items (no trays here!) and trooping down a fire escape stairwell back into the corridors between the lecture halls where there was insufficient seating. At one point our Vice President of Product Development, had expressed an interest in attending Europython, having shown a genuine curiosity in the langauge – we’ve been working hard on internal advocacy. At the point at which I found myself having to stand up to eat my lunch standing up in a corridor, having requisitioned a corner of the coffee serving table with my colleague, I was glad he wasn’t here to witness this somewhat unprofessional scenario (Yes, I know its a community event, and not ‘professionally’ organised, but a touch of professionalism can go a long way in the Python advocacy stakes with the right people). I appreciate there were a mitigating circumstances (a broken lift) but the nature of the facilities alone will make me think twice about who I encourage to attend this event in future.

Preparation and planning

Shortly before Europython I’d returned from a conference called EAGE 2009 which had 7000 student and professional delegates in a major European convention centre (RAI in Amsterdam). I’ll confess that this may have done much to anchor my expection levels vis-à-vis conferences immediately prior to Europython. Venue differences and registration cost aside (Europython was 5× cheaper than EAGE) one clear difference was the poor level of preparedness of speakers at Europython compared to EAGE, even though at both conferences speakers were drawn from professional, student and amateur circles. Time and again at Europython speakers hurriedly arrived in the room with little or no time for setup of equipment such as projectors. At this point I must give a note — no, make that a plea — to conference organizers: Please place a printed notice in 144 point text stating the preferred projector resolution on the table or lectern at the front so we can be spared a linear search through the possible resolutions at the beginning of each talk.

Now, I don’t know about you, but on occasions that I’ve been presenting and a private meeting or a conference, especially if it involves a live demo requiring network connectivity, I’m in the room setting up, connecting, authenticating, testing, and re-testing for up to 30 minutes before hand ensuring that everything will run smoothly. Although this doesn’t ensure a professional delivery, it can usually eliminate the tiresome, and stressful for the speaker, configuration during the talk. Its also a common courtesy to your audience to be ready before they are and not keep them waiting. Doing this means you means you’ll get longer to talk, and time for questions. I’m sure most people would agree that time for questions beats staring Display Settings for the umpteenth time.

Some talks dependend on a ‘live’ Internet connection – always a bad idea but in this case it was fatal. Wireless connectivity was intermittent throughout the conference.

Talk quality

I found the quality of the material presented at Europython to be highly variable; it covered the gamut from borderline incompetent to very good indeed. Many of the talks simply did not have sufficient material behind them to justify the length of the time slot they were given. I wonder if fewer parallel tracks with shorter talks would give a better result; audiences would be larger, the pace quicker and delegates would be a able to see a larger proportion of talks, which would help in technology transfer between the different subgroups within the wider Python community.

Keynotes

The keynotes were engaging as one expects. Cory Doctorow gave the predictable but entertaining DRM is evil and copyright unenforceable polemic, in the moral vacuum we have come to expect. Bruce Eckel delivered an entertaining historical overview of language features across C++, Java, C# and Python. Susan Blackmore and Simon Greenish made an impassioned plea for funds for Saving Bletchley Park, although the talk lacked any technical content about the operations or machines there, which given the technical audience was a little disappointing. Sir Tony Hoare rounded off the keynotes with a comparative study of the disciplines of software engineering and computing science, finishing with the not unreasonable prediction that one day “software will be the most reliable component of every product which contains it”, although he didn’t venture to suggest a time-scale on which this might come to pass.

Conclusion

I’m glad I attended. The conference was good value and on the whole informative. The organisers should seriously consider a better venue for next year and raising the standard of talks would make a return visit a more compelling prospect. The easiest way to do this may be to simply reduce the number of talks, or shift more of the content into a more appropriate format such as lightning talks.

Jython and IronPython run on platforms where strings are unicode capable by default. Both implementations have chosen to make str essentially an alias for unicode in Python source code. The bytes type, introduced in PEP358 as part of transition to fully unicode Python 3.0, is unambiguously a sequence of single byte values. We can see in the table below that Jython and IronPython are caught between what is on the one hand most practical for interopability with existing code and their host platforms, and on the other hand the Right Thing as delivered by Python 3.0.

It seems clear that if you need to write code that is portable between the different Python implementations you should steer clear str and use bytes and unicode to unambigiously express your intent.

Of course, this is impossible since the Python Standard Library is littered with uses of str. For example, in IronPython pickle.dumps() returns str just like Python 2.6 but the str is actually has multibyte storage. IronPython hides this well, but the abstraction can leak, resulting in much confusion. Again Python 3.0 does what is right, and pickle.dumps() returns a bytes instance.

These difficulties are most likely to occur when interfacing with native Java or .NET APIs that expect byte arrays, for example when pickling to database blobs.

In Jython an str instance can be converted to a Java byte array as follows.

>>> import jarray
>>> a = jarray.array("This is  string", 'b')
>>> a
array('b', [84, 104, 105, 115, 32, 105, 115, 32, 32, 115, 116, 114, 105, 110, 103])

The equivalent in IronPython, as provided by Michael Foord, being,

>>> from System import Array, Byte
>>> a = Array[Byte](tuple(Byte(ord(c)) for c in "This is a string"))
>>> a
Array[Byte]((<System.Byte object at 0x000000000000002B [84]>, <System.Byte object at 0x000000000000002C [104]>, <System.Byte object at 0x000000000000002D [105]>, <System.Byte object at 0x000000000000002E [115]>, <System.Byte object at 0x000000000000002F [32]>, <System.Byte object at 0x0000000000000030 [105]>, <System.Byte object at 0x0000000000000031 [115]>, <System.Byte object at 0x0000000000000032 [32]>, <System.Byte object at 0x0000000000000033 [97]>, <System.Byte object at 0x0000000000000034 [32]>, <System.Byte object at 0x0000000000000035 [115]>, <System.Byte object at 0x0000000000000036 [116]>, <System.Byte object at 0x0000000000000037 [114]>, <System.Byte object at 0x0000000000000038 [105]>, <System.Byte object at 0x0000000000000039 [110]>, <System.Byte object at 0x000000000000003A [103]>))

Going back we can use identical code in IronPython and Jython.

>>> s = ''.join(chr(c) for c in a)
>>> s
'This is a string'