Sufficiently Small

Jython is currently, by far the most useful of the the three main Python implementations, Jython, CPython and IronPython. It allows access to the Java standard library which often provides more standard, better defined, and certainly better documented alternatives to much of what is in the Python standard library. It is also more cross-platform than .NET bound IronPython and offers superior multithreading support to CPython.

Its well known within the Jython community that having first-class functions in Python makes development with Swing more productive in Jython than it is with Java, by allowing us to attach functions or methods directly to Swing callbacks such as actionPerformed, without the need for anonymous classes or excessive need for interface implementation. This example taken from What’s Good About Jython? shows this in action:

import javax.swing as swing

def printMessage(event):
    print "Ouch!"
        
if __name__== "__main__":
    frame=swing.JFrame(title="My Frame", size=(300,300))
    frame.defaultCloseOperation=swing.JFrame.EXIT_ON_CLOSE;
    button=swing.JButton("Push Me!", actionPerformed=printMessage)
    frame.contentPane.add(button)
    frame.visible=1

The most powerful result of learning different programming languages is being able to apply concepts learned in one language, in another language. This type of ‘technology transfer’ drives progress in many science and engineering fields. This cross-fertilisation has always been possible, but is becoming both more feasible and popular thanks to multiparadigm languages such as Python and Ruby, and wider use of the STL and Boost in Standard C++ which encourage and facilitate functional and higher-order programming style. There are even books on the subject of Higher Order Perl.

Although its had many notable ongoing and recent successes (e.g. Emacs, Darcs, Pugs) the functional community is unfortunately widely perceived as academic or impractical. Indeed, Guido van Rossum has recently threatened to remove some functional programming support from Python, bizarrely whilst simultaneously adding higher-order features such as decorators. This is unfortunate because functional programming style and idioms can go a long way to improving software quality in any language - functions without side-effects are far easier to test, debug and reason about.

For example, the technique of currying - fixing a function argument and returning a new function which takes the remaining parameters can be very useful for development with the Swing UI toolkit supplied with Java.

In this example extracted from one of my own projects, where the user interface features two radio buttons, we look at how some higher-order techniques can be used to aid user-interface programming with Swing. In the code below,

fixed_radio = JRadioButton(text="Regular columns of fixed width",
        actionPerformed = self.handleFixedOption)

separated_radio = JRadioButton(text="Fields separated by spaces,
        commas, or other characters",
        actionPerformed = self.handleSeparatedOption)

two bound-methods are attached to the actionPerformed event of each button.

def handleFixedOption(self, event=None):
    self.controller.setOption(self.controller.FIXED)

def handleSeparatedOption(self, event=None):
    self.controller.setOption(self.controller.SEPARATED)

We don’t actually use the event parameter in this case, but it will be passed to the handler function we provide from actionPerformed, so we must accommodate it. These two methods are identical apart from the value passed to setOption internally, so we could factor out the common code and these two functions with a single function and pass the option value FIXED or SEPARATED in as an extra argument,

def handleOption(self, option, event=None):
    self.controller.setOption(option)

However, we now have a difficulty in that we can no longer bind our new function directly to the actionPerformed handler of the buttons, since actionPerformed expects a function which takes single event argument. We know at coding time which option value should be attached to each button - but we don’t know until the code is run, which event will be passed from actionPerformed. We need to bind the option argument before the event argument. The solution to this is currying.

Rather than writing our own currying facility, we can use on off-the-shelf implementation provided by the Xoltar Toolkit which provides some functional programming support in Python. In particular we use the curry function to curry our handleOption function into some new functions that only take take a single event argument.

from functional import curry

fixed_radio = JRadioButton(text="Regular columns of fixed width",
        actionPerformed = curry(self.handleOption,
                self.controller.FIXED))

separated_radio = JRadioButton(text="Fields separated by spaces,
        commas, or other characters",
        actionPerformed = curry(self.handleOption,
                self.controller.SEPARATED))

The curry function takes the function to be curried as its first argument. In this case we pass the bound-method self.optionHandler. The second argument to curry is the value of the first parameter to the function to be curried, which want tobe fixed to this value in the created function, for example, self.controller.FIXED. The result of the curry function is a new function which accepts one less argument than the original function - so in this case we get a new function which only accepts the remaining event parameter, and is therefore equivalent to the handleFixedOption function we started with, but which we have now avoided having to write.

The material advantage with such a simple example is minimal, but when such techniques are used throughout a project the savings in space and complexity can be substantial. The combination of functional concepts from borrowed from languages such a Haskell, facilitated by tools provided by the Xoltar Tookit, developed in dynamically-typed Jython and exploiting one of the better designed user interface APIs in the form of Swing can be concise and powerful.

There’s an awful lot of C++ out there. As a industry we’ve been pumping out C++ code since we collectively jumped on the object orientation bandwagon in the early 1990s. There are large systems, each with millions of lines of code created entirely in C++. Every day thousands of hours are spent, at fantastic expense, finding memory leaks, wondering why pointers are dangling and waiting hours for our code to compile. And these are the major contibutory factors to the big C++ problem.

Productivity.

Or lack of it. You see, writing C++ is easy. Writing good C++ is hard. Of course it is possible to write solid C++ but the unfortunate truth is that the huge existing body of code is unlikely to be rebuilt around the good programming techniques facilitated by libraries such as Boost or Qt. We will continue on the long death-march down the road of ongoing mainentance.

So why do programmers continue to subject themselved to working in C++, and why do our managers not only think this is acceptable, but activly encourage it? Of course, there are many reasons, but two stand out. First of all, many developers value the performance of their code, over their personal productivity. The second is the perceived difficulty of learning a different programming language in which to create better code.

My experience is that developers display an almost unnatural obsession with the performance of their code. This need for high performance has become some sort of pseudo-requirement for new code in the mind of many developers. Performance is commonly cited as a reason for using C++ over and above some other language such as Java or Python. But rarely is it necessary; many developers are not aware of exactly how fast their code needs to be, or conversely how slow it can afford to be. In spite of the claim that performance is important, the overwhelming majority have never used profiling tools in order to truly improve performance in the few lines where it counts. Why is performance valued more highly than rapid producing something that works?

This attachment to performace presents a serious problem, since for most programming tasks it is possible to write faster solutions in C++ than is most other programming languages. Any other language is then almost always a notch down in performance, even though this in itself is usually irrelevant for the task at hand. Perl hackers have recognised this for years with ‘You can sometimes write faster code in C, but you can always write code faster in Perl’ - although I fully agree with the spirit of their statement I actually think Perl introduces its own set of syntactic productivity barriers.

So given these topsy-turvy circumstances we should expect managers to be challenging their developers. Our managers should be asking why we are using tools such as C++ which make it unnecessarily awkward and time-consuming to write reliable code, and which do nothing to prevent common categories of errors. They should be encouraging us to write in Python, Ruby or even C# which allow us to much more rapidly create something which works, and which will liekly require much less cumulative maintenance over the life of the software. Furthermore, developers should be volunteering to use these tools in droves since their own productivity will increase massively, which is surely good for their careers and employers, and they will spend less time with mind-numbing debugging.

Although there is certainly biodiversity in the programming language jungle, most developers choose only to sample one or two programming language fruits and usually these are usually very similar - more a case of oranges and lemons than apples and oranges. For example many developers may know C++ (from work) and Java (from college). This citric experience provides a limited perspective on programming solutions. I don’t believe that the best programmers ‘think in C++’ or ‘think in Python’, in spite of some excellent books with that moniker. The best developers - the superstars our teams depend on - ‘think in software’, some sort of well defined, but difficult to expess, higher-level concept of how software should be designed and implemented. These great hackers know several languages which exploit different paradigms - C++ (imperative, strong, static typing), Python (imperative, strong, dynamic typing), Haskell (functional, strong typing) - and they are comfortable with applying techniques from one language in another, for example, by writing reliable code in C++ by avoiding functions with side-effects. Learning languages is relatively easy within a given paradigm - for example a C++ programmer learning Python. Crossing paradigm boundaries can be more challenging I’m a C++ developer by trade and I still struggle with functional Haskell - largely through lack of practical experience. However, the majority or programmers know only one language well enough to apply it commercially, and more remarkably show little curiosity about what other programming languages could offer them.

So why all the fuss about multiple languages and the corporate obsession with C++ ?

I recently had the dubious honour of being asked to review some C++ code which both transformed XML recieved from a socket into standardized SOAP requests. I was asked for comments on how the code could be improved, or refactored. The original was unreliable, leaked memory, and hogged the CPU due to some dubious socket handling. I hacked around in the code for 15 minutes looking for easy opportunities for improvement; hoping to make an impression, or spot some neat refactoring that could be applied. I was struggling - the code was almost unreadable, with five different string representations (don’t ask) and multitudinous conversions between them. It was clear that this particular problem was indeed the proverbial nail being pounded by the proverbial C++ hammer. Nobody volunteers to do XML processing in C++ do they? When I encountered inhumane C++ SOAP bindings statically generated from WSDL files I knew I had seen enough. I wanted to recommend that the code be rewritten in something more suited to the problem at hand- in this case I though Python would be a much better fit to the problem with its nice XML processing and SOAP libraries.

I spend a few evenings trying to understand the intent of the original code and re-implementing it in Python - not line-by-line as that would have achieved nothing - but from a higher level perspective. The result? I had reduced 8000 lines of C++ down to around 800 lines of Python, and I had enjoyed myself to boot. This was the first time I had experienced the fabled 10:1 code reduction of working in Python in a non-trivial example. I was amazed. Surely I had missed some critical piece of functionality? But no, it was all there - and what’s more it nearly worked.

I presented my late-night efforts back to my colleague who had introduced me to the problematic C++ code initially. He responded enthusiastically. We spend a few days with him debugging, and me helping and hinting - a kind of pair programming separated by communicating only through Jabber and source control over the 300 kilometres of mountain and glacier that separated us. Within 36 hours and with some help from tcpdump, he had something that worked. We had slain the leaking, intransigent, C++ beast, and replaced it with a svelt Pythonic beauty.

And then he talked to his manager.

I wasn’t present, but I imagine the conversation went something like this:

Developer: “You know the SOAP agent we’ve been having trouble with?”

Manager: “Yeah - the one we need to fix and deploy by the end of next week?”

Developer: “Well, most of the problems are sorted, its no longer leaking memory and the socket communication is now far more reliable. In fact - I’m confident its now a quality product.”

Manager: “So what was the problem?”

Developer: “Well, we didn’t want to waste more time tracing leaks in the C++ version, so we recoded it in Python; now its 90% smaller, and far more robust. By the way, you know how the older version only worked on Windows? Well now it also works on our Linux, Solaris and Irix too!”

Manager: “We can’t use it.”

Developer: “The only catch is that we don’t have experience of deploying Python applications, so there might be some work to do on the installer, but this program is usually installed by sys-admins anyway, and we’re sure its solvable in the …”

Manager: “We can’t …”

Developer: “… Excuse me?”

Manager: “We don’t use Python here. We use C++. We can’t afford the overhead of developers needing to know more than one language.”

Ouch! So now the Python solution, which is a much better technical fit to the problem than the C++ behemoth is rejected because it is feared the cognitive load of learning a second language may be too high for some developers. Worse still, the hopes for our enthusiastic colleague becoming a topline developer are diminished, because rather than valuing diversity of programming language knowledge, the application of such abilities is frowned upon as ‘too costly’.

There is hope for Python, Ruby and other productive languages though to overturn the C++ monopoly in the realm of proprietary software, non web-based sofware. (Remember real applications?) But with developers on the one hand needlessly obsessing about which functions they should inline, or avoiding dynamic_cast completely because its slow, and on the other hand managers encouraging a language monoculture to protect their developers from having to learn something new, I think change will take an order of magnitude longer to achieve than we have seen in the FOSS world. Achieving escape velocity from the black hole of C++ productivity cannot be achieved without both main engine developer thrust and management boosters. Developers on the ground must accept that using a language slower than C++ can be a good thing, and managers need to permit a degree of linguistic diversity in the interests of multiplying the productivity of their teams.