Have you ever wanted to find the schematic or PCB layout for your Arduino Ethernet shield, but been frustrated by the lack of version or revision information printed on the shield itself? Have you found the version numbering terminology for Ethernet shields confusing and inconsistent? The Atlas of Arduino Ethernet Shields is here to help!

The folks managing the Arduino website seem to have a penchant for showing pictures of one shield and offering schematics for another right along side or even showing pictures of different shields with incorrect labelling. This is compounded by that fact that many shields don’t have useful version or revision information printed on the shield itself. Finally, there seems to be an element of folks in the community second-guessing or inventing shield version numbers (if it came just before 5.0, it must be 4.0, right?) which just makes a bad situation worse. Until the Arduino folks get their change management practices in order, pages like this will be needed to clear up the confusion. I hope you find it useful!

Any errors below are my own, and I’ll be happy to correct them if you can provide information with a more reliable provenance than “off the Internet”.

This guide includes pictures of the Ethernet shields and links to the corresponding schematics and board layouts.

The following shields are presented, in order of age:
Official Arduino Shields

• Ethernet Shield 01
• Ethernet Shield 05 or Ethernet Shield V5
• Ethernet Shield 06 or Ethernet Shield V6
• Ethernet Shield R3

Unofficial or Impostor “Arduino” Shields

Regrettably, some shield have been released using the “Arduino” name, although they are not offical Arduino designs. These shields use a different chipset and are not (yet) documented here. This further confuses an already murky picture and is misleading to consumers.

• Arduino Ethernet Shield V1.0
• Arduino Ethernet Shield V1.1

Ethernet Shield 01

and with the SD Card socket fitted (although note the text on the label saying “SD CARD NOT SUPPORTED”)

Also Known As: V1.0, V1, 01, “Old Version 2009″, V4.0
Date: 2009
Schematic: arduino-ethernet-shield-01-schematic.pdf
Reference design: arduino-ethernet-shield-reference-design.zip
Compatibility: Arduino Duemilanove form factor. Incompatible with the MEGA without hardware modification. This shield uses analogue pins A0 and A1 for SD card detection and write protection capabilities.

The Arduino Ethernet Shield 01 (or possibly V1 or V4.0) uses the Wiznet W5100 Ethernet chip and does not have a six pin ISP/ICSP connector to the left underside of the board. This means that is is not compatible with the Arduino MEGA or Arduino MEGA 2560 without modification.

Ethernet Shield 05

Also Known As: V2.0
Date: 2010
Schematic: arduino-ethernet-shield-05-schematic.pdf
Reference design: arduino-ethernet-shield-05-reference-design.zip
Compatibility: Compatible with regular Uno and Mega form factors. The microSD card slot is connected to the Arduino’s analogue A0 and A1 pins with 10k pull-up resistors, preventing the use of these analogue inputs for other purposes without modifying the Ethernet shield.

The Arduino Ethernet Shield 05 or V5 uses the Wiznet W5100 and includes an microSD card slot. The shield uses the six pin ICSP header for the SPI signals, making it compatible with both the regular Arduino form factor (Duemilanove, Uno) and also the Mega 1280 and Mega 2560 Arduinos. To liberate the A0 and A1 signals from use by this shield cut tracks or remove the pin headers. The microSD functionality can still be used even if such changes are made; the two analogue lines are only used for card detection and write-protect functionality.

Ethernet Shield 06

Date: 2011
Schematic: arduino-ethernet-shield-06-schematic.pdf
EagleCAD reference design: arduino-ethernet-shield-06-reference-design.zip
Compatibility: Compatible with regular Uno and Mega form factors. Unlike the 05 shield, the 06 leaves the A0 and A1 pins free for other uses.

Ethernet Shield R3

Date: 2011
Schematic:: Not yet available?
Reference design: Not yet available?
Compatibility: Uno R3 or Mega 2560 R3 since this shield has the extra pins in the headers (circled in red on the photograph of the component side of the board) which are only present on R3 Arduinos. This particular header pin configuration is also known as the Arduino 1.0 pinout. I don’t know if these R3 shields are physcially compatible with pre-R3 Unos or Megas, although they are electrically compatible.

Followed my recently renewed interest in hardware and electronics—a hobby which has been dormant for about twelve years— I’ve had my first printed circuit board (PCB) made commercially. I’ve always made my own PCBs before, starting with pressing Letraset symbols directly onto copper boards for etching in a ferric chloride solution back in the 1980s. Later I graduated to photoresist based techniques, which at least meant I could re-use my designs. Today there is little point in doing this stuff at home when commercial services are available – you get much better results and you avoid also that messing about with chemistry.

I’ve been using the free Eagle CAD to design my PCBs and used PCB Train in the UK for manufacturing. My first design is a bidirectional optoisolator for the I²C bus. See below for both the design and the reality.

I obviously still have a something to learn because although the component outlines and some of the labels made it through to the end result the component names did not. Fortunately there are only a few components! Also, I notice that my font does not match the manufacturers font.

Eagle CAD design for an I2C optoisolator

I2C optoisolator PCB

In a word, don’t. Or if you do, configure your DHCP server to always lease the same address to your NexentaStor appliance.

A week ago I installed NexentaStor CE 3.1 on a new NAS server I have thrown together. Following the install last week, everything in the web-based GUI seemed to be working fine. Following the install and a few basic tests I shut down the server, knowing that I wouldn’t have time to get back to working with it for a while.

Fast forward one week to the next opportunity I get to configure my new server. Attempts to connect to the server over HTTP on port 2000 are met with a very long wait by the browser – several minutes are needed to display the main page, which is http://nas:2000/status/general/ and once I’m in there, some pages are dog slow whereas others are quite sprightly.

After much wailing and gnashing of teeth on my part I discovered that the time since I last used the server was long enough for the DHCP lease to expire. My router duly handed out a new lease, albeit with a different address. Unfortunately, Nexenta had inserted it’s own hostname and the original IP address into its /etc/hosts file. Something in the NMV web GUI was causing a look-up here, and of course by this time my router had re-allocated the original Nexenta IP address to my wife’s iPhone.

The solution was two-fold. First of all edit /etc/hosts to contain the current IP address of the server, and secondly configure my router’s DHCP server to always dish out the same IP address to the NAS server. After a re-start everything is now working fine and it seems now seems possible that I’ll be able to use NexentaStor.

My faith isn’t quite restored though – NexentaStor 3.1 is a brand new release – and judging from the forums quite a few people have experienced problems with very long browser delays waiting for the NMV to respond. NexentaStor should work with DHCP out-of-the-box and DHCP lease expiry should be tested if Nexenta are claiming that this is an enterprise grade solution.

I’ve just released asq 1.0 – a LINQ-inspired API for performing queries over Python objects. The project had been on the unpublished one-day-I’ll-finish-this back burner for a couple of years now, but recently I found myself wanting it in the course of developing another project. I decided to go public with the incomplete version I had back in January 2011, calling what I had asq 0.5 and hoping that the harsh glare of publicity would force me to make progress. I got 18 downloads without any promotional effort and that spurred me on towards a 0.9 release two months later which was feature complete and pretty stable.

Over the course of the next three months, asq 0.9 clocked up over 200 downloads.

I decided at this point that I’d hold back on any actual marketing as such until I could produce a solid 1.0 including comprehensive documentation. Well, asq 1.0 isn’t even 24 hours old yet and it’s already made 174 downloads.

I’m in the process of writing some code which ideally would work unchanged on CPython 2.x, CPython 3.x IronPython 2.x and Jython 2.x. Given the nature of the code I’m writing, that seems eminently possible with very few workarounds.

One of the changes between Python 2 and Python 3 was the removal of the cmp() function and its corresponding __cmp__() special method. Python 3 requires the use of the rich comparison operators __lt__(), __gt__(), __ge__(), __le__(), __eq__() and __ne__(). This change is well publicised and is fine so far as it goes; it’s usually no problem to modify the code accordingly. However, the same document goes on to say,

Use __lt__() for sorting, __eq__() with __hash__(), and other rich comparisons as needed.

This is clear guidance, if not a specification, that sort routines will use __lt__() for sorting. We don’t have to look much further for a statement closer to a specification though. In the Python Sorting HOWTO we can find the following clear declaration:

The sort routines are guaranteed to use __lt__() when making comparisons between two objects. So, it is easy to add a standard sort order to a class by defining an __lt__() method.

Unfortunately, this fact is not mentioned in the documentation for list.sort() or the sorted() built-in, so it’s difficult to find. Even more unfortunately for me, the documentation for the standard library heapq module does not specify at all what protocols it requires of the objects it will be ordering.

Now all of this became an issue for me today when I implemented an important optimisation which required that I implement an incremental (i.e. lazy) partial sort rather than a complete eager sort of a sequence. Partial sorts can be efficiently implemented using heaps – hence my encounter with the heapq module. My migration from sort() to heapq worked fine on CPython 2.x and CPython 3.x but when I ran my tests with IronPython I got failures for some sorting results.

Soon I determined that the problem was that, as per the documentation for sorting, I’d implemented only __lt__() and __eq__() rather than the full set of six rich comparison operators. This worked fine with my original code and with my heapq based sort in CPython because those sort implementation do indeed only call __lt__() and __eq__(). However, the IronPython implementation of heapq also calls __gt__() and so failed. Note that IronPython hasn’t done anything wrong here – it’s following the letter of the canonical CPython documentation, if not the spirit. I’m not sure it’s fair to consider this a bug in IronPython – rather it exposes a small but important weakness in the CPython documentation.

There are perhaps two morals to this story: The first is to always implement all or nothing when in comes to the rich comparison operators just like your mother taught you. The second is that we’d avoid these sort of difficulties if the Python documentation was a little more explicit about what protocols are expected of objects by its algorithms and containers.

The existence of multiple Python implementations has been extremely effective at defining what it means to be Python as opposed to CPython. Many inconsistencies such as the one noted above have been flushed out over the years. For this reason alone I’m hesitant to suggest that the folks porting the standard library from CPython to IronPython or Jython should actually read the CPython implementations to ensure compatibility. I think it’s much preferable that such things are reimplemented from the documentation and test suite with consequential improvements for both.

Before you get too excited this is an implementation of a type inference algorithm that happens to be written in Python; it has nothing to do with the Python language itself!

I’ve been working on OWL BASIC, a compiler for BBC BASIC for the .NET CLR. The compiler itself is written in IronPython. One of the challenges of compiling BBC BASIC is to infer the types of functions from the type of their return types. The return value of a BBC BASIC function can be an arbitrary expression, including calls to other functions or recursive calls to itself. I’ve implemented a simple type inference scheme which works well in the common cases, but for a fully capable solution my type checker and type inferencer need to be beefed up somewhat. To that end, I’ve been investigating standard type systems such as Hindley-Milner and inferencing algorithms such as Damas-Milner, sometimes known as Algorithm W. These algorithms or derivatives thereof are using in the ML family of languages (Standard ML, Ocaml, F#) and Haskell.

I managed to locate a Modula-2 implementation, a Perl implementation and a Scala implementation of the algorithm, each descended from the previous. With a view to improving my understanding of the algorithm I set about reimplementing in Python, largely guided by the Scala implementation, making mine the fourth in this sequence. I also located a Haskell implementation which seems to have independent ancestry. I’ve gone back to the companion paper (Cardelli 1987) to the original Modula-2 implementation and carried forward some of the code comments which had been omitted from its descendants to assist others who wish to understand the algorithm.

The program implements abstract syntax tree nodes for a small functional language, the type inferencing algorithm and finally exercises the algorithm by inferring the types of some canned expressions in the context of some predefined types. When executed it produces the following output:

> python hindley_milner.py
(letrec factorial = (fn n => (((cond (zero n)) 1) ((times n) (factorial (pred n))))) in (factorial 5)) :  int
(fn x => ((pair (x 3)) (x true))) :  Type mismatch: bool != int
((pair (f 4)) (f true)) :  Undefined symbol f
(let f = (fn x => x) in ((pair (f 4)) (f true))) :  (int * bool)
(fn f => (f f)) :  recursive unification
(let g = (fn f => 5) in (g g)) :  int
(fn g => (let f = (fn x => g) in ((pair (f 3)) (f true)))) :  (a -> (a * a))
(fn f => (fn g => (fn arg => (g (f arg))))) :  ((b -> c) -> ((c -> d) -> (b -> d)))

The Python code is shown below in its entirety or it can be downloaded as hindley_milner.py. It will run without modification on Python 2.6 or Python 3.

#!/usr/bin/env python
'''
.. module:: hindley_milner
   :synopsis: An implementation of the Hindley Milner type checking algorithm
              based on the Scala code by Andrew Forrest, the Perl code by
              Nikita Borisov and the paper "Basic Polymorphic Typechecking"
              by Cardelli.
.. moduleauthor:: Robert Smallshire
'''

from __future__ import print_function

#=======================================================#
# Class definitions for the abstract syntax tree nodes
# which comprise the little language for which types
# will be inferred

class Lambda(object):
    """Lambda abstraction"""

    def __init__(self, v, body):
        self.v = v
        self.body = body

    def __str__(self):
        return "(fn {v} => {body})".format(v=self.v, body=self.body)

class Ident(object):
    """Identfier"""

    def __init__(self, name):
        self.name = name

    def __str__(self):
        return self.name

class Apply(object):
    """Function application"""

    def __init__(self, fn, arg):
        self.fn = fn
        self.arg = arg

    def __str__(self):
        return "({fn} {arg})".format(fn=self.fn, arg=self.arg)

class Let(object):
    """Let binding"""

    def __init__(self, v, defn, body):
        self.v = v
        self.defn = defn
        self.body = body

    def __str__(self):
        return "(let {v} = {defn} in {body})".format(v=self.v, defn=self.defn, body=self.body)

class Letrec(object):
    """Letrec binding"""

    def __init__(self, v, defn, body):
        self.v = v
        self.defn = defn
        self.body = body

    def __str__(self):
        return "(letrec {v} = {defn} in {body})".format(v=self.v, defn=self.defn, body=self.body)

#=======================================================#
# Exception types

class TypeError(Exception):
    """Raised if the type inference algorithm cannot infer types successfully"""

    def __init__(self, message):
        self.__message = message

    message = property(lambda self: self.__message)

    def __str__(self):
        return str(self.message)

class ParseError(Exception):
    """Raised if the type environment supplied for is incomplete"""
    def __init__(self, message):
        self.__message = message

    message = property(lambda self: self.__message)

    def __str__(self):
        return str(self.message)

#=======================================================#
# Types and type constructors

class TypeVariable(object):
    """A type variable standing for an arbitrary type.

    All type variables have a unique id, but names are only assigned lazily,
    when required.
    """

    next_variable_id = 0

    def __init__(self):
        self.id = TypeVariable.next_variable_id
        TypeVariable.next_variable_id += 1
        self.instance = None
        self.__name = None

    next_variable_name = 'a'

    def _getName(self):
        """Names are allocated to TypeVariables lazily, so that only TypeVariables
        present
        """
        if self.__name is None:
            self.__name = TypeVariable.next_variable_name
            TypeVariable.next_variable_name = chr(ord(TypeVariable.next_variable_name) + 1)
        return self.__name

    name = property(_getName)

    def __str__(self):
        if self.instance is not None:
            return str(self.instance)
        else:
            return self.name

    def __repr__(self):
        return "TypeVariable(id = {0})".format(self.id)

class TypeOperator(object):
    """An n-ary type constructor which builds a new type from old"""

    def __init__(self, name, types):
        self.name = name
        self.types = types

    def __str__(self):
        num_types = len(self.types)
        if num_types == 0:
            return self.name
        elif num_types == 2:
            return "({0} {1} {2})".format(str(self.types[0]), self.name, str(self.types[1]))
        else:
            return "{0} {1}" % (self.name, ' '.join(self.types))

class Function(TypeOperator):
    """A binary type constructor which builds function types"""

    def __init__(self, from_type, to_type):
        super(Function, self).__init__("->", [from_type, to_type])

# Basic types are constructed with a nullary type constructor
Integer = TypeOperator("int", [])  # Basic integer
Bool    = TypeOperator("bool", []) # Basic bool

#=======================================================#
# Type inference machinery

def analyse(node, env, non_generic=None):
    """Computes the type of the expression given by node.

    The type of the node is computed in the context of the context of the
    supplied type environment env. Data types can be introduced into the
    language simply by having a predefined set of identifiers in the initial
    environment. environment; this way there is no need to change the syntax or, more
    importantly, the type-checking program when extending the language.

    Args:
        node: The root of the abstract syntax tree.
        env: The type environment is a mapping of expression identifier names
            to type assignments.
            to type assignments.
        non_generic: A set of non-generic variables, or None

    Returns:
        The computed type of the expression.

    Raises:
        TypeError: The type of the expression could not be inferred, for example
            if it is not possible to unify two types such as Integer and Bool
        ParseError: The abstract syntax tree rooted at node could not be parsed
    """

    if non_generic is None:
        non_generic = set()

    if isinstance(node, Ident):
        return getType(node.name, env, non_generic)
    elif isinstance(node, Apply):
        fun_type = analyse(node.fn, env, non_generic)
        arg_type = analyse(node.arg, env, non_generic)
        result_type = TypeVariable()
        unify(Function(arg_type, result_type), fun_type)
        return result_type
    elif isinstance(node, Lambda):
        arg_type = TypeVariable()
        new_env = env.copy()
        new_env[node.v] = arg_type
        new_non_generic = non_generic.copy()
        new_non_generic.add(arg_type)
        result_type = analyse(node.body, new_env, new_non_generic)
        return Function(arg_type, result_type)
    elif isinstance(node, Let):
        defn_type = analyse(node.defn, env, non_generic)
        new_env = env.copy()
        new_env[node.v] = defn_type
        return analyse(node.body, new_env, non_generic)
    elif isinstance(node, Letrec):
        new_type = TypeVariable()
        new_env = env.copy()
        new_env[node.v] = new_type
        new_non_generic = non_generic.copy()
        new_non_generic.add(new_type)
        defn_type = analyse(node.defn, new_env, new_non_generic)
        unify(new_type, defn_type)
        return analyse(node.body, new_env, non_generic)
    assert 0, "Unhandled syntax node {0}".format(type(t))

def getType(name, env, non_generic):
    """Get the type of identifier name from the type environment env.

    Args:
        name: The identifier name
        env: The type environment mapping from identifier names to types
        non_generic: A set of non-generic TypeVariables

    Raises:
        ParseError: Raised if name is an undefined symbol in the type
            environment.
    """
    if name in env:
        return fresh(env[name], non_generic)
    elif isIntegerLiteral(name):
        return Integer
    else:
        raise ParseError("Undefined symbol {0}".format(name))

def fresh(t, non_generic):
    """Makes a copy of a type expression.

    The type t is copied. The the generic variables are duplicated and the
    non_generic variables are shared.

    Args:
        t: A type to be copied.
        non_generic: A set of non-generic TypeVariables
    """
    mappings = {} # A mapping of TypeVariables to TypeVariables

    def freshrec(tp):
        p = prune(tp)
        if isinstance(p, TypeVariable):
            if isGeneric(p, non_generic):
                if p not in mappings:
                    mappings[p] = TypeVariable()
                return mappings[p]
            else:
                return p
        elif isinstance(p, TypeOperator):
            return TypeOperator(p.name, [freshrec(x) for x in p.types])

    return freshrec(t)

def unify(t1, t2):
    """Unify the two types t1 and t2.

    Makes the types t1 and t2 the same.

    Args:
        t1: The first type to be made equivalent
        t2: The second type to be be equivalent

    Returns:
        None

    Raises:
        TypeError: Raised if the types cannot be unified.
    """

    a = prune(t1)
    b = prune(t2)
    if isinstance(a, TypeVariable):
        if a != b:
            if occursInType(a, b):
                raise TypeError("recursive unification")
            a.instance = b
    elif isinstance(a, TypeOperator) and isinstance(b, TypeVariable):
        unify(b, a)
    elif isinstance(a, TypeOperator) and isinstance(b, TypeOperator):
        if (a.name != b.name or len(a.types) != len(b.types)):
            raise TypeError("Type mismatch: {0} != {1}".format(str(a), str(b)))
        for p, q in zip(a.types, b.types):
            unify(p, q)
    else:
        assert 0, "Not unified"

def prune(t):
    """Returns the currently defining instance of t.

    As a side effect, collapses the list of type instances. The function Prune
    is used whenever a type expression has to be inspected: it will always
    return a type expression which is either an uninstantiated type variable or
    a type operator; i.e. it will skip instantiated variables, and will
    actually prune them from expressions to remove long chains of instantiated
    variables.

    Args:
        t: The type to be pruned

    Returns:
        An uninstantiated TypeVariable or a TypeOperator
    """
    if isinstance(t, TypeVariable):
        if t.instance is not None:
            t.instance = prune(t.instance)
            return t.instance
    return t

def isGeneric(v, non_generic):
    """Checks whether a given variable occurs in a list of non-generic variables

    Note that a variables in such a list may be instantiated to a type term,
    in which case the variables contained in the type term are considered
    non-generic.

    Note: Must be called with v pre-pruned

    Args:
        v: The TypeVariable to be tested for genericity
        non_generic: A set of non-generic TypeVariables

    Returns:
        True if v is a generic variable, otherwise False
    """
    return not occursIn(v, non_generic)

def occursInType(v, type2):
    """Checks whether a type variable occurs in a type expression.

    Note: Must be called with v pre-pruned

    Args:
        v:  The TypeVariable to be tested for
        type2: The type in which to search

    Returns:
        True if v occurs in type2, otherwise False
    """
    pruned_type2 = prune(type2)
    if pruned_type2 == v:
        return True
    elif isinstance(pruned_type2, TypeOperator):
        return occursIn(v, pruned_type2.types)
    return False

def occursIn(t, types):
    """Checks whether a types variable occurs in any other types.

    Args:
        v:  The TypeVariable to be tested for
        types: The sequence of types in which to search

    Returns:
        True if t occurs in any of types, otherwise False
    """
    return any(occursInType(t, t2) for t2 in types)

def isIntegerLiteral(name):
    """Checks whether name is an integer literal string.

    Args:
        name: The identifier to check

    Returns:
        True if name is an integer literal, otherwise False
    """
    result = True
    try:
        int(name)
    except ValueError:
        result = False
    return result

#==================================================================#
# Example code to exercise the above

def tryExp(env, node):
    """Try to evaluate a type printing the result or reporting errors.

    Args:
        env: The type environment in which to evaluate the expression.
        node: The root node of the abstract syntax tree of the expression.

    Returns:
        None
    """
    print(str(node) + " : ", end=' ')
    try:
        t = analyse(node, env)
        print(str(t))
    except (ParseError, TypeError) as e:
        print(e)

def main():
    """The main example program.

    Sets up some predefined types using the type constructors TypeVariable,
    TypeOperator and Function.  Creates a list of example expressions to be
    evaluated. Evaluates the expressions, printing the type or errors arising
    from each.

    Returns:
        None
    """

    var1 = TypeVariable()
    var2 = TypeVariable()
    pair_type = TypeOperator("*", (var1, var2))

    var3 = TypeVariable()

    my_env = { "pair" : Function(var1, Function(var2, pair_type)),
               "true" : Bool,
               "cond" : Function(Bool, Function(var3, Function(var3, var3))),
               "zero" : Function(Integer, Bool),
               "pred" : Function(Integer, Integer),
               "times": Function(Integer, Function(Integer, Integer)) }

    pair = Apply(Apply(Ident("pair"), Apply(Ident("f"), Ident("4"))), Apply(Ident("f"), Ident("true")))

    examples = [
            # factorial
            Letrec("factorial", # letrec factorial =
                Lambda("n",    # fn n =>
                    Apply(
                        Apply(   # cond (zero n) 1
                            Apply(Ident("cond"),     # cond (zero n)
                                Apply(Ident("zero"), Ident("n"))),
                            Ident("1")),
                        Apply(    # times n
                            Apply(Ident("times"), Ident("n")),
                            Apply(Ident("factorial"),
                                Apply(Ident("pred"), Ident("n")))
                        )
                    )
                ),      # in
                Apply(Ident("factorial"), Ident("5"))
            ),

            # Should fail:
            # fn x => (pair(x(3) (x(true)))
            Lambda("x",
                Apply(
                    Apply(Ident("pair"),
                        Apply(Ident("x"), Ident("3"))),
                    Apply(Ident("x"), Ident("true")))),

            # pair(f(3), f(true))
            Apply(
                Apply(Ident("pair"), Apply(Ident("f"), Ident("4"))),
                Apply(Ident("f"), Ident("true"))),

            # let f = (fn x => x) in ((pair (f 4)) (f true))
            Let("f", Lambda("x", Ident("x")), pair),

            # fn f => f f (fail)
            Lambda("f", Apply(Ident("f"), Ident("f"))),

            # let g = fn f => 5 in g g
            Let("g",
                Lambda("f", Ident("5")),
                Apply(Ident("g"), Ident("g"))),

            # example that demonstrates generic and non-generic variables:
            # fn g => let f = fn x => g in pair (f 3, f true)
            Lambda("g",
                   Let("f",
                       Lambda("x", Ident("g")),
                       Apply(
                            Apply(Ident("pair"),
                                  Apply(Ident("f"), Ident("3"))
                            ),
                            Apply(Ident("f"), Ident("true"))))),

            # Function composition
            # fn f (fn g (fn arg (f g arg)))
            Lambda("f", Lambda("g", Lambda("arg", Apply(Ident("g"), Apply(Ident("f"), Ident("arg"))))))
    ]

    for example in examples:
        tryExp(my_env, example)

if __name__ == '__main__':
    main()

To compile the code comprising an OWL BASIC procedure, function or main program into CIL, we must linearise the Control Flow Graph (CFG) representing the program statements. The CFG undergoes many transformations during compilation, for example to eliminate unreachable code or convert GOSUB routines into named procedures. Generation of CIL using Reflection.Emit requires that we can define branch targets in advance of generating branch instructions or marking the target instruction and of course we want to do this in a manner which minimises the number of branches required to represent the code. The structure of the graph may be quite complex, especially for traditional BASIC spaghetti code which uses GOTO excessively rather than the more structured alternatives such as procedures and functions or the control structures introduced in BBC BASIC V .

Consider the following procedure from Sphinx Adventure. It contains three loops, one on line 271 formed with a GOTO back to the start of the line, and two REPEAT .. UNTIL loops.

266 DEF PROCL(L)
267 LOCAL I,J:CO=0:CN=0
268 IF L=1 THEN278
269 PRINT’: RESTORE L: IF O?31<>0 THEN O?31=0:DW=1
270 READ R$,R$:R$="You are "+R$
271 IF LEN(R$)+ POS>CO-CN+39 THEN R$= FNS(R$,39+CO-CN):CO=CO+39: GOTO271
272 PRINT R$: IFL=136 OR L=15 THEN O?56=L
273 IF L=16 AND FL=1 THEN PRINT"The walls are very hot!" ELSE IF L=16 THEN PRINT "The walls are steaming!"
274 IF L<>3 AND L<>142 AND L<>143 THEN PROCEX(L): IF ABS(L-19)=1 AND CH=1 THEN PRINT ELSE IF ABS(L-42)=1 AND VO=1 THEN PRINT
275 IF CH=1 AND ABS(L-19)=1 THEN PROC R(22): PRINT "chasm.":O?53=L
276 IF VO=1 AND ABS(L-42)=1 THEN PROCR(22): PRINT"glacier.":O?53=L
277 IF L=26 OR L=27 THEN O?53=L
278 J=0:I=0:CO=0
279 REPEAT:J=J+1: IF O?J=L THEN CO=CO+1
280 UNTIL J=52: IF CO=0 AND L=1 THEN PROCR(L): GOTO 284 ELSE IF CO=0 AND L<>1 THEN 284 ELSE PRINT:MAX=CO
281 IF L=1 THEN PROCR(3) ELSE PROCR(4)
282 CN=0:CO=MAX: REPEAT I=I+1: IF O?I=L THEN PROCOT(I,CO):CO=CO-1
283 UNTIL I=52
284 IF D<>0 THEN O?31=L
285 IF CF=1 AND L=94 THEN PRINT’"The casket is open."
286 IF L=24 AND SA=1 THEN PRINT’"The safe door is open."
287 PRINT: ENDPROC

The CFG for this code is shown below. Each program statement is shown as a purple box, with control flow to the following statement(s). Conditionals are shown in diamond boxes. The numbers in each purple box are source line numbers, where known.

Careful comparison of the source above and the diagram below will reveal some of the transformations that have been applied to the program; for example, READ R$,R$ on line 270 has been transformed into two consecutive assignment statements which actually take the form R$ = READ() where READ() is a function not available in the source language.

The statement level CFG has been analysed to identify basic-blocks, shown as yellow group nodes, thereby defining a higher level basic-block level CFG. Each basic block has only one entry point statement; none of the statement within the basic block are destinations of other jump instructions. Furthermore, each block has only one exit point.

More text follows this long diagram…

Generating the CIL code for a single basic block is straightforward enough – we can simply iterate through the statements comprising the basic block in order and generate the code for each in turn. However, there are many possible orders in which the code for the basic block themselves could be representing in the CIL, since we can branch from the end of any block to the next block, although of course we must start at the entry block for the procedure. Although any order starting with the entry block can be made to work, where possible we would like program control to flow smoothly from the end of a block to one of its successors without requiring a branch.

At first sight, some sort of topological ordering would seem to be appropriate, but a topological ordering is only well defined for a directed acyclic graph (DAG), and a DAG this program is not. The key to this conundrum is to reduce the directed graph to a DAG by identifying strongly connected components. By contracting each SCC to a single node we obtain what is called the condensation of the CFG which will be a DAG. To the resulting DAG we can apply a topological ordering. The ordering of vertices with each SCC is chosen by starting at the vertex with the greatest in-degree.

In order to identify and contract the SCCs we use an implementation of Tarjan’s algorithm during depth first traversal of the CFG. The reverse post ordering of the primary depth first traversal is used to generate the topological ordering of the condensed CFG.

The resulting ordering of basic blocks is shown in the diagram by the numeric labels to the top-left of each. This will be the order in which the CIL code for them is generated, and it can be seen that in about half of the cases, fall through from one block to the next (consecutive block numbers) without explicit branching can be exploited. Future optimisations will focus on further simplifying the generated code by removing vertices, such as block 31, which contain only jumps.

After a long haul, and diversions into other more important projects — including starting a family — OWL BASIC today produced its first executable. Its not much. In fact its hardly anything. Just 2048 bytes of Windows PE executable containing the global variable declarations from Acornsoft’s 1982 Sphinx Adventure. Each file of BASIC source code will be converted to a single .NET static class, with the global variables as private static fields.

The first executable produced from OWL BASIC.

Above you can see the executable loaded up into .NET Reflector, which can be used to introspect the executable, and in this case attempt to disassemble it into C#. Now we see what makes .NET such a great platform for compiler construction; below is the IronPython source code for the embryonic assembly generation function. It clocks in at fewer than ten lines of code to create an assembly, create a module, create a class, add one private static field to it for each global variable, and save the result as an .exe.

def generateAssembly(name, global_symbols):
    domain = Thread.GetDomain()
    assembly_name = AssemblyName(name)
    assembly_builder = domain.DefineDynamicAssembly(assembly_name, AssemblyBuilderAccess.RunAndSave)
    module_builder = assembly_builder.DefineDynamicModule(name + ".exe")
    type_builder = module_builder.DefineType(name, TypeAttributes.Class | TypeAttributes.Public, object().GetType())

    # Add global variables to the class
    for symbol in global_symbols.symbols.values():
        field_builder = type_builder.DefineField(symbol.name, ctsType(symbol),
                                                 FieldAttributes.Private | FieldAttributes.Static)

    result = type_builder.CreateType()
    assembly_builder.Save(name + ".exe")

where global_symbols is the global symbol table constructed during traversal of the Abstract Syntax Tree and the Control Flow Graph and the ctsType function maps OWL BASIC types to their equivalent Common Type System types for .NET. Everything else is provided by Reflection.Emit and other parts of .NET.

Its interesting that no validation was applied to the variable names supplied to Reflection.Emit. As you can see, the variable names still include the sigil suffixes for variable typing (e.g. $ for string) and Reflector happily dissassembles these into invalid C# identifiers. For the final version these names will need to be mangled (Hungarian notation?), or merely de-sigiled if no conflicts result, for compatibility with other .NET languages and tools.

After 15 years programming in C++, I was surprised to discover today that throw in C++ is an expression rather than a statement. As a result, throw may be used as part of larger expressions.

int x = 5;
int y = x > 4 ? x : throw std::out_of_range;

The only use I’ve found for this — and in fact the speculative attempt by which I discovered it — is range checking within constructor initializer lists.

RangeChecked::RangeChecked(int x) :
    int_member(x > 0 ? x : throw std::out_of_range)
{
}

It’s academic what the type of a throw expression is, since it will never be returned, but for type-checking purposes the compiler seems to be happy to use it in place of any type.

Eclipse doesn’t come with an installer for Windows. As a result of this, installing Eclipse into Program Files is very awkward without getting into a tussle with User Account Control, and virtualisation of the Program Files directory.

After far too much effort I have finally found a sequence that Works For Me™ on Windows Vista Ultimate x64.

The procedure

1. Download eclipse-SDK-3.5-win32.zip
2. Extract the zip file into a temporary directory. I used C:\Users\<usename>\Documents\tmp\eclipse
3. Within the temporary directory (important!) edit eclipse.ini, adding the line

   -configuration @user.home\.eclipse_35_config
   

   My eclipse.ini then looked like this:

   -startup
   plugins/org.eclipse.equinox.launcher_1.0.200.v20090520.jar
   --launcher.library
   plugins/org.eclipse.equinox.launcher.win32.win32.x86_1.0.200.v20090519
   -showsplash
   org.eclipse.platform
   -configuration @user.home\.eclipse_35_config
   --launcher.XXMaxPermSize
   256m
   -vmargs
   -Xms40m
   -Xmx256m
   

4. Within the temporary directory create a UTF-8 text file called eclipse.exe.manifest file side-by-side with the eclipse.exe file with the following content:

   <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
   <assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
     <assemblyIdentity version="1.0.0.0"
        processorArchitecture="X86"
        name="eclipse"
        type="win32"/>
     <description>Eclipse</description>
     <!-- Identify the application security requirements. -->
     <trustInfo xmlns="urn:schemas-microsoft-com:asm.v2">
       <security>
         <requestedPrivileges>
           <requestedExecutionLevel level="asInvoker" uiAccess="false"/>
           </requestedPrivileges>
          </security>
     </trustInfo>
   </assembly>
   

5. Using drag and drop copy in Explorer, copy the entire eclipse directory from your temporary location to C:\Program Files (x86)\eclipse. We copy, rather than move simply so you can easily return to this step.
6. Right-click on eclipse.exe in Explorer and choose Properties
7. Click on the Security tab
8. Click Advanced in the lower right
9. In the Advanced Security Settings window that pops up, click on the
   Owner tab
10. Click Edit
11. Click Continue if you get a UAC dialog
12. Click Other users or groups
13. Click Advanced in the lower left corner
14. Click Find Now
15. Scroll through the results and click on your current user account
16. Click OK to all of the remaining windows
17. Right-click the file and select Properties (again)
18. Click on the Security tab
19. Click Edit…
20. Click on the Users group
21. Adjust the permissions for your user using the check boxes at the bottom of the dialog. e.g. enable Full Control
22. Click OK to all of the remaining windows
23. Right-click the file and select Properties (again!)
24. Click Unblock and close the dialog
25. Double click eclipse.exe to run it. You should not get an Open File – Security Warning dialog.
26. Remove the temporary eclipse directory you created when you unzipped the archive.

Credit where it is due

The above post is largely pulled together from various sources I located mixed with a good degree of trial and error. I found the following particularly useful:

• Open File – Security Warning on VistaHeads
• How do you run Eclipse 3.3 on Vista?