The collapse of the banking system has left us with considerable external financing needs. We estimate this need to be about $25 billion during the period from now and until the end of 2010. Of this, about $19 billion are composed by arrears on obligations of the three intervened private banks as well as financing earmarked for payments in relation to foreign deposits, leaving a cash financing need of $5 billion.

In case you missed it - it says that 25 - 19 = 5.

(note: this post makes no judgement of what amounts are needed, just pointing out the math, and yes - I realize it is probably a rounding error)

Update: Almost immediately after posting this, I started receiving helpful comments that perhaps introducing multithreading is an overkill here. While I do like the notion of asynchronous message passing via MVars, the comments do have a strong point. Instead of changing the post, and thereby doing the commenters a disservice by invalidating their comments, I just put a non-threaded version of this on gist.github. Indeed, it is simpler than what is found on this page and even a little bit faster :)

I lurk on several mailing lists, one of them being the MochiKit list. One thing recently discussed there is a tool to let users of MochiKit create their own customized version of it. This entails a GUI that allows a user to select which module he or she needs, and the webapp takes care of including all the dependencies as well.

This led me to a fun exercise. Say we have a dependency graph, which by nature must by a directed acyclic graph (DAG), otherwise you could have circular dependencies. Now, if you have N modules with no interdependencies, then there are 2^N possible ways of choosing a subset. However, if you restrict yourself only to those subsets that work, i.e. those where all dependencies of included modules are also included, then that puts a limit on this number.

For example, if you have two modules Base and Sub, and the latter depends on the former, then you would not allow the set containing just Sub. In mathematical terms, you only allow closed subsets of modules, where closed means:

A subset S of modules is closed if and only if the following holds:

• If M is a module in S, and M depends on module P, then P is also in S

MochiKit contains around 15 modules and their dependency graph looks like this:

I wanted to know if it was feasible to pre-generate all legal combinations of modules, and the main factor of that is exactly how many are there. Clearly there are fewer than the roughly 16.000 which there would be if there were no dependencies. Generating all dependency-closed subsets seemed like a very classical graph theoretical problem so someone must have written an algorithm for it. I asked my friend and professor, Magnús Már, for some terms to put into Google. Instead he gave me a hint on how to solve this.

The hint is this: To every closed subset of modules, there is a distinct anti-chain of modules that represents that set. An anti-chain of modules is a collection of modules such that no two depend on each other, neither directly nor indirectly.

For example, say we want the modules Logging, DOM, Style, Selector and Visual. That means we have to take all the modules in the green area below. The anti-chain for this set of modules are the blue modules.

You can see how the set of the blue modules is the smallest set of modules needed to pull in all the modules that we want.

Since there is a one-to-one correspondence between anti-chains and legal subsets, then we need only find the anti-chains and we can generate the subsets.

First some preliminaries,

> module Main where
> 
> import Control.Concurrent (forkIO)
> import Control.Concurrent.MVar
> import Data.Maybe (fromJust)
> import Data.List (nub, intersect)

We then set up our data as a simple list of pairs, where the first element is the module name and the second is a list of its dependencies. This list must be ordered in topological order, meaning that a module may not appear before any of its dependencies. This is easy to do by hand by looking at the above picture.

> dependencies = [
>  ("Base",[]),
>  ("DateTime",["Base"]),
>  ("Format",["Base"]),
>  ("Iter",["Base"]),
>  ("Async",["Base"]),
>  ("DOM",["Base"]),
>  ("Style",["Base"]),
>  ("Color",["DOM", "Style"]),
>  ("Logging",["Base"]),
>  ("LoggingPane",["Logging"]),
>  ("Selector",["DOM", "Style"]),
>  ("Signal",["DOM", "Style"]),
>  ("Visual",["Color"]),
>  ("DragAndDrop",["Iter","Signal","Visual"]),
>  ("Sortable",["DragAndDrop"])
>  ]
> 
> -- For convenience, derive the list of module names 
> modules = map fst dependencies

The first thing we need to do is to generate all anti-chains. We will do this in a dedicated thread and feed the anti-chains as we find them, one by one, on an MVar. Note that the MVar holds a Maybe value, this is so that we can indicate the end by sending Nothing.

The algorithm is simple backtracking through recursion, we start with the empty set which is always an anti-chain. At each step, we do the following.

1. We check if the candidate element is dependent on any element in the anti-chain already. If it is not, we can add it and recurse.

2. Since the anti-chain is still valid without the element, we also recurse on the unchanged anti-chain but discard the candidate element.

The recursion bottoms up when there are no more candidate elements to consider. Note that the topological order of candidates is crucial here.

The helper function accepts determines if a candidate is allowed in a chain. This is simply done by closing the candidate (see later) and seeing if there is an overlap with the current anti-chain.

> antichains :: MVar (Maybe [String]) -> IO ()
> antichains channel = antichains' [] modules >> putMVar channel Nothing
>     where
>       antichains' :: [String] -> [String] -> IO ()
>       antichains' ac [] = putMVar channel (Just ac) >> return ()
>       antichains' ac (candidate:rest) =
>           do if ac `accepts` candidate
>                 then antichains' (candidate:ac) rest
>                 else return ()
>              antichains' ac rest
>       chain `accepts` candidate = null $ intersect (closure [candidate]) chain

Above we mentioned closing a module, or more precisely a set of modules. This means to take a set of modules and adding all modules needed to make it closed with respect to dependencies. This is used both in the accepts helper above but also to map the anti-chains to what we ultimately want, the closed sets.

> closure :: [String] -> [String]
> closure ms = 
>     let missing = nub $ filter (not . flip elem ms) 
>                   $ concatMap (fromJust . flip lookup dependencies) ms
>     in
>       if null missing
>          then ms
>          else closure (ms ++ missing)

The anti-chain generator above feeds the anti-chains to an MVar. The following IO action is run in a second thread and it eats off the anti-chains from the MVar, closes them up and prints. We use a second MVar to notify the main thread when we are done.

> printer :: MVar () -> MVar (Maybe [String]) -> IO ()
> printer stop in_ = 
>     do v <- takeMVar in_
>        case v of
>          Just xs -> putStrLn $ show $ closure xs
>          Nothing -> putMVar stop ()
>        printer stop in_

Finally, to tie it all together, the main action creates the MVars for communication and sparks separate threads for the anti-chain generation and for the printing.

> main :: IO ()
> main = do chan <- newEmptyMVar
>           stop <- newEmptyMVar
>           forkIO $ printer stop chan
>           forkIO $ antichains chan
>           takeMVar stop
>           return ()

That’s it! The result: For the dependency graph of MochiKit above, there are only 817 legal combinations of modules. Actually - 816 if you discount the empty one :)

I’ll leave it up to someone else to answer the original question about feasibility of generating and storing that number of files.

Ubiquity is an ultra-handy (and geek friendly) tool for Firefox that allows for text commands. Today I created two Ubiquity commands that I find rather handy - especially for redditors :)

Automatic Markdown links

I write this blog in the Markdown markup format, which is the same format that is used for reddit comments (which I have been known to make occasionally). When writing blog posts, I often find that I mention a word or a phrase that is naturally turned into a link. This means I have to open a new tab, Google the word or phrase - open the relevant page and copy the URL into my post or comment, making sure it is in the correct Markdown format for links. Now, Ubiquity is very apt at solving these problems, so I wrote a command that allows me to do one of two things:

1. I write a phrase, say “Reykjavik University”. I then select that phrase, invoke ubiquity (in my case it is Option+Space) and type “mdlink” and hit enter. This performs a Google search on the phrase, picks the first result and uses its URL to make the phrase a proper Markdown link, replacing the selected text: [Reykjavik University](http://www.ru.is/)

2. Instead of typing the phrase and selecting it, I can also just invoke Ubuiquity and type “mdlink Reykjavik University” directly in the prompt - with the same result.

The command can be found and installed from Gist.github. Notice the (very cool) feature of Gist that it recognizes Ubiquity commands and inserts the appropriate markup to make FF offer you to install it.

Follow this link to see and/or install the “Markdown link” command

jQuery injection

When browsing, I often find the need to do it programmatically, executing Javascript in the context of a 3rd party webpage. For this it is very handy to have something like jQuery on hand and thus I’ve used a bookmarklet to inject jQuery into any web-page. That makes it very easy to do fancy stuff (like down-voting every comment by a reddit user) through the Firebug JS console.

I converted the bookmarklet linked above into a simple Ubiquity command, allowing for quicker access to the “Inject jQuery” operation. You can get the bookmarklet on its gist.github entry:

Follow this link to see and/or install the “Inject jQuery” command

While writing a parser for CCS, a process description language (don’t worry too much about it), I found I needed to parse constructs with a suffix operator. To provide a little context, the relevant grammar is this.

P ::= 0 | a.P | ... | P[f] | P \ S

where … indicates I’ve omitted some irrelevant stuff. Here, P is a process - the entity CCS is meant to describe. a is a so-called action label, just think of it as a generic label. a.P stands for a process that can perform an a action and then become the process specified after the dot. 0 (zero) stands for a special process that cannot do anything, the deadlocked process.

f is a mapping from such labels to labels, called a relabelling and is commonly specified with a list of pairs b/a,d/c,... which indicates that a maps to b, c to d etc. S is a set of such labels, represented in the normal manner with curly braces {a,b,c,...}. The construct P \ S is called a restriction, informally it means P cannot perform any of the actions in S.

In summary,

• a.a.0 is a process that can do two a actions and then stop
• P [b/a] is whatever P is, except any a performed by P will be changed to a b
• P \ {a,b} is whatever P is, except a or b actions are not allowed

Now, [f] and \ S can be considered postfix unary operators on processes, with some additional structure, the f and the S. Since they are postfix there is no need to define their relative precedence, their order is enough. I.e.

a.0 [b/a] \ {b} [d/c,e/d] \ {e,g}

is unambiguous. Our Haskell ADT to represent the abstract syntax tree looks like this,

type Action = String
type Relabeling = Map.Map Action Action
type Restriction = Set.Set Action
 
data Process
    = Null
    | Prefix Action Process
    | ...
    | Relabel Process Relabeling
    | Restrict Process Restriction
    deriving (Eq, Show, Ord)

We want the example above to be parsed into this value (indented for clarity),

Restrict (
    Relabel (
        Restrict (
            Relabel (
                Prefix "a" Null
            ) (Map.fromList [("a","b")])
        ) (Set.fromList ["b"])
    ) (Map.fromList [("c","d"),("d","e")]
) (Set.fromList ["e", "g"])

My initial attempt at writing a parser for these constructs was ugly and complex. I was trying in one rule to parse a bunch of relabellings and restrictions and construct the appropriate value from the process that had already been parsed, cluttering the rules for other constructs with this logic. I thus went to the best documentation of Haskell, the #haskellirc channel, and asked for opinions. Christophe Poucet (vincenz) showed me the light and pointed out that a parser could well return a function,

April 13th, 2008
15:24 < vincenz> Arnar: what about...
15:24 < vincenz> process_adapter :== relabeling | restriction
15:24 < vincenz> and have those return the type
15:24 < vincenz> Process -> Process

And therein lies the trick. Instead of constructing the correct value, I simply parse a sequence of adapters, each of which can be a relabeling or a restriction, in each step returning the function that wraps a process in the relevant constructors.

relabeling :: Parser Relabeling
relabeling = do rs <- brackets $ commaSep1 rename
                return $ Map.fromList rs
                where
                  rename :: Parser (Action,Action)
                  rename = do s1 <- lexeme action
                              symbol "/"
                              s2 <- lexeme action
                              return (s2,s1)
 
restriction :: Parser Restriction
restriction = do symbol "\\"
                 ss <- braces $ commaSep1 action
                 return $ Set.fromList ss
 
process_adapter :: Parser (Process -> Process)
process_adapter = do restr <- restriction
                     return (\p -> Restrict p restr)
                  <|>
                  do rel <- relabeling
                     return (\p -> Relabel p rel)

Very simple, very clean. Now, parsing a process followed by a sequence of such adapters is trivial. We then feed the process through the adapter functions in order, i.e.

adaptedProcess :: Parser Process
adaptedProcess = do p <- simpleProcess
                    adapters <- many process_adapter
                    return $ foldr (flip (.)) id adapters p

I especially like that last line, foldr (flip (.)) id adapters p. Remember adapters has the type [Process -> Process], i.e. a list of adapters. If this list is [f1,f2,...,fN], then

foldr (flip (.)) id adapters

will give us the composition of these in the correct order, namely

\x -> fN ( ... (f2 (f1 (id x))) ...)

All that’s left to do is to apply this on the original process and out pops a correctly wrapped one.

I think this is very neat and it showcases the power of first-class functions. If you don’t see the beauty here, feel absolutely free to write me off as eccentric ;o)

You can find the complete code and a PDF writeup of the CCS project right here.

Lazyness in Haskell means that we get infinite lists (or streams) for free - i.e. they don’t need any special handling over finite lists. Granted, other languages have infinite lists, but in Haskell any regular list has the natural potential of being infinite so we don’t need any specific syntax or constructors for them.

So, when I read about a Rutgers graduate student named Eric Rowland that recently discovered a simple prime-number generating formula, I decided to try it out in Haskell. Turns out it is extremely simple.

The generating formula is based on the sequence a(n) that you get by setting a(1) = 7 and then for n ≥ 2:

a(n) = a(n-1) + gcd(n,a(n-1))

In Haskell, running in interactive mode (GHCi), we define this by

Prelude> let a' = (1,7) : (map ( \(n,x) -> (n+1, x + (gcd (n+1) x))) a')
Prelude> take 10 a'
[(1,7),(2,8),(3,9),(4,10),(5,15),(6,18),(7,19),(8,20),(9,21),(10,22)]
Prelude> let a = map snd a'
Prelude> take 23 a
[7,8,9,10,15,18,19,20,21,22,33,36,37,38,39,40,41,42,43,44,45,46,69]

Note that each element of a’ is a tuple of the index n and the value a(n). By taking differences between successive values of this sequence, you always get either the number 1 or a prime number (proving this is Rowland’s contribution).

Prelude> let d = zipWith (-) (tail a) a
Prelude> take 23 d
[1,1,1,5,3,1,1,1,1,11,3,1,1,1,1,1,1,1,1,1,1,23,3]

So far, so good. Now in one step we remove the 1-s by using filter and get rid of duplicates by using nub (which is defined in the List module).

Prelude> :m +List
Prelude List> let primes = nub $ filter (/= 1) d
Prelude List> take 23 primes
[5,3,11,23,47,101,7,13,233,467,941,1889,3779,7559,15131,53,30323,60647,121403,242807,19,37,17]

Neat! The list primes is of course infinite and we could pluck the values off it as long as we like, and Haskell’s laziness will make sure values are only computed on demand. Rowland’s paper contains the details of the math.

I went on the Golden circle excursion with the ICALP attendees today. This is the “standard” tour given to most visitors and involves visiting the area around Geysir, Gullfoss and Þingvellir. After I got back I started selecting photos to put on the ICALP photo-webpage. I’ve been trying out Adobe’s new beta of Lightroom and I must say I like it. It has all the same features as the previous version, but feels considerably snappier.

One new feature is selective retouching, i.e. you paint a mask and can make some exposure/color changes to selected areas. Of course, performance goes out the window as soon as you do that but it is quite powerful and saves you from starting Photoshop.

While at Gullfoss, I took the following picture of Bláfell, a mountain north-east of there (if I’m incorrect and you know better, please correct me in the comments). The picture straight from the camera wasn’t that great…

… so I played with it for about a minute in Lightroom…

While it is still not great, I think it is quite good less than 60 seconds of work. The correctios I made were exposure correction, removing the dirt from the lens with the spot tool and then I made the mountain and sky a little more pronounced by upping the saturation and using the clarity slider (which I think is just the high-pass sharpening technique).

So.. try it out yourself if you like photography.

The activities started on Friday with the DYNAMO 2008 PhD school. I didn’t attend many of the lectures, but the ones I did were very interesting. To name one, Elias Koutsoupias gave informative talks on Game Theory and its use for example in online auctions.

Yesterday, Sunday, there were several workshops. I mostly attended SOS’08, a workshop on Structural Operational Semantics. Dale Miller, a hard-core logician, gave an excellent invited talk on formalizing SOS specifications in logic and although I didn’t understand half of it I somehow found it very interesting. His slides are available at his website.

Today was the first day of the conference proper and started off with an invited talk by Google’s S. Muthukrishnan, where he explained how Google runs auctions to decide what ads are displayed along with search results. Attendance to the talk was excellent, and while I had my hands full with other duties, I can safely say it was interesting on many levels.

After lunch I mostly attended track B, on logic, semantics and theory of programming. I’m afraid I can’t say much of merit about the talks as I had to do my best to understand them properly myself. I can say that this first day of ICALP went very smoothly in my opinion. As a bonus, the weather turned out to be very nice so in the afternoon and during the welcome reception, people could stand outside and enjoy the fresh air.

For photos from ICALP, check out our Picasa web album (updated at the end of each day). For more authoritative information on some of the talks, check out MohammadReza’s guest posts on Luca’s blog

Before I get on with the post, I’m gonna spend a few words on explaining Scala. It is a strictly typed language with a very powerful type inference system. Scala runs on the Java Virtual Machine, a port or the .NET CLR is available. It is a mix between functional and object oriented languages and is almost fully interoperable with Java. Scala replaces some of the constructs and mechanisms that you normally have in Java with more general things. Rather than using anonymous classes you have closures, there are no interfaces, but you have traits and so on. Scala also adds implicit conversions, operator overloading, mutable and immutable types and some more goodies.

Scala access to all of the class libraries that come with Java and everything you throw into its classpath. It also comes with its own class library which has collections, tuples, parsers, xml types, etc etc. Relevant for this post, maps are written as Map(key -> value), tuples as (first, second, third) and lists as List(1, 2, 3).

The first thing that is going to look weird is that the type of things appears after the name or definition and there is a keyword in front.

val i = 3
val j: Int = i
 
def hello: String = "Hello world"

The val keyword placed in front of the variables is to tell the compiler that an immutable value is coming up and var is the mutable counterpart. Methods are defined with the def keyword. The first two lines show the type inference system at work, since you have the number 3 it can only be an integer. If there is any doubt about the type during compilation, the compiler will complain and exit. The same goes for the return type of the “hello” method, the type String is specified but can be omitted. Since the method is a oneliner the = "Hello world" is a short hand for { return "Hello world" }.

Back to the JSON parser, the parsing combinator package has to be imported

import scala.util.parsing.combinator._

In Scala _ is a sequential any reference, or like in this case as * in Java imports. The parser itself is a token based parser, and extends from a trait called JavaTokenParsers which includes parsing for identifiers, numbers and strings.

class JSON extends JavaTokenParsers { 
        def obj: Parser[Map[String, Any]] = 
                "{"~> repsep(member, ",") <~"}" ^^ (Map() ++ _) 
 
        def arr: Parser[List[Any]] = 
                "["~> repsep(value, ",") <~"]" 
 
        def member: Parser[(String, Any)] = 
                stringLiteral~":"~value ^^ 
                        { case name~":"~value => (name, value) } 
 
        def value: Parser[Any] = ( 
                obj 
                | arr 
                | stringLiteral 
                | floatingPointNumber ^^ (_.toInt) 
                | "null" ^^ (x => null) 
                | "true" ^^ (x => true) 
                | "false" ^^ (x => false) 
                ) 
}

The root of the parse tree is defined in the method value which returns a Parser[Any]. Since the short hand notation is being used and it expects are Parser[Any], it uses implicit conversions for the Parser class and from there you get the ( ... | ... ) notation which provides you with parser alternatives. The weird ^^ thing is result conversions, so floatingPointNumber ^^ (_.toInt) means, take that floating number and return it as an integer using the wildcard _. repsep is a shorthand for interleaved repitition. Lastly the wavy dudes with ~, <~ and ~> denotes sequential composition and with the less and more than symbols it keeps to the left or right.

To run the parser and do something meaningful with it we read in a file pass it on to the parser and print the output

import java.io.FileReader 
 
object JSONTest extends JSON { 
        def main(args: Array[String]) { 
                val reader = new FileReader(args(0)) 
                println(parseAll(value, reader)) 
        } 
}

Different from Java, Scala does not have static types per se but allows you to do static like things by using a object instead of a class. Also notice how the FileReader is being imported directly from the Java class libraries.

After we’ve compiled the code and run it on the same file JSON file as Arnar provided in his example, we are presented with

[22.2] parsed: 
    Map(”glossary” -> 
        Map(”title” -> “example glossary”, “GlossDiv” -> 
            Map(”title” -> “S”, 
                “GlossList” -> 
                Map(”GlossEntry” -> 
                    Map(”Acronym” -> “SGML”, 
                        “Abbrev” -> “ISO 8879:1986″, 
                        “GlossSee” -> “markup”, 
                        “GlossTerm” -> “Standard Generalized Markup Language”, 
                        “ID” -> “SGML”, 
                        “GlossDef” -> 
                            Map(”para” -> 
                                “A meta-markup language, used to create markup 
                                languages such as DocBook.”, 
                                “GlossSeeAlso” -> List(”GML”, “XML”)), 
                        “SortAs” -> “SGML”)))))

I am to lazy to format this output nicely, but you get the picture. :) I noticed also that this JSON parser does not handle comments, which I leave as an exercise to my good readers.

I have been meaning to write something on Scala for quite some time now and now I was forced to since Arnar mentioned my crazy ranting. In the coming weeks I am hopefully going to show you some more niceness of Scala, such as implicit conversions, XML types, its functional nature, etc.

A friend of mine was showing me a very simple and elegant parser for JSON, written in Scala. As an excercise, I decided to write one in Haskell, using the Parsec parser combinator library. Here’s a quick walkthrough.

First we need some imports,

> import qualified Data.Map as Map
> import Control.Monad (liftM)
> import Text.ParserCombinators.Parsec
> import qualified Text.ParserCombinators.Parsec.Token as T
> import Text.ParserCombinators.Parsec.Language

Now, to represent a JSON value, we use an algebraic data type. For simplicity, we’ll only worry about integers and not floating point numbers. The definition of the type is simple, to be concise we use single letter constructor names (not a good idea if you want to publish a proper library).

> data JSON = S String 
>           | I Integer 
>           | B Bool 
>           | L [JSON] 
>           | O (Map.Map String JSON)
>           deriving Show

Now, to make our life somewhat easier, we’ll make use of the facilities in the ParsecToken module. This module provides us with ways to create functions for generic lexers, which deal correctly with whitespace (and comments) in a language. The first step is to create a language definition, in our case it only specifies how JSON comments are delimited. From the definition we create a token parser which we call lexer.

> ldef = emptyDef { commentStart = "/*"
>                 , commentEnd = "*/"
>                 , commentLine = "//"
>                 }
> lexer = T.makeTokenParser ldef

We can now pass lexer to the convenience functions of ParsecToken. As is common with such parsers, we rebind their names in our module to spare us referring to lexer every time we use them.

> symbol        = T.symbol        lexer
> stringLiteral = T.stringLiteral lexer
> integer       = T.integer       lexer
> squares       = T.squares       lexer
> braces        = T.braces        lexer
> commaSep      = T.commaSep      lexer

And now the parser itself. The basic element is a JSON value. From now on I’ll include the type signatures to make things easier to follow, but they are of course not necessary.

> value :: Parser JSON
> value = liftM S stringLiteral
>     <|> liftM I integer
>     <|> liftM B boolean
>     <|> liftM L list
>     <|> liftM O object

A value is simply a choice between each of the basic types, a string, an integer, a boolean, a list of values or an object (which is simply a mapping from strings to values). We defer each to its own parser, and simply lift the value constructors to the parser monad. We get parsing of strings and integers completely for free from the ParsecToken module, but the other three we need to handle ourselves. The boolean case is simple,

> boolean :: Parser Bool
> boolean = do symbol "true"
>              return True
>       <|> do symbol "false"
>              return False

Surprisingly, the list is even simpler, due to the helpers from ParsecToken.

> list :: Parser [JSON]
> list = squares $ commaSep value

Parsing objects is also fairly simple, basically just like list except that instead of parsing a comma seperated list of values, we parse a comma seperated list of key/value pairs. Such a pair is handled by the nested keyValue parser.

> object :: Parser (Map.Map String JSON)
> object = liftM Map.fromList $ braces $ commaSep keyValue
>          where
>              keyValue = do key <- stringLiteral
>                            symbol ":"
>                            val <- value
>                            return (key, val)

And that’s it! To test this easily, we finally add a main action that simply parses a JSON value from standard input.

> main = getContents >>= parseTest value

Testing the parser

A simple test, including a comment in the middle:

arnarb-macbook:tmp arnarb$ runhaskell json.hs 
[1,"two",false,{"one":1, /* a comment here */ "two":2, "list":[]}]
< here I pressed ctrl-d to insert an EOF, rest is output >
L [I 1,S "two",B False,O (fromList [("list",L []),(”one”,I 1),(”two”,I 2)])]

As an more complex test, we run it on the first example from json.org (newlines and indent added to the output for clarity).

arnarb-macbook:tmp arnarb$ runhaskell json.hs 
{
  "glossary": {
    "title": "example glossary",
    "GlossDiv": {
      "title": "S",
      "GlossList": {
        "GlossEntry": {
          "ID": "SGML",
          "SortAs": "SGML",
          "GlossTerm": "Standard Generalized Markup Language",
          "Acronym": "SGML",
          "Abbrev": "ISO 8879:1986",
          "GlossDef": {
            "para": "A meta-markup language, used to create markup languages such as DocBook.",
            "GlossSeeAlso": ["GML", "XML"]
          },
          “GlossSee”: “markup”
        }
      }
    }
  }
}
< here I pressed ctrl-d to insert an EOF, rest is output >
O (fromList [
    ("glossary",O (fromList [
        ("GlossDiv",O (fromList [
            ("GlossList",O (fromList [
                ("GlossEntry",O (fromList [
                    ("Abbrev",S "ISO 8879:1986"),
                    ("Acronym",S "SGML"),
                    ("GlossDef",O (fromList [
                        ("GlossSeeAlso",L [S "GML",S "XML"]),
                        (”para”,S “A meta-markup language, used to create markup languages such as DocBook.”)
                    ])),
                    (”GlossSee”,S “markup”),
                    (”GlossTerm”,S “Standard Generalized Markup Language”),
                    (”ID”,S “SGML”),
                    (”SortAs”,S “SGML”)
                ]))
            ])),
            (”title”,S “S”)
        ])),
        (”title”,S “example glossary”)
    ]))
])

Finally, we should note that many people have done this before :)

Edits:

• Added link to Scala version.
• Small revisisons, thanks to kpreid on #haskell for pointing out braces and squares.
• Stop lugging lexer around, thanks to Paczesiowa on reddit for useful suggestions.

However, in addition to improving the form validation code, I suggest (and I am by no means the first to propose this) that we stop requesting so much information in user registration forms, and focus on information that is relevant to the application. For instance, it is very rational to ask for a username and a password that the user wishes to have. It is also rational to ask for an email address which can be used to verify the registration, as well as provide a way to handle forgotten passwords.

On the other hand, I see no reason to require phone numbers, age and so on. In Arnar’s post there is a screen shot from Apple, where they fail to validate the phone number. Why does Apple require you to register your phone number in the first place? What if I do not, under any circumstances, want Apple to call me? If they really need to reach me, they can do so via e-mail. I’m also curious how often Apple actually uses this information, has anyone received a phone call from Apple after signing up for an Apple ID?

The form that led to this post is the account creation form at Digg. I saw an article that I wanted to bump up, an action that requires a Digg account. However, after I viewed the registration form I decided not to sign up.

In addition to the standerd username, password and email address, Digg requires you to provide your first and last name, gender, birthday, and full address. They provide no rational explanation for requesting this information, so I can only assume that they use it to improve their profiling. If I ever feel so strongly about a submitted story on Digg that I feel compelled to sign up, I will most certainly register with false information.

So to wrap this up: request only the information you need in order to provide the service. It leads to simpler form validation, better privacy and less annoyed users. Sounds like a win/win situation to me.