Professional Documents
Culture Documents
Abstract
1.2
1.
Introduction
1.1
We felt that the clients application domain could benefit from using
a more powerful language, and we saw this as an opportunity to
explore functional programming. Convincing the client to do this
took some work, especially since we had no real experience with
functional languages at that point. We took a two-pronged approach
to selling functional programming for this job.
First, we explained that we had significant programming experience in more traditional languages, especially Java. We pointed out
that this made us familiar with the limitations of these languages: so
familiar, in fact, that in the case of Java we had already moved away
from the language to escape those very limitations. We explained
that, for us, switching languages every few years was a normal technology upgrade (much as any other industry will incorporate new
advancements in the state of the art) and we had previous experience with working through these sorts of changes.
Second, we argued that we had been for some time looking to
move on to a new language, had done significant research in that
direction, and had already identified many of the specific things
needed to make the change successful and profitable. As well
as general promises of better productivity and fewer bugs, we
pointed to specific features we wanted in a new language for which
the client himself was looking. In particular, he had expressed a
preference for a system where he would have the ability to examine
and modify the financial algorithms himself; we explained and
demonstrated that the languages we were looking at offered much
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full citation
on the first page. To copy otherwise, to republish, to post on servers or to redistribute
to lists, requires prior specific permission and/or a fee.
ICFP09, August 31September 2, 2009, Edinburgh, Scotland, UK.
c 2009 ACM 978-1-60558-332-7/09/08. . . $5.00
Copyright
185
2.
2.1
The world of functional programming offers a wide, even bewildering variety of features and technologies, spread across many different languages. In some cases a language has has strong support for
an imperative or object-oriented coding style, with merely the possibility of doing functional programming (such as JavaScript); in
other cases one must abandon wholesale ones non-functional style
and adopt a completely different one.
As we had already extensive experience with Java and Ruby and
had become unhappy with the level of expressive power that either
offered, we chose to pursue what we felt were more advanced
language features, at the cost of an increased learning curve and
greater risk.
Functional language features of particular interest to us were
a sophisticated type system (Hindley-Milner is the obvious example here), and advanced, modular structures for control flow. More
generally, we were looking for minimal syntax, concise yet expressive code, powerful abstractions, and facilities that would allow us
easily to use parallelism.
The concise yet expressive code requirement was particularly
important to us, both because weve found that reading and understanding existing code is a substantial part of the programming
process, and because we wanted to be able to develop code that, at
least for the parts in his domain, our client could read and perhaps
modify.
2.2
2.3
Languages Examined
With the above criteria in mind, we looked at some of the more popular free functional language implementations available. Our ability to compare these was limited: we simply didnt know enough
about functional languages in general, and had no extensive experience with any functional language implementation. Thus, we were
forced to rely on what we could learn from textbooks and information on the web. (Blog entries documenting interesting features
of the various languages were particularly influential.) Especially,
we had no way of judging some of the more esoteric and complex
language features as we simply didnt understand them.
This no doubt skewed our evaluation process, but we saw no
reasonable way of dealing with this other than spending several
months building a non-trivial test application in each of several
different languages. This point should be kept in mind by language
promoters: the information you need to communicate to convince
non-functional-programmers to try a language is of a substantially
different sort than what sways those who are already familiar with
at least one functional programming language.
We looked most seriously at the Glasgow Haskell Compiler(2),
Objective Caml(6), and Scala(8). (We considered looking at LISP
and Scheme implementations, but these seemed to be both lacking in certain language features and we had (what are in retrospect
perhaps undeserved) concerns about the available free implementations.)
All three of these language implementations we examined share
some features in common:
fast code;
tools that fit with our current Unix development systems;
the ability to interface with code compiled from other lan-
guages; and
an open source implementation under active development.
186
2.3.1
3.
3.1
Objective Caml
To us, Objective Caml(6) appeared to be a more conservative alternative to GHC, offering many of GHCs and Haskells features,
but without being so radical as to introduce enforced purity or lazy
evaluation.
Advantages particular to the implementation were that it was
well known, mature, reputedly produced very fast code, and we
knew it to be used for large amounts of production code by other
companies in the financial community.
OCaml did have several things that put us off. The language
syntax didnt seem nearly as clean as Haskell, which was partially
inherent (such as double-semicolons to end declarations) and partly
due to the inability to overload functions (due to the lack of type
classes).
As well, the books we had available were just not as good. At
that time wed just finished reading a fairly recent book on OCaml
that, unlike the Haskell books wed read, did not show us any
impressive new programming techniques but instead appeared to
treat the language in a very imperative fashion.
3.2
2.4
3 ByteString
2.3.3
Scala
Selection of Haskell
187
3.3
4.
Speed
With the exception of space leak problems (see next section), speed
was never an issue for us. The application is sensitive to how fast
it can do the calculations to build a pricing model for the current
market, but the code generated by GHC 6.8.3 turned out to be significantly faster than the clients previous Java application. Weve
to this point seen no significant difference between the current systems performance and what we believe we could achieve in C or
C++.
We use multi-core systems, and make extensive use of multithreaded code for both concurrency and parallelism. This has
worked well for us. However, when used for parallelism, with lazy
evaluation one has to be careful about in which thread computations are really occurring: i.e., that one is sending a result, and not
an unevaluated thunk, to another thread. This has brought up problems and used solutions similar in style to the space leak issues.
We have yet to make extensive use of explicitly parallel computation based on the par function. However, it seems clear that
if and when we move in to this area, implementation will be considerably simpler than when using a thread-based model. Haskells
purity by default model helps greatly here.
3.4
4.1
Language Issues
Portability
One pleasant surprise was that, once wed made a few modifications to our build system, building and running our application under Microsoft Windows was no trouble at all. While not an initial
requirement, this was a nice facility to have as it saved our client
setting up a separate Unix machine to run the simulator for himself. Eventually, we ended up being able to write in Haskell a small
amount of Windows code wed initially planned to write in C++,
which saved us considerable time and effort (see below).
3.5
4.2
Refactoring
188
rest broken while testing the change, in Haskell one must fix
every function in the module before even being able to compile.
There are certain techniques to help work around this, such as
copying the module and removing the code not currently of interest
(which is tedious), or minimizing the use of type declarations
(which doesnt always help, and brings its own problems), but we
didnt find these to be very satisfactory.
We feel that a good refactoring tool could significantly ease this
problem, but there seems nothing like this in common use in the
Haskell community at the moment.
That said, the major refactorings we do on a regular basis (major ones being restructuring that crosses many modules and moves
significant amounts of code between them) were less affected by
this problem, and did not seem to take significantly longer than implementing the same sort of change in any other language. As well,
being able to do more work with the type checker and less with testing increased our confidence that our refactorings had not broken
parts of the system.
4.3
4.4
Lazy evaluation turned out to have good and bad points in the end.
However, it caused, and continues to cause, us pain in two particular
areas.
The first is that, when distributing computation amongst multiple co-operating threads, its important to ensure that data structures passed between threads are evaluated to head normal form
when necessary before being used by the receiving thread. Initially
we found figuring out how to do this to be difficult; even determining when this is necessary can be a challenge in itself. (One clue is
a large amount of heap use connected with communications queues
between threads, as the sending thread overruns the receivers ability to evaluate the thunks its receiving.) A profiler that did not slow
down the program and that could help with this would be highly
welcome.
The second is the dreaded space leak, when unevaluated computations retain earlier versions of data in the heap and cause it to
grow to enormous proportions, slowing and eventually halting the
program as it consumes all memory and swap space. This proved to
be a particular problem in our application, which is essentially several loops (processing messages and doing calculations) executed
millions of times over the course of a many-hour run. The profiler
that comes with GHC is of more help here, but we have yet to work
out a truly satisfactory way of dealing with this.
A particularly common cause of space leaks for us was when
using data structures (such as lists and Data.Map) that have many
small changes over time. As one inserts and removes data, old, nolonger-referenced versions of the data structure are kept around to
enable the evaluation of thunks that will create the new version of
the data structure. As mentioned previously, nave approaches to
the problem can backfire, and better approaches were not always
obvious. For Data.Map and similar structures, we resorted to doing an immediate lookup and forced evaluation of data just inserted;
this seemed to fix the problem while avoiding the massive performance penalty of doing an rnf on the root of the data structure.
Another issue relates to interactive programs. Certain lazy idioms (such as using the getContents function) can effectively
deadlock such programs, rendering them unresponsive. This is not
immediately obvious to programmers new to lazy evaluation, and
caused us early on to spend some figuring out what was going on.
Knowing what we know now, we are still hesitant about the
value of lazy evaluation; every time we feel weve finally got a good
handle on it, another problem seems to crop up. We feel that the
subject needs, at the very least, a good book that would both cover
all of the issues well and help the reader develop a good intuition
for the behaviour of lazily-evaluated systems.
189
5.
References
Conclusions
[1] Friedman, Daniel P., and Felleisen, Matthias, The Little Schemer,
Fourth Edition. MIT Press, 1996. ISBN-13: 978-0-262-56099-3.
[2] The Glasgow Haskell Compiler. http://www.haskell.org/ghc/
[3] Graham, Paul, Beating the Averages, from Hackers and Painters
OReilly, 2004, ISBN-13: 978-0-596-00662-4. Also available from
http://paulgraham.com/avg.html
[4] Hudak, Paul, The Haskell School of Expression: Learning Functional
Programming Through Multimedia. Cambridge University Press,
2000. ISBN-13: 978-0-521-64408-2.
[5] Hutton, Graham, Programming in Haskell. Cambridge University
Press, 2007. ISBN-13: 978-0-521-69269-4.
[6] Objective Caml. http://caml.inria.fr/ocaml/index.en.html
[7] Ruby Programming Language. http://www.ruby-lang.org/en/
[8] The Scala Programming Language. http://www.scala-lang.org/
[9] Duncan Coutts, Don Stewart and Roman Leshchinskiy, Rewriting
Haskell Strings.
http://www.cse.unsw.edu.au/ dons/papers/CSL06.html
Our experience with GHC has shown that, while it has its quirks
and issues, these are no worse than those of other commonly-used
programming language implementations when used for commercial software development. However, some of the issues are different enough that one should be prepared to spend a little more time
learning how to deal with them than when moving between more
similar systems.
We found significant advantages to using Haskell, and its clear
to us that there is much more expressive power available of which
weve yet to make use. Learning to take advantage of Haskell takes
a fair amount of work, but benefits are seen fairly quickly and
continuously as one improves.
We were lucky with our choice of Haskell and GHC and, in light
of our experience, we would make the same choice again. However,
we note that, given our lack of experience with other functional
languages and platforms, we cannot really say whether or not it is
significantly better than, say, OCaml or Scala.
It is our opinion that, overall, our switch to a functional language in a commercial environment has been successful, and we
are convinced we will continue to see further benefit over the long
term.
190