The second annual Clojure/conj came to a close last November. I was able to attend and wanted to share some impressions of the conference in general and a couple of the more inspiring talks in particular. I’ll be happy to further detail any of the talks for any interested parties.
Chief among Clojure’s aims are performance, programmer productivity and simplified concurrent programming. Clojure is a functional language that looks beyond Moore’s law to Amdahl’s law, predicting that the increasing number of compute tasks that can benefit from multi-core processing will require the adoption of a programming model that fully embraces concurrency.
A pragmatic, general-purpose language, Clojure is finding purchase in a wide array of pursuits. Being that Clojure is perhaps best known for its success with Big Data and concurrency tasks, I was impressed with the diversity of the topics; the conference was a series of success stories in such areas as machine intelligence, performance engineering, genome processing, web development, and live music coding.
What surprised me about the attendees was how many of them, myself included, were drawn to Clojure despite having little experience with it. It was a pretty even split between Rubyists and Java programmers, with a substantial Lisp contingent intermixed.
As the conference was single-track, I was able to attend every talk. While they were, with a very few exceptions, all very interesting, a couple in particular held immediate practical interest.
Recently acquired by Twitter via BackType, Cascalog is a Clojure-based query language that bills itself as “data processing on Hadoop without the hassle”. Cascalog emphasizes function composition to make complicated social-graph-style queries easy. Its creator Nathan Marz demonstrated a variety of querying tasks to answer questions like “Which Twitter ‘following’ relationships are between two users that each have more than 2 following relationships” that can be awkward in the SQL world. Using Cascalog, it looks very straight-forward (and shorter than the previous sentence in this paragraph)…
(let [many-follows (<- [?person] (follows ?person _) (c/count ?count) (> ?count 2))] (?<- (stdout) [?person1 ?person2] (many-follows ?person1) (many-follows ?person2) (follows ?person1 ?person2)))
To break this down, the let form defines many-follows as a function that takes a person as input, retrieves a count of their followers, and ensures that count is greater than 2.
Then, with stdout as our output, and two arbitrary persons as input, we ensure each person qualifies via the many-follows function and that person1 follows person2.
The above is computed over a series of map/reduce jobs that retrieve the necessary data, break it into chunks, farm those chunks out to nodes, and recombine the output of those nodes.
Chas Emerick’s talk, “Modeling the world probabilistically using Bayesian networks in Clojure” was particularly interesting. Chas described his project Raposo, a Clojure library for Bayesian inference and modeling he’s preparing for release. His use case for Raposo requires document analysis on large bodies of documents with a constantly changing set of ad-hoc formats, such as SEC filings, which need to be parsed into structured data for extraction. FM has clearly done much work in this area and this talk got me very interested in learning more about it.
Of less practical value, but very cool nonetheless, was Sam Aaron’s demonstration of Overtone, a live music coding platform. Typing Clojure forms using the Overtone syntax into Emacs and executing them let Sam programmatically create music on the fly. He used it to assign controls to a Monome, a wired box with lit buttons, allowing him to produce beats, synth sounds, and samples by pressing physical buttons. His impromptu jam session got a standing ovation and the lion’s share of the post-conference press on Twitter.
Also deserving of a shoutout is Daniel Spiewak, whose talk on data structures was so animated, emphatic and incisive that it kept the entire room awake during a talk on data structures. In fact, calling it a ‘talk’ is probably under-selling it; it was more of divinely-inspired Robin-Williams-style conniption fit.
A few of the more notable names in attendance were Rich Hickey himself; Daniel P. Friedman, author of The Little Schemer and a host of other Lisp-related texts; William Byrd, co-author with Prof. Friedman of The Reasoned Schemer; “Uncle Bob” Martin, object-oriented author and speaker; Phil Bagwell, inventor of the VList data structure; Ola Bini of JRuby fame, and a large and boisterous subset of the Clojure/core team. There were certainly others I’m not hip enough to recognize.
That’s my run-through of the more inspiring talks at Clojure/conj 2011. My side-project TODO list is now a few items larger, and I’m already tinkering with the domestic schedule to see if I can attend Clojure/West in San Jose in mid-March 2012 — one week after PyCon 2012 and 15 minutes away, incidentally.
Some links for more information…