cynojure

In the course of my recent work, integrating various java libraries with clojure, I spent a lot of time going back and forth between emacs and firefox, just to read java api documentation. I even tried loading up the "All Classes" index from the java api docs in an emacs/w3m buffer and isearch-ing through it. It was a far from optimal experience, and I decided to hack something using clojure and slime.

Bill Clementson had posted a great intro to javadoc and clojure/slime, but I was looking for something that wouldnt' require me to leave emacs just to read documentation, and that too java api docs.

Ideally, I wanted to be able able to put the cursor on any java class symbol in a clojure-mode buffer in emacs, execute M-x slime-javadoc, and have the api documentation popup in an adjacent emacs window.

Turns out the hardest part of implementing this was in resolving an unqualified class name to its fully qualified form, before looking up its documentation path. For example,

(ns example
  (:import (java.util UUID)))

(defn uuid []
  (.toString (UUID/randomUUID)))

Looking up the javadoc on UUID should really be on java.util.UUID, which clojure knows. So, the core of the hack is a clojure function, which takes the symbol at point ("UUID" in this case), and the current namespace ("example") and returns a fully qualified class name "java.util.UUID". That function, implemented as a slime command looks like the following:

(swank.commands/defslimefn resolve-symbol [sym ns-name]
 (when-let [the-class (ns-resolve (find-ns (symbol ns-name))
                                  (symbol sym))]
   (.getName the-class)))

Now all we need is emacs to clean up the symbol at point, and call this function via slime.

(defun slime-javadoc (symbol-name)
  "Get JavaDoc documentation on Java class at point."
  (interactive (list (slime-read-symbol-name "JavaDoc info for: ")))
  (or symbol-name (error "No symbol given"))
  (let ((class-name (slime-eval
                     (list 'resolve-symbol
                           (first
                            (split-string (trim-trailing-dot symbol-name)
                                          "/"))
                           (slime-current-package)))))
    (if class-name
        (browse-url
         (concat (javadoc-root class-name)
                 (subst-char-in-string ?$
                                       ?.
                                       (subst-char-in-string ?.
                                                             ?/
                                                             class-name))
                 ".html"))
      (message "No javadoc found for %s" symbol-name))))

The `javadoc-root' function is just a way to dispatch to different javadoc paths (local or online), based on the package names. For example, by defining:

(setq javadoc-alist
  '(("^\\(java[x]?\.\\|org\.ietf\.\\|org\.omg\.\\|org\.w3c\.\\|org\.xml\.\\)" .
     "file://opt/java/docs/api/")
    ("^org\.jets3t" . "file://opt/java/jets3t-0.7.0/api-docs/")
    ("^com\.xerox\.amazonws" . "file://opt/java/typica-1.5.2a/apidocs/")
    ("^org\.mortbay" . "http://www.mortbay.org/apidocs/"))

Now, emacs/w3m will load the right documentation, based on the class package prefix.

The code is available on github.

Unity.

Loading mentions Retweet

I've been using Amazon EC2, S3, and SQS services in various projects at work. For one project, I needed to build a little tool that integrated SQS and S3 services, and I decided to build it in Clojure. In the course of building the tool, I learned a bit about the various open source Java libraries for working with Amazon AWS, and the issues with integrating them with Clojure.

The libraries I used are:

• jets3t -- S3 interface
• typica -- SQS and EC2 interface
• Apache commons-httpclient and associated dependencies
• postgres jdbc driver

Having been away from Java development for some time now, I'd forgotten about the convoluted mess of Classpaths and Class loaders. I had a "crash" refresher course in the last week. And, it's all still a bloody mess. I did come across one-jar, which made it pretty easy to package up the various jar dependencies into a single deliverable jar file.

In the end, the tool worked out well and served its purpose. Plus, it was rewarding to be able to write code like this:

(defun process-loop [:key
                     aws-access-key
                     aws-secret-key
                     geo-db-path
                     [message-count 5]
                     [sleep 5000]
                     sqs-queue]
  (with-geo-service (geo-get-service :path geo-db-path)
    (with-sqs-queue (sqs-get-queue aws-access-key aws-secret-key sqs-queue)
      (with-connection *db*
        (let [process (fn [msgs]
                        (if (empty? msgs)
                          (Thread/sleep sleep)
                          (try
                           (transaction
                             (doseq [msg msgs] (process-message msg)))
                           (doseq [msg msgs] (sqs-delete-message msg))
                           (catch java.sql.BatchUpdateException ex
                             (println "* process error:"
                                      (.getNextException ex))))))]
          (while true
            (try
             (process (sqs-receive-messages message-count))
             (catch Exception ex
               (println "* error:" ex)))))))))

and know it's running on the JVM.

I've committed some of the code for the Amazon integration into the cynojure git repository, in case anyone else finds it useful.

Harmony.

Loading mentions Retweet

I finally got around to creating a repository for some of the Clojure code I'm developing on github. It's quite spare right now; just the Common Lisp like `defun' macro, a few odd helper functions, and the SQL query syntax I'd written about before.
 
The repository is online at http://github.com/kriyative/cynojure/tree/master.
 
I'm not all that happy with the SQL macros, but haven't had a whole lot of time to play with it. I've been too distracted with real work.

Loading mentions Retweet

On one of the Common Lisp projects I worked on (a very typical Web app), I used the CLSQL library to integrate with a Postgres database. CLSQL is itself an opensource version of the SQL package which is part of Lispworks. One of the more interesting features of this library was an s-expression based query language, which emitted SQL query strings behind the scenes. So, for example:

(select [count 1] 
        :from '([person]) 
        :where [and [= [role] +role-programmer+] 
                    [< [age] 30]))

would emit the following SQL (and actually runs the query):

SELECT COUNT(1) FROM person WHERE (role = 1 AND age < 30)

Nice. Just being able to write the WHERE clause as a readable s-expression makes CLSQL worthwhile. Of course, it does much more than just that.

Anyway, Clojure has a library in clojure-contrib, for integrating with SQL databases using JDBC. But, as you can see from the examples on that wiki page, SELECT, INSERT and UPDATE more or less require hand built SQL query strings. So, I figured I'd try and emulate the CLSQL approach using Clojure macrology. Here's what I was able to do:

(sql-select-stmt "count(1)"
                 :from '(person)
                 :where (sql-and (sql-= 'role +role-programmer+)
                                 (sql-< 'age 30)))

Not as concise as CLSQL (the "sql-" prefix interrupts the flow), but perhaps the best we can do, considering Clojure doesn't have programmable reader macros (at least not without serious hackery). CLSQL cleverly uses reader macros to do interesting things with expressions which are inside square brackets, like turning [or ...] into (sql-or ...), and [a-table-name] into "a_table_name" etc.

Incidentally, `sql-select-stmt' doesn't run the query, just generates the query string, which can be passed to `with-results', like so:

(with-results rs
    (sql-select-stmt "count(1)" :from from :where where)
  (:count (first rs)))

The code for the sql macros is a bit long to insert here, so I'm thinking I'll create a `cynojure' repo on github, and put all the various bits of clojure code I create in there. I'll post about that separately.

Peace.

Loading mentions Retweet

I like the keyword args support in Common Lisp. So, when I learned that clojure didn't support keyword args in the base language, I was a bit disappointed. But, with clojure's support for destructuring and macros, it should be easy to build keyword args ... right?

Rich Hickey posted a simple example on the clojure mailing list, which I adapted slightly to my taste:

(defmacro defun [sym args & body]
  (let [[pargs [_ & kargs]] (split-with (fn [x] (not (= x '&key))) args)
        gkeys (gensym "gkeys__")
        letk (fn [k]
               (let [[nm val] (if (vector? k) k [k])
                     kname (keyword (name nm))]
                 `(~nm (or (~gkeys ~kname) ~val))))]
    `(defn ~sym [~@pargs & k#]
       (let [~gkeys (apply hash-map k#)
             ~@(apply concat (map letk kargs))]
         ~@body))))

After which:

(defun foo [a b &key c [d 5]]
  [a b c d])

user> (foo 1 2 :c 10)
[1 2 10 5]

Joy.

Loading mentions Retweet

Welcome to cynojure -- a blog about clojure -- a modern Lisp implementation.

I've been hacking Lisp, professionally and as a hobbyist, for about eighteen years now. Starting with elisp and Scheme, and then Common Lisp for the past eight years. At the risk of importing my CV into this post, I've worked in Smalltalk, Objective-C, Java; even some Cadence SKILL. More recently, I got to work on a reasonably extensive project in Erlang. Before you write me off as a mono-lingual snob, let me assure you I've had a fair amount of experience with C and its derivative family of languages as well. I just happen to be a bigger fan of Lisp, and functional programming in general; which leads me to clojure.

I first heard about clojure a year back, and was very impressed with the design motivations of its creator, Rich Hickey. A pure functional Lisp with immutable data structures, STM, actors ... and, efficient Java interop! Sounded too good, so I had to check it out. Since then, I've been following the development of the language and its libraries, read the Programming Clojure book, and played with it enough to be somewhat productive.

And, this week (May 4, 2009) Rich Hickey formally announced the release of Clojure 1.0, which is great news. Congratulations to Rich and all the other contributors. To mark this major milestone in Clojure's history, and to add my own little bit to its growth, I decided to blog about my experiences with clojure as I learn and apply it. Hopefully, my experiences will be of some use or entertainment to others :)

Happy hacking.

PS: About the name - cynojure is a play (in keeping with clojure tradition) on the word cynosure, which according to WordNet has the following meanings:

cynosure

something that provides guidance (as Polaris guides mariners)
something that strongly attracts attention and admiration

Perhaps a bit grandiose, but I thought it was suitable.

Loading mentions Retweet