Package Management in Common Lisp — the CLIM Way

Study the Past to Become Better in the Future

In the early 1990s, some of the brightest minds of the Common Lisp community at the time joined forces to develop a portable GUI framework known as the Common Lisp Interface Manager (CLIM). Whilst CLIM did not catch onto mainstream programming circles at the time, the framework’s specifications provides a masterclass in developing large, complex object-oriented systems in Common Lisp.

The project lives on today and is actively being developed in its McCLIM form and the astute reader is encouraged to get involved in this open source effort for two reasons:

• There are not many high quality, large code bases in Lisp that one can study for self-improvement
• CLIM is an excellent example of CLOS in action. Whilst initially difficult to get one’s head around due to the myriad of classes and functions, after spending a bit of time with CLIM, one is likely to have a revelation on how to best organise large and complex programs.

For today’s discussion, I want to touch upon CLIM’s approach to Package Management, as highlighted by Professor Robert Strandh on IRC, who is a senior computer scientist and primary developer of SICL, a new implementation of Common Lisp that stresses modularity and portability.

Without further ado, let’s get started!

The CLIM Approach to Package Management

Before we begin, I want to flag that we assume the reader has a reasonable understanding of packages in Common Lisp. Feel free to check out my introductory guide on packages, however today’s topic is at an intermediate level, so you may need some time for the ideas to sink in.

Moving along, consider the scenario where we are working with a large codebase. There are a two competing concerns when it comes to package management:

• It would be ideal if all the symbols in the codebase were in the same package, to avoid having to use package qualifiers or otherwise worry too much about which symbol is in which package
• However, having all the symbols in one package, across multiple files, makes it difficult to isolate the key symbols in a particular file. It is for this reason that the one package/one file approach has become quite popular as it makes it easy to note the exported (i.e. most important) symbols in a particular file.
• Whilst this can also be achieved by placing (export ...) forms within each file, it is generally preferable to list all the exported symbols in one place, namely within the package definition

CLIM’s approach resolves these issues and we will illustrate with an example. Let us say we have the following packages:

• An APP-MAIN package that represents the final package shared with users. Let’s say it exports symbols A, B, C, D and E.
• Three sub packages, let’s say SUB-PACKAGE-1, SUB-PACKAGE-2 and SUB-PACKAGE-3. Say all three sub packages use each others symbols, but in terms of exported symbols, SUB-PACKAGE-1 exports A and B, SUB-PACKAGE-2 exports C and SUB-PACKAGE-3 exports D and E

The CLIM approach is as follows:

• For the overarching main package (APP-MAIN), export all symbols as follows

(defpackage :app-main
  (:use)
  (:export #:A #:B #:C #:D #:E))

• For each of the sub packages, :use :app-main and export the relevant symbols. For example:

(defpackage :sub-package-1
  (:use :app-main)
  (:export #:A #:B))

The end result is that

• All of your sub packages have full access to all of the exported symbols of APP-MAIN
• There is a clear distinction between the external and internal symbols of APP-MAIN. A sub package can only access the external symbols of APP-MAIN without a package qualifier and must qualify any internal symbols not interned in its package
• It is very easy to distinguish between the external and internal symbols of any given sub package

The key trick that CLIM does is, using the above as an example, the inverted use of the APP-MAIN package:

• Typically one imports exported symbols from the sub packages into the main package, but here it is the other way around
• The sub packages export symbols (as appropriate) that they inherit from the main package
• However, at the same time, there is no circular dependency between the sub packages and the main package as we are not importing symbols from the sub packages back to the main package

This all may be a bit confusing, but have a read through the above and hopefully you find it a useful trick to use in your programs!

Thanks for reading!