Introduction to ASDF

The De Facto Standard Build Facility for Common Lisp

Today, we will do a basic introduction of ASDF, a clever abbreviation for Another System Definition Facility, and the de facto build facility for Common Lisp.

1: Installing ASDF

Ref: Official documentation on loading ASDF

ASDF generally comes installed with most Common Lisp implementations, so nothing to do here. After all, we did say it was the de facto build facility for Common Lisp :-)

2: Configuring ASDF

Ref: Official documentation on configuring ASDF

ASDF works by loading .asd files. By default, it will search for these files in ~/common-lisp/ and ~/.local/share/common-lisp/source/. We can add our own folders to the above list by:

• Create the directory ~/.config/common-lisp/source-registry.conf.d/
• Inside this directory, create a file with any filename but with an ending of .conf, for example 50-user-lisp.conf
• Within that file, add the following to instruct ASDF to recursively scan all subdirectories within the provided path (replace “/home/user/lisp/” in the below with your chosen folder path):

(:tree “/home/user/lisp/”)

3: Defining an ASDF System

Ref: Official documentation on defining ASDF systems

An ASDF system definition is an .asd file that tells ASDF what are the dependencies of the system and how to load its files (i.e. in what order).

Below is an example. It is mostly self-explanatory, with the following notes:

• :depends-on is where you list your external dependencies
• :serial t loads all files sequentially in the order they appear in the system definition
• We don’t add a .lisp to the end of the filenames — this is automatically assumed by ASDF. For example, It would be an error if you wrote utilities.lisp as ASDF will then look for utilities.lisp.lisp and not find it
• ASDF will compile all files, so no need to refer to your .fasl files (FASL is the file extension for compiled lisp files).
• :module is used for referencing files within subdirectories (subfolder mod in our example) — you can see the nested recursion at play — we again refer to :components with :module and thus we can nested many folders down

(defsystem "my-system"
  :description "My System Description"
  :version "0.0.1"
  :author "Firstname Lastname <email@email.com>"
  :license "GPL"
  :depends-on ("bknr.datastore" "bknr.indices" "bknr.impex")
  :serial t
  :components
        ((:file "utilities")
        (:module "mod"
                 :components ((:file "another-file")))
        (:file "another-file-2")))

To build on the above example, it is recommended to rely on :depends-on to explicitly state dependencies of each file vs. the simpler :serial t option that specifies loading files in order. Below is an example.

(defsystem "foobar"
  :depends-on ("alexandria" "trivia" "trivia.ppcre")
  :components
  ((:file "package")
   (:file "utils" :depends-on ("package"))
   (:file "foo" :depends-on ("utils"))
   (:file "bar" :depends-on ("utils"))
   (:file "foobar" :depends-on ("foo" "bar"))))

4: A More Complicated Example

A more involved example is below (refer to the bolded parts in particular). The key points here:

• :depends-on always searches for files in its current set of components
• Thus, we cannot write (:file “another-file” :depends-on (“utilities”)) in the below because there is no “utilities.lisp” within the subfolder (it is within the folder above)
• Instead, we add the :depends-on clause within the :module clause and not within the :file clause
• Similarly, we cannot write (:file "another-file-2" :depends-on ("another-file")), as “another-file.lisp” is not within this directory, but rather within the subdirectory
• Instead, we can refer to the module directly with (:file "another-file-2" :depends-on ("mod"))

(defsystem "my-system"
  :description "My System Description"
  :version "0.0.1"
  :author "Firstname Lastname <email@email.com>"
  :license "GPL"
  :depends-on ("bknr.datastore" "bknr.indices" "bknr.impex")
  :serial t
  :components
        ((:file "utilities")
        (:module "mod"
                 :depends-on ("utilities")                 
                 :components ((:file "another-file")))
        (:file "another-file-2" :depends-on ("mod"))))

Finally, a useful guide on defining ASDF systems was written by Mario Mommer and can be found here.

5: Loading an ASDF System

Loading an ASDF system is as simple as entering the following command in your REPL (replacing “my-system” with the name of the system you want to load):

(asdf:load-system "my-system")

6: Miscellaneous Notes

To conclude this guide, below are some (hopefully useful) notes I picked up.

Scanning Folders & Duplicates

ASDF will recursively scan through the folder paths in its configuration settings, so you just need to ensure your .asd files are saved somewhere in the directories ASDF scans.

In addition, ASDF loads the first system definition encountered for a given name and any system definition with the same name found thereafter in the scan process are not loaded.

EDIT: THANKS TO DAEWOK ON R/REDDIT for the below clarification

This isn’t guaranteed. ASDF claims the results are undefined if a system could come from multiple files. Currently, it takes the one with the shortest name string (likely to be the first one found, but not necessarily). In the future, it may signal an error.

File Compilation

To force ASDF to recompile all your files, pass in a :force argument:

(asdf:load-system "my-system" :force t)

Bypassing Quicklisp

We can bypass Quicklisp by ensuring we have all the required dependencies saved within the ASDF folder paths. Note that a dependency may have its own dependencies, so this is a recursive process.

Alternatively, use Quicklisp to download all the dependencies then reference the dependencies you need in your system definition. By doing so, you no longer need to call QuickLisp or other library managers within your program as your ASD files will directly load the required dependencies.

At the time of writing, Quicklisp was saving the files in the following (this was on a Mac). Quicklisp installs this path into the ASDF configuration, but you may wish to copy your dependency files from here and into your main Common Lisp project folder (assuming that folder has also been installed into your ASDF configuration).

/Users/username/.quicklisp/dists/quicklisp/software/