Table of ContentsASDF ManualIntroductionASDF is another system definition facility: a tool for describing the sub-systems and files that comprise a system and for operating on these components in the right order so that they can be compiled, loaded, tested, etc. ASDF presents two faces: one for system implementors who need to be able to describe their systems and one for Lisp programmers who want to use those systems. See the getting started guide to learn how to use ASDF to load a system. This document describes how to write your own system definitions for ASDF. Defining systems with defsystemThis chapter describes how to use ASDF to define systems and develop software. First, some definitions:
The defsystem form
Systems are defined with the (defpackage hello-lisp-system (:use :common-lisp :asdf)) (in-package :hello-lisp-system) (defsystem "hello-lisp" :description "hello-lisp: a sample Lisp system." :version "0.2" :author "Joe User <joe@example.com>" :licence "Public Domain" :components ((:file "packages") (:file "macros" :depends-on ("packages")) (:file "hello" :depends-on ("macros")))) Some notes about this example:
A more involved example
Let's illustrate some more involved uses of (defsystem "foo" :version "1.0" :components ((:module "foo" :components ((:file "bar") (:file "baz") (:file "quux")) :perform (compile-op :after (op c) (do-something c)) :explain (compile-op :after (op c) (explain-something c))) (:file "blah")))
The :perform (compile-op :after (op c) (do-something c)) :explain (compile-op :after (op c) (explain-something c)) have the effect of (defmethod perform :after ((op compile-op) (c (eql ...))) (do-something c)) (defmethod explain :after ((op compile-op) (c (eql ...))) (explain-something c))
where The defsystem grammarsystem-definition := ( defsystem system-designator {option}* ) option := :components component-list | :pathname pathname | :default-component-class | :perform method-form | :explain method-form | :output-files method-form | :operation-done-p method-form | :depends-on ( {dependency-def}* ) | :serial [ t | nil ] | :weakly-depends-on ( {dependency-def}* ) | :in-order-to ( {dependency}+ ) | :class symbol component-list := ( {component-def}* ) component-def := component-name | ( component-type name {option}* ) component-type := :module | :file | :system | other-component-type dependency-def := component-name | ( :feature feature-name ) | ( :version component-name version-specifier) dependency := ( dependent-op {requirement}+ ) requirement := ( required-op {required-component}+ ) | ( feature feature-name ) dependent-op := operation-name required-op := operation-name component-name := string | symbol operation-name := symbol feature-name := symbol version-specifier := number[.number]* Serial dependencies
If the :serial t :components ((:file "a") (:file "b") (:file "c")) is equivalent to :components ((:file "a") (:file "b" :depends-on ("a")) (:file "c" :depends-on ("a" "b"))) Weakly-depends-on
The Notes:
Source location
The Instead, ASDF follows a hairy set of rules that are designed so that
If a system is being loaded for the first time, its top-level pathname will be set to:
If a system is being redefined, the top-level pathname will be
Other code in .asd files
Files containing defsystem forms are regular Lisp files that are executed by
System dependenciesWarning: As of July 20, 2009, this aspect of ASDF is in flux. We suggest not relying too heavily on the current syntax. There are several ways in which one system can depend on another:
The following tables illustrate how these dependencies work: Simple
:weakly-depends-on
Contingent
The object model of ASDF
ASDF is entirely object-oriented. Both a system's structure and the operations that can be performed on systems follow a protocol. ASDF is extensible to new operations and to new component types. This allows the addition of behaviours: for example, a new component could be added for Java JAR archives, and methods specialised on This chapter deals with components, the building blocks of a system, and operations, the actions that can be performed on a system. OperationsAn operation object of the appropriate type is instantiated whenever the user wants to do something with a system like:
Operations can be invoked directly, or examined to see what their effects would be without performing them. : document how! There are a bunch of methods specialised on operation and component type that actually do the grunt work. The operation object contains whatever state is relevant for this purpose (perhaps a list of visited nodes, for example) but primarily is a nice thing to specialise operation methods on and easier than having them all be EQL methods.
Operations are invoked on systems via X operate operation-class system &rest args &key (verbose t) version force &allow-other-keys function Operate does three things:
The traverse operation is wrapped in Note that dependencies may cause the operation to invoke other operations on the system or its components: the new operations will be created with the same initargs as the original one. X oos operation-class system &rest args &key force (verbose t) version &allow-other-keys function Short for operate on system and an alias for the operate function. Operate does three things:
The traverse operation is wrapped in Note that dependencies may cause the operation to invoke other operations on the system or its components: the new operations will be created with the same initargs as the original one. Predefined operations of ASDF
All the operations described in this section are in the (asdf:operate 'asdf:operation-name 'system-name {operation-options ...}) – Operation: compile-op &key proclamations This operation compiles the specified component. If proclamations are supplied, they will be proclaimed. This is a good place to specify optimization settings.
When creating a new component type, you should provide methods for
When – Operation: load-op &key proclamations This operation loads a system.
The default methods for – Operation: load-source-op This operation will load the source for the files in a module even if the source files have been compiled. Systems sometimes have knotty dependencies which require that sources are loaded before they can be compiled. This is how you do that. If you are creating a component type, you need to implement this operation - at least, where meaningful. – Operation: test-op
This operation should test the specified component. ASDF does not (currently) provide a method for returning a value from the test-op (or any other operation), so the implementor must ensure that executing
The default method for
The default dependency for
The default method for Creating new operations
ASDF was designed to be extensible in an object-oriented fashion. To teach ASDF new tricks, a programmer can implement the behaviour he wants by creating a subclass of
ASDF's pre-defined operations are in no way “privileged”, but it is requested that developers never use the
An operation must provide methods for the following generic functions when invoked with an object of type
Operations that print output should send that output to the standard CL stream Components
A component represents a source file or (recursively) a collection of components. A system is (roughly speaking) a top-level component that can be found via A system designator is a string or symbol and behaves just like any other component name (including with regard to the case conversion rules for component names). – Function: find-system system-designator &optional (error-p t)
Given a system designator,
To find and update systems,
When system definitions are loaded from .asd files, a new scratch package is created for them to load into, so that different systems do not overwrite each others operations. The user may also wish to (and is recommended to) include
The default value of Common attributes of components
All components, regardless of type, have the following attributes. All attributes except Name
A component name is a string or a symbol. If a symbol, its name is taken and lowercased. The name must be a suitable value for the
The lower-casing-symbols behaviour is unconventional, but was selected after some consideration. Observations suggest that the type of systems we want to support either have lowercase as customary case (Unix, Mac, windows) or silently convert lowercase to uppercase (lpns), so this makes more sense than attempting to use Version identifierThis optional attribute is used by the test-system-version operation. See Predefined operations of ASDF. For the default method of test-system-version, the version should be a string of integers separated by dots, for example '1.0.11'. Required featuresTraditionally defsystem users have used reader conditionals to include or exclude specific per-implementation files. This means that any single implementation cannot read the entire system, which becomes a problem if it doesn't wish to compile it, but instead for example to create an archive file containing all the sources, as it will omit to process the system-dependent sources for other systems.
Each component in an ASDF system may therefore specify features using the same syntax as #+ does, and it will (somehow) be ignored for certain operations unless the feature conditional is a member of DependenciesThis attribute specifies dependencies of the component on its siblings. It is optional but often necessary. There is an excitingly complicated relationship between the initarg and the method that you use to ask about dependencies Dependencies are between (operation component) pairs. In your initargs for the component, you can say :in-order-to ((compile-op (load-op "a" "b") (compile-op "c")) (load-op (load-op "foo"))) This means the following things:
The syntax is approximately (this-op {(other-op required-components)}+) required-components := component-name | (required-components required-components) component-name := string | (:version string minimum-version-object) Side note: This is on a par with what ACL defsystem does. mk-defsystem is less general: it has an implied dependency for all x, (load x) depends on (compile x)
and using a (compile b) depends on (load a) This is insufficient for e.g. the McCLIM system, which requires that all the files are loaded before any of them can be compiled ] End side note
In ASDF, the dependency information for a given component and operation can be queried using ((load-op "a") (load-op "b") (compile-op "c") ...)
pathnameThis attribute is optional and if absent will be inferred from the component's name, type (the subclass of source-file), and the location of its parent. The rules for this inference are: (for source-files)
(for modules)
Note that the DEFSYSTEM operator (used to create a “top-level” system) does additional processing to set the filesystem location of the top component in that system. This is detailed elsewhere, See Defining systems with defsystem. The answer to the frequently asked question “how do I create a system definition where all the source files have a .cl extension” is thus (defmethod source-file-type ((c cl-source-file) (s (eql (find-system 'my-sys)))) "cl") propertiesThis attribute is optional. Packaging systems often require information about files or systems in addition to that specified by ASDF's pre-defined component attributes. Programs that create vendor packages out of ASDF systems therefore have to create “placeholder” information to satisfy these systems. Sometimes the creator of an ASDF system may know the additional information and wish to provide it directly.
Pre-defined subclasses of component– Component: source-file A source file is any file that the system does not know how to generate from other components of the system. Note that this is not necessarily the same thing as “a file containing data that is typically fed to a compiler”. If a file is generated by some pre-processor stage (e.g. a .h file from .h.in by autoconf) then it is not, by this definition, a source file. Conversely, we might have a graphic file that cannot be automatically regenerated, or a proprietary shared library that we received as a binary: these do count as source files for our purposes. Subclasses of source-file exist for various languages. : describe these. – Component: module A module is a collection of sub-components. A module component has the following extra initargs:
The default operation knows how to traverse a module, so most operations will not need to provide methods specialised on modules.
– Component: system
A system is a module with a few extra attributes for documentation purposes; these are given elsewhere. See The defsystem grammar.
Users can create new classes for their systems: the default Creating new component typesNew component types are defined by subclassing one of the existing component classes and specializing methods on the new component class. : this should perhaps be explained more throughly, not only by example …
As an example, suppose we have some implementation-dependent functionality that we want to isolate in one subdirectory per Lisp implementation our system supports. We create a subclass of (defclass unportable-cl-source-file (cl-source-file) ())
A hypothetical function (defmethod component-pathname ((component unportable-cl-source-file)) (let ((pathname (call-next-method)) (name (string-downcase (system-dependent-dirname)))) (merge-pathnames (make-pathname :directory (list :relative name)) pathname)))
The new component type is used in a (defsystem :foo :components ((:file "packages") ... (:unportable-cl-source-file "threads" :depends-on ("packages" ...)) ... ) Controlling where ASDF saves compiled filesASDF includes code to control where the binaries files are places. The location depends on the the Common Lisp implementation and platform. The getting-started guides summaries the variables that control the mapping. The details for each variable can be found below: CustomizationX *centralize-lisp-binaries* variable If true, compiled lisp files without an explicit mapping (see *source-to-target-mappings*) will be placed in subdirectories of *default-toplevel-directory*. If false, then compiled lisp files without an explicitly mapping will be placed in subdirectories of their sources. X *default-toplevel-directory* variable If *centralize-lisp-binaries* is true, then compiled lisp files without an explicit mapping (see *source-to-target-mappings*) will be placed in subdirectories of *default-toplevel-directory*. X *include-per-user-information* variable When *centralize-lisp-binaries* is true this variable controls whether or not to customize the output directory based on the current user. It can be nil, t or a string. If it is nil (the default), then no additional information will be added to the output directory. If it is t, then the user's name (as taken from the return value of #'user-homedir-pathname) will be included into the centralized path (just before the lisp-implementation directory). Finally, if *include-per-user-information* is a string, then this string will be included in the output-directory. X *map-all-source-files* variable If true, then all subclasses of source-file will have their output locations mapped by ASDF-Binary-Locations. If nil (the default), then only subclasses of cl-source-file will be mapped. X *enable-asdf-binary-locations* variable If true, then compiled lisp files will be placed into a directory computed from the Lisp version, Operating System and computer archetecture. See implementation-specific-directory-name for details. X *source-to-target-mappings* variable The *source-to-target-mappings* variable specifies mappings from source to target. If the target is nil, then it means to not map the source to anything. I.e., to leave it as is. This has the effect of turning off ASDF-Binary-Locations for the given source directory. Examples: ;; compile everything in .../src and below into .../cmucl '(("/nfs/home/compbio/d95-bli/share/common-lisp/src/" "/nfs/home/compbio/d95-bli/lib/common-lisp/cmucl/")) ;; leave SBCL innards alone (SBCL specific) (list (list (princ-to-string (sb-ext:posix-getenv "SBCL_HOME")) nil)) X implementation-specific-directory-name function Return a name that can be used as a directory name that is unique to a Lisp implementation, Lisp implementation version, operating system, and hardware architecture. X output-files-for-system-and-operation system operation component source possible-paths function Returns the directory where the componets output files should be placed. This may depends on the system, the operation and the component. The ASDF default input and outputs are provided in the source and possible-paths parameters. Notes and Issues
Special variablesX *asdf-revision* variable No documentation found X *central-registry* variable A list of 'system directory designators' ASDF uses to find systems. A 'system directory designator' is a pathname or a function which evaluates to a pathname. For example: (setf asdf:*central-registry* (list '*default-pathname-defaults* #p"/home/me/cl/systems/" #p"/usr/share/common-lisp/systems/")) X *compile-file-warnings-behaviour* variable No documentation found X *compile-file-failure-behaviour* variable No documentation found X *resolve-symlinks* variable Determine whether or not ASDF resolves symlinks when defining systems.
Defaults to X *system-definition-search-functions* variable No documentation found Error handling
It is an error to define a system incorrectly: an implementation may detect this and signal a generalised instance of
Operations may go wrong (for example when source files contain errors). These are signalled using generalised instances of Compilation error and warning handling
ASDF checks for warnings and errors when a file is compiled. The variables Additional FunctionalityASDF includes several additional features that are generally useful for system definition and development. These include:
It's often handy to locate a file relative to some system. The system-relative-pathname function meets this need. It takes two arguments: the name of a system and a relative pathname. It returns a pathname built from the location of the system's source file and the relative pathname. For example > (asdf:system-relative-pathname 'cl-ppcre "regex.data") #P"/repository/other/cl-ppcre/regex.data" Getting the latest versionTo get the greatest and latest, you can: TODO listOutstanding spec questions, things to add
missing bits in implementation
we should do something inventive when processing a defsystem form, like take the list of kids and setf the slot to nil, then transfer children from old to new list as they're found
Inspirationmk-defsystem (defsystem-3.x)We aim to solve basically the same problems as mk-defsystem does. However, our architecture for extensibility better exploits CL language features (and is documented), and we intend to be portable rather than just widely-ported. No slight on the mk-defsystem authors and maintainers is intended here; that implementation has the unenviable task of supporting pre-ANSI implementations, which is no longer necessary.
The surface defsystem syntax of ASDF is more-or-less compatible with mk-defsystem, except that we do not support the The mk-defsystem code for topologically sorting a module's dependency list was very useful. defsystem-4 proposalMarco and Peter's proposal for defsystem 4 served as the driver for many of the features in here. Notable differences are:
If you want to find out what files an operation would create, ask the operation.
If you want to compile in a particular package, use an in-package form in that file (ilisp / SLIME will like you more if you do this anyway)
A system has a given version which can be used to check dependencies, but that's all. The defsystem 4 proposal tends to look more at the external features, whereas this one centres on a protocol for system introspection. kmp's "The Description of Large Systems", MIT AI Memo 801Available in updated for Common Lisp form on the web. In our implementation we borrow kmp's overall PROCESS-OPTIONS and concept to deal with creating component trees from defsystem surface syntax. [ this is not true right now, though it used to be and probably will be again soon ]. Footnotes
Copyright © 2001-2009 Daniel Barlow and contributors ASDF has an MIT style license Last updated 18 October 2009 at 22:44 |