sbcl.sbcl/doc/manual/intro.texinfo

@c Generated by the sb-manual contrib. Do not edit.

@node introduction
@chapter Introduction

@menu
* ANSI Conformance: ansi conformance.
* Extensions: extensions.
* Idiosyncrasies: idiosyncrasies.
* Development Tools: development tools.
* More SBCL Information: more sbcl information.
* More Common Lisp Information: more common lisp information.
* History and Implementation of SBCL: history and implementation of sbcl.
@end menu

SBCL is a mostly-conforming implementation of the ANSI Common Lisp
standard. This manual focuses on behavior which is specific to SBCL,
not on behavior which is common to all implementations of ANSI Common
Lisp.

@node ansi conformance
@section ANSI Conformance

Essentially every type of non-conformance is considered a bug. (The
exceptions involve internal inconsistencies in the standard.) See
@ref{reporting bugs}.

@itemize
@item @code{prog2} returns the primary value of its second form, as
  specified in the @emph{Arguments and Values} section of the
  specification for that operator, not that of its first form, as
  specified in the @emph{Description}.

@item The @code{string} type is considered to be the union of all types
  @code{(array c (size))} for all non-@code{nil} subtypes @code{c} of
   @code{character}, excluding arrays specialized to the empty
   type.

@item The @code{:order} long form option in @code{define-method-combination} method
  group specifiers accepts the value @code{nil} as well as
  @code{:most-specific-first} and @code{:most-specific-last}, in order to allow
  programmers to declare that the order of methods playing that role
  in the method combination does not matter.
@end itemize

@node extensions
@section Extensions

SBCL comes with numerous extensions, some in core and some in modules
loadable with @code{require}. Unfortunately, not all of these extensions
have proper documentation yet.

@itemize
@item @strong{System Definition Tool:} ASDF is a flexible and popular
  protocol-oriented system definition tool by Daniel Barlow.

@item @strong{Foreign Function Interface:} The @code{sb-alien} package allows
  interfacing with C-code, loading shared object files, etc. See
  @ref{foreign function interface}.

@ref{sb grovel} can be used to partially automate generation of
foreign function interface definitions.

@item @strong{Recursive Event Loop:} SBCL provides a recursive event
  loop (@code{serve-event}) for doing non-blocking IO on multiple streams
  without using threads.

@item @strong{Timeouts and Deadlines:} SBCL allows restricting the execution
  time of individual operations or parts of a computation using
  @code{:timeout} arguments to certain blocking operations, synchronous
  timeouts and asynchronous timeouts. The latter two affect operations
  without explicit timeout support (such as standard functions and
  macros). See @ref{timeouts and deadlines}.

@item @strong{Metaobject Protocol:} The @code{sb-mop} package provides an
  implementation of the metaobject protocol for the Common Lisp
  Object System as described in @emph{The Art of the Metaobject Protocol}
  by Kiczales et al.

@item @strong{Extensible Sequences:} SBCL allows users to define subclasses
  of the @code{sequence} class. See @ref{extensible sequences}.

@item @strong{Native Threads:} SBCL has native threads on numerous platforms,
  capable of taking advantage of SMP on multiprocessor machines. See
  @ref{threading}.

@item @strong{Network Interface:} The @code{sb-bsd-sockets} module is a low-level
  networking interface, providing both TCP and UDP sockets. See
  @ref{networking}.

@item @strong{Introspective Facilities:} The @ref{sb introspect} module offers
  numerous introspective extensions, including access to function
  lambda-lists and a cross referencing facility.

@item @strong{Operating System Interface:} The @code{sb-ext} package contains a
  number of functions for running external processes, accessing
  environment variables, etc.

The @ref{sb posix} module provides a lispy interface to standard
POSIX facilities.

@item @strong{Extensible Streams:} The package @code{sb-gray} provides an
  implementation of @ref{gray streams}.

The @ref{sb simple streams} module is an implementation of the Simple
Streams API proposed by Franz Inc.

@item @strong{Profiling:} The @code{sb-profile} package provides an exact,
  per-function @ref{deterministic profiler}.

The @code{sb-sprof} module is SBCL's @ref{statistical profiler}, capable
of call-graph generation and instruction level profiling, which
also supports allocation profiling.

@item @strong{Customization Hooks:} SBCL contains a number of extra-standard
  customization hooks that can be used to tweak the behaviour of the
  system. See @ref{customization hooks for users}.

@item @strong{sb-aclrepl:} The @ref{sb aclrepl} module provides an Allegro-style
  toplevel for SBCL, as an alternative to the classic CMUCL-style
  one.

@item @strong{CLTL2 Compatibility Layer:} The SB-CLTL2 module provides
  @code{sb-cltl2:compiler-let} and environment access functionality
  described in @emph{Common Lisp The Language, 2nd Edition} which were
  removed from the language during the ANSI standardization process.

@item @strong{Executable Delivery:} The @code{:executable} argument to
  @code{sb-ext:save-lisp-and-die} can produce a "standalone" executable
  containing both an image of the current Lisp session and an SBCL
  runtime.

@item @strong{Bitwise Rotation:} The @ref{sb rotate byte} module provides an
  efficient primitive for bitwise rotation of integers, an operation
  required by e.g. numerous cryptographic algorithms but not
  available as a primitive in ANSI Common Lisp.

@item @strong{Test Harness:} The @code{sb-rt} module is a simple yet attractive
  regression and unit-test framework.

@item @strong{MD5 Sums:} The @ref{sb md5} module provides an implementation of the
  MD5 message digest algorithm for Common Lisp, using the
  @ref{modular arithmetic} optimizations provided by SBCL.
@end itemize

@node idiosyncrasies
@section Idiosyncrasies

@menu
* Declarations: declarations.
* FASL format: fasl format.
* Compiler-only Implementation: compiler only implementation.
* Defining Constants: defining constants.
* Style Warnings: style warnings.
@end menu

The information in this section describes some of the ways that SBCL
deals with choices that the ANSI standard leaves to the
implementation.

@node declarations
@subsection Declarations

Declarations are generally treated as assertions. This general
principle, and its implications, and the bugs which still keep the
compiler from quite satisfying this principle, are discussed in
@ref{declarations as assertions}.

@node fasl format
@subsection FASL format

SBCL fasl-format is binary compatible only with the exact SBCL version
it was generated with. While this is obviously suboptimal, it has
proven more robust than trying to maintain fasl compatibility across
versions: accidentally breaking things is far too easy, and can lead
to hard to diagnose bugs.

The following snippet handles fasl recompilation automatically for
ASDF-based systems, and makes a good candidate for inclusion in the
user or system initialization file (see @ref{initialization files}).

@example
(require :asdf)

;;; If a fasl was stale, try to recompile and load (once).
(defmethod asdf:perform :around ((o asdf:load-op)
                                 (c asdf:cl-source-file))
   (handler-case (call-next-method o c)
      ;; If a fasl was stale, try to recompile and load (once).
      (sb-ext:invalid-fasl ()
         (asdf:perform (make-instance 'asdf:compile-op) c)
         (call-next-method))))
@end example

@node compiler only implementation
@subsection Compiler-only Implementation

SBCL is essentially a compiler-only implementation of Common Lisp.
That is, for all but a few special cases, @code{eval} creates a lambda
expression, calls @code{compile} on the lambda expression to create a
compiled function, and then calls @code{funcall} on the resulting function
object. A more traditional interpreter is also available on default
builds; it is usually only called internally. This is explicitly
allowed by the ANSI standard but leads to some oddities; e.g. at
default settings, @code{functionp} and @code{compiled-function-p} are equivalent,
and they collapse into the same function when SBCL is built without
the interpreter.

@node defining constants
@subsection Defining Constants

SBCL is quite strict about ANSI's definition of @code{defconstant}.
ANSI says that doing @code{defconstant} of the same symbol more than once
is undefined unless the new value is @code{eql} to the old value.
Conforming to this specification is a nuisance when the "constant"
value is only constant under some weaker test like @code{string=} or @code{equal}.

It's especially annoying because, in SBCL, @code{defconstant} takes effect
not only at load time but also at compile time, so that just
compiling and loading reasonable code like

@example
(defconstant +foobyte+ '(1 4))
@end example

runs into this undefined behavior. Many implementations of Common
Lisp try to help the programmer around this annoyance by silently
accepting the undefined code and trying to do what the programmer
probably meant.

SBCL instead treats the undefined behavior as an error. Often such
code can be rewritten in portable ANSI Common Lisp which has the
desired behavior. E.g., the code above can be given an exactly
defined meaning by replacing @code{defconstant} either with @code{defparameter} or
with a customized macro which does the right thing, e.g.

@example
(defmacro define-constant (name value &optional doc)
  `(defconstant ,name (if (boundp ',name) (symbol-value ',name) ,value)
                      ,@@(when doc (list doc))))
@end example

or possibly along the lines of the @code{sb-int:defconstant-eqx} macro used
internally in the implementation of SBCL itself. In circumstances
where this is not appropriate, the programmer can handle the
condition type @code{sb-ext:defconstant-uneql} and choose either the
@code{continue} restart or @code{abort} restart as appropriate.

@node style warnings
@subsection Style Warnings

SBCL gives style warnings about various kinds of perfectly legal code,
e.g.

@itemize
@item multiple @code{defun}s of the same symbol in different units;

@item special variables not named in the conventional @code{*foo*} style, and
  lexical variables unconventionally named in the @code{*foo*} style.
@end itemize

This causes friction with people who point out that other ways of
organizing code (especially avoiding the use of @code{defgeneric}) are just
as aesthetically stylish. However, these warnings should be read not
as @emph{warning, bad aesthetics detected, you have no style} but as
@emph{warning, this style keeps the compiler from understanding the code
as well as you might like}. That is, unless the compiler warns about
such conditions, there's no way for the compiler to warn about some
programming errors which would otherwise be easy to
overlook. (Related bug: The warning about multiple @code{defun}s is
pointlessly annoying when you compile and then load a function
containing @code{defun} wrapped in @code{eval-when}, and ideally should be
suppressed in that case, but still isn't as of SBCL 0.7.6.)

@node development tools
@section Development Tools

@menu
* Editor Integration: editor integration.
* Language Reference: language reference.
* Generating Executables: generating executables.
@end menu

@node editor integration
@subsection Editor Integration

Though SBCL can be used running "bare", the recommended mode of
development is with an editor connected to SBCL, supporting not
only basic lisp editing (paren-matching, etc), but providing among
other features an integrated debugger, interactive compilation, and
automated documentation lookup.

Currently @emph{SLIME} (Superior Lisp Interaction Mode for Emacs)
together with Emacs is recommended for use with SBCL, though other
options exist as well. Historically, the ILISP package at
@url{http://ilisp.cons.org/} provided similar functionality, but it does
not support modern SBCL versions.

SLIME can be downloaded from @url{https://slime.common-lisp.dev/}.

@node language reference
@subsection Language Reference

@emph{CLHS} (Common Lisp HyperSpec) is a hypertext version of the ANSI
standard, made freely available by LispWorks -- an invaluable
reference.

See @url{https://www.lispworks.com/documentation/HyperSpec/Front/index.htm}.

@node generating executables
@subsection Generating Executables

SBCL can generate stand-alone executables. The generated executables
include the SBCL runtime itself, so no restrictions are placed on
program functionality. For example, a deployed program can call
@code{compile} and @code{load}, which requires the compiler to be present in the
executable. For further information, @code{sb-ext:save-lisp-and-die}.

@node more sbcl information
@section More SBCL Information

@menu
* SBCL Homepage: sbcl homepage.
* Online Documentation: online documentation.
* Additional Documentation Files: additional documentation files.
* Internals Documentation: internals documentation.
@end menu

@node sbcl homepage
@subsection SBCL Homepage

The SBCL website at @url{http://www.sbcl.org/} has some general
information, plus links to mailing lists devoted to SBCL, and to
archives of these mailing lists. Subscribing to the mailing lists
@code{sbcl-help} and @code{sbcl-announce} is recommended: both are fairly
low-volume, and help you keep abreast with SBCL development.

@node online documentation
@subsection Online Documentation

Documentation for non-ANSI extensions for various commands is
available online from the SBCL executable itself. The extensions for
functions which have their own command prompts (e.g. the debugger,
and @code{inspect}) are documented in text available by typing @code{help} at
their command prompts. The extensions for functions which don't have
their own command prompt (such as @code{trace}) are described in their
documentation strings, unless your SBCL was compiled with an option
not to include documentation strings, in which case the
documentation strings are only readable in the source code.

@node additional documentation files
@subsection Additional Documentation Files

Besides this user manual both SBCL source and binary distributions
include some other SBCL-specific documentation files, which should
be installed along with this manual on your system, e.g. in
@code{/usr/local/share/doc/sbcl/}.

@itemize
@item @code{copying}: Licence and copyright summary.

@item @code{credits}: Authorship information on various parts of SBCL.

@item @code{install}: Covers installing SBCL from both source and binary
   distributions on your system, and also has some installation
   related troubleshooting information.

@item @code{news}: Summarizes changes between various SBCL versions.
@end itemize

@node internals documentation
@subsection Internals Documentation

If you're interested in the development of the SBCL system itself,
then subscribing to
@uref{mailto:sbcl-devel@@lists.sourceforge.net, sbcl-devel@@lists.sourceforge.net}
is a good idea.

SBCL internals documentation -- besides comments in the source -- is
available in the Web Archive:

@url{https://web.archive.org/web/20120814000933/http://sbcl-internals.cliki.net/index}.

Some low-level information describing the programming details of the
conversion from CMUCL to SBCL is available in the
@code{doc/FOR-CMUCL-DEVELOPERS} file.

@node more common lisp information
@section More Common Lisp Information

@menu
* Internet Community: internet community.
* Third-party Libraries: third party libraries.
* Common Lisp Books: common lisp books.
@end menu

@node internet community
@subsection Internet Community

IRC channels on @url{https://libera.chat/}:

@itemize
@item @code{#common-lisp}: "Common Lisp, the #1=(programmable . #1#)
  programming language"

@item @code{#lispcafe}: "The Lisp Café; sit down, have a drink, chat about
  anything, and enjoy your stay. | @url{https://www.cliki.net/lispcafe} |
  Be insuperable to each other".

@item @code{#sbcl}: "Steel Bank Common Lisp Dev Hangout"
@end itemize

You can use @url{https://web.libera.chat} or a normal IRC client.

Also, see @url{https://www.reddit.com/r/Common_Lisp/}, as well as
@url{https://www.lisp.org} and @url{https://cliki.net}, which contain
numerous pointers places in the net where lispers talks shop.

@node third party libraries
@subsection Third-party Libraries

For a wealth of information about free Common Lisp libraries and tools
we recommend checking out @emph{CLiki}: @url{https://cliki.net/}.

The most popular library manager is Quicklisp:
@url{https://www.quicklisp.org/beta/}.

@node common lisp books
@subsection Common Lisp Books

If you're not a programmer and you're trying to learn, many
introductory Lisp books are available. However, we don't have any
standout favorites.

If you are an experienced programmer in other languages but need to
learn about Common Lisp, some books stand out:

@itemize
@item Practical Common Lisp, by Peter Seibel

An excellent introduction to the language, covering both the
basics and "advanced topics" like macros, CLOS, and packages.
Available both in print format and on the web:
@url{https://gigamonkeys.com/book/}.

@item Paradigms Of Artificial Intelligence Programming, by Peter Norvig

Good information on general Common Lisp programming, and many
nontrivial examples. Whether or not your work is AI, it's a very
good book to look at.

@item On Lisp, by Paul Graham

An in-depth treatment of macros, but not recommended as a first
Common Lisp book, since it is slightly pre-ANSI so you need to
be on your guard against non-standard usages, and since it
doesn't really even try to cover the language as a whole,
focusing solely on macros. Downloadable from
@url{https://www.paulgraham.com/onlisp.html}.

@item Object-Oriented Programming In Common Lisp, by Sonya Keene

With the exception of @emph{Practical Common Lisp}, most introductory
books don't emphasize CLOS. This one does. Even if you're very
knowledgeable about object oriented programming in the abstract,
it's worth looking at this book if you want to do any OO in
Common Lisp. Some abstractions in CLOS (especially multiple
dispatch) go beyond anything you'll see in most OO systems, and
there are a number of lesser differences as well. This book
tends to help with the culture shock.

@item Art Of Metaobject Programming, by Gregor Kiczales et al.

Currently the prime source of information on the Common Lisp
Metaobject Protocol, which is supported by SBCL. Section
2 (Chapters 5 and 6) are freely available at
@url{http://mop.lisp.se/www.alu.org/mop/}.
@end itemize

@node history and implementation of sbcl
@section History and Implementation of SBCL

You can work productively with SBCL without knowing or understanding
anything about where it came from, how it is implemented, or how it
extends the ANSI Common Lisp standard. However, a little knowledge
can be helpful in order to understand error messages, to
troubleshoot problems, to understand why some parts of the system
are better debugged than others, and to anticipate which known bugs,
known performance problems, and missing extensions are likely to be
fixed, tuned, or added.

SBCL is descended from CMUCL, which is itself descended from Spice
Lisp, including early implementations for the Mach operating system on
the IBM RT, back in the 1980s. Some design decisions from that time are
still reflected in the current implementation:

@itemize
@item The system expects to be loaded into a fixed-at-compile-time
  location in virtual memory, and also expects the location of all
  of its heap storage to be specified at compile time.

@item The system overcommits memory, allocating large amounts of address
  space from the system (often more than the amount of virtual
  memory available) and then failing if it ends up using too much of
  the allocated storage.

@item The system is implemented as a C program which is responsible for
  supplying low-level services and loading a Lisp @code{.core} file.
@end itemize

@cindex garbage collector, generational
@cindex generational garbage collector
SBCL also inherited some newer architectural features from CMUCL.
The most important is that on some architectures it has a
generational GC, which has various implications (mostly good) for
performance. These are discussed in another chapter, @ref{efficiency}.

SBCL has diverged from CMUCL in that SBCL is now essentially a
compiler-only implementation of Common Lisp. This is a change in
implementation strategy, taking advantage of the freedom "any of
these facilities might share the same execution strategy"
guaranteed in @code{clhs} @code{3.1} (Evaluation). It does not mean SBCL can't
be used interactively, and in fact the change is largely invisible
to the casual user, since SBCL still can and does execute code
interactively by compiling it on the fly. (It is visible if you know
how to look, like using @code{compiled-function-p}; and it is visible in
the way that SBCL doesn't have many bugs which behave differently in
interpreted code than in compiled code.) What it means is that in
SBCL, the @code{eval} function only truly "interprets" a few easy kinds
of forms, such as symbols which are @code{boundp}. More complicated forms
are evaluated by calling @code{compile} and then calling @code{funcall} on the
returned result.

The direct ancestor of SBCL is the x86 port of CMUCL. This port was in
some ways the most cobbled-together of all the CMUCL ports, since a
number of strange changes had to be made to support the register-poor
x86 architecture. Some things (like tracing and debugging) do not work
particularly well there. SBCL should be able to improve in these areas
(and has already improved in some other areas), but it takes a while.

@cindex garbage collector, conservative
@cindex conservative garbage collector
On the x86 SBCL -- like the x86 port of CMUCL -- uses a
@emph{conservative GC}. This means that it doesn't maintain a strict
separation between tagged and untagged data, instead treating some
untagged data (e.g. raw floating point numbers) as possibly-tagged
data and so not collecting any Lisp objects that they point to. This
has some negative consequences for average time efficiency (though
possibly no worse than the negative consequences of trying to
implement an exact GC on a processor architecture as register-poor
as the X86) and also has potentially unlimited consequences for
worst-case memory efficiency. In practice, conservative GCs work
reasonably well, not getting anywhere near the worst case. But they
can occasionally cause odd patterns of memory usage.

The fork from CMUCL was based on a major rewrite of the system
bootstrap process. CMUCL has for many years tolerated a very unusual
"build" procedure which doesn't actually build the complete system
from scratch, but instead progressively overwrites parts of a
running system with new versions. This quasi-build procedure can
cause various bizarre bootstrapping hangups, especially when a major
change is made to the system. It also makes the connection between
the current source code and the current executable more tenuous than
in other software systems -- it's easy to accidentally build a CMUCL
system containing characteristics not reflected in the current
version of the source code.

Other major changes since the fork from CMUCL include:

@itemize
@item SBCL has removed many CMUCL extensions, (e.g. IP networking,
  remote procedure call, Unix system interface, and X11 interface)
  from the core system. Most of these are available as contributed
  modules (distributed with SBCL) or third-party modules instead.

@item SBCL has deleted or deprecated some nonstandard features and code
  complexity which helped efficiency at the price of
  maintainability. For example, the SBCL compiler no longer
  implements memory pooling internally (and so is simpler and more
  maintainable, but generates more garbage and runs more slowly).
@end itemize