@node Beyond the ANSI Standard @comment node-name, next, previous, up @chapter Beyond the ANSI Standard SBCL is derived from CMUCL, which implements many extensions to the ANSI standard. SBCL doesn't support as many extensions as CMUCL, but it still has quite a few. @xref{Contributed Modules}. @menu * Reader Extensions:: * Package-Local Nicknames:: * Package Variance:: * Garbage Collection:: * Generic Function Dispatch:: * Slot Access:: * Metaobject Protocol:: * Extensible Sequences:: * Support For Unix:: * Unicode Support:: * Customization Hooks for Users:: * Tools To Help Developers:: * Resolution of Name Conflicts:: * Hash Table Extensions:: * Random Number Generation:: * Timeouts and Deadlines:: * Miscellaneous Extensions:: * Stale Extensions:: * Efficiency Hacks:: @end menu @node Reader Extensions @comment node-name, next, previous, up @section Reader Extensions @cindex Reader Extensions @subsection Extended Package Prefix Syntax @cindex Extended Package Prefix Syntax @cindex Package Prefix Syntax, extended @vindex @cl{@earmuffs{package}} @cindex Interning Symbols @cindex Symbols, interning @findex @cl{intern} SBCL supports extended package prefix syntax, which allows specifying an alternate package instead of @code{*package*} for the reader to use as the default package for interning symbols: @lisp @var{package-name}::@var{form-with-interning-into-package} @end lisp Example: @lisp 'foo::(bar quux zot) == '(foo::bar foo::quux foo::zot) @end lisp @cindex Package Locks @vindex @cl{@earmuffs{package}} @code{*package*} is not rebound during the course of reading a form with extended package prefix syntax; if @code{foo::bar} would cause a read-time package lock violation, so does @code{foo::(bar)}. @subsection Symbol Name Normalization @cindex Symbol Name Normalization @cindex Normalization, Symbol Name @cindex Unicode @cindex NFKC SBCL also extends the reader to normalize all symbols to Normalization Form KC in builds with Unicode enabled. Whether symbols are normalized is controlled by @include fun-sb-ext-readtable-normalization.texinfo Symbols created by @findex @cl{intern} @code{intern} and similar functions are not affected by this setting. If @code{sb-ext:readtable-normalization} is @code{t}, symbols that are not normalized are escaped during printing. @subsection Decimal Syntax for Rationals @cindex Decimal Syntax for Rationals @cindex Rational, decimal syntax for @tindex @cl{float} SBCL supports a decimal syntax for rationals, modelled after the standard syntax for floating-point numbers. If a number with floating-point syntax has an exponent marker of @code{r} or @code{R} (rather than one of the standard exponent markers), it is read as the rational with the exact value of the decimal number expressed as a float. @vindex @cl{@earmuffs{read-default-float-format}} @tindex @cl{rational} @findex @cl{read} @findex @cl{read-from-string} In addition, setting or binding the value of @code{*read-default-float-format*} to @code{rational} around a call to @code{read} or @code{read-from-string} has the effect that floating-point numbers without exponent markers are read as rational numbers, as if there had been an explicit @code{r} or @code{R} marker. @vindex @cl{@earmuffs{read-default-float-format}} @tindex @cl{rational} Floating point numbers of all types are printed with an exponent marker while the value of @code{*read-default-float-format*} is @code{rational}; however, rational numbers are printed in their standard syntax, irrespective of the value of @code{*read-default-float-format*}. @node Package-Local Nicknames @comment node-name, next, previous, up @section Package-Local Nicknames @cindex Package-Local Nicknames @cindex Nicknames, Package-local SBCL allows giving packages local nicknames: they allow short and easy-to-use names to be used without fear of name conflict associated with normal nicknames. A local nickname is valid only when inside the package for which it has been specified. Different packages can use same local nickname for different global names, or different local nickname for same global name. @vindex @cl{@earmuffs{features}} Symbol @code{:package-local-nicknames} in @code{*features*} denotes the support for this feature. @findex @cl{defpackage} @defmac @cl{defpackage} name [[option]]* @result{} package Options are extended to include @itemize @item @code{:local-nicknames (@var{local-nickname} @var{actual-package-name})*} The package has the specified local nicknames for the corresponding actual packages. @end itemize Example: @findex @cl{find-symbol} @findex @cl{find-package} @vindex @cl{@earmuffs{package}} @lisp (defpackage :bar (:intern "X")) (defpackage :foo (:intern "X")) (defpackage :quux (:use :cl) (:local-nicknames (:bar :foo) (:foo :bar))) (find-symbol "X" :foo) ; => FOO::X (find-symbol "X" :bar) ; => BAR::X (let ((*package* (find-package :quux))) (find-symbol "X" :foo)) ; => BAR::X (let ((*package* (find-package :quux))) (find-symbol "X" :bar)) ; => FOO::X @end lisp @end defmac @include fun-sb-ext-package-local-nicknames.texinfo @include fun-sb-ext-package-locally-nicknamed-by-list.texinfo @include fun-sb-ext-add-package-local-nickname.texinfo @include fun-sb-ext-remove-package-local-nickname.texinfo @node Package Variance @comment node-name, next, previous, up @section Package Variance Common Lisp standard specifies that ``If the new definition is at variance with the current state of that package, the consequences are undefined;'' SBCL by default signals a full warning and retains as much of the package state as possible. This can be adjusted using @code{sb-ext:*on-package-variance*}: @include var-sb-ext-star-on-package-variance-star.texinfo @node Garbage Collection @comment node-name, next, previous, up @section Garbage Collection @cindex Garbage collection SBCL provides additional garbage collection functionality not specified by ANSI. @include var-sb-ext-star-after-gc-hooks-star.texinfo @include fun-sb-ext-gc.texinfo @subsection Finalization @cindex Finalization Finalization allows code to be executed after an object has been garbage collected. This is useful for example for releasing foreign memory associated with a Lisp object. @include fun-sb-ext-finalize.texinfo @include fun-sb-ext-cancel-finalization.texinfo @subsection Weak Pointers @cindex Weak pointers Weak pointers allow references to objects to be maintained without keeping them from being garbage collected: useful for building caches among other things. Hash tables can also have weak keys and values: @pxref{Hash Table Extensions}. @include fun-sb-ext-make-weak-pointer.texinfo @include fun-sb-ext-weak-pointer-value.texinfo @subsection Introspection and Tuning @include var-sb-ext-star-gc-run-time-star.texinfo @include fun-sb-ext-bytes-consed-between-gcs.texinfo @include fun-sb-ext-dynamic-space-size.texinfo @include fun-sb-ext-get-bytes-consed.texinfo @include fun-sb-ext-gc-logfile.texinfo @include fun-sb-ext-generation-average-age.texinfo @include fun-sb-ext-generation-bytes-allocated.texinfo @include fun-sb-ext-generation-bytes-consed-between-gcs.texinfo @include fun-sb-ext-generation-minimum-age-before-gc.texinfo @include fun-sb-ext-generation-number-of-gcs-before-promotion.texinfo @include fun-sb-ext-generation-number-of-gcs.texinfo @subsection Tracing Live Objects Back to Roots @quotation note This feature is intended to help expert users diagnose rare low-level issues and should not be needed during normal usage. On top of that, the interface and implementation are experimental and may change at any time without further notice. @end quotation It is sometimes important to understand why a given object is retained in the Lisp image instead of being garbage collected. To help with this problem, SBCL provides a mechanism that searches through the different memory spaces, builds a path of references from a root to the object in question and finally reports this paths: @include fun-sb-ext-search-roots.texinfo An example of using this could look like this: @lisp * (defvar *my-string* (list 1 2 "my string")) *MY-STRING* * (sb-ext:search-roots (sb-ext:make-weak-pointer (third *my-string*))) -> ((SIMPLE-VECTOR 3)) #x10004E9EAF[2] -> (SYMBOL) #x5044100F[1] -> (CONS) #x100181FAE7[1] -> (CONS) #x100181FAF7[1] -> (CONS) #x100181FB07[0] -> #x100181F9AF @end lisp The single line of output on @code{cl:*standard-output*} shows the path from a root to @t{"my string"}: the path starts with SBCL's internal package system data structures followed by the symbol (@t{cl-user:*my-string*}) followed the three cons cells of the list. The @code{:print :verbose} argument produces similar behavior but describe the path elements in more detail: @lisp * (sb-ext:search-roots (sb-ext:make-weak-pointer (third *my-string*)) :print :verbose) Path to "my string": 6 10004E9EAF [ 2] a (simple-vector 3) 0 5044100F [ 1] COMMON-LISP-USER::*MY-STRING* 0 100181FAE7 [ 1] a cons 0 100181FAF7 [ 1] a cons 0 100181FB07 [ 0] a cons @end lisp The @code{:print nil} argument is a bit different: @lisp * (sb-ext:search-roots (sb-ext:make-weak-pointer (third *my-string*)) :print nil) (("my string" :STATIC (#(*MY-STRING* 0 0) . 2) (*MY-STRING* . 1) ((1 2 "my string") . 1) ((2 "my string") . 1) (("my string") . 0))) @end lisp There is no output on @code{cl:*standard-output*} and the return value is a single path for the target object @t{"my string"}. As before, the path shows the symbol and the three cons cells. @node Generic Function Dispatch @comment node-name, next, previous, up @section Generic Function Dispatch @findex @sbpcl{no-primary-method} @findex @cl{no-applicable-method} If a generic function with standard or short method combination is called, and the set of applicable methods does not include any primary methods, then the generic function @code{no-primary-method} will be invoked, with the arguments to @code{no-primary-method} being the invoked generic function and its arguments, similar to the standard function @code{no-applicable-method}. As with @code{no-applicable-method}, the default method on @code{no-primary-method} signals an error; programmers may define methods on it. @node Slot Access @comment node-name, next, previous, up @section Slot Access @findex @cl{slot-value} @findex @setf{@cl{slot-value}} @findex @cl{slot-boundp} @findex @cl{slot-makunbound} The slot access functions @code{slot-value}, @code{(setf slot-value)}, @code{slot-boundp} and @code{slot-makunbound} are defined to function as expected on conditions (of metaclass @code{condition-class}) and, with some limitations, on structures (of metaclass @code{structure-class}). For structures: @itemize @item @findex @cl{defstruct} The name of a slot for the purposes of the slot access functions is the symbol used as the slot-name in the slot-description in the @code{defstruct} form; @item @findex @cl{slot-unbound} @findex @cl{slot-value} @findex @cl{slot-boundp} @cindex Unbound slots @code{slot-value} and @code{slot-boundp} function as expected, including (for @code{slot-value}) calling and respecting the return value of @code{slot-unbound} if the slot is unbound; @item @findex @setf{@cl{slot-value}} @code{(setf slot-value)} functions as expected, including performing type checks to verify that the new value is of an appropriate type for the slot; @item @findex @cl{slot-makunbound} @cindex Unbound slots @code{slot-makunbound} makes the slot be unbound only when the slot corresponds to an @code{&aux} argument with no default in a by-order-of-arguments (BOA) constructor. In all other cases calling @code{slot-makunbound} on a structure signals an error. @item @findex @cl{slot-missing} If any of the slot access functions is called with a structure instance which does not have a slot of the given name, @code{slot-missing} is called and the return value of the effective method, if any, is respected. @end itemize @node Metaobject Protocol @comment node-name, next, previous, up @section Metaobject Protocol @subsection AMOP Compatibility of Metaobject Protocol SBCL supports a metaobject protocol which is intended to be compatible with AMOP; present exceptions to this (as distinct from current bugs) are: @itemize @item @findex @sbmop{compute-effective-method} @code{compute-effective-method} only returns one value, not two. There is no record of what the second return value was meant to indicate, and apparently no clients for it. @item @tindex @cl{generic-function} @tindex @cl{standard-generic-function} @tindex @sbmop{funcallable-standard-object} @tindex @cl{standard-object} @tindex @cl{function} The direct superclasses of @code{funcallable-standard-object} are @code{(function standard-object)}, not @code{(standard-object function)}. This is to ensure that the @code{standard-object} class is the last of the standardized classes before @code{t} appearing in the class precedence list of @code{generic-function} and @code{standard-generic-function}, as required by section 1.4.4.5 of the ANSI specification. @item @findex @cl{ensure-generic-function} @findex @sbmop{generic-function-declarations} the arguments @code{:declare} and @code{:declarations} to @code{ensure-generic-function} are both accepted, with the leftmost argument defining the declarations to be stored and returned by @code{generic-function-declarations}. Where AMOP specifies @code{:declarations} as the keyword argument to @code{ensure-generic-function}, the Common Lisp standard specifies @code{:declare}. Portable code should use @code{:declare}. @item @findex @sbmop{validate-superclass} @findex @sbmop{finalize-inheritance} @tindex @cl{standard-class} @tindex @sbmop{funcallable-standard-class} @tindex @cl{function} @findex @sbmop{class-prototype} although SBCL obeys the requirement in AMOP that @code{validate-superclass} should treat @code{standard-class} and @code{funcallable-standard-class} as compatible metaclasses, we impose an additional requirement at class finalization time: a class of metaclass @code{funcallable-standard-class} must have @code{function} in its superclasses, and a class of metaclass @code{standard-class} must not. @findex @cl{typep} @findex @cl{class-of} @findex @cl{subtypep} After a class has been finalized, it is associated with a class prototype which is accessible by a standard mop function @code{class-prototype}. The user can then ask whether this object is a @code{function} or not in several different ways: whether it is a function according to @code{typep}; whether its @code{class-of} is @code{subtypep} @code{function}, or whether @code{function} appears in the superclasses of the class. The additional consistency requirement comes from the desire to make all of these answers the same. The following class definitions are bad, and will lead to errors either immediately or if an instance is created: @lisp (defclass bad-object (funcallable-standard-object) () (:metaclass standard-class)) @end lisp @lisp (defclass bad-funcallable-object (standard-object) () (:metaclass funcallable-standard-class)) @end lisp The following definition is acceptable: @lisp (defclass mixin () ((slot :initarg slot))) (defclass funcallable-object (funcallable-standard-object mixin) () (:metaclass funcallable-standard-class)) @end lisp and leads to a class whose instances are funcallable and have one slot. @tindex @sbmop{funcallable-standard-object} Note that this requirement also applies to the class @code{funcallable-standard-object}, which has metaclass @code{funcallable-standard-class} rather than @code{standard-class} as AMOP specifies. @item the requirement that ``No portable class @math{C_p} may inherit, by virtue of being a direct or indirect subclass of a specified class, any slot for which the name is a symbol accessible in the @code{common-lisp-user} package or exported by any package defined in the ANSI Common Lisp standard.'' is interpreted to mean that the standardized classes themselves should not have slots named by external symbols of public packages. The rationale behind the restriction is likely to be similar to the ANSI Common Lisp restriction on defining functions, variables and types named by symbols in the Common Lisp package: preventing two independent pieces of software from colliding with each other. @item @findex @sbmop{slot-value-using-class} @findex @setf{@sbmop{slot-value-using-class}} @findex @sbmop{slot-boundp-using-class} specializations of the @code{new-value} argument to @code{(setf slot-value-using-class)} are not allowed: all user-defined methods must have a specializer of the class @code{t}. This prohibition is motivated by a separation of layers: the @code{slot-value-using-class} family of functions is intended for use in implementing different and new slot allocation strategies, rather than in performing application-level dispatching. Additionally, with this requirement, there is a one-to-one mapping between metaclass, class and slot-definition-class tuples and effective methods of @code{(setf slot-value-using-class)}, which permits optimization of @code{(setf slot-value-using-class)}'s discriminating function in the same manner as for @code{slot-value-using-class} and @code{slot-boundp-using-class}. Note that application code may specialize on the @code{new-value} argument of slot accessors. @item @findex @cl{defclass} @findex @sbmop{ensure-class} @findex @sbmop{ensure-class-using-class} @findex @cl{find-class} @findex @cl{class-name} the class named by the @code{name} argument to @code{ensure-class}, if any, is only redefined if it is the proper name of that class; otherwise, a new class is created. This is consistent with the description of @code{ensure-class} in AMOP as the functional version of @code{defclass}, which has this behaviour; however, it is not consistent with the weaker requirement in AMOP, which states that any class found by @code{find-class}, no matter what its @code{class-name}, is redefined. @item @findex @sbmop{slot-definition-name} @tindex @cl{structure-class} @findex @cl{defstruct} an error is not signaled in the case of the @code{:name} initialization argument for @code{slot-definition} objects being a constant, when the slot definition is of type @code{structure-slot-definition} (i.e. it is associated with a class of type @code{structure-class}). This allows code which uses constant names for structure slots to continue working as specified in ANSI, while enforcing the constraint for all other types of slot. @item @tindex @cl{t} @tindex @cl{built-in-class} @findex @sbmop{validate-superclass} @findex @cl{defclass} the class named @code{t} is not an instance of the @code{built-in-class} metaclass. AMOP specifies, in the ``Inheritance Structure of Metaobject Classes'' section, that the class named @code{t} should be an instance of @code{built-in-class}. However, it also specifies that @code{validate-superclass} should return true (indicating that a direct superclass relationship is permissible) if the second argument is the class named @code{t}. Also, ANSI specifies that classes with metaclass @code{built-in-class} may not be subclassed using @code{defclass}, and also that the class named @code{t} is the universal superclass, inconsistent with it being a @code{built-in-class}. @item @findex @cl{change-class} @findex @cl{defclass} @findex @sbmop{ensure-class} @findex @sbmop{ensure-class-using-class} @findex @sbmop{slot-value-using-class} @findex @setf{@sbmop{slot-value-using-class}} @findex @sbmop{slot-boundp-using-class} @findex @sbmop{slot-definition-allocation} uses of @code{change-class} and redefinitions of classes with @code{defclass} (or the functional interfaces @code{ensure-class} or @code{ensure-class-using-class}) must ensure that for each slot with allocation @code{:instance} or @code{:class}, the set of applicable methods on the @code{slot-value-using-class} family of generic functions is the same before and after the change. This is required for correct operation of the protocol to update instances for the new or redefined class, and can be seen as part of the contract of the @code{:instance} or @code{:class} allocations. @item @findex @cl{no-applicable-method} @findex @sbpcl{no-primary-method} @findex @sbmop{compute-discriminating-function} @findex @cl{compute-applicable-methods} @findex @sbmop{compute-effective-method} Metaobject protocol users may wish to override @code{compute-discriminating-function} for their own generic function classes. Overriding implementations of @code{compute-discriminating-function} must, in order to participate in the @code{no-applicable-method} and @code{no-primary-method} protocols, perform appropriate checks on the return value of @code{compute-applicable-methods} before processing the effective method; the standard effective method contains error-invoking forms, but those forms have no access to the generic function invocation's arguments. @end itemize @subsection Metaobject Protocol Extensions In addition, SBCL supports extensions to the Metaobject protocol from AMOP; at present, they are: @itemize @item @findex @cl{defmethod} @findex @cl{find-class} @findex @sbmop{intern-eql-specializer} @findex @sbpcl{make-method-specializers-form} @findex @sbmop{make-method-lambda} compile-time support for generating specializer metaobjects from specializer names in @code{defmethod} forms is provided by the @code{make-method-specializers-form} function, which returns a form which, when evaluated in the lexical environment of the @code{defmethod}, returns a list of specializer metaobjects. This operator suffers from similar restrictions to those affecting @code{make-method-lambda}, namely that the generic function must be defined when the @code{defmethod} form is expanded, so that the correct method of @code{make-method-specializers-form} is invoked. The system-provided method on @code{make-method-specializers-form} generates a call to @code{find-class} for each symbol specializer name, and a call to @code{intern-eql-specializer} for each @code{(eql @var{x})} specializer name. @item @findex @cl{find-method} @findex @sbpcl{parse-specializer-using-class} @findex @sbpcl{unparse-specializer-using-class} run-time support for converting between specializer names and specializer metaobjects, mostly for the purposes of @code{find-method}, is provided by @code{parse-specializer-using-class} and @code{unparse-specializer-using-class}, which dispatch on their first argument, the generic function associated with a method with the given specializer. The system-provided methods on those methods convert between classes and proper names and between lists of the form @code{(eql @var{x})} and interned eql specializer objects. @item @vindex @sbpcl{@earstuds{slot-unbound}} @findex @sbmop{standard-instance-access} @findex @sbmop{funcallable-standard-instance-access} @cindex Unbound slots distinguishing unbound instance allocated slots from bound ones when using @code{standard-instance-access} and @code{funcallable-standard-instance-access} is possible by comparison to the symbol-macro @code{+slot-unbound+}. @end itemize @node Extensible Sequences @comment node-name, next, previous, up @section Extensible Sequences @menu * Iterator Protocol:: * Simple Iterator Protocol:: @end menu @tindex @cl{sequence} @tindex @cl{vector} @findex @cl{find} @findex @cl{subseq} ANSI Common Lisp has a class @code{sequence} with subclasses @code{list} and @code{vector} on which the ``sequence functions'' like @code{find}, @code{subseq}, etc. operate. As an extension to the ANSI specification, SBCL allows additional subclasses of @code{sequence} to be defined @footnote{A motivation, rationale and additional examples for the design of this extension can be found in the paper @cite{Rhodes, Christophe (2007): User-extensible sequences in Common Lisp} available for download at @url{http://www.doc.gold.ac.uk/~mas01cr/papers/ilc2007/sequences-20070301.pdf}.}. @findex @cl{coerce} @findex @cl{subseq} @findex @cl{make-instance} @tindex @cl{list} Users of this extension just make instances of @code{sequence} subclasses and transparently operate on them using sequence functions: @lisp (coerce (subseq (make-instance 'my-sequence) 5 10) 'list) @end lisp From this perspective, no distinction between builtin and user-defined @code{sequence} subclasses should be necessary. Providers of the extension, that is of user-defined @code{sequence} subclasses, have to adhere to a ``sequence protocol'' which consists of a set of generic functions in the @code{sequence} package. @tindex @cl{sequence} @tindex @cl{standard-object} A minimal @code{sequence} subclass has to specify @code{standard-object} and @code{sequence} as its superclasses and has to be the specializer of the @code{sequence} parameter of methods on at least the following generic functions: @include fun-sb-sequence-length.texinfo @include fun-sb-sequence-elt.texinfo @include fun-sb-sequence-setf-elt.texinfo @include fun-sb-sequence-adjust-sequence.texinfo @include fun-sb-sequence-make-sequence-like.texinfo @tindex @cl{sequence} @findex @sequence{make-sequence-like} @findex @cl{subseq} @findex @cl{copy-seq} @findex @sequence{adjust-sequence} @code{make-sequence-like} is needed for functions returning freshly-allocated sequences such as @code{subseq} or @code{copy-seq}. @code{adjust-sequence} is needed for functions which destructively modify their arguments such as @code{delete}. In fact, all other sequence functions can be implemented in terms of the above functions and actually are, if no additional methods are defined. However, relying on these generic implementations, in particular not implementing the iterator protocol can incur a high performance penalty @xref{Iterator Protocol}. When the sequence protocol is only partially implemented for a given @code{sequence} subclass, an attempt to apply one of the missing operations to instances of that class signals the following condition: @include condition-sb-sequence-protocol-unimplemented.texinfo In addition to the mandatory functions above, methods on the sequence functions listed below can be defined. There are two noteworthy irregularities: @itemize @item The function @code{sb-sequence:emptyp} does not have a counterpart in the @code{cl} package. It is intended to be used instead of @code{length} when working with lazy or infinite sequences. @item The functions @code{map}, @code{concatenate} and @code{merge} receive a type designator specifying the type of the constructed sequence as their first argument. However, the corresponding generic functions @code{sb-sequence:map}, @code{sb-sequence:concatenate} and @code{sb-sequence:merge} receive a prototype instance of the requested @code{sequence} subclass instead. @end itemize @include fun-sb-sequence-emptyp.texinfo @itemize @item @code{sb-sequence:count}, @code{sb-sequence:count-if}, @code{sb-sequence:count-if-not} @item @code{sb-sequence:find}, @code{sb-sequence:find-if}, @code{sb-sequence:find-if-not} @item @code{sb-sequence:position}, @code{sb-sequence:position-if}, @code{sb-sequence:position-if-not} @item @code{sb-sequence:subseq} @item @code{sb-sequence:copy-seq} @item @code{sb-sequence:fill} @item @include fun-sb-sequence-map.texinfo @item @code{sb-sequence:nsubstitute}, @code{sb-sequence:nsubstitute-if}, @code{sb-sequence:nsubstitute-if-not}, @code{sb-sequence:substitute}, @code{sb-sequence:substitute-if}, @code{sb-sequence:substitute-if-not} @item @code{sb-sequence:replace} @item @code{sb-sequence:nreverse}, @code{sb-sequence:reverse} @item @include fun-sb-sequence-concatenate.texinfo @item @code{sb-sequence:reduce} @item @code{sb-sequence:mismatch} @item @code{sb-sequence:search} @item @code{sb-sequence:delete}, @code{sb-sequence:delete-if}, @code{sb-sequence:delete-if-not}, @code{sb-sequence:remove}, @code{sb-sequence:remove-if}, @code{sb-sequence:remove-if-not}, @item @code{sb-sequence:delete-duplicates}, @code{sb-sequence:remove-duplicates} @item @code{sb-sequence:sort}, @code{sb-sequence:stable-sort} @item @include fun-sb-sequence-merge.texinfo @end itemize @findex @cl{dolist} In the spirit of @code{dolist}, generic sequences can be traversed using the macro @include macro-sb-sequence-dosequence.texinfo @node Iterator Protocol @comment node-name, next, previous, up @subsection Iterator Protocol The iterator protocol allows subsequently accessing some or all elements of a sequence in forward or reverse direction. Users first call @code{make-sequence-iterator} to create an iteration state and receive functions to query and mutate it. These functions allow, among other things, moving to, retrieving or modifying elements of the sequence. An iteration state consists of a state object, a limit object, a from-end indicator and the following six functions to query or mutate this state: @findex @sequence{make-sequence-iterator} @deffn {Function} @code{step function} sequence iterator from-end Moves the iterator one position forward or backward in the associated sequence depending on the iteration direction. @end deffn @deffn {Function} @code{endp function} sequence iterator limit from-end Returns non-@code{nil} when the iterator has reached the end of the associated sequence with respect to the iteration direction. @end deffn @deffn {Function} @code{element function} sequence iterator Returns the sequence element associated to the current position of the iteration. @end deffn @deffn {Function} @code{setf element function} new-value sequence iterator Destructively modifies the associates sequence by replacing the sequence element associated to the current iteration position with a new value. @end deffn @deffn {Function} @code{index function} sequence iterator Returns the position of the iteration in the associated sequence. @end deffn @deffn {Function} @code{copy function} sequence iterator Returns a copy of the iteration state which can be mutated independently of the copied iteration state. @end deffn An iterator is created by calling: @include fun-sb-sequence-make-sequence-iterator.texinfo @findex @sequence{make-sequence-iterator} @findex @sequence{make-simple-sequence-iterator} @tindex @cl{sequence} Note that @code{make-sequence-iterator} calls @code{make-simple-sequence-iterator} when there is no specialized method for a particular @code{sequence} subclass. @xref{Simple Iterator Protocol}. The following convenience macros simplify traversing sequences using iterators: @include macro-sb-sequence-with-sequence-iterator.texinfo @include macro-sb-sequence-with-sequence-iterator-functions.texinfo @node Simple Iterator Protocol @comment node-name, next, previous, up @subsection Simple Iterator Protocol For cases in which the full flexibility and performance of the general sequence iterator protocol is not required, there is a simplified sequence iterator protocol consisting of a few generic functions which can be specialized for iterator classes: @include fun-sb-sequence-iterator-step.texinfo @include fun-sb-sequence-iterator-endp.texinfo @include fun-sb-sequence-iterator-element.texinfo @include fun-sb-sequence-setf-iterator-element.texinfo @include fun-sb-sequence-iterator-index.texinfo @include fun-sb-sequence-iterator-copy.texinfo Iterator objects implementing the above simple iteration protocol are created by calling the following generic function: @include fun-sb-sequence-make-simple-sequence-iterator.texinfo @node Support For Unix @comment node-name, next, previous, up @section Support For Unix @menu * Command-line arguments:: * Querying the process environment:: * Running external programs:: @end menu @node Command-line arguments @subsection Command-line arguments @vindex @sbext{@earmuffs{posix-argv}} The UNIX command line can be read from the variable @code{sb-ext:*posix-argv*}. @node Querying the process environment @subsection Querying the process environment The UNIX environment can be queried with the @code{sb-ext:posix-getenv} function. @include fun-sb-ext-posix-getenv.texinfo @node Running external programs @subsection Running external programs External programs can be run with @code{sb-ext:run-program}. @footnote{In SBCL versions prior to 1.0.13, @code{sb-ext:run-program} searched for executables in a manner somewhat incompatible with other languages. As of this version, SBCL uses the system library routine @code{execvp(3)}, and no longer contains the function, @code{find-executable-in-search-path}, which implemented the old search. Users who need this function may find it in @file{run-program.lisp} versions 1.67 and earlier in SBCL's CVS repository here @url{http://sbcl.cvs.sourceforge.net/sbcl/sbcl/src/code/run-program.lisp?view=log}. However, we caution such users that this search routine finds executables that system library routines do not.} @include fun-sb-ext-run-program.texinfo When @code{sb-ext:run-program} is called with @code{wait} equal to NIL, an instance of class @var{sb-ext:process} is returned. The following functions are available for use with processes: @include fun-sb-ext-process-p.texinfo @include fun-sb-ext-process-input.texinfo @include fun-sb-ext-process-output.texinfo @include fun-sb-ext-process-error.texinfo @include fun-sb-ext-process-alive-p.texinfo @include fun-sb-ext-process-status.texinfo @include fun-sb-ext-process-wait.texinfo @include fun-sb-ext-process-exit-code.texinfo @include fun-sb-ext-process-core-dumped.texinfo @include fun-sb-ext-process-close.texinfo @include fun-sb-ext-process-kill.texinfo @node Unicode Support @comment node-name, next, previous, up @section Unicode Support @cindex Unicode SBCL provides support for working with Unicode text and querying the standard Unicode database for information about individual codepoints. Unicode-related functions are located in the @code{sb-unicode} package. @cindex Character Names @cindex Name, of character @findex @cl{name-char} @findex @cl{char-name} SBCL also extends ANSI character literal syntax to support Unicode codepoints. You can either specify a character by its Unicode name, with spaces replaced by underscores, if a unique name exists @footnote{Please note that the codepoint U+1F5CF (PAGE) introduced in Unicode 7.0 is named @code{UNICODE_PAGE}, since the name ``Page'' is required to be assigned to form-feed (U+0C) by the ANSI standard.} or by giving its hexadecimal codepoint preceded by a ``U'', an optional ``+'', and an arbitrary number of leading zeros. You may also input the character directly into your source code if it can be encoded in your file. If a character had an assigned name in Unicode 1.0 that was distinct from its current name, you may also use that name (with spaces replaced by underscores) to specify the character, unless the name is already associated with a codepoint in the latest Unicode standard (such as ``BELL''). For example, you can specify the codepoint U+00E1 (``Latin Small Letter A With Acute'') as @itemize @bullet @item @code{#\LATIN_SMALL_LETTER_A_WITH_ACUTE} @item @code{#\LATIN_SMALL_LETTER_A_ACUTE} @item @code{#\รก} assuming a Unicode source file @item @code{#\U00E1} @item @code{#\UE1} @item @code{#\U+00E1} @end itemize @subsection Unicode property access The following functions can be used to find information about a Unicode codepoint. @include fun-sb-unicode-general-category.texinfo @include fun-sb-unicode-bidi-class.texinfo @include fun-sb-unicode-combining-class.texinfo @include fun-sb-unicode-decimal-value.texinfo @include fun-sb-unicode-digit-value.texinfo @include fun-sb-unicode-numeric-value.texinfo @include fun-sb-unicode-mirrored-p.texinfo @include fun-sb-unicode-bidi-mirroring-glyph.texinfo @include fun-sb-unicode-age.texinfo @include fun-sb-unicode-hangul-syllable-type.texinfo @include fun-sb-unicode-east-asian-width.texinfo @include fun-sb-unicode-script.texinfo @include fun-sb-unicode-char-block.texinfo @include fun-sb-unicode-unicode-1-name.texinfo @include fun-sb-unicode-proplist-p.texinfo @include fun-sb-unicode-uppercase-p.texinfo @include fun-sb-unicode-lowercase-p.texinfo @include fun-sb-unicode-cased-p.texinfo @include fun-sb-unicode-case-ignorable-p.texinfo @include fun-sb-unicode-alphabetic-p.texinfo @include fun-sb-unicode-ideographic-p.texinfo @include fun-sb-unicode-math-p.texinfo @include fun-sb-unicode-whitespace-p.texinfo @include fun-sb-unicode-soft-dotted-p.texinfo @include fun-sb-unicode-hex-digit-p.texinfo @include fun-sb-unicode-default-ignorable-p.texinfo @include fun-sb-unicode-grapheme-break-class.texinfo @include fun-sb-unicode-word-break-class.texinfo @include fun-sb-unicode-sentence-break-class.texinfo @include fun-sb-unicode-line-break-class.texinfo @subsection String operations @cindex Normalization, String SBCL can normalize strings using: @include fun-sb-unicode-normalize-string.texinfo @include fun-sb-unicode-normalized-p.texinfo SBCL implements the full range of Unicode case operations with the functions @include fun-sb-unicode-uppercase.texinfo @include fun-sb-unicode-lowercase.texinfo @include fun-sb-unicode-titlecase.texinfo @include fun-sb-unicode-casefold.texinfo @findex @cl{string-upcase} @findex @cl{char-downcase} @findex @cl{both-case-p} It also extends standard Common Lisp case functions such as @code{string-upcase} and @code{string-downcase} to support a subset of Unicode's casing behavior. Specifically, a character is @code{both-case-p} if its case mapping in Unicode is one-to-one and invertable. The @code{sb-unicode} package also provides functions for collating/sorting strings according to the Unicode Collation Algorithm. @include fun-sb-unicode-unicode-lt.texinfo @include fun-sb-unicode-unicode=.texinfo @include fun-sb-unicode-unicode-equal.texinfo @include fun-sb-unicode-unicode-lt=.texinfo @include fun-sb-unicode-unicode-gt.texinfo @include fun-sb-unicode-unicode-gt=.texinfo The following functions are provided for detecting visually confusable strings: @include fun-sb-unicode-confusable-p.texinfo @subsection Breaking strings The @code{sb-unicode} package includes several functions for breaking a Unicode string into useful parts. @include fun-sb-unicode-graphemes.texinfo @include fun-sb-unicode-words.texinfo @include fun-sb-unicode-sentences.texinfo @include fun-sb-unicode-lines.texinfo @node Customization Hooks for Users @comment node-name, next, previous, up @section Customization Hooks for Users The toplevel repl prompt may be customized, and the function that reads user input may be replaced completely. @c The behaviour of @code{require} when called with only one argument is implementation-defined. In SBCL, @code{require} behaves in the following way: @include fun-common-lisp-require.texinfo @include var-sb-ext-star-module-provider-functions-star.texinfo Although SBCL does not provide a resident editor, the @code{ed} function can be customized to hook into user-provided editing mechanisms as follows: @include fun-common-lisp-ed.texinfo @include var-sb-ext-star-ed-functions-star.texinfo Conditions of type @code{warning} and @code{style-warning} are sometimes signaled at runtime, especially during execution of Common Lisp defining forms such as @code{defun}, @code{defmethod}, etc. To muffle these warnings at runtime, SBCL provides a variable @code{sb-ext:*muffled-warnings*}: @include var-sb-ext-star-muffled-warnings-star.texinfo @node Tools To Help Developers @comment node-name, next, previous, up @section Tools To Help Developers @findex @cl{trace} @findex @cl{inspect} SBCL provides a profiler and other extensions to the ANSI @code{trace} facility. For more information, see @ref{Macro common-lisp trace}. The debugger supports a number of options. Its documentation is accessed by typing @kbd{help} at the debugger prompt. @xref{Debugger}. Documentation for @code{inspect} is accessed by typing @kbd{help} at the @code{inspect} prompt. @node Resolution of Name Conflicts @section Resolution of Name Conflicts @tindex @sbext{name-conflict} @findex @sbext{name-conflict-symbols} The ANSI standard (section 11.1.1.2.5) requires that name conflicts in packages be resolvable in favour of any of the conflicting symbols. In the interactive debugger, this is achieved by prompting for the symbol in whose favour the conflict should be resolved; for programmatic use, the @code{sb-ext:resolve-conflict} restart should be invoked with one argument, which should be a member of the list returned by the condition accessor @code{sb-ext:name-conflict-symbols}. @node Hash Table Extensions @comment node-name, next, previous, up @section Hash Table Extensions @cindex Hash tables Hash table extensions supported by SBCL are all controlled by keyword arguments to @code{make-hash-table}. @include fun-common-lisp-make-hash-table.texinfo @include macro-sb-ext-define-hash-table-test.texinfo @include macro-sb-ext-with-locked-hash-table.texinfo @include fun-sb-ext-hash-table-synchronized-p.texinfo @include fun-sb-ext-hash-table-weakness.texinfo @node Random Number Generation @comment node-name, next, previous, up @section Random Number Generation @cindex Random Number Generation @vindex @cl{@earmuffs{random-state}} @findex @cl{make-random-state} The initial value of @code{*random-state*} is the same each time SBCL is started. This makes it possible for user code to obtain repeatable pseudo random numbers using only standard-provided functionality. See @code{seed-random-state} below for an SBCL extension that allows to seed the random number generator from given data for an additional possibility to achieve this. Non-repeatable random numbers can always be obtained using @code{(make-random-state t)}. @findex @cl{random} The sequence of numbers produced by repeated calls to @code{random} starting with the same random state and using the same sequence of @code{limit} arguments is guaranteed to be reproducible only in the same version of SBCL on the same platform, using the same code under the same evaluator mode and compiler optimization qualities. Just two examples of differences that may occur otherwise: calls to @code{random} can be compiled differently depending on how much is known about the @code{limit} argument at compile time, yielding different results even if called with the same argument at run time, and the results can differ depending on the machine's word size, for example for limits that are fixnums under 64-bit word size but bignums under 32-bit word size. @include fun-sb-ext-seed-random-state.texinfo @tindex @cl{float} Some notes on random floats: The standard doesn't prescribe a specific method of generating random floats. The following paragraph describes SBCL's current implementation and should be taken as purely informational, that is, user code should not depend on any of its specific properties. The method used has been chosen because it is common, conceptually simple and fast. @findex @cl{expt} @findex @cl{random} To generate random floats, SBCL evaluates code that has an equivalent effect as @lisp (* limit (float (/ (random (expt 2 23)) (expt 2 23)) 1.0f0)) @end lisp (for single-floats) and correspondingly (with @code{52} and @code{1.0d0} instead of @code{23} and @code{1.0f0}) for double-floats. Note especially that this means that zero is a possible return value occurring with probability @code{(expt 2 -23)} respectively @code{(expt 2 -52)}. Also note that there exist twice as many equidistant floats between 0 and 1 as are generated. For example, the largest number that @code{(random 1.0f0)} ever returns is @code{(float (/ (1- (expt 2 23)) (expt 2 23)) 1.0f0)} while @code{(float (/ (1- (expt 2 24)) (expt 2 24)) 1.0f0)} is the largest single-float less than 1. This is a side effect of the fact that the implementation uses the fastest possible conversion from bits to floats. SBCL currently uses the Mersenne Twister as its random number generator, specifically the 32-bit version under both 32- and 64-bit word size. The seeding algorithm has been improved several times by the authors of the Mersenne Twister; SBCL uses the third version (from 2002) which is still the most recent as of June 2012. The implementation has been tested to provide output identical to the recommended C implementation. While the Mersenne Twister generates random numbers of much better statistical quality than other widely used generators, it uses only linear operations modulo 2 and thus fails some statistical tests@footnote{See chapter 7 "Testing widely used RNGs" in @cite{TestU01: A C Library for Empirical Testing of Random Number Generators} by Pierre L'Ecuyer and Richard Simard, ACM Transactions on Mathematical Software, Vol. 33, article 22, 2007.}. For example, the distribution of ranks of (sufficiently large) random binary matrices is much distorted compared to the theoretically expected one when the matrices are generated by the Mersenne Twister. Thus, applications that are sensitive to this aspect should use a different type of generator. @node Timeouts and Deadlines @comment node-name, next, previous, up @section Timeouts and Deadlines SBCL supports three different ways of restricting the execution time available to individual operations or parts of computations: @table @strong @item Timeout Parameters Some operations such as thread synchronization primitives accept a @code{:timeout} parameter. @xref{Timeout Parameters}. @item Synchronous Timeouts (Deadlines) @findex @cl{sleep} Certain operations that may suspend execution for extended periods of time such as @code{cl:sleep}, thread synchronization primitives, IO and waiting for external processes respect deadlines established for a part of a computation. @xref{Synchronous Timeouts (Deadlines)}. @item Asynchronous Timeouts Asynchronous timeouts can interrupt most computations at (almost) any point. Thus, this kind of timeouts is the most versatile but it is also somewhat unsafe. @xref{Asynchronous Timeouts}. @end table @menu * Timeout Parameters:: * Synchronous Timeouts (Deadlines):: * Asynchronous Timeouts:: * Operations Supporting Timeouts and Deadlines:: @end menu @node Timeout Parameters @subsection Timeout Parameters @cindex Timeout @tindex @sbthread{join-thread-error} Certain operations accept @code{:timeout} keyword arguments. These only affect the specific operation and must be specified at each call site by passing a @code{:timeout} keyword argument and a corresponding timeout value to the respective operation. Expiration of the timeout before the operation completes results in either a normal return with a return value indicating the timeout or in the signaling of a specialized condition such as @code{sb-thread:join-thread-error}. Example: @lisp (defun join-thread-within-5-seconds (thread) (multiple-value-bind (value result) (sb-thread:join-thread thread :default nil :timeout 5) (when (eq result :timeout) (error "Could not join ~A within 5 seconds" thread)) value)) @end lisp @findex @sbthread{join-thread} The above code attempts to join the specified thread for up to five seconds, returning its value in case of success. If the thread is still running after the five seconds have elapsed, @code{sb-thread:join-thread} indicates the timeout in its second return value. If a @code{:default} value was not provided, @code{sb-thread:join-thread} would signal a @code{sb-thread:join-thread-error} instead. To wait for an arbitrary condition, optionally with a timeout, the @code{sb-ext:wait-for} macro can be used: @include macro-sb-ext-wait-for.texinfo @c @code{sb-sys:make-fd-stream} also takes a @code{:timeout} argument @c resulting in @code{sb-sys:io-timeout}, but that seems to niche to @c document here. @node Synchronous Timeouts (Deadlines) @subsection Synchronous Timeouts (Deadlines) @cindex Timeout @cindex Synchronous Timeout @cindex Deadline Deadlines, in contrast to timeout parameters, are established for a dynamic scope using the @code{sb-sys:with-deadline} macro and indirectly affect operations within that scope. In case of nested uses, the effective deadline is the one that expires first unless an inner use explicitly overrides outer deadlines. @include macro-sb-sys-with-deadline.texinfo Expiration of deadlines set up this way only has an effect when it happens before or during the execution of a deadline-aware operation (@pxref{Operations Supporting Timeouts and Deadlines}). In this case, a @code{sb-sys:deadline-timeout} is signaled. A handler for this condition type may use the @code{sb-sys:defer-deadline} or @code{sb-sys:cancel-deadline} restarts to defer or cancel the deadline respectively and resume execution of the interrupted operation. @include condition-sb-sys-deadline-timeout.texinfo When a thread is executing the debugger, signaling of @code{sb-sys:deadline-timeout} conditions for that thread is deferred until it exits the debugger. Example: @lisp (defun read-input () (list (read-line) (read-line))) (defun do-it () (sb-sys:with-deadline (:seconds 5)) (read-input) (sleep 2) (sb-ext:run-program "my-program")) @end lisp @findex @cl{read-line} @findex @cl{sleep} @findex @sbext{run-program} The above code establishes a deadline of five seconds within which the body of the @code{do-it} function should execute. All calls of deadline-aware functions in the dynamic scope, in this case two @code{read-line} calls, a @code{sleep} call and a @code{sb-ext:run-program} call, are affected by the deadline. If, for example, the first @code{read-line} call completes in one second and the second @code{read-line} call completes in three seconds, a @code{sb-sys:deadline-timeout} condition will be signaled after the @code{sleep} call has been executing for one second. @node Asynchronous Timeouts @subsection Asynchronous Timeouts @cindex Timeout @cindex Asynchronous Timeout Asynchronous timeouts are established for a dynamic scope using the @code{sb-sys:with-timeout} macro: @include macro-sb-ext-with-timeout.texinfo Expiration of the timeout will cause the operation being executed at that moment to be interrupted by an asynchronously signaled @code{sb-ext:timeout} condition, (almost) irregardless of the operation and its context. @include condition-sb-ext-timeout.texinfo @node Operations Supporting Timeouts and Deadlines @comment node-name, @subsection Operations Supporting Timeouts and Deadlines @multitable @columnfractions .5 .25 .25 @headitem Operation @tab Timeout Parameter @tab Affected by Deadlines @item @code{cl:sleep} @tab - @tab since SBCL 1.4.3 @item @code{cl:read-line}, etc. @tab no @tab yes @item @ref{Macro sb-ext wait-for,,@code{wait-for}}@: @tab yes @tab yes @item @ref{Function sb-ext process-wait,,@code{process-wait}}@: @tab no @tab yes @item @ref{Function sb-thread grab-mutex,,@code{grab-mutex}}@: @tab yes @tab yes @item @ref{Function sb-thread condition-wait,,@code{condition-wait}}@: @tab yes @tab yes @item @ref{Function sb-thread wait-on-semaphore,,@code{wait-on-semaphore}}@: @tab yes @tab yes @item @ref{Function sb-thread join-thread,,@code{join-thread}}@: @tab yes @tab yes @item @ref{Function sb-concurrency receive-message,,@code{receive-message}}@: @tab yes @tab yes? @item @ref{Function sb-concurrency wait-on-gate,,@code{wait-on-gate}}@: @tab yes @tab yes? @item @ref{Macro sb-concurrency frlock-write,,@code{frlock-write}}@: @tab yes @tab yes? @item @ref{Function sb-concurrency grab-frlock-write-lock,,@code{grab-frlock-write-lock}}@: @tab yes @tab yes? @end multitable @node Miscellaneous Extensions @comment node-name, next, previous, up @section Miscellaneous Extensions @include fun-sb-ext-array-storage-vector.texinfo @include fun-sb-ext-delete-directory.texinfo @include fun-sb-ext-get-time-of-day.texinfo @include fun-sb-ext-assert-version-gt=.texinfo @include fun-sb-ext-unencapsulated-function.texinfo @node Stale Extensions @comment node-name, next, previous, up @section Stale Extensions SBCL has inherited from CMUCL various hooks to allow the user to tweak and monitor the garbage collection process. These are somewhat stale code, and their interface might need to be cleaned up. If you have urgent need of them, look at the code in @file{src/code/gc.lisp} and bring it up on the developers' mailing list. @findex @sbext{float-denormalized-p} SBCL has various hooks inherited from CMUCL, like @code{sb-ext:float-denormalized-p}, to allow a program to take advantage of IEEE floating point arithmetic properties which aren't conveniently or efficiently expressible using the ANSI standard. These look good, and their interface looks good, but IEEE support is slightly broken due to a stupid decision to remove some support for infinities (because it wasn't in the ANSI spec and it didn't occur to me that it was in the IEEE spec). If you need this stuff, take a look at the code and bring it up on the developers' mailing list. @node Efficiency Hacks @comment node-name, next, previous, up @section Efficiency Hacks @findex @sbext{purify} The @code{sb-ext:purify} function causes SBCL first to collect all garbage, then to mark all uncollected objects as permanent, never again attempting to collect them as garbage. This can cause a large increase in efficiency when using a primitive garbage collector, or a more moderate increase in efficiency when using a more sophisticated garbage collector which is well suited to the program's memory usage pattern. It also allows permanent code to be frozen at fixed addresses, a precondition for using copy-on-write to share code between multiple Lisp processes. This is less important with modern generational garbage collectors, but not all SBCL platforms use such a garbage collector. @comment @include fun-sb-ext-purify.texinfo The @code{sb-ext:truly-the} special form declares the type of the result of the operations, producing its argument; the declaration is not checked. In short: don't use it. @include special-operator-sb-ext-truly-the.texinfo @cindex @code{freeze-type} declaration @cindex Declaration, @code{freeze-type} The @code{sb-ext:freeze-type} declaration declares that a type will never change, which can make type testing (@code{typep}, etc.) more efficient for structure types.