common-lisp/sbcl.sbcl
1
0
Fork 0
mirror of git://git.code.sf.net/p/sbcl/sbcl synced 2026-09-10 07:26:40 -04:00
Code Issues Actions 8 Packages Projects Releases Wiki Activity

Initial revision

Browse source
This commit is contained in:
William Harold Newman 2000-09-18 01:26:16 +00:00
commit a530bbe337
460 changed files with 218837 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
.cvsignore Normal file
View file

@ -0,0 +1,4 @@
obj
output
ChangeLog
local-target-features.lisp-expr

779
BUGS Normal file
View file

@ -0,0 +1,779 @@
REPORTING BUGS
Bugs can be reported on the help mailing list
sbcl-help@lists.sourceforge.net
or on the development mailing list
sbcl-devel@lists.sourceforge.net
Please please please include enough information in a bug report
that someone reading it can reproduce the problem, i.e. don't write
Subject: apparent bug in PRINT-OBJECT (or *PRINT-LENGTH*?)
PRINT-OBJECT doesn't seem to work with *PRINT-LENGTH*. Is this a bug?
but instead
Subject: apparent bug in PRINT-OBJECT (or *PRINT-LENGTH*?)
Under sbcl-1.2.3, when I compile and load the file
(DEFSTRUCT (FOO (:PRINT-OBJECT (LAMBDA (X Y)
(LET ((*PRINT-LENGTH* 4))
(PRINT X Y)))))
X Y)
then at the command line type
(MAKE-FOO)
the program loops endlessly instead of printing the object.
KNOWN PORT-SPECIFIC BUGS
The breakpoint-based TRACE facility doesn't work properly in the
OpenBSD port of sbcl-0.6.7.
KNOWN BUGS
(There is also some information on bugs in the manual page and in the
TODO file. Eventually more such information may move here.)
* (DESCRIBE NIL) causes an endless loop.
* The FUNCTION special operator doesn't check properly whether its
argument is a function name. E.g. (FUNCTION (X Y)) returns a value
instead of failing with an error.
* (DESCRIBE 'GF) fails where GF is the name of a generic function:
The function SB-IMPL::DESCRIBE-INSTANCE is undefined.
* Failure in initialization files is not handled gracefully -- it's
a throw to TOP-LEVEL-CATCHER, which is not caught until we enter
TOPLEVEL-REPL. Code should be added to catch such THROWs even when
we're not in TOPLEVEL-REPL and do *something* with them (probably
complaining about an error outside TOPLEVEL-REPL, perhaps printing
a BACKTRACE, then terminating execution of SBCL).
* COMPILED-FUNCTION-P bogusly reports T for interpreted functions:
* (DEFUN FOO (X) (- 12 X))
FOO
* (COMPILED-FUNCTION-P #'FOO)
T
* The CL:STEP macro is undefined.
* DEFSTRUCT should almost certainly overwrite the old LAYOUT information
instead of just punting when a contradictory structure definition
is loaded.
* It should cause a STYLE-WARNING, not a full WARNING, when a structure
slot default value does not match the declared structure slot type.
(The current behavior is consistent with SBCL's behavior elsewhere,
and would not be a problem, except that the other behavior is
specifically required by the ANSI spec.)
* It should cause a STYLE-WARNING, not a WARNING, when the system ignores
an FTYPE proclamation for a slot accessor.
* Missing ordinary arguments in a macro call aren't reported when the
macro lambda list contains &KEY:
(DEFMACRO FOO (BAR &KEY) BAR) => FOO
(FOO) => NIL
Also in DESTRUCTURING-BIND:
(DESTRUCTURING-BIND (X Y &REST REST) '(1) (VECTOR X Y REST))
=> #(1 NIL NIL)
Also with &REST lists:
(DEFMACRO FOO (BAR &REST REST) BAR) => FOO
(FOO) => NIL
* Error reporting on various stream-requiring operations is not
very good when the stream argument has the wrong type, because
the operation tries to fall through to Gray stream code, and then
dies because it's undefined. E.g.
(PRINT-UNREADABLE-OBJECT (*STANDARD-OUTPUT* 1))
gives the error message
error in SB-KERNEL::UNDEFINED-SYMBOL-ERROR-HANDLER:
The function SB-IMPL::STREAM-WRITE-STRING is undefined.
It would be more useful and correct to signal a TYPE-ERROR:
not a STREAM: 1
(It wouldn't be terribly difficult to write stubs for all the
Gray stream functions that the old CMU CL code expects, with
each stub just raising the appropriate TYPE-ERROR.)
* bogus warnings about undefined functions for magic functions like
SB!C::%%DEFUN and SB!C::%DEFCONSTANT when cross-compiling files
like src/code/float.lisp
* The "byte compiling top-level form:" output ought to be condensed.
Perhaps any number of such consecutive lines ought to turn into a
single "byte compiling top-level forms:" line.
* The handling of IGNORE declarations on lambda list arguments of DEFMETHOD
is at least weird, and in fact seems broken and useless. I should
fix up another layer of binding, declared IGNORABLE, for typed
lambda list arguments.
* Compiling a file containing the erroneous program
(DEFSTRUCT FOO
A
B)
(DEFSTRUCT (BAR (:INCLUDE FOO))
A
B)
gives only the not-very-useful message
caught ERROR:
(during macroexpansion)
Condition PROGRAM-ERROR was signalled.
(The specific message which says that the problem was duplicate
slot names gets lost.)
* The way that the compiler munges types with arguments together
with types with no arguments (in e.g. TYPE-EXPAND) leads to
weirdness visible to the user:
(DEFTYPE FOO () 'FIXNUM)
(TYPEP 11 'FOO) => T
(TYPEP 11 '(FOO)) => T, which seems weird
(TYPEP 11 'FIXNUM) => T
(TYPEP 11 '(FIXNUM)) signals an error, as it should
The situation is complicated by the presence of Common Lisp types
like UNSIGNED-BYTE (which can either be used in list form or alone)
so I'm not 100% sure that the behavior above is actually illegal.
But I'm 90+% sure, and someday perhaps I'll be motivated to look it up..
* It would be nice if the
caught ERROR:
(during macroexpansion)
said what macroexpansion was at fault, e.g.
caught ERROR:
(during macroexpansion of IN-PACKAGE,
during macroexpansion of DEFFOO)
* The type system doesn't understand the KEYWORD type very well:
(SUBTYPEP 'KEYWORD 'SYMBOL) => NIL, NIL
It might be possible to fix this by changing the definition of
KEYWORD to (AND SYMBOL (SATISFIES KEYWORDP)), but the type system
would need to be a bit smarter about AND types, too:
(SUBTYPEP '(AND SYMBOL KEYWORD) 'SYMBOL) => NIL, NIL
(The type system does know something about AND types already,
(SUBTYPEP '(AND INTEGER FLOAT) 'NUMBER) => T, T
(SUBTYPEP '(AND INTEGER FIXNUM) 'NUMBER) =>T, T
so likely this is a small patch.)
* Floating point infinities are screwed up. [When I was converting CMU CL
to SBCL, I was looking for complexity to delete, and I thought it was safe
to just delete support for floating point infinities. It wasn't: they're
generated by the floating point hardware even when we remove support
for them in software. -- WHN] Support for them should be restored.
* The ANSI syntax for non-STANDARD method combination types in CLOS is
(DEFGENERIC FOO (X) (:METHOD-COMBINATION PROGN))
(DEFMETHOD FOO PROGN ((X BAR)) (PRINT 'NUMBER))
If you mess this up, omitting the PROGN qualifier in in DEFMETHOD,
(DEFGENERIC FOO (X) (:METHOD-COMBINATION PROGN))
(DEFMETHOD FOO ((X BAR)) (PRINT 'NUMBER))
the error mesage is not easy to understand:
INVALID-METHOD-ERROR was called outside the dynamic scope
of a method combination function (inside the body of
DEFINE-METHOD-COMBINATION or a method on the generic
function COMPUTE-EFFECTIVE-METHOD).
It would be better if it were more informative, a la
The method combination type for this method (STANDARD) does
not match the method combination type for the generic function
(PROGN).
Also, after you make the mistake of omitting the PROGN qualifier
on a DEFMETHOD, doing a new DEFMETHOD with the correct qualifier
no longer works:
(DEFMETHOD FOO PROGN ((X BAR)) (PRINT 'NUMBER))
gives
INVALID-METHOD-ERROR was called outside the dynamic scope
of a method combination function (inside the body of
DEFINE-METHOD-COMBINATION or a method on the generic
function COMPUTE-EFFECTIVE-METHOD).
This is not very helpful..
* The message "The top of the stack was encountered." from the debugger
is not helpful when I type "FRAME 0" -- I know I'm going to the top
of the stack.
* (SUBTYPEP '(FUNCTION (T BOOLEAN) NIL)
'(FUNCTION (FIXNUM FIXNUM) NIL)) => T, T
(Also, when this is fixed, we can enable the code in PROCLAIM which
checks for incompatible FTYPE redeclarations.)
* The ANSI spec says that CONS can be a compound type spec, e.g.
(CONS FIXNUM REAL). SBCL doesn't support this.
* from Paolo Amoroso on the CMU CL mailing list 27 Feb 2000:
I use CMU CL 18b under Linux. When COMPILE-FILE is supplied a physical
pathname, the type of the corresponding compiled file is X86F:
* (compile-file "/home/paolo/lisp/tools/foo")
Python version 1.0, VM version Intel x86 on 27 FEB 0 06:00:46 pm.
Compiling: /home/paolo/lisp/tools/foo.lisp 27 FEB 0 05:57:42 pm
Converted SQUARE.
Compiling DEFUN SQUARE:
Byte Compiling Top-Level Form:
/home/paolo/lisp/tools/foo.x86f written.
Compilation finished in 0:00:00.
#p"/home/paolo/lisp/tools/foo.x86f"
NIL
NIL
But when the function is called with a logical pathname, the file type
becomes FASL:
* (compile-file "tools:foo")
Python version 1.0, VM version Intel x86 on 27 FEB 0 06:01:04 pm.
Compiling: /home/paolo/lisp/tools/foo.lisp 27 FEB 0 05:57:42 pm
Converted SQUARE.
Compiling DEFUN SQUARE:
Byte Compiling Top-Level Form:
TOOLS:FOO.FASL written.
Compilation finished in 0:00:00.
#p"/home/paolo/lisp/tools/foo.fasl"
NIL
NIL
* from DTC on the CMU CL mailing list 25 Feb 2000:
;;; Compiler fails when this file is compiled.
;;;
;;; Problem shows up in delete-block within ir1util.lisp. The assertion
;;; (assert (member (functional-kind lambda) '(:let :mv-let :assignment)))
;;; fails within bind node branch.
;;;
;;; Note that if c::*check-consistency* is enabled then an un-reached
;;; entry is also reported.
;;;
(defun foo (val)
(declare (values nil))
nil)
(defun bug (val)
(multiple-value-call
#'(lambda (res)
(block nil
(tagbody
loop
(when res
(return nil))
(go loop))))
(foo val))
(catch 'ccc1
(throw 'ccc1
(block bbbb
(tagbody
(let ((ttt #'(lambda () (go cccc))))
(declare (special ttt))
(return-from bbbb nil))
cccc
(return-from bbbb nil))))))
* (I *think* this is a bug. It certainly seems like strange behavior. But
the ANSI spec is scary, dark, and deep..)
(FORMAT NIL "~,1G" 1.4) => "1. "
(FORMAT NIL "~3,1G" 1.4) => "1. "
* from Marco Antoniotti on cmucl-imp mailing list 1 Mar 2000:
(defclass ccc () ())
(setf (find-class 'ccc1) (find-class 'ccc))
(defmethod zut ((c ccc1)) 123)
DTC's recommended workaround from the mailing list 3 Mar 2000:
(setf (pcl::find-class 'ccc1) (pcl::find-class 'ccc))
* There's probably a bug in the compiler handling of special variables
in closures, inherited from the CMU CL code, as reported on the
CMU CL mailing list. There's a patch for this on the CMU CL
mailing list too:
Message-ID: <38C8E188.A1E38B5E@jeack.com.au>
Date: Fri, 10 Mar 2000 22:50:32 +1100
From: "Douglas T. Crosher" <dtc@jeack.com.au>
* The ANSI spec, in section "22.3.5.2 Tilde Less-Than-Sign: Logical Block",
says that an error is signalled if ~W, ~_, ~<...~:>, ~I, or ~:T is used
inside "~<..~>" (without the colon modifier on the closing syntax).
However, SBCL doesn't do this:
* (FORMAT T "~<munge~wegnum~>" 12)
munge12egnum
NIL
* When too many files are opened, OPEN will fail with an
uninformative error message
error in function OPEN: error opening #P"/tmp/foo.lisp": NIL
instead of saying that too many files are open.
* Right now, when COMPILE-FILE has a read error, it actually pops
you into the debugger before giving up on the file. It should
instead handle the error, perhaps issuing (and handling)
a secondary error "caught ERROR: unrecoverable error during compilation"
and then return with FAILURE-P true,
* The print system doesn't conform to ANSI
"22.1.3.3.1 Package Prefixes for Symbols" for keywords printed when
*PACKAGE* is the KEYWORD package.
from a message by Ray Toy on CMU CL mailing list Fri, 28 Apr 2000:
In a discussion on comp.lang.lisp, the following code was given (by
Erik Naggum):
(let ((*package* (find-package :keyword)))
(write-to-string object :readably t))
If OBJECT is a keyword, CMUCL prints out the keyword, but without a
colon. Hence, it's not readable, as requested.
I think the following patch will make this work as expected. The
patch just basically checks for the keyword package first before
checking the current package.
Ray
--- ../cmucl-18c/src/code/print.lisp Wed Dec 8 14:33:47 1999
+++ ../cmucl-18c/new/code/print.lisp Fri Apr 28 09:21:29 2000
@@ -605,12 +605,12 @@
(let ((package (symbol-package object))
(name (symbol-name object)))
(cond
- ;; If the symbol's home package is the current one, then a
- ;; prefix is never necessary.
- ((eq package *package*))
;; If the symbol is in the keyword package, output a colon.
((eq package *keyword-package*)
(write-char #\: stream))
+ ;; If the symbol's home package is the current one, then a
+ ;; prefix is never necessary.
+ ((eq package *package*))
;; Uninterned symbols print with a leading #:.
((null package)
(when (or *print-gensym* *print-readably*)
* from CMU CL mailing list 01 May 2000
I realize I can take care of this by doing (proclaim (ignore pcl::.slots1.))
but seeing as .slots0. is not-exported, shouldn't it be ignored within the
+expansion
when not used?
In: DEFMETHOD FOO-BAR-BAZ (RESOURCE-TYPE)
(DEFMETHOD FOO-BAR-BAZ
((SELF RESOURCE-TYPE))
(SETF (SLOT-VALUE SELF 'NAME) 3))
--> BLOCK MACROLET PCL::FAST-LEXICAL-METHOD-FUNCTIONS
--> PCL::BIND-FAST-LEXICAL-METHOD-MACROS MACROLET
--> PCL::BIND-LEXICAL-METHOD-FUNCTIONS LET PCL::BIND-ARGS LET* PCL::PV-BINDING
--> PCL::PV-BINDING1 PCL::PV-ENV LET
==>
(LET ((PCL::.SLOTS0. #))
(PROGN SELF)
(BLOCK FOO-BAR-BAZ
(LET #
#)))
Warning: Variable PCL::.SLOTS0. defined but never used.
Compilation unit finished.
1 warning
#<Standard-Method FOO-BAR-BAZ (RESOURCE-TYPE) {480918FD}>
* reported by Sam Steingold on the cmucl-imp mailing list 12 May 2000:
Also, there is another bug: `array-displacement' should return an array
or nil as first value (as per ANSI CL), while CMUCL declares it as
returning an array as first value always.
* Sometimes (SB-EXT:QUIT) fails with
Argh! maximum interrupt nesting depth (4096) exceeded, exiting
Process inferior-lisp exited abnormally with code 1
I haven't noticed a repeatable case of this yet.
* The system accepts DECLAIM in most places where DECLARE would be
accepted, without even issuing a warning. ANSI allows this, but since
it's fairly easy to mistype DECLAIM instead of DECLARE, and the
meaning is rather different, and it's unlikely that the user
has a good reason for doing DECLAIM not at top level, it would be
good to issue a STYLE-WARNING when this happens. A possible
fix would be to issue STYLE-WARNINGs for DECLAIMs not at top level,
or perhaps to issue STYLE-WARNINGs for any EVAL-WHEN not at top level.
* There seems to be some sort of bug in the interaction of the
normal compiler, the byte compiler, and type predicates.
Compiling and loading this file
(IN-PACKAGE :CL-USER)
(DEFSTRUCT FOO A B)
(PROGN
(DECLAIM (FTYPE (FUNCTION (FOO) FOO) FOO-BAR))
(DECLAIM (INLINE FOO-BAR))
(DEFUN FOO-BAR (FOO)
(DECLARE (TYPE FOO FOO))
(LET ((RESULT2605 (BLOCK FOO-BAR (PROGN (THE FOO (FOO-A FOO))))))
(UNLESS (TYPEP RESULT2605 'FOO)
(LOCALLY (ERROR "OOPS")))
(THE FOO RESULT2605)))
'FOO-BAR)
(DEFPARAMETER *FOO* (MAKE-FOO :A (MAKE-FOO)))
(UNLESS (EQ *PRINT-LEVEL* 133)
(DEFUN CK? ()
(LABELS ((FLOOD ()
(WHEN (TYPEP *X* 'FOO)
(FOO-BAR *Y*))))))
(PRINT 11)
(PRINT (FOO-BAR *FOO*))
(PRINT 12))
in sbcl-0.6.5 (or also in CMU CL 18b for FreeBSD) gives a call
to the undefined function SB-C::%INSTANCE-TYPEP. %INSTANCE-TYPEP
is not defined as a function because it's supposed to
be transformed away. My guess is what's happening is that
the mixture of toplevel and non-toplevel stuff and inlining
is confusing the system into compiling an %INSTANCE-TYPEP
form into byte code, where the DEFTRANSFORM which is supposed
to get rid of such forms is not effective.
* some sort of bug in inlining and RETURN-FROM in sbcl-0.6.5: Compiling
(DEFUN BAR? (X)
(OR (NAR? X)
(BLOCK USED-BY-SOME-Y?
(FLET ((FROB (STK)
(DOLIST (Y STK)
(UNLESS (REJECTED? Y)
(RETURN-FROM USED-BY-SOME-Y? T)))))
(DECLARE (INLINE FROB))
(FROB (RSTK X))
(FROB (MRSTK X)))
NIL)))
gives
error in function SB-KERNEL:ASSERT-ERROR:
The assertion (EQ (SB-C::CONTINUATION-KIND SB-C::CONT) :BLOCK-START) failed.
* The CMU CL reader code takes liberties in binding the standard read table
when reading the names of characters. Tim Moore posted a patch to the
CMU CL mailing list Mon, 22 May 2000 21:30:41 -0700.
* In some cases the compiler believes type declarations on array
elements without checking them, e.g.
(DECLAIM (OPTIMIZE (SAFETY 3) (SPEED 1) (SPACE 1)))
(DEFSTRUCT FOO A B)
(DEFUN BAR (X)
(DECLARE (TYPE (SIMPLE-ARRAY CONS 1) X))
(WHEN (CONSP (AREF X 0))
(PRINT (AREF X 0))))
(BAR (VECTOR (MAKE-FOO :A 11 :B 12)))
prints
#S(FOO :A 11 :B 12)
in SBCL 0.6.5 (and also in CMU CL 18b). This does not happen for
all cases, e.g. the type assumption *is* checked if the array
elements are declared to be of some structure type instead of CONS.
* The printer doesn't report closures very well. This is true in
CMU CL 18b as well:
(PRINT #'CLASS-NAME)
gives
#<Closure Over Function "DEFUN STRUCTURE-SLOT-ACCESSOR" {134D1A1}>
It would be nice to make closures have a settable name slot,
and make things like DEFSTRUCT and FLET, which create closures,
set helpful values into this slot.
* And as long as we're wishing, it would be awfully nice if INSPECT could
also report on closures, telling about the values of the bound variables.
* as reported by Robert Strandh on the CMU CL mailing list 12 Jun 2000:
$ cat xx.lisp
(defconstant +a-constant+ (make-instance 'a-class))
(defconstant +another-constant+ (vector +a-constant+))
$ lisp
CMU Common Lisp release x86-linux 2.4.19 8 February 2000 build 456,
running on
bobby
Send bug reports and questions to your local CMU CL maintainer,
or to pvaneynd@debian.org
or to cmucl-help@cons.org. (prefered)
type (help) for help, (quit) to exit, and (demo) to see the demos
Loaded subsystems:
Python 1.0, target Intel x86
CLOS based on PCL version: September 16 92 PCL (f)
* (defclass a-class () ())
#<STANDARD-CLASS A-CLASS {48027BD5}>
* (compile-file "xx.lisp")
Python version 1.0, VM version Intel x86 on 12 JUN 00 08:12:55 am.
Compiling:
/home/strandh/Research/Functional/Common-Lisp/CLIM/Development/McCLIM
/xx.lisp 12 JUN 00 07:47:14 am
Compiling Load Time Value of (PCL::GET-MAKE-INSTANCE-FUNCTION-SYMBOL
'(A-CLASS NIL NIL)):
Byte Compiling Top-Level Form:
Error in function C::DUMP-STRUCTURE: Attempt to dump invalid
structure:
#<A-CLASS {4803A5B5}>
How did this happen?
* The compiler assumes that any time a function of declared FTYPE
doesn't signal an error, its arguments were of the declared type.
E.g. compiling and loading
(DECLAIM (OPTIMIZE (SAFETY 3)))
(DEFUN FACTORIAL (X) (GAMMA (1+ X)))
(DECLAIM (FTYPE (FUNCTION (UNSIGNED-BYTE) FACTORIAL)))
(DEFUN FOO (X)
(COND ((> (FACTORIAL X) 1.0E6)
(FORMAT T "too big~%"))
((INTEGERP X)
(FORMAT T "exactly ~S~%" (FACTORIAL X)))
(T
(FORMAT T "approximately ~S~%" (FACTORIAL X)))))
then executing
(FOO 1.5)
will cause the INTEGERP case to be selected, giving bogus output a la
exactly 1.33..
This violates the "declarations are assertions" principle.
According to the ANSI spec, in the section "System Class FUNCTION",
this is a case of "lying to the compiler", but the lying is done
by the code which calls FACTORIAL with non-UNSIGNED-BYTE arguments,
not by the unexpectedly general definition of FACTORIAL. In any case,
"declarations are assertions" means that lying to the compiler should
cause an error to be signalled, and should not cause a bogus
result to be returned. Thus, the compiler should not assume
that arbitrary functions check their argument types. (It might
make sense to add another flag (CHECKED?) to DEFKNOWN to
identify functions which *do* check their argument types.)
* As pointed out by Martin Cracauer on the CMU CL mailing list
13 Jun 2000, the :FILE-LENGTH operation for
FD-STREAM-MISC-ROUTINE is broken for large files: it says
(THE INDEX SIZE) even though SIZE can be larger than INDEX.
* In SBCL 0.6.5 (and CMU CL 18b) compiling and loading
(in-package :cl-user)
(declaim (optimize (safety 3)
(debug 3)
(compilation-speed 2)
(space 1)
(speed 2)
#+nil (sb-ext:inhibit-warnings 2)))
(declaim (ftype (function * (values)) emptyvalues))
(defun emptyvalues (&rest rest) (declare (ignore rest)) (values))
(defstruct foo x y)
(defgeneric assertoid ((x t)))
(defmethod assertoid ((x t)) "just a placeholder")
(defun bar (ht)
(declare (type hash-table ht))
(let ((res
(block blockname
(progn
(prog1
(emptyvalues)
(assertoid (hash-table-count ht)))))))
(unless (typep res 'foo)
(locally
(common-lisp-user::bad-result-from-assertive-typed-fun
'bar
res)))))
then executing
(bar (make-hash-table))
causes the failure
Error in KERNEL::UNDEFINED-SYMBOL-ERROR-HANDLER:
the function C::%INSTANCE-TYPEP is undefined.
%INSTANCE-TYPEP is always supposed to be IR1-transformed away, but for
some reason -- the (VALUES) return value declaration? -- the optimizer is
confused and compiles a full call to %INSTANCE-TYPEP (which doesn't exist
as a function) instead.
* DEFMETHOD doesn't check the syntax of &REST argument lists properly,
accepting &REST even when it's not followed by an argument name:
(DEFMETHOD FOO ((X T) &REST) NIL)
* On the CMU CL mailing list 26 June 2000, Douglas Crosher wrote
Hannu Rummukainen wrote:
...
> There's something weird going on with the compilation of the attached
> code. Compiling and loading the file in a fresh lisp, then invoking
> (test-it) gives
Thanks for the bug report, nice to have this one fixed. It was a bug
in the x86 backend, the < VOP. A fix has been committed to the main
source, see the file compiler/x86/float.lisp.
Probably the same bug exists in SBCL.
* TYPEP treats the result of UPGRADED-ARRAY-ELEMENT-TYPE as gospel,
so that (TYPEP (MAKE-ARRAY 3) '(VECTOR SOMETHING-NOT-DEFINED-YET))
returns (VALUES T T). Probably it should be an error instead,
complaining that the type SOMETHING-NOT-DEFINED-YET is not defined.
* TYPEP of VALUES types is sometimes implemented very inefficiently, e.g. in
(DEFTYPE INDEXOID () '(INTEGER 0 1000))
(DEFUN FOO (X)
(DECLARE (TYPE INDEXOID X))
(THE (VALUES INDEXOID)
(VALUES X)))
where the implementation of the type check in function FOO
includes a full call to %TYPEP. There are also some fundamental problems
with the interpretation of VALUES types (inherited from CMU CL, and
from the ANSI CL standard) as discussed on the cmucl-imp@cons.org
mailing list, e.g. in Robert Maclachlan's post of 21 Jun 2000.
* The definitions of SIGCONTEXT-FLOAT-REGISTER and
%SET-SIGCONTEXT-FLOAT-REGISTER in x86-vm.lisp say they're not
supported on FreeBSD because the floating point state is not saved,
but at least as of FreeBSD 4.0, the floating point state *is* saved,
so they could be supported after all. Very likely
SIGCONTEXT-FLOATING-POINT-MODES could now be supported, too.
* (as discussed by Douglas Crosher on the cmucl-imp mailing list ca.
Aug. 10, 2000): CMUCL currently interprets 'member as '(member); same issue
with 'union, 'and, 'or etc. So even though according to the ANSI spec,
bare 'MEMBER, 'AND, and 'OR are not legal types, CMUCL (and now
SBCL) interpret them as legal types.
* ANSI specifies DEFINE-SYMBOL-MACRO, but it's not defined in SBCL.
CMU CL added it ca. Aug 13, 2000, after some discussion on the mailing
list, and it is probably possible to use substantially the same
patches to add it to SBCL.
* a slew of floating-point-related errors reported by Peter Van Eynde
on July 25, 2000:
* (SQRT -9.0) fails, because SB-KERNEL::COMPLEX-SQRT is undefined.
Similarly, COMPLEX-ASIN, COMPLEX-ACOS, COMPLEX-ACOSH, and others
aren't found.
* SBCL's value for LEAST-POSITIVE-SHORT-FLOAT is bogus, and
should probably be 1.4012985e-45. In SBCL,
(/ LEAST-POSITIVE-SHORT-FLOAT 2) returns a number smaller
than LEAST-POSITIVE-SHORT-FLOAT. Similar problems
exist for LEAST-NEGATIVE-SHORT-FLOAT, LEAST-POSITIVE-LONG-FLOAT,
and LEAST-NEGATIVE-LONG-FLOAT.
* Many expressions generate floating infinity:
(/ 1 0.0)
(/ 1 0.0d0)
(EXPT 10.0 1000)
(EXPT 10.0d0 1000)
PVE's regression tests want them to raise errors. SBCL
generates the infinities instead, which may or may not be
conforming behavior, but then blow it by being unable to
output the infinities, since support for infinities is generally
broken, and in particular SB-IMPL::OUTPUT-FLOAT-INFINITY is
undefined.
* (in section12.erg) various forms a la
(FLOAT 1 DOUBLE-FLOAT-EPSILON) don't give the right behavior.
* type safety errors reported by Peter Van Eynde July 25, 2000:
* (COERCE (QUOTE (A B C)) (QUOTE (VECTOR * 4)))
=> #(A B C)
In general lengths of array type specifications aren't
checked by COERCE, so it fails when the spec is
(VECTOR 4), (STRING 2), (SIMPLE-BIT-VECTOR 3), or whatever.
* CONCATENATE has the same problem of not checking the length
of specified output array types. MAKE-SEQUENCE and MAP and
MERGE also have the same problem.
* (COERCE 'AND 'FUNCTION) returns something related to
(MACRO-FUNCTION 'AND), but ANSI says it should raise an error.
* ELT signals SIMPLE-ERROR if its index argument
isn't a valid index for its sequence argument, but should
signal TYPE-ERROR instead.
* FILE-LENGTH is supposed to signal a type error when its
argument is not a stream associated with a file, but doesn't.
* (FLOAT-RADIX 2/3) should signal an error instead of
returning 2.
* (LOAD "*.lsp") should signal FILE-ERROR.
* (MAKE-CONCATENATED-STREAM (MAKE-STRING-OUTPUT-STREAM))
should signal TYPE-ERROR.
* MAKE-TWO-WAY-STREAM doesn't check that its arguments can
be used for input and output as needed. It should fail with
TYPE-ERROR when handed e.g. the results of MAKE-STRING-INPUT-STREAM
or MAKE-STRING-OUTPUT-STREAM in the inappropriate positions,
but doesn't.
* (PARSE-NAMESTRING (COERCE (LIST #\f #\o #\o (CODE-CHAR 0) #\4 #\8)
(QUOTE STRING)))
should probably signal an error instead of making a pathname with
a null byte in it.
* READ-BYTE is supposed to signal TYPE-ERROR when its argument is
not a binary input stream, but instead cheerfully reads from
character streams, e.g. (MAKE-STRING-INPUT-STREAM "abc").
* DEFCLASS bugs reported by Peter Van Eynde July 25, 2000:
* (DEFCLASS FOO () (A B A)) should signal a PROGRAM-ERROR, and doesn't.
* (DEFCLASS FOO () (A B A) (:DEFAULT-INITARGS X A X B)) should
signal a PROGRAM-ERROR, and doesn't.
* (DEFCLASS FOO07 NIL ((A :ALLOCATION :CLASS :ALLOCATION :CLASS))),
and other DEFCLASS forms with duplicate specifications in their
slots, should signal a PROGRAM-ERROR, and doesn't.
* (DEFGENERIC IF (X)) should signal a PROGRAM-ERROR, but instead
causes a COMPILER-ERROR.
* SYMBOL-MACROLET bugs reported by Peter Van Eynde July 25, 2000:
* (SYMBOL-MACROLET ((T TRUE)) ..) should probably signal
PROGRAM-ERROR, but SBCL accepts it instead.
* SYMBOL-MACROLET should refuse to bind something which is
declared as a global variable, signalling PROGRAM-ERROR.
* SYMBOL-MACROLET should signal PROGRAM-ERROR if something
it binds is declared SPECIAL inside.
* LOOP bugs reported by Peter Van Eynde July 25, 2000:
* (LOOP WITH (A B) DO (PRINT 1)) is a syntax error according to
the definition of WITH clauses given in the ANSI spec, but
compiles and runs happily in SBCL.
* a messy one involving package iteration:
interpreted Form: (LET ((PACKAGE (MAKE-PACKAGE "LOOP-TEST"))) (INTERN "blah" PACKAGE) (LET ((BLAH2 (INTERN "blah2" PACKAGE))) (EXPORT BLAH2 PACKAGE)) (LIST (SORT (LOOP FOR SYM BEING EACH PRESENT-SYMBOL OF PACKAGE FOR SYM-NAME = (SYMBOL-NAME SYM) COLLECT SYM-NAME) (FUNCTION STRING<)) (SORT (LOOP FOR SYM BEING EACH EXTERNAL-SYMBOL OF PACKAGE FOR SYM-NAME = (SYMBOL-NAME SYM) COLLECT SYM-NAME) (FUNCTION STRING<))))
Should be: (("blah" "blah2") ("blah2"))
SBCL: (("blah") ("blah2"))
* (LET ((X 1)) (LOOP FOR I BY (INCF X) FROM X TO 10 COLLECT I))
doesn't work -- SBCL's LOOP says BY isn't allowed in a FOR clause.
* type system errors reported by Peter Van Eynde July 25, 2000:
* (SUBTYPEP 'BIGNUM 'INTEGER) => NIL, NIL
but should be (VALUES T T) instead.
* (SUBTYPEP 'EXTENDED-CHAR 'CHARACTER) => NIL, NIL
but should be (VALUES T T) instead.
* (SUBTYPEP '(INTEGER (0) (0)) 'NIL) dies with nested errors.
* In general, the system doesn't like '(INTEGER (0) (0)) -- it
blows up at the level of SPECIFIER-TYPE with
"Lower bound (0) is greater than upper bound (0)." Probably
SPECIFIER-TYPE should return NIL instead.
* (TYPEP 0 '(COMPLEX (EQL 0)) fails with
"Component type for Complex is not numeric: (EQL 0)."
This might be easy to fix; the type system already knows
that (SUBTYPEP '(EQL 0) 'NUMBER) is true.
* The type system doesn't know about the condition system,
so that e.g. (TYPEP 'SIMPLE-ERROR 'ERROR)=>NIL.
* The type system isn't all that smart about relationships
between hairy types, as shown in the type.erg test results,
e.g. (SUBTYPEP 'CONS '(NOT ATOM)) => NIL, NIL.
* miscellaneous errors reported by Peter Van Eynde July 25, 2000:
* (PROGN
(DEFGENERIC FOO02 (X))
(DEFMETHOD FOO02 ((X NUMBER)) T)
(LET ((M (FIND-METHOD (FUNCTION FOO02)
NIL
(LIST (FIND-CLASS (QUOTE NUMBER))))))
(REMOVE-METHOD (FUNCTION FOO02) M)
(DEFGENERIC FOO03 (X))
(ADD-METHOD (FUNCTION FOO03) M)))
should give an error, but SBCL allows it.
* READ should probably return READER-ERROR, not the bare
arithmetic error, when input a la "1/0" or "1e1000" causes
an arithmetic error.
* There are several metaobject protocol "errors". (In order to fix
them, we might need to document exactly what metaobject
protocol specification we're following -- the current code is
just inherited from PCL.)
* (BUTLAST NIL) should return NIL. (This appears to be a compiler
bug, since the definition of BUTLAST, when interpreted, does
give (BUTLAST NIL)=>NIL.)
* another error from Peter Van Eynde 5 September 2000:
(FORMAT NIL "~F" "FOO") should work, but instead reports an error.
PVE submitted a patch to deal with this bug, but it exposes other
comparably serious bugs, so I didn't apply it. It looks as though
the FORMAT code needs a fair amount of rewriting in order to comply
with the various details of the ANSI spec.
* The bug discussed on the cmucl-imp@cons.org mailing list ca. 5 September,
simplified by Douglas Crosher down to
(defun tickle-bug ()
(labels ((fun1 ()
(fun2))
(fun2 ()
(when nil
(tagbody
tag
(fun2)
(go tag)))
(when nil
(tagbody
tag
(fun1)
(go tag)))))
(fun1)
nil))
causes the same problem on SBCL: compiling it fails with
:LET fell through ECASE expression.
Very likely the patch discussed there is appropriate for SBCL
as well, but I don't understand it, so I didn't apply it.

16
COPYING Normal file
View file

@ -0,0 +1,16 @@
SBCL is derived from CMU CL, which was released into the public
domain, subject only to the BSD-style "free, but credit must be given
and copyright notices must be retained" licenses in the LOOP macro
(from MIT and Symbolics) and in the PCL implementation of CLOS (from
Xerox).
After CMU CL was was released into the public domain, it was
maintained by volunteers, who continued the tradition of releasing
their work into the public domain.
All changes to SBCL since the fork from CMU CL have been released
into the public domain.
Thus, there are no known obstacles to copying, using, and modifying
SBCL freely, as long as the MIT, Symbolics, and Xerox copyright
notices are retained.

493
CREDITS Normal file
View file

@ -0,0 +1,493 @@
The programmers of old were mysterious and profound. We
cannot fathom their thoughts, so all we do is describe their
appearance.
Aware, like a fox crossing the water. Alert, like a general
on the battlefield. Kind, like a hostess greeting her guests.
Simple, like uncarved blocks of wood. Opaque, like black
pools in darkened caves.
Who can tell the secrets of their hearts and minds?
The answer exists only in the Tao.
-- Geoffrey James, "The Tao of Programming"
BROAD OUTLINE
SBCL is derived from the 18b version of CMU CL.
Most of CMU CL was originally written as part of the CMU Common Lisp
project at Carnegie Mellon University. According to the documentation
in CMU CL 18b,
Organizationally, CMU Common Lisp was a small, mostly autonomous
part within the Mach operating system project. The CMU CL project
was more of a tool development effort than a research project.
The project started out as Spice Lisp, which provided a modern
Lisp implementation for use in the CMU community.
and
CMU CL has been under continuous development since the early 1980's
(concurrent with the Common Lisp standardization effort.)
Apparently most of the CMU Common Lisp implementors moved on to
work on the Gwydion environment for Dylan.
CMU CL's CLOS implementation is derived from the PCL reference
implementation written at Xerox PARC.
CMU CL's implementation of the LOOP macro was derived from code
from Symbolics, which was derived from code from MIT.
CMU CL had many individual author credits in the source files. In the
sometimes-extensive rearrangements which were required to make SBCL
bootstrap itself cleanly, it was tedious to try keep such credits
attached to individual source files, so they have been moved here
instead.
William Harold Newman <william.newman@airmail.net> did this
transformation, and so any errors made are probably his. Corrections
would be appreciated.
MORE DETAILS ON SBCL'S CLOS CODE
The original headers of the PCL files contained the following text:
;;; Any person obtaining a copy of this software is requested to send their
;;; name and post office or electronic mail address to:
;;; CommonLoops Coordinator
;;; Xerox PARC
;;; 3333 Coyote Hill Rd.
;;; Palo Alto, CA 94304
;;; (or send Arpanet mail to CommonLoops-Coordinator.pa@Xerox.arpa)
;;;
;;; Suggestions, comments and requests for improvements are also welcome.
This was intended for the original incarnation of the PCL code as a
portable reference implementation. Since our version of the code has
had its portability hacked out of it, it's no longer particularly
relevant to any coordinated PCL effort (which probably doesn't exist
any more anyway). Therefore, this contact information has been deleted
from the PCL file headers.
A few files in the original CMU CL 18b src/pcl/ directory did not
carry such Xerox copyright notices:
* Some code was originally written by Douglas T. Crosher for CMU CL:
** the Gray streams implementation
** the implementation of DOCUMENTATION as methods of a generic
function
* generic-functions.lisp seems to have been machine-generated.
The comments in the CMU CL 18b version of the PCL code walker,
src/pcl/walk.lisp, said in part
;;; a simple code walker, based IN PART on: (roll the credits)
;;; Larry Masinter's Masterscope
;;; Moon's Common Lisp code walker
;;; Gary Drescher's code walker
;;; Larry Masinter's simple code walker
;;; .
;;; .
;;; boy, thats fair (I hope).
MORE DETAILS ON SBCL'S LOOP CODE
The src/code/loop.lisp file from CMU CL 18b had the following
credits-related information in it:
;;; The LOOP iteration macro is one of a number of pieces of code
;;; originally developed at MIT for which free distribution has been
;;; permitted, as long as the code is not sold for profit, and as long
;;; as notification of MIT's interest in the code is preserved.
;;;
;;; This version of LOOP, which is almost entirely rewritten both as
;;; clean-up and to conform with the ANSI Lisp LOOP standard, started
;;; life as MIT LOOP version 829 (which was a part of NIL, possibly
;;; never released).
;;;
;;; A "light revision" was performed by me (Glenn Burke) while at
;;; Palladian Software in April 1986, to make the code run in Common
;;; Lisp. This revision was informally distributed to a number of
;;; people, and was sort of the "MIT" version of LOOP for running in
;;; Common Lisp.
;;;
;;; A later more drastic revision was performed at Palladian perhaps a
;;; year later. This version was more thoroughly Common Lisp in style,
;;; with a few miscellaneous internal improvements and extensions. I
;;; have lost track of this source, apparently never having moved it to
;;; the MIT distribution point. I do not remember if it was ever
;;; distributed.
;;;
;;; The revision for the ANSI standard is based on the code of my April
;;; 1986 version, with almost everything redesigned and/or rewritten.
The date of the M.I.T. copyright statement falls around the time
described in these comments. The dates on the Symbolics copyright
statement are all later -- the earliest is 1989.
MORE DETAILS ON OTHER SBCL CODE FROM CMU CL
CMU CL's symbol (but not package) code (code/symbol.lisp) was
originally written by Scott Fahlman and updated and maintained
by Skef Wholey.
The CMU CL reader (code/reader.lisp) was originally the Spice Lisp
reader, written by David Dill and with support for packages added by
Lee Schumacher. David Dill also wrote the sharpmacro support
(code/sharpm.lisp).
CMU CL's package code was rewritten by Rob MacLachlan based on an
earlier version by Lee Schumacher. It also includes DEFPACKAGE by Dan
Zigmond, and WITH-PACKAGE-ITERATOR written by Blaine Burks. William
Lott also rewrote the DEFPACKAGE and DO-FOO-SYMBOLS stuff.
CMU CL's string code (code/string.lisp) was originally written by
David Dill, then rewritten by Skef Wholey, Bill Chiles, and Rob
MacLachlan.
Various code in the system originated with "Spice Lisp", which was
apparently a predecessor to the CMU CL project. Much of that was
originally written by Skef Wholey:
code/seq.lisp, generic sequence functions, and COERCE
code/array.lisp, general array stuff
SXHASH
code/list.lisp, list functions (based on code from Joe Ginder and
Carl Ebeling)
The CMU CL seq.lisp code also gave credits for later work by Jim Muller
and Bill Chiles.
The modules system (code/module.lisp, containing REQUIRE, PROVIDE,
and friends, now deprecated by ANSI) was written by Jim Muller and
rewritten by Bill Chiles.
The CMU CL garbage collector was credited to "Christopher Hoover,
Rob MacLachlan, Dave McDonald, et al." in the CMU CL code/gc.lisp file,
with some extra code for the MIPS port credited to Christopher Hoover
alone.
Guy Steele wrote the original character functions
code/char.lisp
They were subsequently rewritten by David Dill, speeded up by Scott Fahlman,
and rewritten without fonts and with a new type system by Rob MachLachlan.
Lee Schumacher made the Spice Lisp version of backquote. The comment
in the CMU CL sources suggests he based it on someone else's code for
some other Lisp system, but doesn't say which. A note in the CMU CL
code to pretty-print backquote expressions says that unparsing support
was provided by Miles Bader.
The CMU implementations of the Common Lisp query functions Y-OR-N-P
and YES-OR-NO-P were originally written by Walter van Roggen, and
updated and modified by Rob MacLachlan and Bill Chiles.
The CMU CL sort functions (code/sort.lisp) were written by Jim Large,
hacked on and maintained by Skef Wholey, and rewritten by Bill Chiles.
Most of the internals of the Python compiler seem to have been
originally written by Robert MacLachlan:
the type system and associated "cold load hack magic"
code/typedefs.lisp
code/class.lisp
code/type-init.lisp
etc.
the lexical environment database
compiler/globaldb.lisp, etc.
the IR1 representation and optimizer
compiler/ir1*.lisp, etc.
the IR2 representation and optimizer
compiler/ir2*.lisp, etc.
many concrete optimizations
compiler/srctran.lisp (with some code adapted from
CLC by Wholey and Fahlman)
compiler/float-tran.lisp, etc.
information about optimization of known functions
compiler/fndb.lisp
debug information representation
compiler/debug.lisp, compiler/debug-dump.lisp
memory pools to reduce consing by reusing compiler objects
compiler/alloc.lisp
toplevel interface functions and drivers
compiler/main.lisp
Besides writing the compiler, and various other work mentioned elsewhere,
Robert MacLachlan was also credited with tuning the implementation of
streams for Unix files, and writing
various floating point support code
code/float-trap.lisp, floating point traps
code/float.lisp, misc. support a la INTEGER-DECODE-FLOAT
low-level time functions
code/time.lisp
William Lott is also credited with writing or heavily maintaining some
parts of the CMU CL compiler. He was responsible for lifting
compiler/meta-vmdef.lisp out of compiler/vmdef.lisp, and also wrote
various optimizations
compiler/array-tran.lisp
compiler/saptran.lisp
compiler/seqtran.lisp (with some code adapted from an older
seqtran written by Wholey and Fahlman)
the separable compiler backend
compiler/backend.lisp
compiler/generic/utils.lisp
the implementation of LOAD-TIME-VALUE
compiler/ltv.lisp
the most recent version of the assembler
compiler/new-assem.lisp
vop statistics gathering
compiler/statcount.lisp
centralized information about machine-dependent and..
..machine-independent FOO, with
compiler/generic/vm-fndb.lisp, FOO=function signatures
compiler/generic/vm-typetran.lisp, FOO=type ops
compiler/generic/objdef.lisp, FOO=object representation
compiler/generic/primtype.lisp, FOO=primitive types
Also, Christopher Hoover and William Lott wrote compiler/generic/vm-macs.lisp
to centralize information about machine-dependent macros and constants.
Sean Hallgren converted compiler/generic/primtype.lisp for the Alpha.
The CMU CL machine-independent disassembler (compiler/disassem.lisp)
was written by Miles Bader.
Parts of the CMU CL system were credited to Skef Wholey and Rob
MacLachlan jointly, perhaps because they were originally part of Spice
Lisp and were then heavily modified:
code/load.lisp, the loader, including all the FASL stuff
code/macros.lisp, various fundamental macros
code/mipsstrops.lisp, primitives for hacking strings
code/purify.lisp, implementation of PURIFY
code/stream.lisp, stream functions
code/lispinit.lisp, cold startup
code/profile.lisp, the profiler
Bill Chiles also modified code/macros.lisp. Much of the implementation
of PURIFY was rewritten in C by William Lott.
The CMU CL number functions (code/number.lisp) were written by Rob
MacLachlan, but acknowledge much code "derived from code written by
William Lott, Dave Mcdonald, Jim Large, Scott Fahlman, etc."
CMU CL's weak pointer support (code/weak.lisp) was written by
Christopher Hoover.
The CMU CL DEFSTRUCT system was credited to Rob MacLachlan, William
Lott and Skef Wholey jointly.
The FDEFINITION system for handling arbitrary function names (a la
(SETF FOO)) was originally written by Rob MacLachlan. It was modified
by Bill Chiles to add encapsulation, and modified more by William Lott
to add FDEFN objects.
The CMU CL condition system (code/error.lisp) was based on
some prototyping code written by Ken Pitman at Symbolics.
The CMU CL HASH-TABLE system was originally written by Skef Wholey
for Spice Lisp, then rewritten by William Lott, then rewritten
again by Douglas T. Crosher.
The support code for environment queries (a la LONG-SITE-NAME),
the DOCUMENTATION function, and the DRIBBLE function was written
and maintained "mostly by Skef Wholey and Rob MacLachlan. Scott
Fahlman, Dan Aronson, and Steve Handerson did stuff here too."
The same credit statement was given for the original Mach OS interface code.
The CMU CL printer, print.lisp, was credited as "written by
Neal Feinberg, Bill Maddox, Steven Handerson, and Skef Wholey, and
modified by various CMU Common Lisp maintainers."
The comments in the main body of the CMU CL debugger
code/debug.lisp
say that it was written by Bill Chiles. Some other related files
code/debug-int.lisp, programmer's interface to the debugger
code/ntrace.lisp, tracing facility based on breakpoints
say they were written by Bill Chiles and Rob MacLachlan.
The related file
src/debug-vm.lisp, low-level support for :FUNCTION-END breakpoints
was written by William Lott.
The CMU CL GENESIS cold load system,
compiler/generic/new-genesis.lisp, was originally written by Skef
Wholey, then jazzed up for packages by Rob MacLachlan, then completely
rewritten by William Lott for the MIPS port.
The CMU CL IR1 interpreter was written by Bill Chiles and Robert
MacLachlan.
Various CMU CL support code was written by William Lott:
the bytecode interpreter
code/byte-interp.lisp
bitblt-ish operations a la SYSTEM-AREA-COPY
code/bit-bash.lisp
Unix interface
code/fd-stream.lisp, Unix file descriptors as Lisp streams
code/filesys.lisp, other Unix filesystem interface stuff
handling errors signalled from assembly code
code/interr.lisp
compiler/generic/interr.lisp
finalization based on weak pointers
code/final.lisp
irrational numeric functions
code/irrat.lisp
the pretty printer
code/pprint.lisp
predicates (both type predicates and EQUAL and friends)
code/pred.lisp
saving the current Lisp image as a core file
code/save.lisp
handling Unix signals
code/signal.lisp
implementing FORMAT
code/format.lisp
The ALIEN facility seems to have been written largely by Rob
MacLachlan and William Lott. The CMU CL comments say "rewritten again,
this time by William Lott and Rob MacLachlan," but don't identify who
else might have been involved in earlier versions.
The comments in CMU CL's code/final.lisp say "the idea really was
Chris Hoover's". The comments in CMU CL's code/pprint.lisp say "Algorithm
stolen from Richard Waters' XP." The comments in CMU CL's code/format.lisp
say "with lots of stuff stolen from the previous version by David Adam
and later rewritten by Bill Maddox."
Jim Muller was credited with fixing seq.lisp.
CMU CL's time printing logic, in code/format-time.lisp, was written
by Jim Healy.
Bill Chiles was credited with fixing/updating seq.lisp after Jim Muller.
The CMU CL machine/filesystem-independent pathname functions
(code/pathname.lisp) were written by William Lott, Paul Gleichauf, and
Rob MacLachlan, based on an earlier version written by Jim Large and
Rob MacLachlan.
Besides writing the original versions of the things credited to him
above, William Lott rewrote, updated, and cleaned up various stuff:
code/array.lisp
code/serve-event.lisp
The INSPECT function was originally written by Blaine Burks.
The CMU CL DESCRIBE facility was originally written by "Skef Wholey or
Rob MacLachlan", according to the comments in the CMU CL sources. It
was cleaned up and reorganized by Blaine Burks, then ported and
cleaned up more by Rob MacLachlan. Also, since the split from CMU CL,
the SBCL DESCRIBE facility was rewritten as a generic function and so
become entangled with some DESCRIBE code which was distributed as part
of PCL.
The implementation of the Mersenne Twister RNG used in SBCL is based
on an implementation written by Douglas T. Crosher and Raymond Toy,
which was placed in the public domain with permission from M.
Matsumoto.
Comments in the CMU CL version of FreeBSD-os.c said it came from
an OSF version by Sean Hallgren, later hacked by Paul Werkowski,
with generational conservative GC support added by Douglas Crosher.
Comments in the CMU CL version of linux-os.c said it came from the
FreeBSD-os.c version, morfed to Linux by Peter Van Eynde in July 1996.
Comments in the CMU CL version of backtrace.c said it was "originally
from Rob's version" (presumably Robert Maclachlan).
Comments in the CMU CL version of purify.c said it had stack direction
changes, x86/CGC stack scavenging, and static blue bag stuff (all for
x86 port?) by Paul Werkowski, 1995, 1996; and bug fixes, x86 code
movement support, and x86/gencgc stack scavenging by Douglas Crosher,
1996, 1997, 1998.
According to comments in the source files, much of the CMU CL version
of the x86 support code
assembly/x86/alloc.lisp
assembly/x86/arith.lisp
assembly/x86/array.lisp
assembly/x86/assem-rtns.lisp
compiler/x86/alloc.lisp
compiler/x86/arith.lisp
compiler/x86/c-call.lisp
compiler/x86/call.lisp
compiler/x86/cell.lisp
compiler/x86/char.lisp
compiler/x86/debug.lisp
compiler/x86/float.lisp
compiler/x86/insts.lisp
compiler/x86/macros.lisp
compiler/x86/memory.lisp
compiler/x86/move.lisp
compiler/x86/nlx.lisp
compiler/x86/parms.lisp
compiler/x86/pred.lisp
compiler/x86/print.lisp
compiler/x86/sap.lisp
compiler/x86/static-fn.lisp
compiler/x86/subprim.lisp
compiler/x86/system.lisp
compiler/x86/type-vops.lisp
compiler/x86/values.lisp
compiler/x86/vm.lisp
was originally written by William Lott, then debugged by Paul
Werkowski, and in some cases later enhanced and further debugged by
Douglas T. Crosher; and the x86 runtime support code,
x86-assem.S
was written by Paul F. Werkowski and Douglas T. Crosher.
The CMU CL user manual (doc/cmu-user/cmu-user.tex) says that the X86
FreeBSD port was originally contributed by Paul Werkowski, and Peter
VanEynde took the FreeBSD port and created a Linux version.
According to comments in src/code/bsd-os.lisp, work on the generic BSD
port was done by Skef Wholey, Rob MacLachlan, Scott Fahlman, Dan
Aronson, and Steve Handerson.
Douglas Crosher wrote code to support Gray streams, added X86 support
for the debugger and relocatable code, wrote a conservative
generational GC for the X86 port, and added X86-specific extensions to
support stack groups and multiprocessing.
The CMU CL user manual credits Robert MacLachlan as editor. A chapter
on the CMU CL interprocess communication extensions (not supported in
SBCL) was contributed by William Lott and Bill Chiles.
Peter VanEynde also contributed a variety of #+HIGH-SECURITY patches
to CMU CL, to provide additional safety, especially through runtime
checking on various tricky cases of standard functions (e.g. MAP with
complicated result types, and interactions of various variants of
STREAM).
Raymond Toy wrote the propagate-float-type extension and various
other floating point optimizations.
CMU CL's long float support was written by Douglas T. Crosher.
Paul Werkowski turned the Mach OS support code into Linux OS support code.
Versions of the RUN-PROGRAM extension were written first by David
McDonald, then by Jim Healy and Bill Chiles, then by William Lott.
MORE DETAILS ON THE TRANSITION FROM CMU CL
Bill Newman did the original conversion from CMU CL 18b to a form
which could bootstrap itself cleanly, on Linux/x86 only. Although they
may not have realized it at the time, Rob Maclachlan and Peter Van
Eynde were very helpful, RAM by posting a clear explanation of what
GENESIS is supposed to be doing and PVE by maintaining a version of
CMU CL which worked on Debian, so that I had something to refer to
whenever I got stuck.
CREDITS SINCE THE RELEASE OF SBCL
The PSXHASH code used to implement EQUALP hash tables was originally
copyright (C) 2000 by Cadabra, Inc., then released into the public
domain.
Daniel Barlow contributed sblisp.lisp, a set of patches to make SBCL
play nicely with ILISP. (Those patches have since disappeared from the
SBCL distribution because ILISP has since been patched to play nicely
with SBCL.) He also figured out how to get the CMU CL dynamic object
file loading code to work under SBCL.
Raymond Wiker ported sbcl-0.6.3 back to FreeBSD, restoring the
ancestral CMU CL support for FreeBSD and updating it for the changes
made from FreeBSD version 3 to FreeBSD version 4.

134
INSTALL Normal file
View file

@ -0,0 +1,134 @@
IF YOU HAVE A BINARY DISTRIBUTION:
The two files that SBCL needs to run are sbcl and sbcl.core.
They are in
src/runtime/sbcl
and
output/sbcl.core
sbcl is a standard executable, built by compiling and linking an
ordinary C program. It provides the runtime environment for the
running Lisp image, but it doesn't know much about high-level Lisp
stuff (like symbols and printing and objects) so it's pretty useless
by itself. sbcl.core is a dump file written in a special SBCL format
which only sbcl understands, and it contains all the high-level Lisp
stuff.
In order to get a usable system, you need to run sbcl in a way that
it can find sbcl.core. There are three ways for it to find
sbcl.core:
1. by default, in /usr/lib/sbcl.core or /usr/local/lib/sbcl.core
2. by environment variable:
$ export SBCL_HOME=/foo/bar/
$ sbcl
3. by command line option:
$ sbcl --core /foo/bar/sbcl.core"
The usual, recommended approach is method #1. Method #2 is useful if
you're installing SBCL on a system in your user account, instead of
installing SBCL on an entire system. Method #3 is mostly useful for
testing or other special cases.
So: the standard installation procedure is
1. Copy sbcl.core to /usr/lib or /usr/local/lib.
2. Copy sbcl to /usr/bin or /usr/local/bin.
3. Optionally copy sbcl.1 to /usr/man/man1 or /usr/local/man/man1.
The script install.sh does these for you (choosing the /usr/local
subdirectory) in each case.
IF YOU HAVE A SOURCE DISTRIBUTION:
This software has been built successfully on these systems:
cpu = x86 (Intel 386 or higher, or compatibles like the AMD K6)
os = Debian GNU/Linux 2.1 with libc >= 2.1
host lisp = CMU CL 2.4.17
host lisp = SBCL itself
os = RedHat Linux 6.2
host lisp = SBCL itself
os = FreeBSD 3.4 or 4.0
host lisp = CMU CL
host lisp = SBCL itself
os = OpenBSD 2.6
host lisp = SBCL itself
It is known not to build under CLISP, because CLISP doesn't support
MAKE-LOAD-FORM. Reports of other systems that it works on, or help in
making it run on more systems, would be appreciated.
CAUTION CAUTION CAUTION CAUTION CAUTION
SBCL, like CMU CL, overcommits memory. That is, it
asks the OS for more virtual memory address space than
it actually intends to use, and the OS is expected to
optimistically give it this address space even if the OS
doesn't have enough RAM+swap to back it up. This works
fine as long as SBCL's memory usage pattern is sparse
enough that the OS can actually implement the requested
VM usage. Unfortunately, if the OS runs out of RAM+swap to
implement the requested VM usage, things get bad. On many
systems, including the Linux 2.2.13 kernel that I used for
development of SBCL up to version 0.6.0, the kernel kills
processes more-or-less randomly when it runs out of
resources. You may think your Linux box is very stable, but
it is unlikely to be stable if this happens.:-| So be sure
to have enough memory available when you build the system.
(This can be considered a bug in SBCL, or a bug in the
Unix overcommitment-of-memory architecture, or both. It's
not clear what the best fix is. On the SBCL side, Peter Van
Eynde has a lazy-allocation patch for CMU CL that lets
it run without overcommitting memory, and that could be
ported to SBCL, but unfortunately that might introduce
new issues, e.g. alien programs allocating memory in the
address space that SBCL thinks of as its own, and later
getting trashed when SBCL lazily allocates the memory.
On the OS side, there might be some way to address the
problem with quotas, I don't know.)
To build the system binaries:
1. Make sure that you have enough RAM+swap to build SBCL, as
per the CAUTION note above. (As of version 0.6.0, the most
memory-intensive operation in make.sh is the second call to
GENESIS, which makes the Lisp image grow to nearly 128 Mb RAM+swap.
This will probably be reduced somewhat in some later version
by allowing cold load of byte-compiled files, so that the cold
image can be smaller.)
2. If the GNU make command is not available under the name "gmake",
then define the environment variable GNUMAKE to a name where it can
be found.
3. If you like, you can edit the base-features.lisp-expr file
to customize the resulting Lisp system. By enabling or disabling
features in this file, you can create a smaller system, or one
with extra code for debugging output or error-checking or other things.
4. Run "sh make.sh" in the same directory where you unpacked the
tarball. If you don't already have a SBCL binary installed
as "sbcl" in your path, you'll need to tell make.sh what Lisp
system to use as the cross-compilation host. (To use CMU CL
as the cross-compilation host, run "sh make.sh 'lisp -batch'",
assuming CMU CL has been installed under its default name "lisp".)
5. Wait. This can be a slow process. On my test machines, the
wall clock time for a build of sbcl-0.6.7 was approximately
1.5 hours on a 450MHz K6/3 with 248Mb RAM, running RH Linux 6.2;
4 hours on a 200MHz Pentium (P54C) with 64Mb RAM, running FreeBSD 4.0;
13 hours on a 133MHz Pentium (P54C) with 48Mb RAM, running OpenBSD 2.6.
Around the 48Mb mark, the build process is starved for RAM:
on my 48Mb OpenBSD machine with nothing else running, it
spent about 2/3 of its wall clock time swapping. Anything which
substantially increases memory use, like running X11, Emacs, or,
God forbid, Netscape, can increase the build time substantially.
Now you should have the same src/runtime/sbcl and output/sbcl.core
files that come with the binary distribution, and you can install
them as in the "IF YOU HAVE A BINARY DISTRIBUTION" instructions (above).
To convert the DocBook version of the system documentation (files
ending in .sgml) to more-readable form (HTML or text):
DocBook is an abstract markup system based on SGML. It's intended
to be automatically translated to other formats. Tools to do this
exist on the web, and are becoming increasingly easy to find as
more free software projects move their documentation to DocBook.
Any one of these systems should work with the SBCL documentation.
If you'd like to have the documentation produced in the same
format as appears in the binary distribution, and you have
the jade binary and Norman Walsh's modular DSSSL stylesheets
installed, you can try the doc/make-doc.sh script. Otherwise,
your formatted copy of the SBCL documentation should have the
same content as in the binary distribution, but details of
presentation will probably vary.

508
NEWS Normal file
View file

@ -0,0 +1,508 @@
changes in sbcl-0.6.0 relative to sbcl-0.5.0:
* tidied up "make.sh" script
* tidied up system directory structure
* better "clean.sh" behavior
* added doc/FOR-CMUCL-DEVELOPERS
* many many small tweaks to output format, e.g. removing possibly-confusing
trailing #\. character in DESCRIBE-INSTANCE
* (EQUALP #\A 'A) no longer signals an error.
* new hashing code, including EQUALP hashing
* tidied up Lisp initialization and toplevel
* initialization files (e.g. /etc/sbclrc and $HOME/.sbclrc)
* command line argument processing
* added POSIX-GETENV function to deal with Unix-ish environment variables
* more-Unixy handling of *STANDARD-INPUT* and other Lisp streams, e.g.
terminating SBCL on EOF
* non-verbose GC by default
* There is no more "sbcl" shell script; the sbcl file is now the C
runtime executable (just like CMU CL).
* removed some unused fops, e.g. FOP-UNIFORM-VECTOR, FOP-CHARACTER, and
FOP-POP-FOR-EFFECT
* tweaked debug-info.lisp and debug-int.lisp to make the debugger store
symbol and package information as Lisp native symbol and package objects
instead of strings naming symbols and strings naming packages. This way,
whenever packages are renamed (as in warm init), debug information is
transformed along with everything else.
* tweaked the optimization policy declarations which control the building
of SBCL itself. Now, among other things, the system no longer saves
source location debugging information. (This helps two problems at once
by reducing SBCL size and by keeping SBCL from trying to look for its
sources -- which may not exist -- when reporting errors.)
* added src/cold/chill.lisp, to let SBCL read its own cold sources for
debugging and testing purposes
* cleaned up printing, making the printer call PRINT-OBJECT for
instances, and using PRINT-UNREADABLE-OBJECT for most PRINT-OBJECT
methods, giving nearly-ANSI behavior
* converted almost all special variables to use *FOO* naming convention
* deleted PARSE-TIME functionality, since it can be done portably
* moved some files out of cold init into warm init
* deleted DEFUN UNDEFINED-VALUE, replaced (UNDEFINED-VALUE) forms
with (VALUES) forms
* regularized formatting of source files
* added an install.sh script
* fixed ridiculous memory usage of cross-compiler by making
compiler/alloc.lisp not try to do pooling unless it can hook
itself into the GC of the cross-compilation host. Now the system
builds nicely on my old laptop.
* added :SB-ALLOC in target-features.lisp-expr
* deleted mention of :ANSI-DOC from target-features.lisp-expr (since it
was not implemented)
* re-did condition handling and note reporting in the compiler. Notes
are no longer handled by signalling conditions. Style warnings
and warnings are handled more correctly and reported in such a way
that it's easy to find one or the other in your output (so that you
can e.g. figure out which of many problems caused COMPILE-FILE to
return FAILURE-P).
* changed the severity of several compiler warnings from full WARNING
to STYLE-WARNING in order to conform with the ANSI spec; also changed
compiler note reporting so that it doesn't use the condition system
at all (and hence affects neither FAILURE-P nor WARNINGS-P in the
COMPILE-FILE command)
* made PROCLAIM and DECLAIM conform to ANSI. PROCLAIM is now an ordinary
function. As a consequence, START-BLOCK and END-BLOCK declarations are
no longer supported, since their implementation was deeply intertwingled
with the magical, non-ANSI treatment that PROCLAIM received in CMU CL.
* removed bogus "support" for compiler macros named (SETF FOO), and
removed the compiler macro for SETF INFO (but only after making a fool
of myself on the cmucl-imp mailing list by posting a bogus patch for
DEFINE-COMPILER-MACRO..)
* Compiled files containing forms which have side effects on the Lisp
reader (such as DEFPACKAGE forms) are now handled more correctly.
(Compiler queuing of top level lambdas has been suppressed by setting
*TOP-LEVEL-LAMBDA-MAX* to 0. )
* deleted various currently-unused source files, e.g. gengc.lisp. They
may be added back at some point e.g. when porting to other architectures,
but until they are it's distracting to distribute them and to try to
maintain them.
* deleted "UNCROSS couldn't recurse through.." style warnings, since
there were so many of them they're just distractions, and UNCROSS is
known to be able to handle the current sources
* moved PROFILE functionality into TRACE, so that it will be clear
how the wrapping and unwrapping of functions when you profile them
interacts with the wrapping and unwrapping of functions when you
trace them. (Actually, the functionality isn't there yet, but at least
the interface specification is there. Hopefully, the functionality will
arrive with some future maintenance release.)
* removed host-oops.lisp
* changed signature of QUIT function to allow UNIX-CODE argument
* fixed READ-SEQUENCE bug
* tweaked verbose GC output so that it looks more like the progress
output that ANSI specifies for functions like LOAD
* set up the system on sourceforge.com, with home pages, mailing lists, etc.
* added <http://sbcl.sourceforge.com> to the banner information printed by
the sbcl executable
changes in sbcl-0.6.1 relative to sbcl-0.6.0:
* changed build optimization from (SAFETY 1) to (SAFETY 3) as a short-term
fix for various type-unsafety bugs, e.g. failures with (LENGTH 123) and
(MAKE-LIST -1). In the longer term, it ought to become true
that declarations are assertions even at SAFETY 1. For now, it's not
quite true even at SAFETY 3, but it's at least more nearly true..
(Note that this change seems to increases the size of the system by
O(5%) and to decrease the speed of the compiler by 20% or more.)
* changed ALIEN printing to be much more abbreviated, as a short-term fix
for the problem of printing dozens of lines of distracting information
about low-level system machinery as part of the top stack frame
on entry to the debugger when an undefined function was called.
* tweaked the debugger's use of WITH-STANDARD-IO-SYNTAX so that *PACKAGE*
is not reset to COMMON-LISP-USER.
* Compilation of stuff related to dyncount.lisp has been made conditional
on the :SB-DYNCOUNT target feature, so that the ordinary core system is
smaller. The various dyncount-related symbols have been moved into
a new "SB-DYNCOUNT" package.
* tty-inspect.lisp has been renamed to inspect.lisp.
* unix-glibc2.lisp has been renamed to unix.lisp, and the :GLIBC2
feature has gone away. (When we eventually port to other flavors of
libc and/or Unix, we'll try to make the differences between flavors
invisible at the user level.)
* Various other *FEATURES* tags, and/or their associated conditionals,
have been removed if obsolescent, or given better documentation, or
sometimes given more-mnemonic names.
changes in sbcl-0.6.2 relative to sbcl-0.6.1:
* (Note that the way that the PCL macroexpansions were rewritten
to accommodate the change in DEFGENERIC below breaks binary
compatibility. That is, fasl files compiled under sbcl-0.6.1 may
not run under sbcl-0.6.2. Once we get out of alpha releases,
i.e. hit release 1.0.0, we'll probably try to maintain binary
compatibility between maintenance releases, e.g. between sbcl-1.4.3
and sbcl-1.4.4. Until then, however, it might be fairly common
for maintenance releases to break binary compatibility.)
* A bug in the calculation of WARNINGS-P and FAILURE-P in COMPILE-FILE
has been fixed.
* The reporting of unhandled signals has been changed to print some
explanatory text as well as the report form. (Previously only
the report form was printed.)
* The macroexpansion for DEFGENERIC now DECLAIMs the function that
it defines, so that the compiler no longer issues undefined function
warnings for compiled-but-not-yet-loaded generic functions.
* The CLTL-style "LISP" and "USER" nicknames for the "COMMON-LISP"
and "COMMON-LISP-USER" packages have been removed. Now only the "CL"
and "CL-USER" standard nicknames from the "11.1.2 Standardized Packages"
section of the ANSI spec are supported.
* The "" nickname for the "KEYWORD" package has been removed.
The reader still handles symbol tokens which begin with a package marker
as keywords, but it doesn't expose its mechanism for doing so in the
(PACKAGE-NICKNAMES (FIND-PACKAGE "KEYWORD")) list.
* The system now issues STYLE-WARNINGs for contradictory TYPE
proclamations. (Warnings for contradictory FTYPE proclamations would
be nice too, but those can't be done usefully unless the type system
is made smarter about FUNCTION types.)
* The names of source files "*host-*.lisp" and "*target-*.lisp" have been
systematized, so that "*target-*.lisp is supposed to exist only on the
target and imply that there's a related file which exists on the
host, and *host-*.lisp is supposed to exist only on the host and imply
that there's a related file which exists on the target. This involves a
lot of renaming. Hopefully the acute confusion caused by the renaming
will be justified by the reduction in chronic confusion..
** runtime-type.lisp -> early-target-type.lisp
** target-type.lisp -> late-target-type.lisp
** early-host-format.lisp -> early-format.lisp
** late-host-format.lisp -> late-format.lisp
** host-error.lisp -> misc-error.lisp
** early-error.lisp -> early-target-error.lisp
** late-error.lisp -> late-target-error.lisp
** host-defboot.lisp -> early-defboot.lisp
** code/misc.lisp -> code/target-misc.lisp
** code/host-misc.lisp -> code/misc.lisp
** code/numbers.lisp -> code/target-numbers.lisp
** code/early-numbers.lisp -> numbers.lisp
** early-host-type.lisp -> early-type.lisp
** late-host-type.lisp -> late-type.lisp
** host-typep.lisp -> typep.lisp
** load.lisp -> target-load.lisp
** host-load.lisp -> load.lisp
** host-disassem.lisp -> disassem.lisp
** host-insts.lisp -> insts.lisp
** byte-comp.lisp -> target-byte-comp.lisp
** host-byte-comp.lisp -> byte-comp.lisp
** host-signal.lisp -> signal.lisp
** host-defstruct.lisp -> defstruct.lisp
** late-target-type.lisp -> deftypes-for-target.lisp
Furthermore, several other previously target-only files foo.lisp (e.g.
hash-table.lisp and random.lisp) have been split into a target-and-host
foo.lisp file and a target-only target-foo.lisp file, with their key type
definitions in the target-and-host part, so that the cross-compiler will
know more about target types.
* DEFSTRUCT BACKEND, and the BACKEND-valued *BACKEND* variable, have
gone away. In their place are various *BACKEND-FOO* variables
corresponding to the slots of the old structure.
* A bug which caused the SB-COLD bootstrap-time package to be propagated
into the target SBCL has been fixed.
* The chill.lisp system for loading cold code into a running SBCL
now works better.
* Support for the CMU CL "scavenger hook" extension has been removed.
(It was undocumented and unused in the CMU CL sources that SBCL was
derived from, and stale in sbcl-0.6.1.)
* Various errors in the cross-compiler type system were detected
by running the cross-compiler with *TYPE-SYSTEM-INITIALIZED*
(enabling various consistency checks). Many of them were fixed,
but some hard problems remain, so the compiler is back to
running without *TYPE-SYSTEM-INITIALIZED* for now.
* As part of the cross-compiler type system cleanup, I implemented
DEF!TYPE and got rid of early-ugly-duplicates.lisp.
* I have started adding UNCROSS calls throughout the type system
and the INFO database. (Thus perhaps eventually the blanket UNCROSS
on cross-compiler input files will be able to go away, and various
kludges with it).
* CONSTANTP now returns true for quoted forms (as explicitly required
by the ANSI spec).
changes in sbcl-0.6.3 relative to sbcl-0.6.2:
* The system still can't cross-compile itself with
*TYPE-SYSTEM-INITIALIZED* (and all the consistency checks that
entails), but at least it can compile more of itself that way
than it used to be able to, and various buglets which were uncovered
by trying to cross-compile itself that way have now been fixed.
* This release breaks binary compatibility again. This time
at least I've incremented the FASL file format version to 2, so that the
problem can be detected reliably instead of just causing weird errors.
* various new style warnings:
** using DEFUN, DEFMETHOD, or DEFGENERIC to overwrite an old definition
** using the deprecated EVAL/LOAD/COMPILE situation names in EVAL-WHEN
** using the lexical binding of a variable named in the *FOO* style
* DESCRIBE has been substantially rewritten. It now calls DESCRIBE-OBJECT
as specified by ANSI.
* *RANDOM-STATE* is no longer automatically initialized from
(GET-UNIVERSAL-TIME), but instead from a constant seed. Thus, the
default behavior of the system is to repeat its behavior every time
it's run. If you'd like to change this behavior, you can always
explicitly set the seed from (GET-UNIVERSAL-TIME); whereas under the
old convention there was no comparably easy way to get the system to
repeat its behavior every time it was run.
* Support for the pre-CLTL2 interpretation of FUNCTION declarations as
FTYPE declarations has been removed, in favor of their ANSI
interpretation as TYPE FUNCTION declarations. (See p. 228 of CLTL2.)
* The quantifiers SOME, EVERY, NOTANY, and NOTEVERY no longer cons when
the types of their sequence arguments can be determined at compile time.
This is done through a new open code expansion for MAP which eliminates
consing for (MAP NIL ..), and reduces consing otherwise, when sequence
argument types can be determined at compile time.
* The optimizer now transforms COERCE into an identity operation when it
can prove that the coerced object is already of the correct type. (This
can be a win for machine generated code, including the output of other
optimization transforms, such as the MAP transform above.)
* Credit information has been moved from source file headers into CREDITS.
* Source file headers have been made more standard.
* The CASE macro now compiles without complaining even when it has
no clauses.
changes in sbcl-0.6.4 relative to sbcl-0.6.3:
* There is now a partial SBCL user manual (with some new text and some
text cribbed from the CMU CL manual).
* The beginnings of a profiler have been added (starting with the
CMU CL profiler and simplifying and cleaning up). Eventually the
main interface should be through the TRACE macro, but for now,
it's still accessed through vaguely CMU-CL-style functions and macros
exported from the package SB-PROFILE.
* Some problems left over from porting CMU CL to the new
cross-compilation bootstrap process have been cleaned up:
** DISASSEMBLE now works. (There was a problem in using DEFMACRO
instead of SB!XC:DEFMACRO, compounded by an oversight on my
part when getting rid of the compiler *BACKEND* stuff.)
** The value of *NULL-TYPE* was screwed up, because it was
being initialized before the type system knew the final
definition of the 'NULL type. This screwed up several key
optimizations in the compiler, causing inefficiency in all sorts
of places. (I found it because I wanted to understand why
GET-INTERNAL-RUN-TIME was consing.)
* fixed a bug in DEFGENERIC which was causing it to overwrite preexisting
PROCLAIM FTYPE information. Unfortunately this broke binary
compatibility again, since now the forms output by DEFGENERIC
to refer to functions which didn't exist in 0.6.3.
* added declarations so that SB-PCL::USE-CACHING-DFUN-P
can use the new (as of 0.6.3) transform for SOME into MAP into
inline code
* changed (MOD 1000000) type declarations for Linux timeval.tv_usec slot
values to (INTEGER 0 1000000), so that the time code will no longer
occasionally get blown up by Linux returning 1000000 microseconds
* PRINT-UNREADABLE-OBJECT has been tweaked to make the spacing of
its output conform to the ANSI spec. (Alas, this makes its output
uglier in the :TYPE T :IDENTITY NIL case, but them's the breaks.)
* A full call to MAP NIL with a single sequence argument no longer conses.
* fixes to problems pointed out by Martin Atzmueller:
* The manual page no longer talks about multiprocessing as though
it were currently supported.
* The ILISP support patches have been removed from the distribution,
because as of version 5.10.1, ILISP now supports SBCL without us
having to maintain patches.
* added a modified version of Raymond Toy's recent CMU CL patch for
EQUALP comparison of HASH-TABLE
changes in sbcl-0.6.5 relative to sbcl-0.6.4:
* Raymond Wiker's patches to port the system to FreeBSD have been merged.
* The build process now looks for GNU make under the default name "gmake",
instead of "make" as it used to. If GNU make is not available as "gmake"
on your system, you can change this default behavior by setting the
GNUMAKE environment variable.
* Replace #+SB-DOC with #!+SB-DOC in seq.lisp so that the system
can build without error under CMU CL.
changes in sbcl-0.6.6 relative to sbcl-0.6.5:
* DESCRIBE no longer tries to call itself recursively to describe
bound/fbound values, so that it no longer fails on symbols which are
bound to themselves (like keywords, T, and NIL).
* DESCRIBE now works on generic functions.
* The printer now prints less-screwed-up representations of closures
(not naively trying to bogusly use the %FUNCTION-NAME accessor on them).
* A private symbol is used instead of the :EMPTY keyword previously
used to mark empty slots in hash tables. Thus
(DEFVAR *HT* (MAKE-HASH-TABLE))
(SETF (GETHASH :EMPTY *HT*) :EMPTY)
(MAPHASH (LAMBDA (K V) (FORMAT T "~&~S ~S~%" K V)))
now does what ANSI says that it should. (You can still get
similar noncompliant behavior if bang on the hash table
implementation with all the symbols you get back from
DO-ALL-SYMBOLS, but at least that's a little harder to do.)
This breaks binary compatibility, since tests for equality to
:EMPTY are wired into things like the macroexpansion of
WITH-HASH-TABLE-ITERATOR in FASL files produced by earlier
implementations.
* There's now a minimal placeholder implementation for CL:STEP,
as required by ANSI.
* An obscure bug in the interaction of the normal compiler, the byte
compiler, inlining, and structure predicates has been patched
by setting the flags for the DEFTRANSFORM of %INSTANCE-TYPEP as
:WHEN :BOTH (as per Raymond Toy's suggestion on the cmucl-imp@cons.org
mailing list).
* Missing ordinary arguments in a macro call are now detected even
when the macro lambda list contains &KEY or &REST.
* The debugger no longer complains about encountering the top of the
stack when you type "FRAME 0" to explicitly instruct it to go to
the top of the stack. And it now prints the frame you request even
if it's the current frame (instead of saying "You are here.").
* As specified by ANSI, the system now always prints keywords
as #\: followed by SYMBOL-NAME, even when *PACKAGE* is the
KEYWORD package.
* The default initial SIZE of HASH-TABLEs is now smaller.
* Type information from CLOS class dispatch is now propagated
into DEFMETHOD bodies, so that e.g.
(DEFMETHOD FOO ((X SINGLE-FLOAT))
(+ X 123.0))
is now basically equivalent to
(DEFMETHOD FOO ((X SINGLE-FLOAT))
(DECLARE (TYPE SINGLE-FLOAT X))
(+ X 123.0))
and the compiler can compile (+ X 123.0) as a SINGLE-FLOAT-only
operation, without having to do run-time type dispatch.
* The macroexpansion of DEFMETHOD has been tweaked so that it has
reasonable behavior when arguments are declared IGNORE or IGNORABLE.
* Since I don't seem to be making big file reorganizations very often
any more (and since my archive of sbcl-x.y.zv.tar.bz2 snapshots
is overflowing my ability to conveniently back them up), I've finally
checked the system into CVS. (The CVS repository is on my home system,
not at SourceForge -- putting it on SourceForge might come later.)
* SB-EXT:*GC-NOTIFY-STREAM* has been added, to control where the
high-level GC-NOTIFY-FOO functions send their output. (There's
still very little control of where low-level verbose GC functions
send their output.) The SB-EXT:*GC-VERBOSE* variable now controls
less than it used to -- the GC-NOTIFY-FOO functions are now under
the control of *GC-NOTIFY-STREAM*, not *GC-VERBOSE*.
* The system now stores the version string (LISP-IMPLEMENTATION-VERSION)
in only one place in the source code, and propagates it automatically
everywhere that it's needed. Thus e.g. when I bump the version from
0.6.6 to 0.6.7, I'll only need to modify the sources in one place.
* The C source files now include boilerplate legalese and documentation
at the head of each file (just as the Lisp source files already did).
* At Dan Barlow's suggestion, the hyperlink from the SBCL website
to his page will be replaced with a link to his new CLiki service.
changes in sbcl-0.6.7 relative to sbcl-0.6.6:
* The system has been ported to OpenBSD.
* The system now compiles with a simple "sh make.sh" on the systems
that it's supported on. I.e., now you no longer need to tweak
text in the target-features.lisp-expr and symlinks in src/runtime/
by hand, the make.sh takes care of it for you.
* The system is no longer so grossly inefficient when compiling code
involving vectors implemented as general (not simple) vectors (VECTOR T),
so code which dares to use VECTOR-PUSH-EXTEND and FILL-POINTER, or
which dares to use the various sequence functions on non-simple
vectors, takes less of a performance hit.
* There is now a primitive type predicate VECTOR-T-P
to test for the (VECTOR T) type, so that e.g.
(DEFUN FOO (V) (DECLARE (TYPE (VECTOR T) V)) (AREF V 3))
can now be compiled with some semblance of efficiency. (The old code
turned the type declaration into a full call to %TYPEP at runtime!)
* AREF on (VECTOR T) is still not fast, since it's still compiled
as a full call to SB-KERNEL:DATA-VECTOR-REF, but at least the
ETYPECASE used in DATA-VECTOR-REF is now compiled reasonably
efficiently. (The old version made full calls to SUBTYPEP at runtime!)
* (MAKE-ARRAY 12 :FILL-POINTER T) is now executed less inefficiently,
without making full calls to SUBTYPEP at runtime.
(Some analogous efficiency issues for non-simple vectors specialized to
element types other than T, or for non-simple multidimensional arrays,
have not been addressed. They could almost certainly be handled the
same way if anyone is motivated to do so.)
* The changes in array handling break binary compatibility, so
*BACKEND-FASL-FILE-VERSION* has been bumped to 4.
* (TYPEP (MAKE-ARRAY 12 :FILL-POINTER 4) 'VECTOR) now returns (VALUES T)
instead of (VALUES T T).
* By following the instructions that Dan Barlow posted to sbcl-devel
on 2 July 2000, I was able to enable primitive dynamic object
file loading code for Linux. The full-blown CMU CL LOAD-FOREIGN
functionality is not implemented (since it calls ld to resolve
library references automatically, requiring RUN-PROGRAM for its
implementation), but a simpler SB-EXT:LOAD-1-FOREIGN (which doesn't
try to resolve library references) is now supported.
* The system now flushes the standard output streams when it terminates,
unless QUIT is used with the RECKLESSLY-P option set. It also flushes
them at several other probably-convenient times, e.g. in each pass of
the toplevel read-eval-print loop, and after evaluating a form given
as an "--eval" command-line option. (These changes were motivated by a
discussion of stream flushing issues on cmucl-imp in August 2000.)
* The source transform for TYPEP of array types no longer assumes
that an array whose element type is a not-yet-defined type
is implemented as an array of T, but instead punts, so that the
type will be interpreted at runtime.
* There is now some support for cross-compiling in make.sh: each of
the phases of make.sh has its own script. (This should be transparent
to people doing ordinary, non-cross-compile builds.)
* Since my laptop doesn't have hundreds of megabytes of memory like
my desktop machine, I became more motivated to do some items on
my to-do list in order to reduce the size of the system a little:
** Arrange for various needed-only-at-cold-init things to be
uninterned after cold init. To support this, those things have
been renamed from FOO and *FOO* to !FOO and *!FOO* (i.e., all
symbols with such names are now uninterned after cold init).
** Bind SB!C::*TOP-LEVEL-LAMBDA-MAX* to a nonzero value when building
fasl files for cold load.
** Remove the old compiler structure pooling code (which used to
be conditional on the target feature :SB-ALLOC) completely.
** Redo the representation of some data in cold init to be more compact.
(I also looked into supporting byte compiled code at bootstrap time,
which would probably reduce the size of the system a lot, but that
looked too complicated, so I punted for now.)
* The maximum signal nesting depth in the src/runtime/ support code has
been reduced from 4096 to 256. (I don't know any reason for the very
large old value. If the new smaller value turns out to break something,
I'll probably just bump it back up.)
* PPRINT-LOGICAL-BLOCK is now pickier about the types of its arguments,
as per ANSI.
* Many, many bugs reported by Peter Van Eynde have been added to
the BUGS list; some have even been fixed.
* While enabling dynamic object file loading, I tried to make the
code easier to understand, renaming various functions and variables
with less ambiguous names, and changing some function calling
conventions to be Lispier (e.g. returning NIL instead of 0 for failure).
* While trying to figure out how to do the OpenBSD port, I tried to
clean up some of the code in src/runtime/. In particular, I dropped
support for non-POSIX signal handling, added various comments,
tweaked the code to reduce the number of compilation warnings, and
renamed some files to increase consistency.
* To support the new automatic configuration functionality in make.sh,
the source file target-features.lisp-expr has been replaced with the
source file base-target-features.lisp-expr and the machine-generated
file local-target-features.lisp-expr.
* fixed a stupid quoting error in make.sh so that using CMU CL
"lisp -batch" as cross-compilation host works again
changes in sbcl-0.6.8 relative to sbcl-0.6.7:
?? The system is now under CVS at SourceForge (instead of the
CVS repository on my home machine).
?? The INSTALL file has been updated with some information
about using anonymous CVS to download the most recent version
from SourceForge.
?? There's now code in the tests/ subdirectory to run the system
through the clocc/ansi-tests/ suite, and to run additional
SBCL-specific regression tests as well. (It's not particularly
mature right now, but it's a start.)
?? The system now uses code based on Colin Walters' O(N)
implementation of MAP (from the cmucl-imp@cons.org mailing
list, 2 September 2000) when it can't use a DEFTRANSFORM to
inline the MAP operation, and there is more than one
sequence argument to the MAP call (so that it can't just
do ETYPECASE once and for all based on the type of the
single sequence argument). (The old non-inline implementation
of the general M-argument sequence-of-length-N case required
O(M*N*N) time when any of the sequence arguments were LISTs.)
?? Raymond Wiker's port of CMU CL's RUN-PROGRAM has been added.
(?? Don't forget to mention Colin Walters and Raymond Wiker in the
CREDITS file.)
?? The debugger now flushes standard output streams before it begins
its output ("debugger invoked" and so forth).
?? The two problem cases reported by Peter Van Eynde on 8 Sep 2000,
(BUTLAST '(1 2 3) -1) and (MAKE-LIST -1), now work, and test cases
have now been added to the regression test suite to keep them
from appearing again. (This was a repeat appearance, alas!)
As the regression test system gets more mature, I intend to add
most future fixed bugs to it, but at this point I'm still playing
with it.
?? The patch for the SUBSEQ bug reported on the cmucl-imp mailing
list 12 September 2000 has been applied to SBCL.
?? Martin Atzmueller's versions of two CMU CL patches, as posted on
sbcl-devel 13 September 2000, have been installed. (The patches fix
a bug in SUBSEQ and <a bug in ??>.)
?? A bug in signal handling which kept TRACE from working on OpenBSD
has been fixed.
?? The signal handling bug reported by Martin Atzmueller on
sbcl-devel 13 September 2000, which caused the debugger to
get confused after a Ctrl-C interrupt under ILISP, has been fixed.

173
PRINCIPLES Normal file
View file

@ -0,0 +1,173 @@
"In truth, I found myself incorrigible with respect to *Order*; and
now I am grown old and my memory bad, I feel very sensibly the want of
it. But, on the whole, though I never arrived at the perfection I had
been so ambitious of obtaining, but fell far short of it, yet I was,
by the endeavour, a better and happier man than I otherwise should
have been if I had not attempted it; as those who aim at perfect
writing by imitating the engraved copies, though they never reach the
wished-for excellence of those copies, their hand is mended by the
endeavor, and is tolerable while it continues fair and legible."
-- Benjamin Franklin in his autobiography
"'Signs make humans do things,' said Nisodemus, 'or stop doing things.
So get to work, good Dorcas. Signs. Um. Signs that say *No*.'"
-- Terry Pratchett, _Diggers_
There are some principles which I'd like to see used in the
maintenance of SBCL:
1. conforming to the standard
2. being maintainable
a. removing stale code
b. When practical, important properties should be made manifest in
the code. (Putting them in the comments is a distant second best.)
i. Perhaps most importantly, things being the same (in the strong
sense that if you cut X, Y should bleed) should be manifest in
the code. Having code in more than one place to do the same
thing is bad. Having a bunch of manifest constants with hidden
relationships to each other is inexcusable. (Some current
heinous offenders against this principle are the memoizing
caches for various functions, and the LONG-FLOAT code.)
ii. Enforcing nontrivial invariants, e.g. by declaring the
types of variables, or by making assertions, can be very
helpful.
c. using clearer internal representations
i. clearer names
A. more-up-to-date names, e.g. PACKAGE-DESIGNATOR instead
of PACKAGELIKE (in order to match terminology used in ANSI spec)
B. more-informative names, e.g. SAVE-LISP-AND-DIE instead
of SAVE-LISP or WRAPPER-INVALID rather than WRAPPER-STATE
C. families of names which correctly suggest parallelism,
e.g. CONS-TO-CORE instead of ALLOCATE-CONS, in order to
suggest the parallelism with other FOO-TO-CORE functions
ii. clearer encodings, e.g. it's confusing that WRAPPER-STATE in PCL
returns T for valid and any other value for invalid; could
be clarified by changing to WRAPPER-INVALID returning a
generalized boolean; or e.g. it's confusing to encode things
as symbols and then use STRING= SYMBOL-NAME instead of EQ
to compare them.
iii. clearer implementations, e.g. cached functions being
done with HASH-TABLE instead of hand-coded caches
d. informative comments and other documentation
i. documenting things like the purposes and required properties
of functions, objects, *FEATURES* options, memory layouts, etc.
ii. not using terms like "new" without reference to when.
(A smart source code control system which would let you
find when the comment was written would help here, but
there's no reason to write comments that require a smart
source code control system to understand..)
e. using functions instead of macros where appropriate
f. maximizing the amount of stuff that's (broadly speaking) "table
driven". I find this particularly helpful when the table describes
the final shape of the result (e.g. the package-data-list.lisp-expr
file), replacing a recipe for constructing the result (e.g. various
in-the-flow-of-control package-manipulation forms) in which the
final shape of the result is only implicit. But it can also be very
helpful any time the table language can be just expressive enough
for the problem at hand.
g. using functional operators instead of side-effecting operators
where practical
h. making it easy to find things in the code
i. defining things using constructs which can be understood by etags
i. using the standard library where possible
i. instead of hand-coding stuff
(My package-data-list.lisp-expr stuff may be a bad example as of
19991208, since the system has evolved to the point where it
might be possible to replace my hand-coded machinery with some
calls to DEFPACKAGE.)
j. more-ambitious dreams..
i. fixing the build process so that the system can be bootstrapped
from scratch, so that the source code alone, and not bits and
pieces inherited from the previous executable, determine the
properties of the new executable
ii. making package dependencies be a DAG instead of a mess, so
the system could be understood (and rebuilt) in pieces
iii. moving enough of the system into C code that the Common Lisp
LOAD operator (and all the symbol table and FOP and other
machinery that it depends on) is implemented entirely in C, so
that GENESIS would become unnecessary (because all files could
now be warm loaded)
3. being portable
a. In this vale of tears, some tweaking may be unavoidably required
when making software run on more than one machine. But we should
try to minimize it, not embrace it. And to the extent that it's
unavoidable, where possible it should be handled by making an
abstract value or operation which is used on all systems, then
making separate implementations of those values and operations
for the various systems. (This is very analogous to object-oriented
programming, and is good for the same reasons that method dispatch
is better than a bunch of CASE statements.)
4. making a better programming environment
a. Declarations *are* assertions! (For function return values, too!)
b. Making the debugger, the profiler, and TRACE work better.
c. Making extensions more comprehensible.
i. Making a smaller set of core extensions. IMHO the high level
ones like ONCE-ONLY and LETF belong in a portable library
somewhere, not in the core system.
ii. Making more-orthogonal extensions. (e.g. removing the
PURIFY option from SAVE-LISP-AND-DIE, on the theory that
you can always call PURIFY yourself if you like)
iii. If an extension must be complicated, if possible make the
complexity conform to some existing standard. (E.g. if SBCL
supplied a command-line argument parsing facility, I'd want
it to be as much like existing command-line parsing utilities
as possible.)
5. other nice things
a. improving compiled code
i. faster CLOS
ii. bigger heap
iii. better compiler optimizations
iv. DYNAMIC-EXTENT
b. increasing the performance of the system
i. better GC
ii. improved ability to compile prototype programs fast, even
at the expense of performance of the compiled program
c. improving safety
i. more graceful handling of stack overflow and memory exhaustion
ii. improving interrupt safety by e.g. locking symbol tables
d. decreasing the size of the SBCL executable
e. not breaking old extensions which are likely to make it into the
new ANSI standard
6. other maybe not-so-nice things
a. adding whizzy new features which make it harder to maintain core
code. (Support for the debugger is important enough that I'll
cheerfully make an exception. Multithreading might also be
sufficiently important that it's probably worth making an exception.)
The one other class of extensions that I am particularly interested
is CORBA or other standard interface support, so that programs can
more easily break out of the Lisp/GC box to do things like graphics.
("So why did you drop all the socket support, Bill?" I hear you
ask. Fundamentally, because I have 'way too much to maintain
already; but also because I think it's too low-level to add much
value. People who are prepared to work at that level of abstraction
and non-portability could just code their own wrapper layer
in C and talk to it through the ALIEN stuff.)
7. judgment calls
a. Sharp, rigid tools are safer than dull or floppy tools. I'm
inclined to avoid complicated defaulting behavior (e.g. trying
to decide what file to LOAD when extension is not specified) or
continuable errors, preferring functions which have simple behavior
with no surprises (even surprises which are arguably pleasant).
CMU CL maintenance has been conservative in ways that I would prefer to
be flexible, and flexible in ways that I'd prefer to be conservative.
CMU CL maintainers have been conservative about keeping old code and
maintaining the old structure, and flexible about allowing a bunch of
additional stuff to be tacked onto the old structure.
There are some good things about the way that CMU CL has been
maintained that I nonetheless propose to jettison. In particular,
binary compatibility between releases. This is a very handy feature,
but it's a pain to maintain. At least for a while, I intend to just
require that programs be recompiled any time they're to be used with a
new version of the system. After a while things might settle down to
where recompiles will only be required for new major releases, so
either all 3.3.x fasl files will work with any 3.3.y runtime, or all
3.w.x fasl files will work with any 3.y.z runtime. But before trying
to achieve that kind of stability, I think it's more important to
be able to clean up things about the internal structure of the system.
Aiming for that kind of stability would impair our ability to make
changes like
* cleaning up DEFUN and DEFMACRO to use EVAL-WHEN instead of IR1 magic;
* reducing the separation between PCL classes and COMMON-LISP classes;
* fixing bad FOPs (e.g. the CMU CL fops which interact with the *PACKAGE*
variable)

22
README Normal file
View file

@ -0,0 +1,22 @@
Welcome to SBCL.
To find out more about who created the system, see the "CREDITS" file.
If you'd like information about the legalities of copying the system,
see the "COPYING" file.
If you'd like to install or build the system, see the "INSTALL" file.
If you'd like more information about using the system, see the man
page, "sbcl.1", or the user manual in the "doc/" subdirectory of the
distribution. (The user manual is maintained as DocBook SGML in the
source distribution; there is an HTML version in the binary
distribution.)
The system is a work in progress. See the "TODO" file in the source
distribution for some highlights.
If you'd like to make suggestions, report a bug, or help to improve the
system, please send mail to one of the mailing lists:
sbcl-help@lists.sourceforge.net
sbcl-devel@lists.sourceforge.net

99
STYLE Normal file
View file

@ -0,0 +1,99 @@
Most of the style hints in the Lisp FAQ apply.
When porting the system, I would really prefer code which factors
dependencies into a set of interface functions and constants and
includes implementations of the interface for the different systems.
Patches which require conditional compilation (like all the old
#T+HPUX or #T-X86 tests in the sources inherited from CMUCL) might be
accepted if they're simple, in hopes of factoring out the differences
more cleanly later, but even if accepted, such code may not be
maintained for very long.
grammatical fussiness:
Phrases are not capitalized.
Sentences are capitalized.
Periods terminate sentences.
Periods separate phrases from succeeding sentences, e.g.
;;; the maximum number of transformations we'll make before
;;; concluding we're in an infinite loop and bailing. This can
;;; be changed, but it is an error to change it while we're
;;; solving a system.
(defvar *max-n-transformations* 10)
Lisp in comments is capitalized.
usage fussiness:
Function documentation can be a description of what the function
does, e.g.
;;; Parse the arguments for a BDEFSTRUCT call, and return
;;; (VALUES NAME DEFSTRUCT-ARGS MAKE-LOAD-FORM-FUN BDEFSTRUCT-STYPE),
;;; where NAME is the name of the new type, DEFSTRUCT-ARGS is the
;;; munged result suitable for passing on to DEFSTRUCT,
;;; MAKE-LOAD-FORM-FUN is the make load form function, or NIL if
;;; there's none, and BDEFSTRUCT-SUPERTYPE is the direct supertype
;;; of the type if it is another BDEFSTRUCT-defined type, or NIL
;;; otherwise.
(defun parse-bdefstruct-args (nameoid &rest rest)
..)
or a remark about the function, e.g.
;;; a helper function for BDEFSTRUCT in the #+XC-HOST case
(defun uncross-defstruct-args (defstruct-args)
..)
If you're talking about what the function does, ordinarily you
should just say what the function does, e.g.
;;; Return the first prime number greater than or equal to X.
(defun primify (x) ..)
instead of telling the reader that you're going to tell him what
the function does, e.g.
;;; PRIMIFY returns the first prime number greater than or
;;; equal to X.
(defun primify (x) ..)
or
;;; When you call this function on X, you get back the first
;;; prime number greater than or equal to X.
(defun primify (x) ..)
In general, if you can express it in the code instead of the comments,
do so. E.g. the old CMUCL code has many comments above functions foo
that say things like
;;; FOO -- interface
If we were going to do something like that, we would prefer to do it by
writing
(EXPORT 'FOO)
(Instead, for various other reasons, we centralize all the exports
in package declarations.) The old "FOO -- interface" comments are bad
style because they duplicate information (and they illustrate one
of the evils of duplicating information by the way that they have
drifted out of sync with the code).
There are a number of style practices on display in the code
which are not good examples to follow:
* using conditional compilation to support different architectures,
instead of factoring the dependencies into interfaces and providing
implementations of the interface for different architectures;
* in conditional compilation, using a common subexpression over and
over again, e.g. #+(OR GENGC GENCGC), when the important thing is
that GENGC and GENCGC are (currently) the GCs which support scavenger
hooks. If you have to do that, define a SCAVHOOK feature,
write #+SCAVHOOK in many places, and arrange for the SCAVHOOK feature
to be set once and only once in terms of GENGC and GENCGC. (That way
future maintainers won't curse you.)
* putting the defined symbol, and information about whether it's
exported or not, into the comments around the definition of the symbol;
* naming anything DO-FOO if it isn't an iteration macro
* exposing a lot of high-level functionality not in the ANSI standard
to the user (as discussed above)
* not using a consistent abbreviation style in global names (e.g.
naming some things DEFINE-FOO and other things DEF-BAR, with
no rule to determine whether the abbreviation is used)
* using lots of single-colon package prefixes (distracting and hard
to read, and obstacles to reaching package nirvana where
package dependencies are a directed acyclic graph) or even
double-colon package prefixes (hard to understand and hard
to maintain). (One exception: I've sometimes been tempted to
add a CL: prefix to the definition of every CL symbol (e.g.
(DEFUN CL:CADDDR (..) ..) as reminders that they're required by
ANSI and can't be deleted no matter how obscure and useless some
of them might look.:-)
Most of these are common in the code inherited from CMUCL. I've
eliminated them in some places, but there's a *lot* of code inherited
from CMUCL..

189
TODO Normal file
View file

@ -0,0 +1,189 @@
Accumulation of half-understood design decisions eventually
chokes a program as a water weed chokes a canal. By refactoring
you can ensure that your full understanding of how the program
should be designed is always reflected in the program. As a
water weed quickly spreads its tendrils, partially understood
design decisions quickly spread their effects throughout your
program. No one or two or even ten individual actions will be
enough to eradicate the problem.
-- Martin Fowler, _Refactoring: Improving the Design
of Existing Code_, p. 360
===============================================================================
some things that I'd like to do in 0.6.x, in no particular order:
-------------------------------------------------------------------------------
PROBLEM:
The batch-related command line options for SBCL don't work
properly.
A small part of making them work properly is making sure that
verbose GC messages end up piped to error output.
Make sure that when the system dies due to an unhandled error
in batch mode, the error is printed successfully, whether
FINISH-OUTPUT or an extra newline or whatever is required.
Make sure that make.sh dies gracefully when one of the SBCLs
it's running dies with an error.
MUSING:
Actually, the ANSI *DEBUGGER-HOOK* variable might be a better
place to put the die-on-unhandled-error functionality.
FIX:
??
-------------------------------------------------------------------------------
PROBLEM:
As long as I'm working on the batch-related command-line options,
it would be reasonable to add one more option to "do what I'd want",
testing standard input for TTY-ness and running in no-programmer
mode if so.
FIX:
?? Do it.
-------------------------------------------------------------------------------
PROBLEM:
In order to make a well-behaved backtrace when a batch program
terminates abnormally, it should be limited in length.
FIX:
?? Add a *DEBUG-BACKTRACE-COUNT* variable, initially set to 64,
to provide a default for the COUNT argument to BACKTRACE.
-------------------------------------------------------------------------------
PROBLEM:
I used CMU CL for years, and dozens of times I cursed the
inadequate breakpoint-based TRACE facility which doesn't work on
some functions, and I never realized that there's a wrapper-based
facility too until I was wading through the source code for SBCL.
Yes, I know I should have RTFM, but there is a lot of M..
FIX:
?? possibility 1: Add error-handling code in ntrace.lisp to
catch failure to set breakpoints and retry using
wrapper-based tracing.
?? possibility 2: Add error-handling code in ntrace.lisp to
catch failure to catch failure to set breakpoints and output
a message suggesting retrying with wrapper-based breakpoints
?? possibility 3: Fix the breakpoint-based TRACE facility so that
it always works.
-------------------------------------------------------------------------------
PROBLEM:
When cross-compiling host-byte-comp.lisp, I get bogus
warnings
caught STYLE-WARNING:
undefined function: %%DEFCONSTANT
caught STYLE-WARNING:
This function is undefined:
%%DEFCONSTANT
MUSING:
The best way to clean this up would be as a side-effect of
a larger cleanup, making all the %%DEFFOO stuff use EVAL-WHEN
instead of IR1 magic.
There's probably some way to do it with a quick local hack too.
FIX:
??
-------------------------------------------------------------------------------
PROBLEM:
My system of parallel build directories doesn't seem to add value.
FIX:
?? Replace it with a system where fasl output files live in the
same directories as the sources and have names a la
"foo.fasl-from-host and "foo.fasl-from-xc".
-------------------------------------------------------------------------------
PROBLEM:
It might be good to use the syntax (DEBUGGER-SPECIAL *PRINT-LEVEL*)
etc. to control the in-the-debug-context special variables. Then we
wouldn't have to pick and choose which variables we shadow in the
debugger.
The shadowing values could also be made persistent between
debugger invocations, so that entering the debugger, doing
(SETF *PRINT-LEVEL* 2), and exiting the debugger would leave
(DEBUGGER-SPECIAL *PRINT-LEVEL*) set to 2, and upon reentry to the
debugger, *PRINT-LEVEL* would be set back to 2.
FIX:
??
-------------------------------------------------------------------------------
PROBLEM:
The :SB-TEST target feature should do something.
FIX:
??
-------------------------------------------------------------------------------
PROBLEM:
I still haven't cleaned up the cut-and-paste programming in
* DEF-BOOLEAN-ATTRIBUTE, DELETEF-IN, and PUSH-IN
* SB!SYS:DEF!MACRO ASSEMBLE and SB!XC:DEFMACRO ASSEMBLE
FIX:
??
-------------------------------------------------------------------------------
PROBLEM:
We be able to get rid of the IR1 interpreter, which would
not only get rid of all the code in *eval*.lisp, but also allow us to
reduce the number of special cases elsewhere in the system. (Try
grepping for 'interpret' sometime.:-) Making this usable might
require cleaning up %DEFSTRUCT, %DEFUN, etc. to use EVAL-WHEN
instead of IR1 transform magic, which would be a good
thing in itself, but might be a fair amount of work.)
FIX:
?? Delete, delete, delete.
-------------------------------------------------------------------------------
PROBLEM:
The hashing code is new and should be tested.
FIX:
?? Enable the existing test code.
-------------------------------------------------------------------------------
PROBLEM:
My ad hoc system of revision control is looking pretty clunky,
and I've pretty much stopped doing stuff to confuse CVS (like moving
directories around).
FIX:
?? Check into CVS.
?? Make sure that the tags in FILE-COMMENTs expand correctly.
?? See about automatically propagating version information
from CVS into the runtime.c banner message and the
LISP-IMPLEMENTATION-VERSION string.
===============================================================================
other known issues with no particular target date:
user manual including, at a minimum, updated versions of the
CMU CL user manual information on the compiler and the alien
interface
bugs listed on the man page
more regression tests
various bugs fixed in CMUCL since this code was forked off of it
ca. 19980801, since most of these haven't been fixed yet in SBCL
byte compilation of appropriate parts of the system, so that the
system core isn't so big
uninterning needed-only-at-init-time stuff after init is complete,
so that the system core isn't so big
Search for unused external symbols (ones which are not bound, fbound,
types, or whatever, and also have no other uses as e.g. flags) and
delete them. This should make the system core a little smaller, but
is mostly useful just to make the source code smaller and simpler.
The eventual plan is for SBCL to bootstrap itself in two phases. In
the first phase, the cross-compilation host is any old ANSI Common
Lisp (not necessarily SBCL) and the cross-compiler won't handle some
optimizations because the code it uses to implement them is not
portable. In the second phase, the cross-compilation host will be
required to be a compatible version of SBCL, and the cross-compiler
will take advantage of that to implement all optimizations. The
current version of SBCL only knows how to do the first of those two
phases, with a fully-portable cross-compiler, so some optimizations
are not done. Probably the most important consequence of this is that
because the fully-portable cross-compiler isn't very smart about
dealing with immediate values which are of specialized array type
(e.g. (SIMPLE-ARRAY (UNSIGNED-BYTE 4) 1)) the system sometimes has to
use unnecessarily-general array types internally.
adding new FOPs to provide something like CMU CL's FOP-SYMBOL-SAVE and
FOP-SMALL-SYMBOL-SAVE functionality, so that fasl files will be more
compact. (FOP-SYMBOL-SAVE used *PACKAGE*, which was concise but allowed
obscure bugs. Something like FOP-LAST-PACKAGE-SYMBOL-SAVE could have
much of the same conciseness advantage without the bugs.)
hundreds of FIXME notes in the sources from WHN
various other unfinished business from CMU CL and before, marked with
"XX" or "XXX" or "###" or "***" or "???" or "pfw" or "@@@@" or "zzzzz"
or probably also other codes that I haven't noticed or have forgotten.
(Things marked as KLUDGE are in general things which are ugly or
confusing, but that, for whatever reason, may stay that way
indefinitely.)

35
UGLINESS Normal file
View file

@ -0,0 +1,35 @@
There are a number of hacks that I've used to make the system work
that even I can see are ugly. Some which come to mind..
It's dependent on being compiled in a rigid sequence, all in a single
compilation pass, particularly in the cross-compilation phase.
There's very little support for compiling modules in parallel
or recompiling the system incrementally.
The way the cross-compiler uses UNCROSS is ugly.
The heavy use of %PYTHON:DEFMACRO to construct basic macros is
arguably ugly. But it's better than what I tried before that, and the
system is still slightly contaminated with fallout from what I tried..
When I was first trying to bootstrap the system, I went off on a wild
goose chase of trying to define everything (even fundamental macros
like DEFUN and DEFMACRO) in terms of ordinary functions and Lisp
special operators. I later realized that I could do without this, but
a number of the changes that I made to the code while on that chase
still live on, and the code is unnecessarily unclear because of them.
The contrapuntal intertwingling of the cross-compiler and
target Lisp build sequences is, well, baroque.
Using host floating point numbers to represent target floating point
numbers, or host characters to represent target characters, is theoretically
shaky. (The characters are OK as long as the characters are
in the ANSI-guaranteed character set, though.)
Despite my attempts to make the compiler portable, it still makes assumptions
about the cross-compilation host Common Lisp:
Simple bit vectors are distinct from simple vectors (in
DEFINE-STORAGE-BASE and elsewhere). (Actually, I'm not sure
that things would really break if this weren't so, but I
strongly suspect that they would.)
SINGLE-FLOAT is distinct from DOUBLE-FLOAT.

284
base-target-features.lisp-expr Normal file
View file

@ -0,0 +1,284 @@
;;;; tags which are set during the build process and which end up in
;;;; CL:*FEATURES* in the target SBCL, plus some comments about other
;;;; CL:*FEATURES* tags which have special meaning to SBCL or which
;;;; have a special conventional meaning
;;;; This software is part of the SBCL system. See the README file for
;;;; more information.
;;;;
;;;; This software is derived from the CMU CL system, which was
;;;; written at Carnegie Mellon University and released into the
;;;; public domain. The software is in the public domain and is
;;;; provided with absolutely no warranty. See the COPYING and CREDITS
;;;; files for more information.
(
;;
;; features present in all builds
;;
;; our standard
:ansi-cl :common-lisp
;; FIXME: Isn't there a :x3jsomething feature which we should set too?
;; our dialect
:sbcl
;; Douglas Thomas Crosher's conservative generational GC (the only one
;; we currently support)
:gencgc
;; We're running under a UNIX. This is sort of redundant, and it was also
;; sort of redundant under CMU CL, which we inherited it from: neither SBCL
;; nor CMU CL supports anything but UNIX (and "technically not UNIX"es
;; such as *BSD and Linux). But someday, maybe we might, and in that case
;; we'd presumably remove this, so its presence conveys the information
;; that the system isn't one which follows such a change.
:unix
;;
;; features present in this particular build
;;
;; Setting this enables the compilation of documentation strings
;; from the system sources into the target Lisp executable.
;; Traditional Common Lisp folk will want this option set.
;; I (WHN) made it optional because I came to Common Lisp from
;; C++ through Scheme, so I'm accustomed to asking
;; Emacs about things that I'm curious about instead of asking
;; the executable I'm running.
:sb-doc
;; When this is set, EVAL is implemented as an "IR1 interpreter":
;; code is compiled into the compiler's first internal representation,
;; then the IR1 is interpreted. When this is not set, EVAL is implemented
;; as a little bit of hackery wrapped around a call to COMPILE, i.e.
;; the system becomes a "compiler-only implementation" of Common Lisp.
;; As of sbcl-0.6.7, the compiler-only implementation is prototype code,
;; and much less mature than the old IR1 interpreter. Thus, the safe
;; thing is to leave :SB-INTERPRETER set. However, the compiler-only
;; system is noticeably smaller, so you might want to omit
;; :SB-INTERPRETER if you have a small machine.
;;
;; Probably, the compiler-only implementation will become more
;; stable someday, and support for the IR1 interpreter will then be
;; dropped. This will make the system smaller and easier to maintain
;; not only because we no longer need to support the interpreter,
;; but because code elsewhere in the system (the dumper, the debugger,
;; etc.) no longer needs special cases for interpreted code.
:sb-interpreter
;; Do regression and other tests when building the system. You
;; might or might not want this if you're not a developer,
;; depending on how paranoid you are. You probably do want it if
;; you are a developer.
:sb-test
;; Setting this makes more debugging information available.
;; If you aren't hacking or troubleshooting SBCL itself, you
;; probably don't want this set.
;;
;; At least two varieties of debugging information are enabled by this
;; option:
;; * SBCL is compiled with a higher level of OPTIMIZE DEBUG, so that
;; the debugger can tell more about the state of the system.
;; * Various code to print debugging messages, and similar debugging code,
;; is compiled only when this feature is present.
;;
;; Note that the extra information recorded by the compiler at
;; this higher level of OPTIMIZE DEBUG includes the source location
;; forms. In order for the debugger to use this information, it has to
;; re-READ the source file. In an ordinary installation of SBCL, this
;; re-READing may not work very well, for either of two reasons:
;; * The sources aren't present on the system in the same location that
;; they were on the system where SBCL was compiled.
;; * SBCL is using the standard readtable, without the added hackage
;; which allows it to handle things like target features.
;; If you want to be able to use the extra debugging information,
;; therefore, be sure to keep the sources around, and run with the
;; readtable configured so that the system sources can be read.
; :sb-show
;; Enable extra debugging output in the assem.lisp assembler/scheduler
;; code. (This is the feature which was called :DEBUG in the
;; original CMU CL code.)
; :sb-show-assem
;; Setting this makes SBCL more "fluid", i.e. more amenable to
;; modification at runtime, by suppressing various INLINE declarations,
;; compiler macro definitions, FREEZE-TYPE declarations; and by
;; suppressing various burning-our-ships-behind-us actions after
;; initialization is complete; and so forth. This tends to clobber the
;; performance of the system, so unless you have some special need for
;; this when hacking SBCL itself, you don't want this set.
; :sb-fluid
;; Enable code for collecting statistics on usage of various operations,
;; useful for performance tuning of the SBCL system itself. This code
;; is probably pretty stale (having not been tested since the fork from
;; base CMU CL) but might nonetheless be a useful starting point for
;; anyone who wants to collect such statistics in the future.
; :sb-dyncount
;; Peter Van Eynde's increase-bulletproofness code
;;
;; This is not maintained or tested in current SBCL, but I haven't
;; gone out of my way to remove or break it, either.
;;
; :high-security
; :high-security-support
;; multiprocessing support
;;
;; This is not maintained or tested in current SBCL. I haven't gone out
;; of my way to break it, but since it's derived from an old version of
;; CMU CL where multiprocessing was pretty shaky, it's likely to be very
;; flaky now.
;; :MP enables multiprocessing
;; :MP-I486 is used, only within the multiprocessing code, to control
;; what seems to control processor-version-specific code. It's
;; probably for 486 or later, i.e. could be set as long as
;; you know you're not running on a 386, but it doesn't seem
;; to be documented anywhere, so that's just a guess.
; :mp
; :mp-i486
;; KLUDGE: used to suppress stale code related to floating point infinities.
;; I intend to delete this code completely some day, since it was a pain
;; for me to try to work with and since all benefits it provides are
;; non-portable. Until I actually pull the trigger, though, I've left
;; various stale code in place protected with #!-SB-INFINITIES.
; :sb-infinities
;; This affects the definition of a lot of things in bignum.lisp. It
;; doesn't seem to be documented anywhere what systems it might apply to.
;; It doesn't seem to be needed for X86 systems anyway.
; :32x16-divide
;; This is probably true for some processor types, but not X86. It affects
;; a lot of floating point code.
; :negative-zero-is-not-zero
;; This is mentioned in cmu-user.tex, which says that it enables
;; the compiler to reason about integer arithmetic. It also seems to
;; control other fancy numeric reasoning, e.g. knowing the result type of
;; a remainder calculation given the type of its inputs.
;;
;; KLUDGE: Even when this is implemented for the target feature list,
;; the code to implement this feature will not generated in the
;; cross-compiler (i.e. will only be generated in the target compiler).
;; The reason for this is that the interval arithmetic routines used
;; to implement this feature are written under the assumption that
;; Lisp arithmetic supports plus and minus infinity, which isn't guaranteed by
;; ANSI Common Lisp. I've tried to mark the conditionals which implement
;; this kludge with the string CROSS-FLOAT-INFINITY-KLUDGE so that
;; sometime it might be possible to undo them (perhaps by using
;; nice portable :PLUS-INFINITY and :MINUS-INFINITY values instead of
;; implementation dependent floating infinity values, which would
;; admittedly involve extra consing; or perhaps by finding some cleaner
;; way of suppressing the construction of this code in the cross-compiler).
;;
;; KLUDGE: Even after doing the KLUDGE above, the cross-compiler doesn't work,
;; because some interval operations are conditional on PROPAGATE-FUN-TYPE
;; instead of PROPAGATE-FLOAT-TYPE. So for now, I've completely turned off
;; both PROPAGATE-FUN-TYPE and PROPAGATE-FLOAT-TYPE. (After I build
;; a compiler which works, then I can think about getting the optimization
;; to work.) -- WHN 19990702
; :propagate-float-type
;; According to cmu-user.tex, this enables the compiler to infer result
;; types for mathematical functions a la SQRT, EXPT, and LOG, allowing
;; it to e.g. eliminate the possibility that a complex result will be
;; generated.
;;
;; KLUDGE: turned off as per the comments for PROPAGATE-FLOAT-TYPE above
; :propagate-fun-type
;; It's unclear to me what this does (but it was enabled in the code that I
;; picked up from Peter Van Eynde). -- WHN 19990224
:constrain-float-type
;; This is set in classic CMU CL, and presumably there it means
;; that the floating point arithmetic implementation
;; conforms to IEEE's standard. Here it definitely means that the
;; floating point arithmetic implementation conforms to IEEE's standard.
;; I (WHN 19990702) haven't tried to verify
;; that it does conform, but it should at least mostly conform (because
;; the underlying x86 hardware tries).
:ieee-floating-point
;; This seems to be the pre-GENCGC garbage collector for CMU CL, which was
;; AFAIK never supported for the X86.
; :gengc
;; CMU CL had, and we inherited, code to support 80-bit LONG-FLOAT on the x86
;; architecture. Nothing has been done to actively destroy the long float
;; support, but it hasn't been thoroughly maintained, and needs at least
;; some maintenance before it will work. (E.g. the LONG-FLOAT-only parts of
;; genesis are still implemented in terms of unportable CMU CL functions
;; which are not longer available at genesis time in SBCL.) A deeper
;; problem is SBCL's bootstrap process implicitly assumes that the
;; cross-compilation host will be able to make the same distinctions
;; between floating point types that it does. This assumption is
;; fundamentally sleazy, even though in practice it's unlikely to break down
;; w.r.t. distinguishing SINGLE-FLOAT from DOUBLE-FLOAT; it's much more
;; likely to break down w.r.t. distinguishing DOUBLE-FLOAT from LONG-FLOAT.
;; Still it's likely to be quite doable to get LONG-FLOAT support working
;; again, if anyone's sufficiently motivated.
; :long-float
;;
;; miscellaneous notes on other things which could have special significance
;; in the *FEATURES* list
;;
;; notes on the :NIL and :IGNORE features:
;;
;; #+NIL is used to comment out forms. Occasionally #+IGNORE is used
;; for this too. So don't use :NIL or :IGNORE as the names of features..
;; notes on :SB-XC and :SB-XC-HOST features (which aren't controlled by this
;; file, but are instead temporarily pushed onto *FEATURES* or
;; *TARGET-FEATURES* during some phases of cross-compilation):
;;
;; :SB-XC-HOST stands for "cross-compilation host" and is in *FEATURES*
;; during the first phase of cross-compilation bootstrapping, when the
;; host Lisp is being used to compile the cross-compiler.
;;
;; :SB-XC stands for "cross compiler", and is in *FEATURES* during the second
;; phase of cross-compilation bootstrapping, when the cross-compiler is
;; being used to create the first target Lisp.
;; notes on the :SB-ASSEMBLING feature (which isn't controlled by
;; this file):
;;
;; This is a flag for whether we're in the assembler. It's
;; temporarily pushed onto the *FEATURES* list in the setup for
;; the ASSEMBLE-FILE function. It would be a bad idea
;; to use it as a name for a permanent feature.
;; notes on local features (which are set automatically by the
;; configuration script, and should not be set here unless you
;; really, really know what you're doing):
;;
;; machine architecture features:
;; :x86 ; any Intel 386 or better, or compatibles like the AMD K6 or K7
;; (No others are supported by SBCL as of 0.6.7, but :alpha or
;; :sparc support could be ported from CMU CL if anyone is
;; sufficiently motivated to do so.)
;; (CMU CL also had a :pentium feature, which affected the definition
;; of some floating point vops. It was present but not enabled in the
;; CMU CL code that SBCL is derived from, and is present but stale
;; in SBCL as of 0.6.7.)
;;
;; operating system features:
;; :linux = We're intended to run under some version of Linux.
;; :bsd = We're intended to run under some version of BSD Unix. (This
;; is not exclusive with the features which indicate which
;; particular version of BSD we're intended to run under.)
;; :freebsd = We're intended to run under FreeBSD.
;; :openbsd = We're intended to run under FreeBSD.
;; (No others are supported by SBCL as of 0.6.7, but :hpux or
;; :solaris support could be ported from CMU CL if anyone is
;; sufficiently motivated to do so.)
)

13
binary-distribution.sh Executable file
View file

@ -0,0 +1,13 @@
#!/bin/sh
# Create a binary distribution. (make.sh should be run first to create
# the various binary files, and make-doc.sh, or possibly some other
# DocBook-to-HTML converter, should also be run to create the
# HTML version of the documentation.)
tar cf ../sbcl-x.y.z-binary.tar \
output/sbcl.core src/runtime/sbcl \
BUGS COPYING CREDITS INSTALL NEWS README \
install.sh \
doc/sbcl.1 doc/cmucl/cmu-user doc/*.htm* \
pubring.pgp

64
clean.sh Executable file
View file

@ -0,0 +1,64 @@
#!/bin/sh
# Remove everything in directories which are only used for output.
# In most cases, we can remove the directories, too.
#
# (We don't remove all the directories themselves for a stupid technical
# reason: "gmake clean" in the src/runtime directory gets unhappy if the
# output/ directory doesn't exist, because it tries to build Depends
# before it cleans itself, and src/c-runtime/sbcl.h is a symlink into
# the output/ directory, and it gets the gcc dependency processing gets
# all confused trying to figure out a header file which is a symlink
# into a directory which doesn't exist. We'd like to be able to run
# this script (including "gmake clean" in the src/runtime directory)
# several times in a row without failure.. so we leave the output/
# directory in place.)
rm -rf obj/* output/* doc/user-manual/ \
doc/user-manual.junk/ doc/DBTOHTML_OUTPUT_DIR*
# (The doc/user-manual.junk/ and doc/DBTOHTML_OUTPUT_DIR* directories
# are created when the Cygnus db2html script when it formats the the
# user manual, and since this db2html script is the one which is
# currently used to format the manual for the standard binary
# distribution, we automatically clean up after it here in the
# standard clean.sh file.)
# Within other directories, remove things which don't look like source
# files. Some explanations:
# (symlinks)
# are never in the sources; they must've been created
# sbcl
# the runtime environment, created by compiling C code
# sbcl.h
# information about Lisp code needed to build the runtime environment,
# created by running GENESIS
# Config, target
# architecture-dependent or OS-dependent symlinks
# *.htm, *.html
# probably machine-generated translation of DocBook (*.sgml) files
# core
# probably a core dump -- not part of the sources anyway
# *~, #*#, TAGS
# common names for editor temporary files
find . \( \
-type l -or \
-name '*~' -or \
-name '#*#' -or \
-name '?*.x86f' -or \
-name '?*.lbytef' -or \
-name 'core' -or \
-name '?*.core' -or \
-name '*.map' -or \
-name '*.nm' -or \
-name '*.host-obj' -or \
-name '*.lisp-obj' -or \
-name '*.target-obj' -or \
-name '*.lib' -or \
-name '*.tmp' -or \
-name '*.o' -or \
-name 'sbcl' -or \
-name 'sbcl.h' -or \
-name 'depend' -or \
-name '*.htm' -or \
-name '*.html' -or \
-name 'TAGS' -or \
-name 'local-target-features.lisp-expr' \) -print | xargs rm -f

477
common-lisp-exports.lisp-expr Normal file
View file

@ -0,0 +1,477 @@
;;; symbols exported from the COMMON-LISP package (from the ANSI spec,
;;; section 1.9, figures 1-4 to 1-15, inclusive)
(
;; from figure 1-4:
"&ALLOW-OTHER-KEYS" "*PRINT-MISER-WIDTH*"
"&AUX" "*PRINT-PPRINT-DISPATCH*"
"&BODY" "*PRINT-PRETTY*"
"&ENVIRONMENT" "*PRINT-RADIX*"
"&KEY" "*PRINT-READABLY*"
"&OPTIONAL" "*PRINT-RIGHT-MARGIN*"
"&REST" "*QUERY-IO*"
"&WHOLE" "*RANDOM-STATE*"
"*" "*READ-BASE*"
"**" "*READ-DEFAULT-FLOAT-FORMAT*"
"***" "*READ-EVAL*"
"*BREAK-ON-SIGNALS*" "*READ-SUPPRESS*"
"*COMPILE-FILE-PATHNAME*" "*READTABLE*"
"*COMPILE-FILE-TRUENAME*" "*STANDARD-INPUT*"
"*COMPILE-PRINT*" "*STANDARD-OUTPUT*"
"*COMPILE-VERBOSE*" "*TERMINAL-IO*"
"*DEBUG-IO*" "*TRACE-OUTPUT*"
"*DEBUGGER-HOOK*" "+"
"*DEFAULT-PATHNAME-DEFAULTS*" "++"
"*ERROR-OUTPUT*" "+++"
"*FEATURES*" "-"
"*GENSYM-COUNTER*" "/"
"*LOAD-PATHNAME*" "//"
"*LOAD-PRINT*" "///"
"*LOAD-TRUENAME*" "/="
"*LOAD-VERBOSE*" "1+"
"*MACROEXPAND-HOOK*" "1-"
"*MODULES*" "<"
"*PACKAGE*" "<="
"*PRINT-ARRAY*" "="
"*PRINT-BASE*" ">"
"*PRINT-CASE*" ">="
"*PRINT-CIRCLE*" "ABORT"
"*PRINT-ESCAPE*" "ABS"
"*PRINT-GENSYM*" "ACONS"
"*PRINT-LENGTH*" "ACOS"
"*PRINT-LEVEL*" "ACOSH"
"*PRINT-LINES*" "ADD-METHOD"
;; from figure 1-5:
"ADJOIN" "ATOM" "BOUNDP"
"ADJUST-ARRAY" "BASE-CHAR" "BREAK"
"ADJUSTABLE-ARRAY-P" "BASE-STRING" "BROADCAST-STREAM"
"ALLOCATE-INSTANCE" "BIGNUM" "BROADCAST-STREAM-STREAMS"
"ALPHA-CHAR-P" "BIT" "BUILT-IN-CLASS"
"ALPHANUMERICP" "BIT-AND" "BUTLAST"
"AND" "BIT-ANDC1" "BYTE"
"APPEND" "BIT-ANDC2" "BYTE-POSITION"
"APPLY" "BIT-EQV" "BYTE-SIZE"
"APROPOS" "BIT-IOR" "CAAAAR"
"APROPOS-LIST" "BIT-NAND" "CAAADR"
"AREF" "BIT-NOR" "CAAAR"
"ARITHMETIC-ERROR" "BIT-NOT" "CAADAR"
"ARITHMETIC-ERROR-OPERANDS" "BIT-ORC1" "CAADDR"
"ARITHMETIC-ERROR-OPERATION" "BIT-ORC2" "CAADR"
"ARRAY" "BIT-VECTOR" "CAAR"
"ARRAY-DIMENSION" "BIT-VECTOR-P" "CADAAR"
"ARRAY-DIMENSION-LIMIT" "BIT-XOR" "CADADR"
"ARRAY-DIMENSIONS" "BLOCK" "CADAR"
"ARRAY-DISPLACEMENT" "BOOLE" "CADDAR"
"ARRAY-ELEMENT-TYPE" "BOOLE-1" "CADDDR"
"ARRAY-HAS-FILL-POINTER-P" "BOOLE-2" "CADDR"
"ARRAY-IN-BOUNDS-P" "BOOLE-AND" "CADR"
"ARRAY-RANK" "BOOLE-ANDC1" "CALL-ARGUMENTS-LIMIT"
"ARRAY-RANK-LIMIT" "BOOLE-ANDC2" "CALL-METHOD"
"ARRAY-ROW-MAJOR-INDEX" "BOOLE-C1" "CALL-NEXT-METHOD"
"ARRAY-TOTAL-SIZE" "BOOLE-C2" "CAR"
"ARRAY-TOTAL-SIZE-LIMIT" "BOOLE-CLR" "CASE"
"ARRAYP" "BOOLE-EQV" "CATCH"
"ASH" "BOOLE-IOR" "CCASE"
"ASIN" "BOOLE-NAND" "CDAAAR"
"ASINH" "BOOLE-NOR" "CDAADR"
"ASSERT" "BOOLE-ORC1" "CDAAR"
"ASSOC" "BOOLE-ORC2" "CDADAR"
"ASSOC-IF" "BOOLE-SET" "CDADDR"
"ASSOC-IF-NOT" "BOOLE-XOR" "CDADR"
"ATAN" "BOOLEAN" "CDAR"
"ATANH" "BOTH-CASE-P" "CDDAAR"
;; from figure 1-6:
"CDDADR" "CLEAR-INPUT" "COPY-TREE"
"CDDAR" "CLEAR-OUTPUT" "COS"
"CDDDAR" "CLOSE" "COSH"
"CDDDDR" "CLRHASH" "COUNT"
"CDDDR" "CODE-CHAR" "COUNT-IF"
"CDDR" "COERCE" "COUNT-IF-NOT"
"CDR" "COMPILATION-SPEED" "CTYPECASE"
"CEILING" "COMPILE" "DEBUG"
"CELL-ERROR" "COMPILE-FILE" "DECF"
"CELL-ERROR-NAME" "COMPILE-FILE-PATHNAME" "DECLAIM"
"CERROR" "COMPILED-FUNCTION" "DECLARATION"
"CHANGE-CLASS" "COMPILED-FUNCTION-P" "DECLARE"
"CHAR" "COMPILER-MACRO" "DECODE-FLOAT"
"CHAR-CODE" "COMPILER-MACRO-FUNCTION" "DECODE-UNIVERSAL-TIME"
"CHAR-CODE-LIMIT" "COMPLEMENT" "DEFCLASS"
"CHAR-DOWNCASE" "COMPLEX" "DEFCONSTANT"
"CHAR-EQUAL" "COMPLEXP" "DEFGENERIC"
"CHAR-GREATERP" "COMPUTE-APPLICABLE-METHODS" "DEFINE-COMPILER-MACRO"
"CHAR-INT" "COMPUTE-RESTARTS" "DEFINE-CONDITION"
"CHAR-LESSP" "CONCATENATE" "DEFINE-METHOD-COMBINATION"
"CHAR-NAME" "CONCATENATED-STREAM" "DEFINE-MODIFY-MACRO"
"CHAR-NOT-EQUAL" "CONCATENATED-STREAM-STREAMS" "DEFINE-SETF-EXPANDER"
"CHAR-NOT-GREATERP" "COND" "DEFINE-SYMBOL-MACRO"
"CHAR-NOT-LESSP" "CONDITION" "DEFMACRO"
"CHAR-UPCASE" "CONJUGATE" "DEFMETHOD"
"CHAR/=" "CONS" "DEFPACKAGE"
"CHAR<" "CONSP" "DEFPARAMETER"
"CHAR<=" "CONSTANTLY" "DEFSETF"
"CHAR=" "CONSTANTP" "DEFSTRUCT"
"CHAR>" "CONTINUE" "DEFTYPE"
"CHAR>=" "CONTROL-ERROR" "DEFUN"
"CHARACTER" "COPY-ALIST" "DEFVAR"
"CHARACTERP" "COPY-LIST" "DELETE"
"CHECK-TYPE" "COPY-PPRINT-DISPATCH" "DELETE-DUPLICATES"
"CIS" "COPY-READTABLE" "DELETE-FILE"
"CLASS" "COPY-SEQ" "DELETE-IF"
"CLASS-NAME" "COPY-STRUCTURE" "DELETE-IF-NOT"
"CLASS-OF" "COPY-SYMBOL" "DELETE-PACKAGE"
;; from figure 1-7:
"DENOMINATOR" "EQ"
"DEPOSIT-FIELD" "EQL"
"DESCRIBE" "EQUAL"
"DESCRIBE-OBJECT" "EQUALP"
"DESTRUCTURING-BIND" "ERROR"
"DIGIT-CHAR" "ETYPECASE"
"DIGIT-CHAR-P" "EVAL"
"DIRECTORY" "EVAL-WHEN"
"DIRECTORY-NAMESTRING" "EVENP"
"DISASSEMBLE" "EVERY"
"DIVISION-BY-ZERO" "EXP"
"DO" "EXPORT"
"DO*" "EXPT"
"DO-ALL-SYMBOLS" "EXTENDED-CHAR"
"DO-EXTERNAL-SYMBOLS" "FBOUNDP"
"DO-SYMBOLS" "FCEILING"
"DOCUMENTATION" "FDEFINITION"
"DOLIST" "FFLOOR"
"DOTIMES" "FIFTH"
"DOUBLE-FLOAT" "FILE-AUTHOR"
"DOUBLE-FLOAT-EPSILON" "FILE-ERROR"
"DOUBLE-FLOAT-NEGATIVE-EPSILON" "FILE-ERROR-PATHNAME"
"DPB" "FILE-LENGTH"
"DRIBBLE" "FILE-NAMESTRING"
"DYNAMIC-EXTENT" "FILE-POSITION"
"ECASE" "FILE-STREAM"
"ECHO-STREAM" "FILE-STRING-LENGTH"
"ECHO-STREAM-INPUT-STREAM" "FILE-WRITE-DATE"
"ECHO-STREAM-OUTPUT-STREAM" "FILL"
"ED" "FILL-POINTER"
"EIGHTH" "FIND"
"ELT" "FIND-ALL-SYMBOLS"
"ENCODE-UNIVERSAL-TIME" "FIND-CLASS"
"END-OF-FILE" "FIND-IF"
"ENDP" "FIND-IF-NOT"
"ENOUGH-NAMESTRING" "FIND-METHOD"
"ENSURE-DIRECTORIES-EXIST" "FIND-PACKAGE"
"ENSURE-GENERIC-FUNCTION" "FIND-RESTART"
;; from figure 1-8:
"FIND-SYMBOL" "GET-INTERNAL-RUN-TIME"
"FINISH-OUTPUT" "GET-MACRO-CHARACTER"
"FIRST" "GET-OUTPUT-STREAM-STRING"
"FIXNUM" "GET-PROPERTIES"
"FLET" "GET-SETF-EXPANSION"
"FLOAT" "GET-UNIVERSAL-TIME"
"FLOAT-DIGITS" "GETF"
"FLOAT-PRECISION" "GETHASH"
"FLOAT-RADIX" "GO"
"FLOAT-SIGN" "GRAPHIC-CHAR-P"
"FLOATING-POINT-INEXACT" "HANDLER-BIND"
"FLOATING-POINT-INVALID-OPERATION" "HANDLER-CASE"
"FLOATING-POINT-OVERFLOW" "HASH-TABLE"
"FLOATING-POINT-UNDERFLOW" "HASH-TABLE-COUNT"
"FLOATP" "HASH-TABLE-P"
"FLOOR" "HASH-TABLE-REHASH-SIZE"
"FMAKUNBOUND" "HASH-TABLE-REHASH-THRESHOLD"
"FORCE-OUTPUT" "HASH-TABLE-SIZE"
"FORMAT" "HASH-TABLE-TEST"
"FORMATTER" "HOST-NAMESTRING"
"FOURTH" "IDENTITY"
"FRESH-LINE" "IF"
"FROUND" "IGNORABLE"
"FTRUNCATE" "IGNORE"
"FTYPE" "IGNORE-ERRORS"
"FUNCALL" "IMAGPART"
"FUNCTION" "IMPORT"
"FUNCTION-KEYWORDS" "IN-PACKAGE"
"FUNCTION-LAMBDA-EXPRESSION" "INCF"
"FUNCTIONP" "INITIALIZE-INSTANCE"
"GCD" "INLINE"
"GENERIC-FUNCTION" "INPUT-STREAM-P"
"GENSYM" "INSPECT"
"GENTEMP" "INTEGER"
"GET" "INTEGER-DECODE-FLOAT"
"GET-DECODED-TIME" "INTEGER-LENGTH"
"GET-DISPATCH-MACRO-CHARACTER" "INTEGERP"
"GET-INTERNAL-REAL-TIME" "INTERACTIVE-STREAM-P"
;; from figure 1-9:
"INTERN" "LISP-IMPLEMENTATION-TYPE"
"INTERNAL-TIME-UNITS-PER-SECOND" "LISP-IMPLEMENTATION-VERSION"
"INTERSECTION" "LIST"
"INVALID-METHOD-ERROR" "LIST*"
"INVOKE-DEBUGGER" "LIST-ALL-PACKAGES"
"INVOKE-RESTART" "LIST-LENGTH"
"INVOKE-RESTART-INTERACTIVELY" "LISTEN"
"ISQRT" "LISTP"
"KEYWORD" "LOAD"
"KEYWORDP" "LOAD-LOGICAL-PATHNAME-TRANSLATIONS"
"LABELS" "LOAD-TIME-VALUE"
"LAMBDA" "LOCALLY"
"LAMBDA-LIST-KEYWORDS" "LOG"
"LAMBDA-PARAMETERS-LIMIT" "LOGAND"
"LAST" "LOGANDC1"
"LCM" "LOGANDC2"
"LDB" "LOGBITP"
"LDB-TEST" "LOGCOUNT"
"LDIFF" "LOGEQV"
"LEAST-NEGATIVE-DOUBLE-FLOAT" "LOGICAL-PATHNAME"
"LEAST-NEGATIVE-LONG-FLOAT" "LOGICAL-PATHNAME-TRANSLATIONS"
"LEAST-NEGATIVE-NORMALIZED-DOUBLE-FLOAT" "LOGIOR"
"LEAST-NEGATIVE-NORMALIZED-LONG-FLOAT" "LOGNAND"
"LEAST-NEGATIVE-NORMALIZED-SHORT-FLOAT" "LOGNOR"
"LEAST-NEGATIVE-NORMALIZED-SINGLE-FLOAT" "LOGNOT"
"LEAST-NEGATIVE-SHORT-FLOAT" "LOGORC1"
"LEAST-NEGATIVE-SINGLE-FLOAT" "LOGORC2"
"LEAST-POSITIVE-DOUBLE-FLOAT" "LOGTEST"
"LEAST-POSITIVE-LONG-FLOAT" "LOGXOR"
"LEAST-POSITIVE-NORMALIZED-DOUBLE-FLOAT" "LONG-FLOAT"
"LEAST-POSITIVE-NORMALIZED-LONG-FLOAT" "LONG-FLOAT-EPSILON"
"LEAST-POSITIVE-NORMALIZED-SHORT-FLOAT" "LONG-FLOAT-NEGATIVE-EPSILON"
"LEAST-POSITIVE-NORMALIZED-SINGLE-FLOAT" "LONG-SITE-NAME"
"LEAST-POSITIVE-SHORT-FLOAT" "LOOP"
"LEAST-POSITIVE-SINGLE-FLOAT" "LOOP-FINISH"
"LENGTH" "LOWER-CASE-P"
"LET" "MACHINE-INSTANCE"
"LET*" "MACHINE-TYPE"
;; from figure 1-10:
"MACHINE-VERSION" "MASK-FIELD"
"MACRO-FUNCTION" "MAX"
"MACROEXPAND" "MEMBER"
"MACROEXPAND-1" "MEMBER-IF"
"MACROLET" "MEMBER-IF-NOT"
"MAKE-ARRAY" "MERGE"
"MAKE-BROADCAST-STREAM" "MERGE-PATHNAMES"
"MAKE-CONCATENATED-STREAM" "METHOD"
"MAKE-CONDITION" "METHOD-COMBINATION"
"MAKE-DISPATCH-MACRO-CHARACTER" "METHOD-COMBINATION-ERROR"
"MAKE-ECHO-STREAM" "METHOD-QUALIFIERS"
"MAKE-HASH-TABLE" "MIN"
"MAKE-INSTANCE" "MINUSP"
"MAKE-INSTANCES-OBSOLETE" "MISMATCH"
"MAKE-LIST" "MOD"
"MAKE-LOAD-FORM" "MOST-NEGATIVE-DOUBLE-FLOAT"
"MAKE-LOAD-FORM-SAVING-SLOTS" "MOST-NEGATIVE-FIXNUM"
"MAKE-METHOD" "MOST-NEGATIVE-LONG-FLOAT"
"MAKE-PACKAGE" "MOST-NEGATIVE-SHORT-FLOAT"
"MAKE-PATHNAME" "MOST-NEGATIVE-SINGLE-FLOAT"
"MAKE-RANDOM-STATE" "MOST-POSITIVE-DOUBLE-FLOAT"
"MAKE-SEQUENCE" "MOST-POSITIVE-FIXNUM"
"MAKE-STRING" "MOST-POSITIVE-LONG-FLOAT"
"MAKE-STRING-INPUT-STREAM" "MOST-POSITIVE-SHORT-FLOAT"
"MAKE-STRING-OUTPUT-STREAM" "MOST-POSITIVE-SINGLE-FLOAT"
"MAKE-SYMBOL" "MUFFLE-WARNING"
"MAKE-SYNONYM-STREAM" "MULTIPLE-VALUE-BIND"
"MAKE-TWO-WAY-STREAM" "MULTIPLE-VALUE-CALL"
"MAKUNBOUND" "MULTIPLE-VALUE-LIST"
"MAP" "MULTIPLE-VALUE-PROG1"
"MAP-INTO" "MULTIPLE-VALUE-SETQ"
"MAPC" "MULTIPLE-VALUES-LIMIT"
"MAPCAN" "NAME-CHAR"
"MAPCAR" "NAMESTRING"
"MAPCON" "NBUTLAST"
"MAPHASH" "NCONC"
"MAPL" "NEXT-METHOD-P"
"MAPLIST" "NIL"
;; from figure 1-11:
"NINTERSECTION" "PACKAGE-ERROR"
"NINTH" "PACKAGE-ERROR-PACKAGE"
"NO-APPLICABLE-METHOD" "PACKAGE-NAME"
"NO-NEXT-METHOD" "PACKAGE-NICKNAMES"
"NOT" "PACKAGE-SHADOWING-SYMBOLS"
"NOTANY" "PACKAGE-USE-LIST"
"NOTEVERY" "PACKAGE-USED-BY-LIST"
"NOTINLINE" "PACKAGEP"
"NRECONC" "PAIRLIS"
"NREVERSE" "PARSE-ERROR"
"NSET-DIFFERENCE" "PARSE-INTEGER"
"NSET-EXCLUSIVE-OR" "PARSE-NAMESTRING"
"NSTRING-CAPITALIZE" "PATHNAME"
"NSTRING-DOWNCASE" "PATHNAME-DEVICE"
"NSTRING-UPCASE" "PATHNAME-DIRECTORY"
"NSUBLIS" "PATHNAME-HOST"
"NSUBST" "PATHNAME-MATCH-P"
"NSUBST-IF" "PATHNAME-NAME"
"NSUBST-IF-NOT" "PATHNAME-TYPE"
"NSUBSTITUTE" "PATHNAME-VERSION"
"NSUBSTITUTE-IF" "PATHNAMEP"
"NSUBSTITUTE-IF-NOT" "PEEK-CHAR"
"NTH" "PHASE"
"NTH-VALUE" "PI"
"NTHCDR" "PLUSP"
"NULL" "POP"
"NUMBER" "POSITION"
"NUMBERP" "POSITION-IF"
"NUMERATOR" "POSITION-IF-NOT"
"NUNION" "PPRINT"
"ODDP" "PPRINT-DISPATCH"
"OPEN" "PPRINT-EXIT-IF-LIST-EXHAUSTED"
"OPEN-STREAM-P" "PPRINT-FILL"
"OPTIMIZE" "PPRINT-INDENT"
"OR" "PPRINT-LINEAR"
"OTHERWISE" "PPRINT-LOGICAL-BLOCK"
"OUTPUT-STREAM-P" "PPRINT-NEWLINE"
"PACKAGE" "PPRINT-POP"
;; from figure 1-12:
"PPRINT-TAB" "READ-CHAR"
"PPRINT-TABULAR" "READ-CHAR-NO-HANG"
"PRIN1" "READ-DELIMITED-LIST"
"PRIN1-TO-STRING" "READ-FROM-STRING"
"PRINC" "READ-LINE"
"PRINC-TO-STRING" "READ-PRESERVING-WHITESPACE"
"PRINT" "READ-SEQUENCE"
"PRINT-NOT-READABLE" "READER-ERROR"
"PRINT-NOT-READABLE-OBJECT" "READTABLE"
"PRINT-OBJECT" "READTABLE-CASE"
"PRINT-UNREADABLE-OBJECT" "READTABLEP"
"PROBE-FILE" "REAL"
"PROCLAIM" "REALP"
"PROG" "REALPART"
"PROG*" "REDUCE"
"PROG1" "REINITIALIZE-INSTANCE"
"PROG2" "REM"
"PROGN" "REMF"
"PROGRAM-ERROR" "REMHASH"
"PROGV" "REMOVE"
"PROVIDE" "REMOVE-DUPLICATES"
"PSETF" "REMOVE-IF"
"PSETQ" "REMOVE-IF-NOT"
"PUSH" "REMOVE-METHOD"
"PUSHNEW" "REMPROP"
"QUOTE" "RENAME-FILE"
"RANDOM" "RENAME-PACKAGE"
"RANDOM-STATE" "REPLACE"
"RANDOM-STATE-P" "REQUIRE"
"RASSOC" "REST"
"RASSOC-IF" "RESTART"
"RASSOC-IF-NOT" "RESTART-BIND"
"RATIO" "RESTART-CASE"
"RATIONAL" "RESTART-NAME"
"RATIONALIZE" "RETURN"
"RATIONALP" "RETURN-FROM"
"READ" "REVAPPEND"
"READ-BYTE" "REVERSE"
;; from figure 1-13:
"ROOM" "SIMPLE-BIT-VECTOR"
"ROTATEF" "SIMPLE-BIT-VECTOR-P"
"ROUND" "SIMPLE-CONDITION"
"ROW-MAJOR-AREF" "SIMPLE-CONDITION-FORMAT-ARGUMENTS"
"RPLACA" "SIMPLE-CONDITION-FORMAT-CONTROL"
"RPLACD" "SIMPLE-ERROR"
"SAFETY" "SIMPLE-STRING"
"SATISFIES" "SIMPLE-STRING-P"
"SBIT" "SIMPLE-TYPE-ERROR"
"SCALE-FLOAT" "SIMPLE-VECTOR"
"SCHAR" "SIMPLE-VECTOR-P"
"SEARCH" "SIMPLE-WARNING"
"SECOND" "SIN"
"SEQUENCE" "SINGLE-FLOAT"
"SERIOUS-CONDITION" "SINGLE-FLOAT-EPSILON"
"SET" "SINGLE-FLOAT-NEGATIVE-EPSILON"
"SET-DIFFERENCE" "SINH"
"SET-DISPATCH-MACRO-CHARACTER" "SIXTH"
"SET-EXCLUSIVE-OR" "SLEEP"
"SET-MACRO-CHARACTER" "SLOT-BOUNDP"
"SET-PPRINT-DISPATCH" "SLOT-EXISTS-P"
"SET-SYNTAX-FROM-CHAR" "SLOT-MAKUNBOUND"
"SETF" "SLOT-MISSING"
"SETQ" "SLOT-UNBOUND"
"SEVENTH" "SLOT-VALUE"
"SHADOW" "SOFTWARE-TYPE"
"SHADOWING-IMPORT" "SOFTWARE-VERSION"
"SHARED-INITIALIZE" "SOME"
"SHIFTF" "SORT"
"SHORT-FLOAT" "SPACE"
"SHORT-FLOAT-EPSILON" "SPECIAL"
"SHORT-FLOAT-NEGATIVE-EPSILON" "SPECIAL-OPERATOR-P"
"SHORT-SITE-NAME" "SPEED"
"SIGNAL" "SQRT"
"SIGNED-BYTE" "STABLE-SORT"
"SIGNUM" "STANDARD"
"SIMPLE-ARRAY" "STANDARD-CHAR"
"SIMPLE-BASE-STRING" "STANDARD-CHAR-P"
;; from figure 1-14:
"STANDARD-CLASS" "SUBLIS"
"STANDARD-GENERIC-FUNCTION" "SUBSEQ"
"STANDARD-METHOD" "SUBSETP"
"STANDARD-OBJECT" "SUBST"
"STEP" "SUBST-IF"
"STORAGE-CONDITION" "SUBST-IF-NOT"
"STORE-VALUE" "SUBSTITUTE"
"STREAM" "SUBSTITUTE-IF"
"STREAM-ELEMENT-TYPE" "SUBSTITUTE-IF-NOT"
"STREAM-ERROR" "SUBTYPEP"
"STREAM-ERROR-STREAM" "SVREF"
"STREAM-EXTERNAL-FORMAT" "SXHASH"
"STREAMP" "SYMBOL"
"STRING" "SYMBOL-FUNCTION"
"STRING-CAPITALIZE" "SYMBOL-MACROLET"
"STRING-DOWNCASE" "SYMBOL-NAME"
"STRING-EQUAL" "SYMBOL-PACKAGE"
"STRING-GREATERP" "SYMBOL-PLIST"
"STRING-LEFT-TRIM" "SYMBOL-VALUE"
"STRING-LESSP" "SYMBOLP"
"STRING-NOT-EQUAL" "SYNONYM-STREAM"
"STRING-NOT-GREATERP" "SYNONYM-STREAM-SYMBOL"
"STRING-NOT-LESSP" "T"
"STRING-RIGHT-TRIM" "TAGBODY"
"STRING-STREAM" "TAILP"
"STRING-TRIM" "TAN"
"STRING-UPCASE" "TANH"
"STRING/=" "TENTH"
"STRING<" "TERPRI"
"STRING<=" "THE"
"STRING=" "THIRD"
"STRING>" "THROW"
"STRING>=" "TIME"
"STRINGP" "TRACE"
"STRUCTURE" "TRANSLATE-LOGICAL-PATHNAME"
"STRUCTURE-CLASS" "TRANSLATE-PATHNAME"
"STRUCTURE-OBJECT" "TREE-EQUAL"
"STYLE-WARNING" "TRUENAME"
;; from figure 1-15:
"TRUNCATE" "VALUES-LIST"
"TWO-WAY-STREAM" "VARIABLE"
"TWO-WAY-STREAM-INPUT-STREAM" "VECTOR"
"TWO-WAY-STREAM-OUTPUT-STREAM" "VECTOR-POP"
"TYPE" "VECTOR-PUSH"
"TYPE-ERROR" "VECTOR-PUSH-EXTEND"
"TYPE-ERROR-DATUM" "VECTORP"
"TYPE-ERROR-EXPECTED-TYPE" "WARN"
"TYPE-OF" "WARNING"
"TYPECASE" "WHEN"
"TYPEP" "WILD-PATHNAME-P"
"UNBOUND-SLOT" "WITH-ACCESSORS"
"UNBOUND-SLOT-INSTANCE" "WITH-COMPILATION-UNIT"
"UNBOUND-VARIABLE" "WITH-CONDITION-RESTARTS"
"UNDEFINED-FUNCTION" "WITH-HASH-TABLE-ITERATOR"
"UNEXPORT" "WITH-INPUT-FROM-STRING"
"UNINTERN" "WITH-OPEN-FILE"
"UNION" "WITH-OPEN-STREAM"
"UNLESS" "WITH-OUTPUT-TO-STRING"
"UNREAD-CHAR" "WITH-PACKAGE-ITERATOR"
"UNSIGNED-BYTE" "WITH-SIMPLE-RESTART"
"UNTRACE" "WITH-SLOTS"
"UNUSE-PACKAGE" "WITH-STANDARD-IO-SYNTAX"
"UNWIND-PROTECT" "WRITE"
"UPDATE-INSTANCE-FOR-DIFFERENT-CLASS" "WRITE-BYTE"
"UPDATE-INSTANCE-FOR-REDEFINED-CLASS" "WRITE-CHAR"
"UPGRADED-ARRAY-ELEMENT-TYPE" "WRITE-LINE"
"UPGRADED-COMPLEX-PART-TYPE" "WRITE-SEQUENCE"
"UPPER-CASE-P" "WRITE-STRING"
"USE-PACKAGE" "WRITE-TO-STRING"
"USE-VALUE" "Y-OR-N-P"
"USER-HOMEDIR-PATHNAME" "YES-OR-NO-P"
"VALUES" "ZEROP")

21
contrib/README Normal file
View file

@ -0,0 +1,21 @@
This directory is for extensions to SBCL. They aren't necessary for
core SBCL functionality, or else they'd be built into the main SBCL
binary automatically. And they're not portable Common Lisp, or they'd
be put elsewhere (e.g. http://clocc.sourceforge.net/).
Some good candidates for future extensions here are:
* bindings to existing foreign libraries (e.g. to a regexp library
like PCRE, or to a compression library like zlib, or to a graphics
library like Tk)
* new libraries (e.g. a CORBA interface, or a port of the CMU CL
POSIX functions, or a new higher-level POSIX functions)
* low-level hooks into SBCL needed to interface it to some wrapper
system (e.g. to interface to a graphical debugger of some sort)
* a too-alpha-to-be-supported-yet tree shaker
SBCL extensions of less general interest, e.g. a binding to the C
interface of the Oracle RDBMS, or particularly large extensions, e.g.
big graphics frameworks, can also be associated with the SBCL project,
but instead of being included in this directory as part of the
distribution, they will be made available on the SBCL project web
site.

252
contrib/scriptoids Normal file
View file

@ -0,0 +1,252 @@
From sbcl-devel-admin@lists.sourceforge.net Sun Jul 16 12:10:07 2000
Received: from localhost (IDENT:newman@localhost.localdomain [127.0.0.1])
by rootless.localdomain (8.9.3/8.9.3) with ESMTP id MAA07245
for <newman@localhost>; Sun, 16 Jul 2000 12:10:05 -0500 (CDT)
Received: from mail.airmail.net
by localhost with POP3 (fetchmail-5.1.1)
for newman@localhost (single-drop); Sun, 16 Jul 2000 12:10:06 -0500 (CDT)
Received: from lists.sourceforge.net from [198.186.203.35] by mail.airmail.net
(/\##/\ Smail3.1.30.16 #30.438) with esmtp for <william.newman@airmail.net> sender: <sbcl-devel-admin@lists.sourceforge.net>
id <mn/13DanY-000GXOn@mail.airmail.net>; Sat, 15 Jul 2000 17:52:40 -0500 (CDT)
Received: from mail1.sourceforge.net (localhost [127.0.0.1])
by lists.sourceforge.net (8.9.3/8.9.3) with ESMTP id PAA03497;
Sat, 15 Jul 2000 15:52:33 -0700
Received: from tninkpad.telent.net (detached.demon.co.uk [194.222.13.128])
by lists.sourceforge.net (8.9.3/8.9.3) with ESMTP id PAA03477
for <sbcl-devel@lists.sourceforge.net>; Sat, 15 Jul 2000 15:52:28 -0700
Received: from dan by tninkpad.telent.net with local (Exim 3.12 #1 (Debian))
id 13Daly-0002eu-00; Sat, 15 Jul 2000 23:51:02 +0100
To: sbcl-devel@lists.sourceforge.net
From: Daniel Barlow <dan@telent.net>
Date: 15 Jul 2000 23:51:02 +0100
Message-ID: <87og3zvwh5.fsf@tninkpad.telent.net>
User-Agent: Gnus/5.0803 (Gnus v5.8.3) Emacs/20.7
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
Subject: [Sbcl-devel] LINK-SYSTEM - "How big is a `hello world' program in SBCL?"
Sender: sbcl-devel-admin@lists.sourceforge.net
Errors-To: sbcl-devel-admin@lists.sourceforge.net
X-Mailman-Version: 1.1
Precedence: bulk
List-Id: <sbcl-devel.lists.sourceforge.net>
X-BeenThere: sbcl-devel@lists.sourceforge.net
X-Airmail-Delivered: Sat, 15 Jul 2000 17:52:40 -0500 (CDT)
X-Airmail-Spooled: Sat, 15 Jul 2000 17:52:40 -0500 (CDT)
Status: RO
Content-Length: 8179
Lines: 80
--=-=-=
1103 bytes. :-)
The problem I wanted to solve here is that of making sbcl programs
that run from the command line and look superficially like normal unix
executables (in, say, the same way as shell scripts or Perl programs
do). The programs in question are expected to run on a system with
sbcl installed (there's a core file, and a runtime, etc) but have to
share the same core file and not each dump their own. Disk may be
cheap but it's not _that_ cheap ...
This is achieved using shell #! magic and concatenation of fasl files.
STANDALONEIZE-FILE, given a collection of x86f files, makes a single
file that can be run from the shell prompt. The file consists of
the concatenation of all the x86f files, appended to #! magic which
invokes sbcl on them.
LINK-SYSTEM operates with mk-defsystem (get it from CLOCC) to build a similar
file from a system definition. It currently breaks if the system has
non-Lisp components (e.g. db-sockets, which loads .so objects)
Here's how you use it:
:; cat hello.lisp
(in-package :cl-user)
(format t "hello world ~%")
(quit)
:; sbcl --noinform --core testcore.core --eval '(progn (compile-file "hello.lisp") (standaloneize:standaloneize-file "hello" "hello.x86f") (quit))'
compiling "/home/dan/src/telent/lisploader/hello.lisp" (written 15 JUL 2000 10:27:45 PM):
byte compiling top-level form:
byte compiling top-level form:
byte compiling top-level form:
hello.x86f written
compilation finished in 0:00:00
:; ls -l hello
-rwxr-xr-x 1 dan dan 1103 Jul 15 22:43 hello
:; time ./hello
hello world
real 0m0.116s
user 0m0.040s
sys 0m0.060s
It also understands search paths ...
:; cp hello ~/bin
:; type hello
hello is /home/dan/bin/hello
:; hello
hello world
So how about that? 1k executables and 1/10th second startup times.
It helps that I already have another instance of sbcl open, of course :-)
The whole thing is only about 5k, so I enclose it here as an
attachment. Build instructions are in the comment at the top. You
have to dump a core file with it compiled in, but the point is that
you only have to do so once per sbcl, not once per application.
I hope this will (eventually, anyway) encourage use of SBCL by people
wanting to solve "scripting" problems. The unix shell may be ugly,
but it's not going away any time soon, so it helps if we play nice
with it.
--=-=-=
Content-Disposition: attachment; filename=heuristic-fasload.lisp
(eval-when (:compile-toplevel :load-toplevel)
(defpackage "STANDALONEIZE"
(:use :sb-alien :sb-c-call :common-lisp)
(:export standaloneize-file)))
(in-package :standaloneize)
;;;; Functions useful for making sbcl do sensible stuff with #!
;;;; (STANDALONEIZE-FILE output-file input-files) gloms the input files
;;;; together and sticks shell magic on top. FIND-AND-LOAD-FASL and its
;;;; supporting functions are called when the file is executed
;;;; How to use it. Compile this file. Load it into a fresh SBCL image.
;;;; Dump a core file. Use that core file.
(defun find-fasl-in-stream (stream)
"Search forwards in STREAM for a line starting with the value of sb-c:*fasl-header-string-start-string*. Leave the stream at the offset of the start of that line, and return the offset"
(let ((fasl-cookie sb-c:*fasl-header-string-start-string*))
(loop for position = (file-position stream)
for text = (read-line stream)
;;do (format t "~A ~A ~A ~%" position text fasl-cookie)
if (and text
(>= (length (the simple-string text))
(length fasl-cookie))
(string= text fasl-cookie :end1 (length fasl-cookie)))
return (progn (file-position stream position) position))))
;;; This courtesy of Pierre Mai in comp.lang.lisp 08 Jan 1999 00:51:44 +0100
;;; Message-ID: <87lnjebq0f.fsf@orion.dent.isdn.cs.tu-berlin.de>
(defun split (string &optional max (ws '(#\Space #\Tab)))
"Split `string' along whitespace as defined by the sequence `ws'.
The whitespace is elided from the result. The whole string will be
split, unless `max' is a non-negative integer, in which case the
string will be split into `max' tokens at most, the last one
containing the whole rest of the given `string', if any."
(flet ((is-ws (char) (find char ws)))
(loop for start = (position-if-not #'is-ws string)
then (position-if-not #'is-ws string :start index)
for index = (and start
(if (and max (= (1+ word-count) max))
nil
(position-if #'is-ws string :start start)))
while start
collect (subseq string start index)
count 1 into word-count
while index)))
(defun find-name-on-path (name)
(let* ((search-string (or (sb-ext:posix-getenv "PATH")
":/bin:/usr/bin"))
(search-list (split search-string nil '(#\:))))
(or
(loop for p in search-list
for directory = (merge-pathnames (make-pathname :directory p))
if (probe-file (merge-pathnames name directory))
return (merge-pathnames name directory))
name)))
(defun find-and-load-fasl (name)
"Attempt to find and load a FASL file from NAME. FASL data in the file may be preceded by any number of lines of arbitrary text. If NAME contains no directory portion, it is searched for on the system path in a manner similar to that of execvp(3)"
(let ((path
(if (pathname-directory name)
name
(find-name-on-path name))))
(with-open-file (i path :direction :input)
(find-fasl-in-stream i)
(sb-impl::fasload i nil nil))))
;;;; and now some functions for more easily creating these scuffed fasl files
(defun copy-stream (from to)
"Copy into TO from FROM until end of file, without translating or otherwise mauling anything"
(let ((buf (make-array 4096 :element-type (stream-element-type from)
:initial-element #\Space)))
(do ((pos (read-sequence buf from) (read-sequence buf from)))
((= 0 pos) nil)
(write-sequence buf to :end pos))))
(defparameter *standalone-magic*
"#!/bin/sh
exec /usr/local/bin/sbcl --core testcore.core --noinform --noprint --eval \"(standaloneize::find-and-load-fasl \\\"$0\\\")\" $*
"
"This text is prepended to the output file created by STANDALONEIZE-FILE")
;;; this syscall seems to have been removed from SBCL.
(def-alien-routine chmod int (path c-string) (mode int))
(defun standaloneize-file (output-filename &rest objects)
"Make a standalone executable(sic) called OUTPUT-FILENAME out of OBJECTS, through the magic of hash bang."
(with-open-file (out output-filename :direction :output)
(write-sequence *standalone-magic* out)
(dolist (obj objects)
(with-open-file (in obj)
(copy-stream in out))))
(chmod (namestring output-filename) #o755))
;;;; Another way of doing it would be to create a "link" operation for
;;;; systems defined with mk-defsystem -
#+mk-defsystem
(defun print-binary-file-operation (component force)
"Spit the binary file associated with COMPONENT to *STANDARD-OUTPUT*"
(with-open-file (i (compile-file-pathname
(make::component-pathname component :binary))
:direction :input)
(copy-stream i *standard-output*))
nil)
#+mk-defsystem
(defun link-system (system output-file)
"Create a single executable file from all the files in SYSTEM"
(make::component-operation 'print-binary 'print-binary-file-operation)
(with-open-file (o output-file :direction :output
:if-exists :rename)
(write-sequence *standalone-magic* o)
(let ((*standard-output* o))
(make::operate-on-system system 'print-binary))))
--=-=-=
-dan
--
http://ww.telent.net/cliki/ - CLiki: CL/Unix free software link farm
--=-=-=--
_______________________________________________
Sbcl-devel mailing list
Sbcl-devel@lists.sourceforge.net
http://lists.sourceforge.net/mailman/listinfo/sbcl-devel

204
doc/FOR-CMUCL-DEVELOPERS Normal file
View file

@ -0,0 +1,204 @@
This document was motivated by a request from Paolo Amoroso for notes
or other documentation on my work on SBCL. It's intended for
developers who are familiar with the guts of CMU CL, as an overview of
the changes made to CMU CL in order to produce SBCL. It was written
for the initial release (sbcl-0.5.0) and has not been updated since
then.
There are two sections in this report:
I. non-fundamental changes
II. fundamental changes
In this context, fundamental changes are changes which were
directly driven by the goal of making the system bootstrap itself.
Section I: non-fundamental changes
Before I describe the fundamental changes I had to make in order to
get the system to bootstrap itself, let me emphasize that there are
many non-fundamental changes as well. I won't try to summarize them
all, but I'll mention some to give some idea. (Some more information
about why I made some of these changes is in the PRINCIPLES file in
the distribution.)
Many, many extensions have been removed.
Packages have all been renamed; in the final system,
the system packages have names which begin with "SB-".
Mostly these correspond closely to CMU CL packages,
e.g. the "C" package of CMU CL has become the "SB-C" package,
and the "EXTENSIONS" package of CMU CL has become the "SB-EXT"
package.
Some other definitions and declarations have been centralized, too.
E.g. the build order is defined in one place, and all the COMMON-LISP
special variables are declared in one place.
I've made various reformatting changes in the comments, and
added a number of comments.
INFO is now implemented as a function instead of a macro,
using keywords as its first and second arguments, and is
no longer in the extensions package, but is considered a
private implementation detail.
The expected Lisp function arguments and command line arguments
for SAVE-LISP (now called SAVE-LISP-AND-DIE) and loading
the core back into a new Lisp have changed completely.
The SB-UNIX package no longer attempts to be a complete user interface
to Unix. Instead, it's considered a private part of the implementation
of SBCL, and tries to implement only what's needed by the current
implementation of SBCL.
Lots of stale conditional code was deleted, e.g. code to support
portability to archaic systems in the LOOP and PCL packages. (The
SB-PCL and SB-LOOP packages no longer aspire to portability.)
Various internal symbols, and even some externally-visible extensions,
have been given less-ambiguous or more-modern names, with more to
follow. (E.g. SAVE-LISP becoming SAVE-LISP-AND-DIE, both to avoid
surprising the user and to reserve the name SAVE-LISP in case we ever
manage to implement a SAVE-LISP which doesn't cause the system to die
afterwards. And GIVE-UP and ABORT-TRANSFORM have been renamed
to GIVE-UP-IR1-TRANSFORM and ABORT-IR1-TRANSFORM. And so on.)
Various internal names "NEW-FOO" have been changed to FOO, generally
after deleting the obsolete old version of FOO. This has happened both
with names at the Lisp level (e.g. "NEW-ASSEM") and at the Unix
filesystem level (e.g. "new-hash.lisp" and "new-assem.lisp").
A cultural change, rather than a technical one: The system no longer
tries to be binary compatible between releases.
Per-file credits for programs should move into a single
centralized CREDITS file Real Soon Now.
A lot of spelling errors have been corrected.:-)
Section II. fundamental changes
There were a number of things which I changed in order to get the
system to boot itself.
The source files have been extensively reordered to fix broken forward
references. In many cases, this required breaking one CMU CL source
file into more than one SBCL source file, and scattering the multiple
SBCL source files into multiple places in the build order. (Some of
the breakups were motivated by reasons which no longer exist, and
could be undone now, e.g. "class.lisp" could probably go back into
"classes.lisp". But I think most of the reasons still apply.)
The assembler and genesis were rewritten for portability, using
vectors for scratch space instead of using SAPs.
We define new readmacro syntax #!+ and #!- which acts like
the standard #+ and #- syntax, except that it switches on the
target feature list instead of the host feature list. We also
introduce temporary new features like :XC-HOST ("in the cross-compilation
host") and :XC ("in the cross-compiler") which will be used
to control some of the behavior below.
A new package SB-XC ("cross-compiler") was introduced to hold
affecting-the-target versions of various things like DEFMACRO,
DEFTYPE, FIND-CLASS, CONSTANTP, CLASS, etc. So e.g. when you're
building the cross-compiler in the cross-compilation host Lisp,
SB-XC:DEFMACRO defines a macro in the target Lisp; SB-XC:CONSTANTP
tells you whether something is known to be constant in the target
Lisp; and SB-XC:CLASS is the class of an object which represents a
class in the target Lisp. In order to make everything work out later
when running the cross-compiler to produce code for the target Lisp,
SB-XC turns into a sort of nickname for the COMMON-LISP package.
Except it's a little more complicated than that..
It doesn't quite work to make SB-XC into a nickname for COMMON-LISP
while building code for the target, because then much of the code in
EVAL-WHEN (:COMPILE-TOPLEVEL :EXECUTE) forms would break. Instead, we
read in code using the ordinary SB-XC package, and then when we
process code in any situation other than :COMPILE-TOPLEVEL, we run it
through the function UNCROSS to translate any SB-XC symbols into the
corresponding CL symbols. (This doesn't seem like a very elegant
solution, but it does seem to work.:-)
Even after we've implemented the UNCROSS hack, a lot of the code inside
EVAL-WHEN forms is still broken, because it does things like CL:DEFMACRO
to define macros which are intended to show up in the target, and
under the new system we really need it to do SB-XC:DEFMACRO instead
in order to achieve the desired effect. So we have to go through
all the EVAL-WHEN forms and convert various CL:FOO operations
to the corresponding SB-XC:FOO operations. Or sometimes instead we
convert code a la
(EVAL-WHEN (COMPILE EVAL)
(DEFMACRO FOO ..))
(code-using-foo)
into code a la
(MACROLET ((FOO ..))
(code-using-foo))
Or sometimes we even give up and write
(DEFMACRO FOO ..)
(code-using-foo)
instead, figuring it's not *that* important to try to save a few bytes
in the target Lisp by keeping FOO from being defined. And in a few
shameful instances we even did things like
#+XC (DEFMACRO FOO ..)
#-XC (DEFMACRO FOO ..
or
#+XC (code-using-foo)
#-XC (other-code-using-foo)
even though we know that we will burn in hell for it. (The really
horribly unmaintainable stuff along those lines is three compiler-building
macros which I hope to fix before anyone else notices them.:-)
In order to avoid trashing the host Common Lisp when cross-compiling
under another instance of ourself (and in order to avoid coming to
depend on its internals in various weird ways, like some systems we
could mention but won't:-) we make the system use different package
names at cold init time than afterwards. The internal packages are
named "SB!FOO" while we're building the system, and "SB-FOO"
afterwards.
In order to make the system work even when we're renaming its packages
out from underneath it, we need to seek out and destroy any nasty
hacks which refer to particular package names, like the one in
%PRIMITIVE which wants to reintern the symbols in its arguments into
the "C"/"SB-C"/"SB!C" package.
Incidentally, because of the #! readmacros and the "SB!FOO" package
names, the system sources are unreadable to the running system. (The
undefined readmacros and package names cause READ-ERRORs.) I'd like
to make a little hack to fix this for use when experimenting with
and maintaining the system, but I haven't gotten around to it,
despite several false starts. Real Soon Now..
In order to keep track of layouts and other type and structure
information set up under the cross-compiler, we use a system built
around the DEF!STRUCT macro. (The #\! character is used to name a lot
of cold-boot-related stuff.) When building the cross-compiler, the
DEF!STRUCT macro is a wrapper around portable DEFSTRUCT which builds
its own portable information about the structures being created, and
arranges for host Lisp instances of the structures to be dumpable as
target Lisp instances as necessary. (This system uses MAKE-LOAD-FORM
heavily and is the reason that I say that bootstrapping under CLISP is
not likely to happen until CLISP supports MAKE-LOAD-FORM.) When
running the cross-compiler, DEF!STRUCT basically reduces to the
DEFSTRUCT macro.
In order to be able to make this system handle target Lisp code,
we need to be able to test whether a host Lisp value matches a
target Lisp type specifier. With the information available from
DEF!STRUCT, and various hackery, we can do that, implementing things
like SB-XC:TYPEP.
Now that we know how to represent target Lisp objects in the
cross-compiler running under vanilla ANSI Common Lisp, we need to make
the dump code portable. This is not too hard given that the cases
which would be hard tend not to be used in the implementation of SBCL
itself, so the cross-compiler doesn't need to be able to handle them
anyway. Specialized arrays are an exception, and currently we dodge
the issue by making the compiler use not-as-specialized-as-possible
array values. Probably this is fixable by bootstrapping in two passes,
one pass under vanilla ANSI Common Lisp and then another under the
SBCL created by the first pass. That way, the problem goes away in the
second pass pass, since we know that all types represented by the
target SBCL can be represented in the cross-compilation host SBCL.

8
doc/README Normal file
View file

@ -0,0 +1,8 @@
SBCL is -- ahem! -- not particularly well documented at this point.
What can I say? Help with documentation might not be refused.:-)
The old CMUCL documentation, in the cmucl/ subdirectory, is still
somewhat useful. The old user's manual is very useful. Most of the
CMUCL extensions to Common Lisp have gone away, but the general
information about how to use the Python compiler is still very
relevant.

232
doc/beyond-ansi.sgml Normal file
View file

@ -0,0 +1,232 @@
<chapter id="beyond-ansi"><title>Beyond the &ANSI; Standard</>
<para>Besides &ANSI;, we have other stuff..</para>
<sect1 id="non-conformance"><title>Non-Conformance with the &ANSI; Standard</>
<para>&SBCL; is derived from code which was written before the &ANSI;
standard, and some incompatibilities remain.</para>
<para>The &ANSI; standard defines constructs like
<function>defstruct</>, <function>defun</>, and <function>declaim</>
so that they can be implemented as macros which expand into ordinary
code wrapped in <function>eval-when</> forms. However, the pre-&ANSI;
&CMUCL; implementation handled these (and some related functions like
<function>proclaim</>) as special cases in the compiler, with subtly
(or sometimes not-so-subtly) different semantics. Much of this
weirdness has been removed in the years since the &ANSI; standard was
released, but bits and pieces remain, so that e.g., as of &SBCL; 0.6.3
compiling the function
<programlisting>(defun foo () (defstruct bar))</>
will cause the class <type>BAR</> to be defined, even when the
function is not executed. These remaining nonconforming behaviors are
considered bugs, and clean patches will be gratefully accepted, but as
long as they don't cause as many problems in practice as other known
issues, they tend not to be actively fixed.</para>
<para>More than any other &Lisp; system I am aware of, &SBCL; (and its
parent &CMUCL;) store and use a lot of compile-time static type
information. By and large they conform to the standard in doing so,
but in one regard they do not &mdash; they consider <function>defun</>s to,
in effect, implicitly <function>proclaim</> type information about the
signature of the function being defined. Thus, if you compile and load
<programlisting>(defun foo-p (x)
(error "stub, foo-p ~s isn't implemented yet!" x))
(defun foolike-p (x)
(or (foo-p x) (foo-p (car x))))</programlisting>
everything will appear to work correctly, but if you subsequently
redefine <function>foo-p</>
<programlisting>(defun foo-p (x) (or (null x) (symbolp (car x))))</>
and call
<programlisting>(foolike-p nil)</>
you will not get the correct result, but an error,
<screen>debugger invoked on SB-DEBUG::*DEBUG-CONDITION* of type
SB-KERNEL:SIMPLE-CONTROL-ERROR:
A function with declared result type NIL returned:
FOO-P</screen>
because when &SBCL; compiled <function>foolike-p</>, &SBCL; thought it
knew that <function>foo-p</> would never return. More insidious
problems are quite possible when &SBCL; thinks it can optimize away e.g.
particular branches of a <function>case</> because of what it's proved
to itself about the function's return type. This will probably be
fixed in the foreseeable future, either with a quick fix, or ideally
in conjunction with some related fixes to generalize the principle
that declarations are assertions (see below). But for now it remains a
gross violation of the &ANSI; spec (and reasonable user
expectations).</para>
<para>The &CMUCL; <function>defstruct</> implementation treated
structure accessors and other <function>defstruct</>-related functions
(e.g. predicates) as having some special properties, not quite like
ordinary functions. This specialness has been reduced in &SBCL;, but
some still remains. In particular, redefining a structure accessor
function may magically cause the entire structure class to be deleted.
This, too, will probably be fixed in the foreseeable future.</para>
<para>The CLOS implementation used in &SBCL; is based on the
<application>Portable Common Loops</> (PCL) reference implementation
from Xerox. Unfortunately, PCL seems never to have quite conformed to
the final CLOS specification. Moreover, despite the "Portable" in its
name, it wasn't quite portable. Various implementation-specific hacks
were made to make it run on &CMUCL;, and then more hacks were added to
make it less inefficient. The result is a system with mostly tolerable
performance which mostly conforms to the standard, but which has a few
remaining weirdnesses which seem to be hard to fix. The most important
remaining weirdness is that the <type>CL:CLASS</> class is not the
same as the <type>SB-PCL:CLASS</> type used internally in PCL; and
there are several other symbols maintained in parallel (e.g.
<type>SB-PCL:FIND-CLASS</> vs. <type>CL:FIND-CLASS</>). So far, any
problems this has caused have had workarounds involving consistently
using the SB-PCL versions or the CL versions of the class hierarchy.
This is admittedly ugly, but it may not be fixed in the foreseeable
future, since the required cleanup looks nontrivial, and we don't have
anyone sufficiently motivated to do it.</para>
</sect1>
<sect1 id="idiosyncrasies"><title>Idiosyncrasies</>
<para>Declarations are generally treated as assertions. This general
principle, and its implications, and the bugs which still keep the
compiler from quite satisfying this principle, are discussed in the
<link linkend="compiler">chapter on the compiler</link>.</para>
<note><para>It's not an idiosyncrasy yet, since we haven't done
it, but someday soon &SBCL; may become a compiler-only implementation.
That is, essentially, <function>eval</> will be defined to create
a lambda expression, call <function>compile</> on the lambda
expression to create a compiled function, and then
<function>funcall</> the resulting function. This would allow
a variety of simplifications in the implementation, while introducing
some other complexities. It remains to be seen when it will be
possible to try this, or whether it will work well when it's tried,
but it looks appealing right now.</para></note>
</sect1>
<sect1 id="extensions"><title>Extensions</>
<para>&SBCL; is derived from &CMUCL;, which implements many extensions to the
&ANSI; standard. &SBCL; doesn't support as many extensions as &CMUCL;, but
it still has quite a few.</para>
<sect2><title>Things Which Might Be in the Next &ANSI; Standard</>
<para>&SBCL; provides extensive support for
calling external C code, described
<link linkend="ffi">in its own chapter</link>.</para>
<para>&SBCL; provides additional garbage collection functionality not
specified by &ANSI;. Weak pointers allow references to objects to be
maintained without keeping them from being GCed. And "finalization"
hooks are available to cause code to be executed when an object is
GCed.</para>
<para>&SBCL; does not currently provide Gray streams, but may do so in
the near future. (It has unmaintained code inherited from &CMUCL; to
do so.) <!-- FIXME: Add citation to Gray streams.-->
</para>
<para>&SBCL; does not currently support multithreading (traditionally
called <wordasword>multiprocessing</> in &Lisp;) but contains unmaintained
code from &CMUCL; to do so. A sufficiently motivated maintainer
could probably make it work.</para>
</sect2>
<sect2><title>Support for Unix</>
<para>The UNIX command line can be read from the variable
<varname>sb-ext:*posix-argv*</>. The UNIX environment can be queried with the
<function>sb-ext:posix-getenv</> function.</para>
<para>The &SBCL; system can be terminated with <function>sb-ext:quit</>,
optionally returning a specified numeric value to the calling Unix
process. The normal Unix idiom of terminating on end of file on input
is also supported.</para>
</sect2>
<sect2><title>Tools to Help Developers</title>
<para>&SBCL; provides a profiler and other extensions to the &ANSI;
<function>trace</> facility. See the online function documentation for
<function>trace</> for more information.</para>
<para>The debugger supports a number of options. Its documentation is
accessed by typing <userinput>help</> at the debugger prompt.</para>
<para>Documentation for <function>inspect</> is accessed by typing
<userinput>help</> at the <function>inspect</> prompt.</para>
</sect2>
<sect2><title>Interface to Low-Level &SBCL; Implementation</title>
<para>&SBCL; has the ability to save its state as a file for later
execution. This functionality is important for its bootstrapping
process, and is also provided as an extension to the user See the
documentation for <function>sb-ext:save-lisp-and-die</> for more
information.</para>
<note><para>&SBCL; has inherited from &CMUCL; various hooks to allow
the user to tweak and monitor the garbage collection process. These
are somewhat stale code, and their interface might need to be cleaned
up. If you have urgent need of them, look at the code in
<filename>src/code/gc.lisp</filename> and bring it up on the
developers' mailing list.</para></note>
<note><para>&SBCL; has various hooks inherited from &CMUCL;, like
<function>sb-ext:float-denormalized-p</>, to allow a program to take
advantage of &IEEE; floating point arithmetic properties which aren't
conveniently or efficiently expressible using the &ANSI; standard. These
look good, and their interface looks good, but &IEEE; support is
slightly broken due to a stupid decision to remove some support for
infinities (because it wasn't in the &ANSI; spec and it didn't occur to
me that it was in the &IEEE; spec). If you need this stuff, take a look
at the ecode and bring it up on the developers' mailing
list.</para></note>
</sect2>
<sect2><title>Efficiency Hacks</title>
<para>The <function>sb-ext:purify</function> function causes &SBCL;
first to collect all garbage, then to mark all uncollected objects as
permanent, never again attempting to collect them as garbage. (This
can cause a large increase in efficiency when using a primitive
garbage collector, but is less important with modern generational
garbage collectors.)</para>
<para>The <function>sb-ext:truly-the</> operator does what the
<function>cl:the</> operator does in a more conventional
implementation of &CommonLisp;, declaring the type of its argument
without any runtime checks. (Ordinarily in &SBCL;, any type declaration
is treated as an assertion and checked at runtime.)</para>
<para>The <function>sb-ext:freeze-type</> declaration declares that a
type will never change, which can make type testing
(<function>typep</>, etc.) more efficient for structure types.</para>
<para>The <function>sb-ext:constant-function</> declaration specifies
that a function will always return the same value for the same
arguments. This is appropriate for functions like <function>sqrt</>.
It is not appropriate for functions like <function>aref</>, which can
change their return values when the underlying data are
changed.</para>
</sect2>
</sect1>
</chapter>

460
doc/cmucl/cmu-user/cmu-user.dict Normal file
View file

@ -0,0 +1,460 @@
'BAR
VARREF
'TEST
UPCASE
ENDLISP
SUBSEQ
ENDDEFUN
FUNARGS
GENSYM
VARS
UNINTERNED
VAR
VSOURCE
CLISP
COND
MYSTUFF
TRADEOFFS
PATHNAME
LLISP
CMUCL
REF
YETMOREKEYS
CLEANUP
ARGS
DEFUN
ZOQ
FOO
'S
CLTL
MACROEXPANDS
MACROEXPANSION
PROXY
ERRORFUL
EQ
ECASE
PYTHON
DEFMACRO
PROMISCUOUS
FLAMAGE
DEBUGGABILITY
FEATUREFULNESS
DEBUGGABLE
ENDDEFVAR
MACROEXPANDED
DEFVAR
ENDDEFMAC
KWD
MGROUP
MSTAR
DEFMAC
OFFS
NOTINLINE
TRADEOFF
FUNCALL
SOMEVAL
SOMEFUN
CM
DEFTYPE
CONSING
FIXNUMS
BIGNUMS
FROB
'FOO
RECOMPILES
FTYPE
TYPECASE
TYPEP
UNTYPED
UNIONED
GLOBALS
MODICUM
MACREF
SLEAZING
ES
STEELE
ETYPECASE
'EQL
'IDENTITY
'FUN
LOCALFUN
ISQRT
ODDP
MYFUN
POS
ZOW
YOW
'YOW
CADR
ZEROP
RES
EXPT
PARED
PUSHING
'ING
RPLACD
IOTA
NTHCDR
NTH
CADDDR
RPLACA
CADDR
FIENDS
SQRT
'SQRT
LISPY
BLANKSPACE
MYCHAPTER
UNENCAPSULATED
ENCAPSULATIONS
UNENCAPSULATE
UNTRACED
UNTRACE
EVALED
SPEC
PUSHES
TRUENAME
MYMAC
UNINFORMATIVE
FOOBAR
BAZ
BACKQUOTE
MALFORMED
MOREKEYS
FUNREF
QUIRKS
UNDILUTED
DISASSEMBLY
NAN
DENORMALIZED
ENDDEFCONST
DEFCONST
HASHTABLES
EFF
OBFUSCATING
SNOC
GRUE
GORP
FLO
NUM
VEC
MULTBY
SOMEOTHERFUN
'CHAR
NOTP
TESTP
FUNVAR
RAZ
ZUG
XFF
IO
GC'ING
EXT
MEGABYTE
SYS
UX
ED
MATCHMAKER
DIRED
PCL
CLOS
CONFORMANCE
ENDDEFCON
DEFCON
DECLAIM
DEFSTRUCT
ENUM
EXTERN
LOWERCASING
DEREFERENCED
MOPT
STRUCT
DEFTP
ENDDEFTP
MALLOC
CSH
PXLREF
ATYPE
CONSTRUCTUED
ANAME
PXREF
ENV
ONECOLUMN
TP
VR
FN
PRINTINDEX
UNNUMBERED
TWOCOLUMN
TLF
UNCOMPILED
DEACTIVATE
CALLABLE
UNREFERENCED
SUPPLIEDP
INTERNING
UNHANDLED
BACKTRACING
TEX
OOB
OBJ
PRIN
OBJS
GP
LINKERS
CC
AR
CFUN
INTS
SIZEOF
PRINTF
CFOO
SUBFORM
SVREF
STASH
FOOS
LC
LD
'N
'X
ERRNO
UPPERCASING
EXPR
ADDR
'STR
STR
DEREF
PTR
SWINDOW
IWINDOW
'SLIDER
DRAWABLE
'KEY
'EXT
TIMEOUTS
'MY
ID
PIXMAPS
'EQ
FUNCALLED
XWINDOW
'IH
SIGSTOP
GETPID
SIGTSTP
SCP
SIGINT
IH
CNT
GENERALRETURN
DEFMACX
'NUKEGARBAGE
GR
HASSLE
PREPENDS
TIMEOUT
FD
MSG
SYSCALL
UNHELPFUL
PREPENDED
VM
PAGEREF
INT
PORTSID
PORTSNAME
SERVPORT
KERN
DATATYPES
TTY
STDERR
STDOUT
STDIN
CMD
AUX
PS
UNACCOUNTED
RUNTIMES
PROFILER
UNPROFILE
REPROFILED
UNPROFILED
CF
ELT
VOPS
MAPCAR
OPTIONALS
CONSES
CONTORTIONS
ALISTS
ALIST
ASSOC
EXP
MYEXP
DEFCONSTANT
INCF
MEMQ
COERCIONS
EQL
LOGAND
AREF
CONSP
TYPEN
LOGIOR
EQUIV
SUPERTYPE
DEFMETHOD
SUBFORMS
CERROR
PSETQ
TAGBODY
DOTIMES
PLOQ
ROQ
SPECS
MPLUS
STEPPER
FDEFINITION
FUNCALLABLE
ST
BR
DB
LB
LL
HFILL
PP
VPRINT
TH
ARGLISTS
SETQ
NAMESPACE
SUBFUNCTION
BACKTRACE
'B
FLET
ARG
'A
CPSUBINDEX
PROGN
CONTRIB
WEEKDAYS
GREENWICH
TIMEZONE
DEST
WEEKDAY
JAN
CINDEX
NAMESTRING
PATHNAMES
FASL
SIGSEGV
PLIST
'ABLE
SETF
PID
EXECVE
DEV
SUBPROCESS
PTY
'TH
UNSUPPLIED
DEFVARX
GCS
CONSED
GC'ED
GC
TRASHING
XLIB
CL
HI
COMMONLOOPS
CTRL
XLREF
DEFUNX
DEFCONSTX
SUBSUBSECTION
VINDEXED
TINDEXED
RESEARCHCREDIT
EM
WHOLEY
SKEF
KAUFMANN
TODD
KOLOJEJCHICK
BUSDIECKER
''
NOINDENT
MOORE
TIM
LOTT
LEINEN
HALLGREN
GLEICHAUF
DUNNING
TED
BADER
MYLISP
NOINIT
FINDEXED
INIT
EVAL
SUBDIRECTORIES
COPYRIGHTED
FTP
LANG
COMP
MEG
MEGABYTES
UNCOMPRESS
CD
OS
USERNAME
SLISP
RT
LIB
SETENV
SAMP
SETPATH
LOGIN
MISC
USR
MODMISC
TXT
DOC
EXECUTABLES
PERQ
UNTAGGED
BENCHMARKING
WINDOWING
INTRO
DOCS
EDU
AFS
VSPACE
IFINFO
DIR
SETFILENAME
TABLEOFCONTENTS
PAGENUMBERING
CLEARPAGE
MAKETITLE
ARPASUPPORT
CITATIONINFO
TRNUMBER
IFTEX
SUNOS
SPARC
DECSTATIONS
THEABSTRACT
DEF
KY
CP
NEWINDEX
ALWAYSREFILL
PAGESTYLE
CMULISP
TITLEPAGE
ELISP
LATEXINFO
DOCUMENTSTYLE

13321
doc/cmucl/cmu-user/cmu-user.tex Normal file
View file

File diff suppressed because it is too large Load diff

2
doc/cmucl/internals/SBCL-README Normal file
View file

@ -0,0 +1,2 @@
things from here which are invaluable for understanding current SBCL:
object.tex

16
doc/cmucl/internals/addenda Normal file
View file

@ -0,0 +1,16 @@
the function calling convention
%ECX is used for a count of function argument words, represented as a
fixnum, so it can also be thought of as a count of function argument
bytes.
The first three arguments are stored in registers. The remaining
arguments are stored on the stack.
The comments at the head of DEFINE-VOP (MORE-ARG) explain that
;;; More args are stored contiguously on the stack, starting immediately at the
;;; context pointer. The context pointer is not typed, so the lowtag is 0.
?? Once we switch into more-arg arrangement, %ecx no longer seems to be
used for argument count (judging from my walkthrough of kw arg parsing
code while troubleshooting cold boot problems)

308
doc/cmucl/internals/architecture.tex Normal file
View file

@ -0,0 +1,308 @@
\part{System Architecture}% -*- Dictionary: int:design -*-
\chapter{Package and File Structure}
\section{RCS and build areas}
The CMU CL sources are maintained using RCS in a hierarchical directory
structure which supports:
\begin{itemize}
\item shared RCS config file across a build area,
\item frozen sources for multiple releases, and
\item separate system build areas for different architectures.
\end{itemize}
Since this organization maintains multiple copies of the source, it is somewhat
space intensive. But it is easy to delete and later restore a copy of the
source using RCS snapshots.
There are three major subtrees of the root \verb|/afs/cs/project/clisp|:
\begin{description}
\item[rcs] holds the RCS source (suffix \verb|,v|) files.
\item[src] holds ``checked out'' (but not locked) versions of the source files,
and is subdivided by release. Each release directory in the source tree has a
symbolic link named ``{\tt RCS}'' which points to the RCS subdirectory of the
corresponding directory in the ``{\tt rcs} tree. At top-level in a source tree
is the ``{\tt RCSconfig}'' file for that area. All subdirectories also have a
symbolic link to this RCSconfig file, allowing the configuration for an area to
be easily changed.
\item[build] compiled object files are placed in this tree, which is subdivided
by machine type and version. The CMU CL search-list mechanism is used to allow
the source files to be located in a different tree than the object files. C
programs are compiled by using the \verb|tools/dupsrcs| command to make
symbolic links to the corresponding source tree.
\end{description}
On order to modify an file in RCS, it must be checked out with a lock to
produce a writable working file. Each programmer checks out files into a
personal ``play area'' subtree of \verb|clisp/hackers|. These tree duplicate
the structure of source trees, but are normally empty except for files actively
being worked on.
See \verb|/afs/cs/project/clisp/pmax_mach/alpha/tools/| for
various tools we use for RCS hacking:
\begin{description}
\item[rcs.lisp] Hemlock (editor) commands for RCS file manipulation
\item[rcsupdate.c] Program to check out all files in a tree that have been
modified since last checkout.
\item[updates] Shell script to produce a single listing of all RCS log
entries in a tree since a date.
\item[snapshot-update.lisp] Lisp program to generate a shell script which
generates a listing of updates since a particular RCS snapshot ({\tt RCSSNAP})
file was created.
\end{description}
You can easily operate on all RCS files in a subtree using:
\begin{verbatim}
find . -follow -name '*,v' -exec <some command> {} \;
\end{verbatim}
\subsection{Configuration Management}
config files are useful, especially in combinarion with ``{\tt snapshot}''. You
can shapshot any particular version, giving an RCSconfig that designates that
configuration. You can also use config files to specify the system as of a
particular date. For example:
\begin{verbatim}
<3-jan-91
\end{verbatim}
in the the config file will cause the version as of that 3-jan-91 to be checked
out, instead of the latest version.
\subsection{RCS Branches}
Branches and named revisions are used together to allow multiple paths of
development to be supported. Each separate development has a branch, and each
branch has a name. This project uses branches in two somewhat different cases
of divergent development:
\begin{itemize}
\item For systems that we have imported from the outside, we generally assign a
``{\tt cmu}'' branch for our local modifications. When a new release comes
along, we check it in on the trunk, and then merge our branch back in.
\item For the early development and debugging of major system changes, where
the development and debugging is expected to take long enough that we wouldn't
want the trunk to be in an inconsistent state for that long.
\end{itemize}
\section{Releases}
We name releases according to the normal alpha, beta, default convention.
Alpha releases are frequent, intended primarily for internal use, and are thus
not subject to as high high documentation and configuration management
standards. Alpha releases are designated by the date on which the system was
built; the alpha releases for different systems may not be in exact
correspondence, since they are built at different times.
Beta and default releases are always based on a snapshot, ensuring that all
systems are based on the same sources. A release name is an integer and a
letter, like ``15d''. The integer is the name of the source tree which the
system was built from, and the letter represents the release from that tree:
``a'' is the first release, etc. Generally the numeric part increases when
there are major system changes, whereas changes in the letter represent
bug-fixes and minor enhancements.
\section{Source Tree Structure}
A source tree (and the master ``{\tt rcs}'' tree) has subdirectories for each
major subsystem:
\begin{description}
\item[{\tt assembly/}] Holds the CMU CL source-file assembler, and has machine
specific subdirectories holding assembly code for that architecture.
\item[{\tt clx/}] The CLX interface to the X11 window system.
\item[{\tt code/}] The Lisp code for the runtime system and standard CL
utilities.
\item[{\tt compiler/}] The Python compiler. Has architecture-specific
subdirectories which hold backends for different machines. The {\tt generic}
subdirectory holds code that is shared across most backends.
\item[{\tt hemlock/}] The Hemlock editor.
\item[{\tt lisp/}] The C runtime system code and low-level Lisp debugger.
\item[{\tt pcl/}] CMU version of the PCL implementation of CLOS.
\item[{\tt tools/}] System building command files and source management tools.
\end{description}
\section{Package structure}
Goals: with the single exception of LISP, we want to be able to export from the
package that the code lives in.
\begin{description}
\item[Mach, CLX...] --- These Implementation-dependent system-interface
packages provide direct access to specific features available in the operating
system environment, but hide details of how OS communication is done.
\item[system] contains code that must know about the operating system
environment: I/O, etc. Hides the operating system environment. Provides OS
interface extensions such as {\tt print-directory}, etc.
\item[kernel] hides state and types used for system integration: package
system, error system, streams (?), reader, printer. Also, hides the VM, in
that we don't export anything that reveals the VM interface. Contains code
that needs to use the VM and SYSTEM interface, but is independent of OS and VM
details. This code shouldn't need to be changed in any port of CMU CL, but
won't work when plopped into an arbitrary CL. Uses SYSTEM, VM, EXTENSIONS. We
export "hidden" symbols related to implementation of CL: setf-inverses,
possibly some global variables.
The boundary between KERNEL and VM is fuzzy, but this fuzziness reflects the
fuzziness in the definition of the VM. We can make the VM large, and bring
everything inside, or we make make it small. Obviously, we want the VM to be
as small as possible, subject to efficiency constraints. Pretty much all of
the code in KERNEL could be put in VM. The issue is more what VM hides from
KERNEL: VM knows about everything.
\item[lisp] Originally, this package had all the system code in it. The
current ideal is that this package should have {\it no} code in it, and only
exist to export the standard interface. Note that the name has been changed by
x3j13 to common-lisp.
\item[extensions] contains code that any random user could have written: list
operations, syntactic sugar macros. Uses only LISP, so code in EXTENSIONS is
pure CL. Exports everything defined within that is useful elsewhere. This
package doesn't hide much, so it is relatively safe for users to use
EXTENSIONS, since they aren't getting anything they couldn't have written
themselves. Contrast this to KERNEL, which exports additional operations on
CL's primitive data structures: PACKAGE-INTERNAL-SYMBOL-COUNT, etc. Although
some of the functionality exported from KERNEL could have been defined in CL,
the kernel implementation is much more efficient because it knows about
implementation internals. Currently this package contains only extensions to
CL, but in the ideal scheme of things, it should contain the implementations of
all CL functions that are in KERNEL (the library.)
\item[VM] hides information about the hardware and data structure
representations. Contains all code that knows about this sort of thing: parts
of the compiler, GC, etc. The bulk of the code is the compiler back-end.
Exports useful things that are meaningful across all implementations, such as
operations for examining compiled functions, system constants. Uses COMPILER
and whatever else it wants. Actually, there are different {\it machine}{\tt
-VM} packages for each target implementation. VM is a nickname for whatever
implementation we are currently targeting for.
\item[compiler] hides the algorithms used to map Lisp semantics onto the
operations supplied by the VM. Exports the mechanisms used for defining the
VM. All the VM-independent code in the compiler, partially hiding the compiler
intermediate representations. Uses KERNEL.
\item[eval] holds code that does direct execution of the compiler's ICR. Uses
KERNEL, COMPILER. Exports debugger interface to interpreted code.
\item[debug-internals] presents a reasonable, unified interface to
manipulation of the state of both compiled and interpreted code. (could be in
KERNEL) Uses VM, INTERPRETER, EVAL, KERNEL.
\item[debug] holds the standard debugger, and exports the debugger
\end{description}
\chapter{System Building}
It's actually rather easy to build a CMU CL core with exactly what you want in
it. But to do this you need two things: the source and a working CMU CL.
Basically, you use the working copy of CMU CL to compile the sources,
then run a process call ``genesis'' which builds a ``kernel'' core.
You then load whatever you want into this kernel core, and save it.
In the \verb|tools/| directory in the sources there are several files that
compile everything, and build cores, etc. The first step is to compile the C
startup code.
{\bf Note:} {\it the various scripts mentioned below have hard-wired paths in
them set up for our directory layout here at CMU. Anyone anywhere else will
have to edit them before they will work.}
\section{Compiling the C Startup Code}
There is a circular dependancy between lisp/internals.h and lisp/lisp.map that
causes bootstrapping problems. To the easiest way to get around this problem
is to make a fake lisp.nm file that has nothing in it by a version number:
\begin{verbatim}
% echo "Map file for lisp version 0" > lisp.nm
\end{verbatim}
and then run genesis with NIL for the list of files:
\begin{verbatim}
* (load ".../compiler/generic/new-genesis") ; compile before loading
* (lisp::genesis nil ".../lisp/lisp.nm" "/dev/null"
".../lisp/lisp.map" ".../lisp/lisp.h")
\end{verbatim}
It will generate
a whole bunch of warnings about things being undefined, but ignore
that, because it will also generate a correct lisp.h. You can then
compile lisp producing a correct lisp.map:
\begin{verbatim}
% make
\end{verbatim}
and the use \verb|tools/do-worldbuild| and \verb|tools/mk-lisp| to build
\verb|kernel.core| and \verb|lisp.core| (see section \ref[building-cores].)
\section{Compiling the Lisp Code}
The \verb|tools| directory contains various lisp and C-shell utilities for
building CMU CL:
\begin{description}
\item[compile-all*] Will compile lisp files and build a kernel core. It has
numerous command-line options to control what to compile and how. Try -help to
see a description. It runs a separate Lisp process to compile each
subsystem. Error output is generated in files with ``{\tt .log}'' extension in
the root of the build area.
\item[setup.lisp] Some lisp utilities used for compiling changed files in batch
mode and collecting the error output Sort of a crude defsystem. Loads into the
``user'' package. See {\tt with-compiler-log-file} and {\tt comf}.
\item[{\it foo}com.lisp] Each system has a ``\verb|.lisp|'' file in
\verb|tools/| which compiles that system.
\end{description}
\section{Building Core Images}
\label{building-cores}
Both the kernel and final core build are normally done using shell script
drivers:
\begin{description}
\item[do-worldbuild*] Builds a kernel core for the current machine. The
version to build is indicated by an optional argument, which defaults to
``alpha''. The \verb|kernel.core| file is written either in the \verb|lisp/|
directory in the build area, or in \verb|/usr/tmp/|. The directory which
already contains \verb|kernel.core| is chosen. You can create a dummy version
with e.g. ``touch'' to select the initial build location.
\item[mk-lisp*] Builds a full core, with conditional loading of subsystems.
The version is the first argument, which defaults to ``alpha''. Any additional
arguments are added to the \verb|*features*| list, which controls system
loading (among other things.) The \verb|lisp.core| file is written in the
current working directory.
\end{description}
These scripts load Lisp command files. When \verb|tools/worldbuild.lisp| is
loaded, it calls genesis with the correct arguments to build a kernel core.
Similarly, \verb|worldload.lisp|
builds a full core. Adding certain symbols to \verb|*features*| before
loading worldload.lisp suppresses loading of different parts of the
system. These symbols are:
\begin{description}
\item[:no-compiler] don't load the compiler.
\item[:no-clx] don't load CLX.
\item[:no-hemlock] don't load hemlock.
\item[:no-pcl] don't load PCL.
\item[:runtime] build a runtime code, implies all of the above, and then some.
\end{description}
Note: if you don't load the compiler, you can't (successfully) load the
pretty-printer or pcl. And if you compiled hemlock with CLX loaded, you can't
load it without CLX also being loaded.

725
doc/cmucl/internals/back.tex Normal file
View file

@ -0,0 +1,725 @@
% -*- Dictionary: design -*-
\chapter{Copy propagation}
File: {\tt copyprop}
This phase is optional, but should be done whenever speed or space is more
important than compile speed. We use global flow analysis to find the reaching
definitions for each TN. This information is used here to eliminate
unnecessary TNs, and is also used later on by loop invariant optimization.
In some cases, VMR conversion will unnecessarily copy the value of a TN into
another TN, since it may not be able to tell that the initial TN has the same
value at the time the second TN is referenced. This can happen when ICR
optimize is unable to eliminate a trivial variable binding, or when the user
does a setq, or may also result from creation of expression evaluation
temporaries during VMR conversion. Whatever the cause, we would like to avoid
the unnecessary creation and assignment of these TNs.
What we do is replace TN references whose only reaching definition is a Move
VOP with a reference to the TN moved from, and then delete the Move VOP if the
copy TN has no remaining references. There are several restrictions on copy
propagation:
\begin{itemize}
\item The TNs must be ``ordinary'' TNs, not restricted or otherwise
unusual. Extending the life of restricted (or wired) TNs can make register
allocation impossible. Some other TN kinds have hidden references.
\item We don't want to defeat source-level debugging by replacing named
variables with anonymous temporaries.
\item We can't delete moves that representation selected might want to change
into a representation conversion, since we need the primitive types of both TNs
to select a conversion.
\end{itemize}
Some cleverness reduces the cost of flow analysis. As for lifetime analysis,
we only need to do flow analysis on global packed TNs. We can't do the real
local TN assignment pass before this, since we allocate TNs afterward, so we do
a pre-pass that marks the TNs that are local for our purposes. We don't care
if block splitting eventually causes some of them to be considered global.
Note also that we are really only are interested in knowing if there is a
unique reaching definition, which we can mash into our flow analysis rules by
doing an intersection. Then a definition only appears in the set when it is
unique. We then propagate only definitions of TNs with only one write, which
allows the TN to stand for the definition.
\chapter{Representation selection}
File: {\tt represent}
Some types of object (such as {\tt single-float}) have multiple possible
representations. Multiple representations are useful mainly when there is a
particularly efficient non-descriptor representation. In this case, there is
the normal descriptor representation, and an alternate non-descriptor
representation.
This possibility brings up two major issues:
\begin{itemize}
\item The compiler must decide which representation will be most efficient for
any given value, and
\item Representation conversion code must be inserted where the representation
of a value is changed.
\end{itemize}
First, the representations for TNs are selected by examining all the TN
references and attempting to minimize reference costs. Then representation
conversion code is introduced.
This phase is in effect a pre-pass to register allocation. The main reason for
its existence is that representation conversions may be farily complex (e.g.
involving memory allocation), and thus must be discovered before register
allocation.
VMR conversion leaves stubs for representation specific move operations.
Representation selection recognizes {\tt move} by name. Argument and return
value passing for call VOPs is controlled by the {\tt :move-arguments} option
to {\tt define-vop}.
Representation selection is also responsible for determining what functions use
the number stack. If any representation is chosen which could involve packing
into the {\tt non-descriptor-stack} SB, then we allocate the NFP register
throughout the component. As an optimization, permit the decision of whether a
number stack frame needs to be allocated to be made on a per-function basis.
If a function doesn't use the number stack, and isn't in the same tail-set as
any function that uses the number stack, then it doesn't need a number stack
frame, even if other functions in the component do.
\chapter{Lifetime analysis}
File: {\tt life}
This phase is a preliminary to Pack. It involves three passes:
-- A pre-pass that computes the DEF and USE sets for live TN analysis, while
also assigning local TN numbers, splitting blocks if necessary. \#\#\# But
not really...
-- A flow analysis pass that does backward flow analysis on the
component to find the live TNs at each block boundary.
-- A post-pass that finds the conflict set for each TN.
\#|
Exploit the fact that a single VOP can only exhaust LTN numbers when there are
large more operands. Since more operand reference cannot be interleaved with
temporary reference, the references all effectively occur at the same time.
This means that we can assign all the more args and all the more results the
same LTN number and the same lifetime info.
|\#
\section{Flow analysis}
It seems we could use the global-conflicts structures during compute the
inter-block lifetime information. The pre-pass creates all the
global-conflicts for blocks that global TNs are referenced in. The flow
analysis pass just adds always-live global-conflicts for the other blocks the
TNs are live in. In addition to possibly being more efficient than SSets, this
would directly result in the desired global-conflicts information, rather that
having to create it from another representation.
The DFO sorted per-TN global-conflicts thread suggests some kind of algorithm
based on the manipulation of the sets of blocks each TN is live in (which is
what we really want), rather than the set of TNs live in each block.
If we sorted the per-TN global-conflicts in reverse DFO (which is just as good
for determining conflicts between TNs), then it seems we could scan though the
conflicts simultaneously with our flow-analysis scan through the blocks.
The flow analysis step is the following:
If a TN is always-live or read-before-written in a successor block, then we
make it always-live in the current block unless there are already
global-conflicts recorded for that TN in this block.
The iteration terminates when we don't add any new global-conflicts during a
pass.
We may also want to promote TNs only read within a block to always-live when
the TN is live in a successor. This should be easy enough as long as the
global-conflicts structure contains this kind of info.
The critical operation here is determining whether a given global TN has global
conflicts in a given block. Note that since we scan the blocks in DFO, and the
global-conflicts are sorted in DFO, if we give each global TN a pointer to the
global-conflicts for the last block we checked the TN was in, then we can
guarantee that the global-conflicts we are looking for are always at or after
that pointer. If we need to insert a new structure, then the pointer will help
us rapidly find the place to do the insertion.]
\section{Conflict detection}
[\#\#\# Environment, :more TNs.]
This phase makes use of the results of lifetime analysis to find the set of TNs
that have lifetimes overlapping with those of each TN. We also annotate call
VOPs with information about the live TNs so that code generation knows which
registers need to be saved.
The basic action is a backward scan of each block, looking at each TN-Ref and
maintaining a set of the currently live TNs. When we see a read, we check if
the TN is in the live set. If not, we:
-- Add the TN to the conflict set for every currently live TN,
-- Union the set of currently live TNs with the conflict set for the TN, and
-- Add the TN to the set of live TNs.
When we see a write for a live TN, we just remove it from the live set. If we
see a write to a dead TN, then we update the conflicts sets as for a read, but
don't add the TN to the live set. We have to do this so that the bogus write
doesn't clobber anything.
[We don't consider always-live TNs at all in this process, since the conflict
of always-live TNs with other TNs in the block is implicit in the
global-conflicts structures.
Before we do the scan on a block, we go through the global-conflicts structures
of TNs that change liveness in the block, assigning the recorded LTN number to
the TN's LTN number for the duration of processing of that block.]
Efficiently computing and representing this information calls for some
cleverness. It would be prohibitively expensive to represent the full conflict
set for every TN with sparse sets, as is done at the block-level. Although it
wouldn't cause non-linear behavior, it would require a complex linked structure
containing tens of elements to be created for every TN. Fortunately we can
improve on this if we take into account the fact that most TNs are "local" TNs:
TNs which have all their uses in one block.
First, many global TNs will be either live or dead for the entire duration of a
given block. We can represent the conflict between global TNs live throughout
the block and TNs local to the block by storing the set of always-live global
TNs in the block. This reduces the number of global TNs that must be
represented in the conflicts for local TNs.
Second, we can represent conflicts within a block using bit-vectors. Each TN
that changes liveness within a block is assigned a local TN number. Local
conflicts are represented using a fixed-size bit-vector of 64 elements or so
which has a 1 for the local TN number of every TN live at that time. The block
has a simple-vector which maps from local TN numbers to TNs. Fixed-size
vectors reduce the hassle of doing allocations and allow operations to be
open-coded in a maximally tense fashion.
We can represent the conflicts for a local TN by a single bit-vector indexed by
the local TN numbers for that block, but in the global TN case, we need to be
able to represent conflicts with arbitrary TNs. We could use a list-like
sparse set representation, but then we would have to either special-case global
TNs by using the sparse representation within the block, or convert the local
conflicts bit-vector to the sparse representation at the block end. Instead,
we give each global TN a list of the local conflicts bit-vectors for each block
that the TN is live in. If the TN is always-live in a block, then we record
that fact instead. This gives us a major reduction in the amount of work we
have to do in lifetime analysis at the cost of some increase in the time to
iterate over the set during Pack.
Since we build the lists of local conflict vectors a block at a time, the
blocks in the lists for each TN will be sorted by the block number. The
structure also contains the local TN number for the TN in that block. These
features allow pack to efficiently determine whether two arbitrary TNs
conflict. You just scan the lists in order, skipping blocks that are in only
one list by using the block numbers. When we find a block that both TNs are
live in, we just check the local TN number of one TN in the local conflicts
vector of the other.
In order to do these optimizations, we must do a pre-pass that finds the
always-live TNs and breaks blocks up into small enough pieces so that we don't
run out of local TN numbers. If we can make a block arbitrarily small, then we
can guarantee that an arbitrarily small number of TNs change liveness within
the block. We must be prepared to make the arguments to unbounded arg count
VOPs (such as function call) always-live even when they really aren't. This is
enabled by a panic mode in the block splitter: if we discover that the block
only contains one VOP and there are still too many TNs that aren't always-live,
then we promote the arguments (which we'd better be able to do...).
This is done during the pre-scan in lifetime analysis. We can do this because
all TNs that change liveness within a block can be found by examining that
block: the flow analysis only adds always-live TNs.
When we are doing the conflict detection pass, we set the LTN number of global
TNs. We can easily detect global TNs that have not been locally mapped because
this slot is initially null for global TNs and we null it out after processing
each block. We assign all Always-Live TNs to the same local number so that we
don't need to treat references to them specially when making the scan.
We also annotate call VOPs that do register saving with the TNs that are live
during the call, and thus would need to be saved if they are packed in
registers.
We adjust the costs for TNs that need to be saved so that TNs costing more to
save and restore than to reference get packed on the stack. We would also like
more often saved TNs to get higher costs so that they are packed in more
savable locations.
\chapter{Packing}
File: {\tt pack}
\#|
Add lifetime/pack support for pre-packed save TNs.
Fix GTN/VMR conversion to use pre-packed save TNs for old-cont and return-PC.
(Will prevent preference from passing location to save location from ever being
honored?)
We will need to make packing of passing locations smarter before we will be
able to target the passing location on the stack in a tail call (when that is
where the callee wants it.) Currently, we will almost always pack the passing
location in a register without considering whether that is really a good idea.
Maybe we should consider schemes that explicitly understand the parallel
assignment semantics, and try to do the assignment with a minimum number of
temporaries. We only need assignment temps for TNs that appear both as an
actual argument value and as a formal parameter of the called function. This
only happens in self-recursive functions.
Could be a problem with lifetime analysis, though. The write by a move-arg VOP
would look like a write in the current env, when it really isn't. If this is a
problem, then we might want to make the result TN be an info arg rather than a
real operand. But this would only be a problem in recursive calls, anyway.
[This would prevent targeting, but targeting across passing locations rarely
seems to work anyway.] [\#\#\# But the :ENVIRONMENT TN mechanism would get
confused. Maybe put env explicitly in TN, and have it only always-live in that
env, and normal in other envs (or blocks it is written in.) This would allow
targeting into environment TNs.
I guess we would also want the env/PC save TNs normal in the return block so
that we can target them. We could do this by considering env TNs normal in
read blocks with no successors.
ENV TNs would be treated totally normally in non-env blocks, so we don't have
to worry about lifetime analysis getting confused by variable initializations.
Do some kind of TN costing to determine when it is more trouble than it is
worth to allocate TNs in registers.
Change pack ordering to be less pessimal. Pack TNs as they are seen in the LTN
map in DFO, which at least in non-block compilations has an effect something
like packing main trace TNs first, since control analysis tries to put the good
code first. This could also reduce spilling, since it makes it less likely we
will clog all registers with global TNs.
If we pack a TN with a specified save location on the stack, pack in the
specified location.
Allow old-cont and return-pc to be kept in registers by adding a new "keep
around" kind of TN. These are kind of like environment live, but are only
always-live in blocks that they weren't referenced in. Lifetime analysis does
a post-pass adding always-live conflicts for each "keep around" TN to those
blocks with no conflict for that TN. The distinction between always-live and
keep-around allows us to successfully target old-cont and return-pc to passing
locations. MAKE-KEEP-AROUND-TN (ptype), PRE-PACK-SAVE-TN (tn scn offset).
Environment needs a KEEP-AROUND-TNS slot so that conflict analysis can find
them (no special casing is needed after then, they can be made with :NORMAL
kind). VMR-component needs PRE-PACKED-SAVE-TNS so that conflict analysis or
somebody can copy conflict info from the saved TN.
Note that having block granularity in the conflict information doesn't mean
that a localized packing scheme would have to do all moves at block boundaries
(which would clash with the desire the have saving done as part of this
mechanism.) All that it means is that if we want to do a move within the
block, we would need to allocate both locations throughout that block (or
something).
Load TN pack:
A location is out for load TN packing if:
The location has TN live in it after the VOP for a result, or before the VOP
for an argument, or
The location is used earlier in the TN-ref list (after) the saved results ref
or later in the TN-Ref list (before) the loaded argument's ref.
To pack load TNs, we advance the live-tns to the interesting VOP, then
repeatedly scan the vop-refs to find vop-local conflicts for each needed load
TN. We insert move VOPs and change over the TN-Ref-TNs as we go so the TN-Refs
will reflect conflicts with already packed load-TNs.
If we fail to pack a load-TN in the desired SC, then we scan the Live-TNs for
the SB, looking for a TN that can be packed in an unbounded SB. This TN must
then be repacked in the unbounded SB. It is important the load-TNs are never
packed in unbounded SBs, since that would invalidate the conflicts info,
preventing us from repacking TNs in unbounded SBs. We can't repack in a finite
SB, since there might have been load TNs packed in that SB which aren't
represented in the original conflict structures.
Is it permissible to "restrict" an operand to an unbounded SC? Not impossible
to satisfy as long as a finite SC is also allowed. But in practice, no
restriction would probably be as good.
We assume all locations can be used when an sc is based on an unbounded sb.
]
TN-Refs are be convenient structures to build the target graph out of. If we
allocated space in every TN-Ref, then there would certainly be enough to
represent arbitrary target graphs. Would it be enough to allocate a single
Target slot? If there is a target path though a given VOP, then the Target of
the write ref would be the read, and vice-versa. To find all the TNs that
target us, we look at the TN for the target of all our write refs.
We separately chain together the read refs and the write refs for a TN,
allowing easy determination of things such as whether a TN has only a single
definition or has no reads. It would also allow easier traversal of the target
graph.
Represent per-location conflicts as vectors indexed by block number of
per-block conflict info. To test whether a TN conflicts on a location, we
would then have to iterate over the TNs global-conflicts, using the block
number and LTN number to check for a conflict in that block. But since most
TNs are local, this test actually isn't much more expensive than indexing into
a bit-vector by GTN numbers.
The big win of this scheme is that it is much cheaper to add conflicts into the
conflict set for a location, since we never need to actually compute the
conflict set in a list-like representation (which requires iterating over the
LTN conflicts vectors and unioning in the always-live TNs). Instead, we just
iterate over the global-conflicts for the TN, using BIT-IOR to combine the
conflict set with the bit-vector for that block in that location, or marking
that block/location combination as being always-live if the conflict is
always-live.
Generating the conflict set is inherently more costly, since although we
believe the conflict set size to be roughly constant, it can easily contain
tens of elements. We would have to generate these moderately large lists for
all TNs, including local TNs. In contrast, the proposed scheme does work
proportional to the number of blocks the TN is live in, which is small on
average (1 for local TNs). This win exists independently from the win of not
having to iterate over LTN conflict vectors.
[\#\#\# Note that since we never do bitwise iteration over the LTN conflict
vectors, part of the motivation for keeping these a small fixed size has been
removed. But it would still be useful to keep the size fixed so that we can
easily recycle the bit-vectors, and so that we could potentially have maximally
tense special primitives for doing clear and bit-ior on these vectors.]
This scheme is somewhat more space-intensive than having a per-location
bit-vector. Each vector entry would be something like 150 bits rather than one
bit, but this is mitigated by the number of blocks being 5-10x smaller than the
number of TNs. This seems like an acceptable overhead, a small fraction of the
total VMR representation.
The space overhead could also be reduced by using something equivalent to a
two-dimensional bit array, indexed first by LTN numbers, and then block numbers
(instead of using a simple-vector of separate bit-vectors.) This would
eliminate space wastage due to bit-vector overheads, which might be 50% or
more, and would also make efficient zeroing of the vectors more
straightforward. We would then want efficient operations for OR'ing LTN
conflict vectors with rows in the array.
This representation also opens a whole new range of allocation algorithms: ones
that store allocate TNs in different locations within different portions of the
program. This is because we can now represent a location being used to hold a
certain TN within an arbitrary subset of the blocks the TN is referenced in.
Pack goals:
Pack should:
Subject to resource constraints:
-- Minimize use costs
-- "Register allocation"
Allocate as many values as possible in scarce "good" locations,
attempting to minimize the aggregate use cost for the entire program.
-- "Save optimization"
Don't allocate values in registers when the save/restore costs exceed
the expected gain for keeping the value in a register. (Similar to
"opening costs" in RAOC.) [Really just a case of representation
selection.]
-- Minimize preference costs
Eliminate as many moves as possible.
"Register allocation" is basically an attempt to eliminate moves between
registers and memory. "Save optimization" counterbalances "register
allocation" to prevent it from becoming a pessimization, since saves can
introduce register/memory moves.
Preference optimization reduces the number of moves within an SC. Doing a good
job of honoring preferences is important to the success of the compiler, since
we have assumed in many places that moves will usually be optimized away.
The scarcity-oriented aspect of "register allocation" is handled by a greedy
algorithm in pack. We try to pack the "most important" TNs first, under the
theory that earlier packing is more likely to succeed due to fewer constraints.
The drawback of greedy algorithms is their inability to look ahead. Packing a
TN may mess up later "register allocation" by precluding packing of TNs that
are individually "less important", but more important in aggregate. Packing a
TN may also prevent preferences from being honored.
Initial packing:
Pack all TNs restricted to a finite SC first, before packing any other TNs.
One might suppose that Pack would have to treat TNs in different environments
differently, but this is not the case. Pack simply assigns TNs to locations so
that no two conflicting TNs are in the same location. In the process of
implementing call semantics in conflict analysis, we cause TNs in different
environments not to conflict. In the case of passing TNs, cross environment
conflicts do exist, but this reflects reality, since the passing TNs are
live in both the caller and the callee. Environment semantics has already been
implemented at this point.
This means that Pack can pack all TNs simultaneously, using one data structure
to represent the conflicts for each location. So we have only one conflict set
per SB location, rather than separating this information by environment
environment.
Load TN packing:
We create load TNs as needed in a post-pass to the initial packing. After TNs
are packed, it may be that some references to a TN will require it to be in a
SC other than the one it was packed in. We create load-TNs and pack them on
the fly during this post-pass.
What we do is have an optional SC restriction associated with TN-refs. If we
pack the TN in an SC which is different from the required SC for the reference,
then we create a TN for each such reference, and pack it into the required SC.
In many cases we will be able to pack the load TN with no hassle, but in
general we may need to spill a TN that has already been packed. We choose a
TN that isn't in use by the offending VOP, and then spill that TN onto the
stack for the duration of that VOP. If the VOP is a conditional, then we must
insert a new block interposed before the branch target so that the value TN
value is restored regardless of which branch is taken.
Instead of remembering lifetime information from conflict analysis, we rederive
it. We scan each block backward while keeping track of which locations have
live TNs in them. When we find a reference that needs a load TN packed, we try
to pack it in an unused location. If we can't, we unpack the currently live TN
with the lowest cost and force it into an unbounded SC.
The per-location and per-TN conflict information used by pack doesn't
need to be updated when we pack a load TN, since we are done using those data
structures.
We also don't need to create any TN-Refs for load TNs. [??? How do we keep
track of load-tn lifetimes? It isn't really that hard, I guess. We just
remember which load TNs we created at each VOP, killing them when we pass the
loading (or saving) step. This suggests we could flush the Refs thread if we
were willing to sacrifice some flexibility in explicit temporary lifetimes.
Flushing the Refs would make creating the VMR representation easier.]
The lifetime analysis done during load-TN packing doubles as a consistency
check. If we see a read of a TN packed in a location which has a different TN
currently live, then there is a packing bug. If any of the TNs recorded as
being live at the block beginning are packed in a scarce SB, but aren't current
in that location, then we also have a problem.
The conflict structure for load TNs is fairly simple, the load TNs for
arguments and results all conflict with each other, and don't conflict with
much else. We just try packing in targeted locations before trying at random.
\chapter{Code generation}
This is fairly straightforward. We translate VOPs into instruction sequences
on a per-block basis.
After code generation, the VMR representation is gone. Everything is
represented by the assembler data structures.
\chapter{Assembly}
In effect, we do much of the work of assembly when the compiler is compiled.
The assembler makes one pass fixing up branch offsets, then squeezes out the
space left by branch shortening and dumps out the code along with the load-time
fixup information. The assembler also deals with dumping unboxed non-immediate
constants and symbols. Boxed constants are created by explicit constructor
code in the top-level form, while immediate constants are generated using
inline code.
[\#\#\# The basic output of the assembler is:
A code vector
A representation of the fixups along with indices into the code vector for
the fixup locations
A PC map translating PCs into source paths
This information can then be used to build an output file or an in-core
function object.
]
The assembler is table-driven and supports arbitrary instruction formats. As
far as the assembler is concerned, an instruction is a bit sequence that is
broken down into subsequences. Some of the subsequences are constant in value,
while others can be determined at assemble or load time.
Assemble Node Form*
Allow instructions to be emitted during the evaluation of the Forms by
defining Inst as a local macro. This macro caches various global
information in local variables. Node tells the assembler what node
ultimately caused this code to be generated. This is used to create the
pc=>source map for the debugger.
Assemble-Elsewhere Node Form*
Similar to Assemble, but the current assembler location is changed to
somewhere else. This is useful for generating error code and similar
things. Assemble-Elsewhere may not be nested.
Inst Name Arg*
Emit the instruction Name with the specified arguments.
Gen-Label
Emit-Label (Label)
Gen-Label returns a Label object, which describes a place in the code.
Emit-Label marks the current position as being the location of Label.
\chapter{Dumping}
So far as input to the dumper/loader, how about having a list of Entry-Info
structures in the VMR-Component? These structures contain all information
needed to dump the associated function objects, and are only implicitly
associated with the functional/XEP data structures. Load-time constants that
reference these function objects should specify the Entry-Info, rather than the
functional (or something). We would then need to maintain some sort of
association so VMR conversion can find the appropriate Entry-Info.
Alternatively, we could initially reference the functional, and then later
clobber the reference to the Entry-Info.
We have some kind of post-pass that runs after assembly, going through the
functions and constants, annotating the VMR-Component for the benefit of the
dumper:
Resolve :Label load-time constants.
Make the debug info.
Make the entry-info structures.
Fasl dumper and in-core loader are implementation (but not instruction set)
dependent, so we want to give them a clear interface.
open-fasl-file name => fasl-file
Returns a "fasl-file" object representing all state needed by the dumper.
We objectify the state, since the fasdumper should be reentrant. (but
could fail to be at first.)
close-fasl-file fasl-file abort-p
Close the specified fasl-file.
fasl-dump-component component code-vector length fixups fasl-file
Dump the code, constants, etc. for component. Code-Vector is a vector
holding the assembled code. Length is the number of elements of Vector
that are actually in use. Fixups is a list of conses (offset . fixup)
describing the locations and things that need to be fixed up at load time.
If the component is a top-level component, then the top-level lambda will
be called after the component is loaded.
load-component component code-vector length fixups
Like Fasl-Dump-Component, but directly installs the code in core, running
any top-level code immediately. (???) but we need some way to glue
together the componenents, since we don't have a fasl table.
Dumping:
Dump code for each component after compiling that component, but defer dumping
of other stuff. We do the fixups on the code vectors, and accumulate them in
the table.
We have to grovel the constants for each component after compiling that
component so that we can fix up load-time constants. Load-time constants are
values needed my the code that are computed after code generation/assembly
time. Since the code is fixed at this point, load-time constants are always
represented as non-immediate constants in the constant pool. A load-time
constant is distinguished by being a cons (Kind . What), instead of a Constant
leaf. Kind is a keyword indicating how the constant is computed, and What is
some context.
Some interesting load-time constants:
(:label . <label>)
Is replaced with the byte offset of the label within the code-vector.
(:code-vector . <component>)
Is replaced by the component's code-vector.
(:entry . <function>)
(:closure-entry . <function>)
Is replaced by the function-entry structure for the specified function.
:Entry is how the top-level component gets a handle on the function
definitions so that it can set them up.
We also need to remember the starting offset for each entry, although these
don't in general appear as explicit constants.
We then dump out all the :Entry and :Closure-Entry objects, leaving any
constant-pool pointers uninitialized. After dumping each :Entry, we dump some
stuff to let genesis know that this is a function definition. Then we dump all
the constant pools, fixing up any constant-pool pointers in the already-dumped
function entry structures.
The debug-info *is* a constant: the first constant in every constant pool. But
the creation of this constant must be deferred until after the component is
compiled, so we leave a (:debug-info) placeholder. [Or maybe this is
implicitly added in by the dumper, being supplied in a VMR-component slot.]
Work out details of the interface between the back-end and the
assembler/dumper.
Support for multiple assemblers concurrently loaded? (for byte code)
We need various mechanisms for getting information out of the assembler.
We can get entry PCs and similar things into function objects by making a
Constant leaf, specifying that it goes in the closure, and then
setting the value after assembly.
We have an operation Label-Value which can be used to get the value of a
label after assembly and before the assembler data structures are
deallocated.
The function map can be constructed without any special help from the
assembler. Codegen just has to note the current label when the function
changes from one block to the next, and then use the final value of these
labels to make the function map.
Probably we want to do the source map this way too. Although this will
make zillions of spurious labels, we would have to effectively do that
anyway.
With both the function map and the source map, getting the locations right
for uses of Elsewhere will be a bit tricky. Users of Elsewhere will need
to know about how these maps are being built, since they must record the
labels and corresponding information for the elsewhere range. It would be
nice to have some cooperation from Elsewhere so that this isn't necessary,
otherwise some VOP writer will break the rules, resulting in code that is
nowhere.
The Debug-Info and related structures are dumped by consing up the
structure and making it be the value of a constant.
Getting the code vector and fixups dumped may be a bit more interesting. I
guess we want a Dump-Code-Vector function which dumps the code and fixups
accumulated by the current assembly, returning a magic object that will
become the code vector when it is dumped as a constant.
]

540
doc/cmucl/internals/compiler-overview.tex Normal file
View file

@ -0,0 +1,540 @@
\chapter{Compiler Overview} % -*- Dictionary: design -*-
The structure of the compiler may be broadly characterized by describing the
compilation phases and the data structures that they manipulate. The steps in
the compilation are called phases rather than passes since they don't
necessarily involve a full pass over the code. The data structure used to
represent the code at some point is called an {\it intermediate
representation.}
Two major intermediate representations are used in the compiler:
\begin{itemize}
\item The Implicit Continuation Representation (ICR) represents the lisp-level
semantics of the source code during the initial phases. Partial evaluation and
semantic analysis are done on this representation. ICR is roughly equivalent
to a subset of Common Lisp, but is represented as a flow-graph rather than a
syntax tree. Phases which only manipulate ICR comprise the "front end". It
would be possible to use a different back end such as one that directly
generated code for a stack machine.
\item The Virtual Machine Representation (VMR) represents the implementation of
the source code on a virtual machine. The virtual machine may vary depending
on the the target hardware, but VMR is sufficiently stylized that most of the
phases which manipulate it are portable.
\end{itemize}
Each phase is briefly described here. The phases from ``local call analysis''
to ``constraint propagation'' all interact; for maximum optimization, they
are generally repeated until nothing new is discovered. The source files which
primarily contain each phase are listed after ``Files: ''.
\begin{description}
\item[ICR conversion]
Convert the source into ICR, doing macroexpansion and simple source-to-source
transformation. All names are resolved at this time, so we don't have to worry
about name conflicts later on. Files: {\tt ir1tran, srctran, typetran}
\item[Local call analysis] Find calls to local functions and convert them to
local calls to the correct entry point, doing keyword parsing, etc. Recognize
once-called functions as lets. Create {\it external entry points} for
entry-point functions. Files: {\tt locall}
\item[Find components]
Find flow graph components and compute depth-first ordering. Separate
top-level code from run-time code, and determine which components are top-level
components. Files: {\tt dfo}
\item[ICR optimize] A grab-bag of all the non-flow ICR optimizations. Fold
constant functions, propagate types and eliminate code that computes unused
values. Special-case calls to some known global functions by replacing them
with a computed function. Merge blocks and eliminate IF-IFs. Substitute let
variables. Files: {\tt ir1opt, ir1tran, typetran, seqtran, vm/vm-tran}
\item[Type constraint propagation]
Use global flow analysis to propagate information about lexical variable
types. Eliminate unnecessary type checks and tests. Files: {\tt constraint}
\item[Type check generation]
Emit explicit ICR code for any necessary type checks that are too complex to be
easily generated on the fly by the back end. Files: {\tt checkgen}
\item[Event driven operations]
Various parts of ICR are incrementally recomputed, either eagerly on
modification of the ICR, or lazily, when the relevant information is needed.
\begin{itemize}
\item Check that type assertions are satisfied, marking places where type
checks need to be done.
\item Locate let calls.
\item Delete functions and variables with no references
\end{itemize}
Files: {\tt ir1util}, {\tt ir1opt}
\item[ICR finalize]
This phase is run after all components have been compiled. It scans the
global variable references, looking for references to undefined variables
and incompatible function redefinitions. Files: {\tt ir1final}, {\tt main}.
\item[Environment analysis]
Determine which distinct environments need to be allocated, and what
context needed to be closed over by each environment. We detect non-local
exits and set closure variables. We also emit cleanup code as funny
function calls. This is the last pure ICR pass. Files: {\tt envanal}
\item[Global TN allocation (GTN)]
Iterate over all defined functions, determining calling conventions
and assigning TNs to local variables. Files: {\tt gtn}
\item[Local TN allocation (LTN)]
Use type and policy information to determine which VMR translation to use
for known functions, and then create TNs for expression evaluation
temporaries. We also accumulate some random information needed by VMR
conversion. Files: {\tt ltn}
\item[Control analysis]
Linearize the flow graph in a way that minimizes the number of branches. The
block-level structure of the flow graph is basically frozen at this point.
Files: {\tt control}
\item[Stack analysis]
Maintain stack discipline for unknown-values continuation in the presence
of local exits. Files: {\tt stack}
\item[Entry analysis]
Collect some back-end information for each externally callable function.
\item[VMR conversion] Convert ICR into VMR by translating nodes into VOPs.
Emit type checks. Files: {\tt ir2tran, vmdef}
\item[Copy propagation] Use flow analysis to eliminate unnecessary copying of
TN values. Files: {\tt copyprop}
\item[Representation selection]
Look at all references to each TN to determine which representation has the
lowest cost. Emit appropriate move and coerce VOPS for that representation.
\item[Lifetime analysis]
Do flow analysis to find the set of TNs whose lifetimes
overlap with the lifetimes of each TN being packed. Annotate call VOPs with
the TNs that need to be saved. Files: {\tt life}
\item[Pack]
Find a legal register allocation, attempting to minimize unnecessary moves.
Files: {\tt pack}
\item[Code generation]
Call the VOP generators to emit assembly code. Files: {\tt codegen}
\item[Pipeline reorganization] On some machines, move memory references
backward in the code so that they can overlap with computation. On machines
with delayed branch instructions, locate instructions that can be moved into
delay slots. Files: {\tt assem-opt}
\item[Assembly]
Resolve branches and convert in to object code and fixup information.
Files: {\tt assembler}
\item[Dumping] Convert the compiled code into an object file or in-core
function. Files: {\tt debug-dump}, {\tt dump}, {\tt vm/core}
\end{description}
\chapter{The Implicit Continuation Representation}
The set of special forms recognized is exactly that specified in the Common
Lisp manual. Everything that is described as a macro in CLTL is a macro.
Large amounts of syntactic information are thrown away by the conversion to an
anonymous flow graph representation. The elimination of names eliminates the
need to represent most environment manipulation special forms. The explicit
representation of control eliminates the need to represent BLOCK and GO, and
makes flow analysis easy. The full Common Lisp LAMBDA is implemented with a
simple fixed-arg lambda, which greatly simplifies later code.
The elimination of syntactic information eliminates the need for most of the
"beta transformation" optimizations in Rabbit. There are no progns, no
tagbodys and no returns. There are no "close parens" which get in the way of
determining which node receives a given value.
In ICR, computation is represented by Nodes. These are the node types:
\begin{description}
\item[if] Represents all conditionals.
\item[set] Represents a {\tt setq}.
\item[ref] Represents a constant or variable reference.
\item[combination] Represents a normal function call.
\item[MV-combination] Represents a {\tt multiple-value-call}. This is used to
implement all multiple value receiving forms except for {\tt
multiple-value-prog1}, which is implicit.
\item[bind]
This represents the allocation and initialization of the variables in
a lambda.
\item[return]
This collects the return value from a lambda and represents the
control transfer on return.
\item[entry] Marks the start of a dynamic extent that can have non-local exits
to it. Dynamic state can be saved at this point for restoration on re-entry.
\item[exit] Marks a potentially non-local exit. This node is interposed
between the non-local uses of a continuation and the {\tt dest} so that code to
do a non-local exit can be inserted if necessary.
\end{description}
Some slots are shared between all node types (via defstruct inheritance.) This
information held in common between all nodes often makes it possible to avoid
special-casing nodes on the basis of type. This shared information is
primarily concerned with the order of evaluation and destinations and
properties of results. This control and value flow is indicated in the node
primarily by pointing to continuations.
The {\tt continuation} structure represents information sufficiently related
to the normal notion of a continuation that naming it so seems sensible.
Basically, a continuation represents a place in the code, or alternatively the
destination of an expression result and a transfer of control. These two
notions are bound together for the same reasons that they are related in the
standard functional continuation interpretation.
A continuation may be deprived of either or both of its value or control
significance. If the value of a continuation is unused due to evaluation for
effect, then the continuation will have a null {\tt dest}. If the {\tt next}
node for a continuation is deleted by some optimization, then {\tt next} will
be {\tt :none}.
[\#\#\# Continuation kinds...]
The {\tt block} structure represents a basic block, in the the normal sense.
Control transfers other than simple sequencing are represented by information
in the block structure. The continuation for the last node in a block
represents only the destination for the result.
It is very difficult to reconstruct anything resembling the original source
from ICR, so we record the original source form in each node. The location of
the source form within the input is also recorded, allowing for interfaces such
as "Edit Compiler Warnings". See section \ref{source-paths}.
Forms such as special-bind and catch need to have cleanup code executed at all
exit points from the form. We represent this constraint in ICR by annotating
the code syntactically within the form with a Cleanup structure describing what
needs to be cleaned up. Environment analysis determines the cleanup locations
by watching for a change in the cleanup between two continuations. We can't
emit cleanup code during ICR conversion, since we don't know which exits will
be local until after ICR optimizations are done.
Special binding is represented by a call to the funny function %Special-Bind.
The first argument is the Global-Var structure for the variable bound and the
second argument is the value to bind it to.
Some subprimitives are implemented using a macro-like mechanism for translating
%PRIMITIVE forms into arbitrary lisp code. Subprimitives special-cased by VMR
conversion are represented by a call to the funny function %%Primitive. The
corresponding Template structure is passed as the first argument.
We check global function calls for syntactic legality with respect to any
defined function type function. If the call is illegal or we are unable to
tell if it is legal due to non-constant keywords, then we give a warning and
mark the function reference as :notinline to force a full call and cause
subsequent phases to ignore the call. If the call is legal and is to a known
function, then we annotate the Combination node with the Function-Info
structure that contains the compiler information for the function.
\section{Tail sets}
\#|
Probably want to have a GTN-like function result equivalence class mechanism
for ICR type inference. This would be like the return value propagation being
done by Propagate-From-Calls, but more powerful, less hackish, and known to
terminate. The ICR equivalence classes could probably be used by GTN, as well.
What we do is have local call analysis eagerly maintain the equivalence classes
of functions that return the same way by annotating functions with a Tail-Info
structure shared between all functions whose value could be the value of this
function. We don't require that the calls actually be tail-recursive, only
that the call deliver its value to the result continuation. [\#\#\# Actually
now done by ICR-OPTIMIZE-RETURN, which is currently making ICR optimize
mandatory.]
We can then use the Tail-Set during ICR type inference. It would have a type
that is the union across all equivalent functions of the types of all the uses
other than in local calls. This type would be recomputed during optimization
of return nodes. When the type changes, we would propagate it to all calls to
any of the equivalent functions. How do we know when and how to recompute the
type for a tail-set? Recomputation is driven by type propagation on the result
continuation.
This is really special-casing of RETURN nodes. The return node has the type
which is the union of all the non-call uses of the result. The tail-set is
found though the lambda. We can then recompute the overall union by taking the
union of the type per return node, rather than per-use.
How do result type assertions work? We can't intersect the assertions across
all functions in the equivalence class, since some of the call combinations may
not happen (or even be possible). We can intersect the assertion of the result
with the derived types for non-call uses.
When we do a tail call, we obviously can't check that the returned value
matches our assertion. Although in principle, we would like to be able to
check all assertions, to preserve system integrity, we only need to check
assertions that we depend on. We can afford to lose some assertion information
as long as we entirely lose it, ignoring it for type inference as well as for
type checking.
Things will work out, since the caller will see the tail-info type as the
derived type for the call, and will emit a type check if it needs a stronger
result.
A remaining question is whether we should intersect the assertion with
per-RETURN derived types from the very beginning (i.e. before the type check
pass). I think the answer is yes. We delay the type check pass so that we can
get our best guess for the derived type before we decide whether a check is
necessary. But with the function return type, we aren't committing to doing
any type check when we intersect with the type assertion; the need to type
check is still determined in the type check pass by examination of the result
continuation.
What is the relationship between the per-RETURN types and the types in the
result continuation? The assertion is exactly the Continuation-Asserted-Type
(note that the asserted type of result continuations will never change after
ICR conversion). The per-RETURN derived type is different than the
Continuation-Derived-Type, since it is intersected with the asserted type even
before Type Check runs. Ignoring the Continuation-Derived-Type probably makes
life simpler anyway, since this breaks the potential circularity of the
Tail-Info-Type will affecting the Continuation-Derived-Type, which affects...
When a given return has no non-call uses, we represent this by using
*empty-type*. This consistent with the interpretation that a return type of
NIL means the function can't return.
\section{Hairy function representation}
Non-fixed-arg functions are represented using Optional-Dispatch. An
Optional-Dispatch has an entry-point function for each legal number of
optionals, and one for when extra args are present. Each entry point function
is a simple lambda. The entry point function for an optional is passed the
arguments which were actually supplied; the entry point function is expected to
default any remaining parameters and evaluate the actual function body.
If no supplied-p arg is present, then we can do this fairly easily by having
each entry point supply its default and call the next entry point, with the
last entry point containing the body. If there are supplied-p args, then entry
point function is replaced with a function that calls the original entry
function with T's inserted at the position of all the supplied args with
supplied-p parameters.
We want to be a bit clever about how we handle arguments declared special when
doing optional defaulting, or we will emit really gross code for special
optionals. If we bound the arg specially over the entire entry-point function,
then the entry point function would be caused to be non-tail-recursive. What
we can do is only bind the variable specially around the evaluation of the
default, and then read the special and store the final value of the special
into a lexical variable which we then pass as the argument. In the common case
where the default is a constant, we don't have to special-bind at all, since
the computation of the default is not affected by and cannot affect any special
bindings.
Keyword and rest args are both implemented using a LEXPR-like "more args"
convention. The More-Entry takes two arguments in addition to the fixed and
optional arguments: the argument context and count. (ARG <context> <n>)
accesses the N'th additional argument. Keyword args are implemented directly
using this mechanism. Rest args are created by calling %Listify-Rest-Args with
the context and count.
The More-Entry parses the keyword arguments and passes the values to the main
function as positional arguments. If a keyword default is not constant, then
we pass a supplied-p parameter into the main entry and let it worry about
defaulting the argument. Since the main entry accepts keywords in parsed form,
we can parse keywords at compile time for calls to known functions. We keep
around the original parsed lambda-list and related information so that people
can figure out how to call the main entry.
\section{ICR representation of non-local exits}
All exits are initially represented by EXIT nodes:
How about an Exit node:
(defstruct (exit (:include node))
value)
The Exit node uses the continuation that is to receive the thrown Value.
During optimization, if we discover that the Cont's home-lambda is the same is
the exit node's, then we can delete the Exit node, substituting the Cont for
all of the Value's uses.
The successor block of an EXIT is the entry block in the entered environment.
So we use the Exit node to mark the place where exit code is inserted. During
environment analysis, we need only insert a single block containing the entry
point stub.
We ensure that all Exits that aren't for a NLX don't have any Value, so that
local exits never require any value massaging.
The Entry node marks the beginning of a block or tagbody:
(defstruct (entry (:include node))
(continuations nil :type list))
It contains a list of all the continuations that the body could exit to. The
Entry node is used as a marker for the the place to snapshot state, including
the control stack pointer. Each lambda has a list of its Entries so
that environment analysis can figure out which continuations are really being
closed over. There is no reason for optimization to delete Entry nodes,
since they are harmless in the degenerate case: we just emit no code (like a
no-var let).
We represent CATCH using the lexical exit mechanism. We do a transformation
like this:
(catch 'foo xxx) ==>
(block \#:foo
(%catch \#'(lambda () (return-from \#:foo (%unknown-values))) 'foo)
(%within-cleanup :catch
xxx))
%CATCH just sets up the catch frame which points to the exit function. %Catch
is an ordinary function as far as ICR is concerned. The fact that the catcher
needs to be cleaned up is expressed by the Cleanup slots in the continuations
in the body. %UNKNOWN-VALUES is a dummy function call which represents the
fact that we don't know what values will be thrown.
%WITHIN-CLEANUP is a special special form that instantiates its first argument
as the current cleanup when converting the body. In reality, the lambda is
also created by the special special form %ESCAPE-FUNCTION, which gives the
lambda a special :ESCAPE kind so that the back end knows not to generate any
code for it.
We use a similar hack in Unwind-Protect to represent the fact that the cleanup
forms can be invoked at arbitrarily random times.
(unwind-protect p c) ==>
(flet ((\#:cleanup () c))
(block \#:return
(multiple-value-bind
(\#:next \#:start \#:count)
(block \#:unwind
(%unwind-protect \#'(lambda (x) (return-from \#:unwind x)))
(%within-cleanup :unwind-protect
(return-from \#:return p)))
(\#:cleanup)
(%continue-unwind \#:next \#:start \#:count))))
We use the block \#:unwind to represent the entry to cleanup code in the case
where we are non-locally unwound. Calling of the cleanup function in the
drop-through case (or any local exit) is handled by cleanup generation. We
make the cleanup a function so that cleanup generation can add calls at local
exits from the protected form. \#:next, \#:start and \#:count are state used in
the case where we are unwound. They indicate where to go after doing the
cleanup and what values are being thrown. The cleanup encloses only the
protected form. As in CATCH, the escape function is specially tagged as
:ESCAPE. The cleanup function is tagged as :CLEANUP to inhibit let conversion
(since references are added in environment analysis.)
Notice that implementing these forms using closures over continuations
eliminates any need to special-case ICR flow analysis. Obviously we don't
really want to make heap-closures here. In reality these functions are
special-cased by the back-end according to their KIND.
\section{Block compilation}
One of the properties of ICR is that supports "block compilation" by allowing
arbitrarily large amounts of code to be converted at once, with actual
compilation of the code being done at will.
In order to preserve the normal semantics we must recognize that proclamations
(possibly implicit) are scoped. A proclamation is in effect only from the time
of appearance of the proclamation to the time it is contradicted. The current
global environment at the end of a block is not necessarily the correct global
environment for compilation of all the code within the block. We solve this
problem by closing over the relevant information in the ICR at the time it is
converted. For example, each functional variable reference is marked as
inline, notinline or don't care. Similarly, each node contains a structure
known as a Cookie which contains the appropriate settings of the compiler
policy switches.
We actually convert each form in the file separately, creating a separate
"initial component" for each one. Later on, these components are merged as
needed. The main reason for doing this is to cause EVAL-WHEN processing to be
interleaved with reading.
\section{Entry points}
\#|
Since we need to evaluate potentially arbitrary code in the XEP argument forms
(for type checking), we can't leave the arguments in the wired passing
locations. Instead, it seems better to give the XEP max-args fixed arguments,
with the passing locations being the true passing locations. Instead of using
%XEP-ARG, we reference the appropriate variable.
Also, it might be a good idea to do argument count checking and dispatching
with explicit conditional code in the XEP. This would simplify both the code
that creates the XEP and the VMR conversion of XEPs. Also, argument count
dispatching would automatically benefit from any cleverness in compilation of
case-like forms (jump tables, etc). On the downside, this would push some
assumptions about how arg dispatching is done into ICR. But then we are
currently violating abstraction at least as badly in VMR conversion, which is
also supposed to be implementation independent.
|\#
As a side-effect of finding which references to known functions can be
converted to local calls, we find any references that cannot be converted.
References that cannot be converted to a local call must evaluate to a
"function object" (or function-entry) that can be called using the full call
convention. A function that can be called from outside the component is called
an "entry-point".
Lots of stuff that happens at compile-time with local function calls must be
done at run-time when an entry-point is called.
It is desirable for optimization and other purposes if all the calls to every
function were directly present in ICR as local calls. We cannot directly do
this with entry-point functions, since we don't know where and how the
entry-point will be called until run-time.
What we do is represent all the calls possible from outside the component by
local calls within the component. For each entry-point function, we create a
corresponding lambda called the external entry point or XEP. This is a
function which takes the number of arguments passed as the first argument,
followed by arguments corresponding to each required or optional argument.
If an optional argument is unsupplied, the value passed into the XEP is
undefined. The XEP is responsible for doing argument count checking and
dispatching.
In the case of a fixed-arg lambda, we emit a call to the %VERIFY-ARGUMENT-COUNT
funny function (conditional on policy), then call the real function on the
passed arguments. Even in this simple case, we benefit several ways from
having a separate XEP:
-- The argument count checking is factored out, and only needs to be done in
full calls.
-- Argument type checking happens automatically as a consequence of passing
the XEP arguments in a local call to the real function. This type checking
is also only done in full calls.
-- The real function may use a non-standard calling convention for the benefit
of recursive or block-compiled calls. The XEP converts arguments/return
values to/from the standard convention. This also requires little
special-casing of XEPs.
If the function has variable argument count (represented by an
OPTIONAL-DISPATCH), then the XEP contains a COND which dispatches off of the
argument count, calling the appropriate entry-point function (which then does
defaulting). If there is a more entry (for keyword or rest args), then the XEP
obtains the more arg context and count by calling the %MORE-ARG-CONTEXT funny
function.
All non-local-call references to functions are replaced with references to the
corresponding XEP. ICR optimization may discover a local call that was
previously a non-local reference. When we delete the reference to the XEP, we
may find that it has no references. In this case, we can delete the XEP,
causing the function to no longer be an entry-point.

6
doc/cmucl/internals/compiler.tex Normal file
View file

@ -0,0 +1,6 @@
\part{Compiler Organization}
\include{compiler-overview}
\include{front}
\include{middle}
\include{back}
\include{interface}

537
doc/cmucl/internals/debugger.tex Normal file
View file

@ -0,0 +1,537 @@
% -*- Dictionary: design; Package: C -*-
\#|
\chapter{Debugger Information}
\index{debugger information}
\label{debug-info}
Although the compiler's great freedom in choice of function call conventions
and variable representations has major efficiency advantages, it also has
unfortunate consequences for the debugger. The debug information that we need
is even more elaborate than for conventional "compiled" languages, since we
cannot even do a simple backtrace without some debug information. However,
once having gone this far, it is not that difficult to go the extra distance,
and provide full source level debugging of compiled code.
Full debug information has a substantial space penalty, so we allow different
levels of debug information to be specified. In the extreme case, we can
totally omit debug information.
\section{The Debug-Info Structure}
\index{debug-info structure}
The Debug-Info structure directly represents information about the
source code, and points to other structures that describe the layout of
run-time data structures.
Make some sort of minimal debug-info format that would support at least the
common cases of level 1 (since that is what we would release), and perhaps
level 0. Actually, it seems it wouldn't be hard to crunch nearly all of the
debug-function structure and debug-info function map into a single byte-vector.
We could have an uncrunch function that restored the current format. This
would be used by the debugger, and also could be used by purify to delete parts
of the debug-info even when the compiler dumps it in crunched form.
[Note that this isn't terribly important if purify is smart about
debug-info...]
|\#
Compiled source map representation:
[\#\#\# store in debug-function PC at which env is properly initialized, i.e.
args (and return-pc, etc.) in internal locations. This is where a
:function-start breakpoint would break.]
[\#\#\# Note that that we can easily cache the form-number => source-path or
form-number => form translation using a vector indexed by form numbers that we
build during a walk.]
Instead of using source paths in the debug-info, use "form numbers". The form
number of a form is the number of forms that we walk to reach that form when
doing a pre-order walk of the source form. [Might want to use a post-order
walk, as that would more closely approximate evaluation order.]
We probably want to continue using source-paths in the compiler, since they are
quick to compute and to get you to a particular form. [\#\#\# But actually, I
guess we don't have to precompute the source paths and annotate nodes with
them: instead we could annotate the nodes with the actual original source form.
Then if we wanted to find the location of that form, we could walk the root
source form, looking that original form. But we might still need to enter all
the forms in a hashtable so that we can tell during IR1 conversion that a given
form appeared in the original source.]
Note that form numbers have an interesting property: it is quite efficient to
determine whether an arbitrary form is a subform of some other form, since the
form number of B will be > than A's number and < A's next sibling's number iff
B is a subform of A.
This should be quite useful for doing the source=>pc mapping in the debugger,
since that problem reduces to finding the subset of the known locations that
are for subforms of the specified form.
Assume a byte vector with a standard variable-length integer format, something
like this:
0..253 => the integer
254 => read next two bytes for integer
255 => read next four bytes for integer
Then a compiled debug block is just a sequence of variable-length integers in a
particular order, something like this:
number of successors
...offsets of each successor in the function's blocks vector...
first PC
[offset of first top-level form (in forms) (only if not component default)]
form number of first source form
first live mask (length in bytes determined by number of VARIABLES)
...more <PC, top-level form offset, form-number, live-set> tuples...
We determine the number of locations recorded in a block by the finding the
start of the next compiled debug block in the blocks vector.
[\#\#\# Actually, only need 2 bits for number of successors {0,1,2}. We might
want to use other bits in the first byte to indicate the kind of location.]
[\#\#\# We could support local packing by having a general concept of "alternate
locations" instead of just regular and save locations. The location would have
a bit indicating that there are alternate locations, in which case we read the
number of alternate locations and then that many more SC-OFFSETs. In the
debug-block, we would have a second bit mask with bits set for TNs that are in
an alternate location. We then read a number for each such TN, with the value
being interpreted as an index into the Location's alternate locations.]
It looks like using structures for the compiled-location-info is too bulky.
Instead we need some packed binary representation.
First, let's represent a SC/offset pair with an "SC-Offset", which is an
integer with the SC in the low 5 bits and the offset in the remaining bits:
----------------------------------------------------
| Offset (as many bits as necessary) | SC (5 bits) |
----------------------------------------------------
Probably the result should be constrained to fit in a fixnum, since it will be
more efficient and gives more than enough possible offsets.
We can the represent a compiled location like this:
single byte of boolean flags:
uninterned name
packaged name
environment-live
has distinct save location
has ID (name not unique in this fun)
name length in bytes (as var-length integer)
...name bytes...
[if packaged, var-length integer that is package name length]
...package name bytes...]
[If has ID, ID as var-length integer]
SC-Offset of primary location (as var-length integer)
[If has save SC, SC-Offset of save location (as var-length integer)]
But for a whizzy breakpoint facility, we would need a good source=>code map.
Dumping a complete code=>source map might be as good a way as any to represent
this, due to the one-to-many relationship between source and code locations.
We might be able to get away with just storing the source locations for the
beginnings of blocks and maintaining a mapping from code ranges to blocks.
This would be fine both for the profiler and for the "where am I running now"
indication. Users might also be convinced that it was most interesting to
break at block starts, but I don't really know how easily people could develop
an understanding of basic blocks.
It could also be a bit tricky to map an arbitrary user-designated source
location to some "closest" source location actually in the debug info.
This problem probably exists to some degree even with a full source map, since
some forms will never appear as the source of any node. It seems you might
have to negotiate with the user. He would mouse something, and then you would
highlight some source form that has a common prefix (i.e. is a prefix of the
user path, or vice-versa.) If they aren't happy with the result, they could
try something else. In some cases, the designated path might be a prefix of
several paths. This ambiguity might be resolved by picking the shortest path
or letting the user choose.
At the primitive level, I guess what this means is that the structure of source
locations (i.e. source paths) must be known, and the source=>code operation
should return a list of <source,code> pairs, rather than just a list of code
locations. This allows the debugger to resolve the ambiguity however it wants.
I guess the formal definition of which source paths we would return is:
All source paths in the debug info that have a maximal common prefix with
the specified path. i.e. if several paths have the complete specified path
as a prefix, we return them all. Otherwise, all paths with an equally
large common prefix are returned: if the path with the most in common
matches only the first three elements, then we return all paths that match
in the first three elements. As a degenerate case (which probably
shouldn't happen), if there is no path with anything in common, then we
return *all* of the paths.
In the DEBUG-SOURCE structure we may ultimately want a vector of the start
positions of each source form, since that would make it easier for the debugger
to locate the source. It could just open the file, FILE-POSITION to the form,
do a READ, then loop down the source path. Of course, it could read each form
starting from the beginning, but that might be too slow.
Do XEPs really need Debug-Functions? The only time that we will commonly end
up in the debugger on an XEP is when an argument type check fails. But I
suppose it would be nice to be able to print the arguments passed...
Note that assembler-level code motion such as pipeline reorganization can cause
problems with our PC maps. The assembler needs to know that debug info markers
are different from real labels anyway, so I suppose it could inhibit motion
across debug markers conditional on policy. It seems unworthwhile to remember
the node for each individual instruction.
For tracing block-compiled calls:
Info about return value passing locations?
Info about where all the returns are?
We definitely need the return-value passing locations for debug-return. The
question is what the interface should be. We don't really want to have a
visible debug-function-return-locations operation, since there are various
value passing conventions, and we want to paper over the differences.
Probably should be a compiler option to initialize stack frame to a special
uninitialized object (some random immediate type). This would aid debugging,
and would also help GC problems. For the latter reason especially, this should
be locally-turn-onable (off of policy? the new debug-info quality?).
What about the interface between the evaluator and the debugger? (i.e. what
happens on an error, etc.) Compiler error handling should be integrated with
run-time error handling. Ideally the error messages should look the same.
Practically, in some cases the run-time errors will have less information. But
the error should look the same to the debugger (or at least similar).
;;;; Debugger interface:
How does the debugger interface to the "evaluator" (where the evaluator means
all of native code, byte-code and interpreted IR1)? It seems that it would be
much more straightforward to have a consistent user interface to debugging
all code representations if there was a uniform debugger interface to the
underlying stuff, and vice-versa.
Of course, some operations might not be supported by some representations, etc.
For example, fine-control stepping might not be available in native code.
In other cases, we might reduce an operation to the lowest common denominator,
for example fetching lexical variables by string and admitting the possibility
of ambiguous matches. [Actually, it would probably be a good idea to store the
package if we are going to allow variables to be closed over.]
Some objects we would need:
Location:
The constant information about the place where a value is stored,
everything but which particular frame it is in. Operations:
location name, type, etc.
location-value frame location (setf'able)
monitor-location location function
Function is called whenever location is set with the location,
frame and old value. If active values aren't supported, then we
dummy the effect using breakpoints, in which case the change won't
be noticed until the end of the block (and intermediate changes
will be lost.)
debug info:
All the debug information for a component.
Frame:
frame-changed-locations frame => location*
Return a list of the locations in frame that were changed since the
last time this function was called. Or something. This is for
displaying interesting state changes at breakpoints.
save-frame-state frame => frame-state
restore-frame-state frame frame-state
These operations allow the debugger to back up evaluation, modulo
side-effects and non-local control transfers. This copies and
restores all variables, temporaries, etc, local to the frame, and
also the current PC and dynamic environment (current catch, etc.)
At the time of the save, the frame must be for the running function
(not waiting for a call to return.) When we restore, the frame
becomes current again, effectively exiting from any frames on top.
(Of course, frame must not already be exited.)
Thread:
Representation of which stack to use, etc.
Block:
What successors the block has, what calls there are in the block.
(Don't need to know where calls are as long as we know called function,
since can breakpoint at the function.) Whether code in this block is
wildly out of order due to being the result of loop-invariant
optimization, etc. Operations:
block-successors block => code-location*
block-forms block => (source-location code-location)*
Return the corresponding source locations and code locations for
all forms (and form fragments) in the block.
Variable maps:
There are about five things that the debugger might want to know about a
variable:
Name
Although a lexical variable's name is "really" a symbol (package and
all), in practice it doesn't seem worthwhile to require all the symbols
for local variable names to be retained. There is much less VM and GC
overhead for a constant string than for a symbol. (Also it is useful
to be able to access gensyms in the debugger, even though they are
theoretically ineffable).
ID
Which variable with the specified name is this? It is possible to have
multiple variables with the same name in a given function. The ID is
something that makes Name unique, probably a small integer. When
variables aren't unique, we could make this be part of the name, e.g.
"FOO\#1", "FOO\#2". But there are advantages to keeping this separate,
since in many cases lifetime information can be used to disambiguate,
making qualification unnecessary.
SC
When unboxed representations are in use, we must have type information
to properly read and write a location. We only need to know the
SC for this, which would be amenable to a space-saving
numeric encoding.
Location
Simple: the offset in SC. [Actually, we need the save location too.]
Lifetime
In what parts of the program does this variable hold a meaningful
value? It seems prohibitive to record precise lifetime information,
both in space and compiler effort, so we will have to settle for some
sort of approximation.
The finest granularity at which it is easy to determine liveness is the
the block: we can regard the variable lifetime as the set of blocks
that the variable is live in. Of course, the variable may be dead (and
thus contain meaningless garbage) during arbitrarily large portions of
the block.
Note that this subsumes the notion of which function a variable belongs
to. A given block is only in one function, so the function is
implicit.
The variable map should represent this information space-efficiently and with
adequate computational efficiency.
The SC and ID can be represented as small integers. Although the ID can in
principle be arbitrarily large, it should be <100 in practice. The location
can be represented by just the offset (a moderately small integer), since the
SB is implicit in the SC.
The lifetime info can be represented either as a bit-vector indexed by block
numbers, or by a list of block numbers. Which is more compact depends both on
the size of the component and on the number of blocks the variable is live in.
In the limit of large component size, the sparse representation will be more
compact, but it isn't clear where this crossover occurs. Of course, it would
be possible to use both representations, choosing the more compact one on a
per-variable basis. Another interesting special case is when the variable is
live in only one block: this may be common enough to be worth picking off,
although it is probably rarer for named variables than for TNs in general.
If we dump the type, then a normal list-style type descriptor is fine: the
space overhead is small, since the shareability is high.
We could probably save some space by cleverly representing the var-info as
parallel vectors of different types, but this would be more painful in use.
It seems better to just use a structure, encoding the unboxed fields in a
fixnum. This way, we can pass around the structure in the debugger, perhaps
even exporting it from the the low-level debugger interface.
[\#\#\# We need the save location too. This probably means that we need two slots
of bits, since we need the save offset and save SC. Actually, we could let the
save SC be implied by the normal SC, since at least currently, we always choose
the same save SC for a given SC. But even so, we probably can't fit all that
stuff in one fixnum without squeezing a lot, so we might as well split and
record both SCs.
In a localized packing scheme, we would have to dump a different var-info
whenever either the main location or the save location changes. As a practical
matter, the save location is less likely to change than the main location, and
should never change without the main location changing.
One can conceive of localized packing schemes that do saving as a special case
of localized packing. If we did this, then the concept of a save location
might be eliminated, but this would require major changes in the IR2
representation for call and/or lifetime info. Probably we will want saving to
continue to be somewhat magical.]
How about:
(defstruct var-info
;;
;; This variable's name. (symbol-name of the symbol)
(name nil :type simple-string)
;;
;; The SC, ID and offset, encoded as bit-fields.
(bits nil :type fixnum)
;;
;; The set of blocks this variable is live in. If a bit-vector, then it has
;; a 1 when indexed by the number of a block that it is live in. If an
;; I-vector, then it lists the live block numbers. If a fixnum, then that is
;; the number of the sole live block.
(lifetime nil :type (or vector fixnum))
;;
;; The variable's type, represented as list-style type descriptor.
type)
Then the debug-info holds a simple-vector of all the var-info structures for
that component. We might as well make it sorted alphabetically by name, so
that we can binary-search to find the variable corresponding to a particular
name.
We need to be able to translate PCs to block numbers. This can be done by an
I-Vector in the component that contains the start location of each block. The
block number is the index at which we find the correct PC range. This requires
that we use an emit-order block numbering distinct from the IR2-Block-Number,
but that isn't any big deal. This seems space-expensive, but it isn't too bad,
since it would only be a fraction of the code size if the average block length
is a few words or more.
An advantage of our per-block lifetime representation is that it directly
supports keeping a variable in different locations when in different blocks,
i.e. multi-location packing. We use a different var-info for each different
packing, since the SC and offset are potentially different. The Name and ID
are the same, representing the fact that it is the same variable. It is here
that the ID is most significant, since the debugger could otherwise make
same-name variables unique all by itself.
Stack parsing:
[\#\#\# Probably not worth trying to make the stack parseable from the bottom up.
There are too many complications when we start having variable sized stuff on
the stack. It seems more profitable to work on making top-down parsing robust.
Since we are now planning to wire the bottom-up linkage info, scanning from the
bottom to find the top frame shouldn't be too inefficient, even when there was
a runaway recursion. If we somehow jump into hyperspace, then the debugger may
get confused, but we can debug this sort of low-level system lossage using
ADB.]
There are currently three relevant context pointers:
-- The PC. The current PC is wired (implicit in the machine). A saved
PC (RETURN-PC) may be anywhere in the current frame.
-- The current stack context (CONT). The current CONT is wired. A saved
CONT (OLD-CONT) may be anywhere in the current frame.
-- The current code object (ENV). The current ENV is wired. When saved,
this is extra-difficult to locate, since it is saved by the caller, and is
thus at an unknown offset in OLD-CONT, rather than anywhere in the current
frame.
We must have all of these to parse the stack.
With the proposed Debug-Function, we parse the stack (starting at the top) like
this:
1] Use ENV to locate the current Debug-Info
2] Use the Debug-Info and PC to determine the current Debug-Function.
3] Use the Debug-Function to find the OLD-CONT and RETURN-PC.
4] Find the old ENV by searching up the stack for a saved code object
containing the RETURN-PC.
5] Assign old ENV to ENV, OLD-CONT to CONT, RETURN-PC to PC and goto 1.
If we changed the function representation so that the code and environment were
a single object, then the location of the old ENV would be simplified. But we
still need to represent ENV as separate from PC, since interrupts and errors
can happen when the current PC isn't positioned at a valid return PC.
It seems like it might be a good idea to save OLD-CONT, RETURN-PC and ENV at
the beginning of the frame (before any stack arguments). Then we wouldn't have
to search to locate ENV, and we also have a hope of parsing the stack even if
it is damaged. As long as we can locate the start of some frame, we can trace
the stack above that frame. We can recognize a probable frame start by
scanning the stack for a code object (presumably a saved ENV).
Probably we want some fairly general
mechanism for specifying that a TN should be considered to be live for the
duration of a specified environment. It would be somewhat easier to specify
that the TN is live for all time, but this would become very space-inefficient
in large block compilations.
This mechanism could be quite useful for other debugger-related things. For
example, when debuggability is important, we could make the TNs holding
arguments live for the entire environment. This would guarantee that a
backtrace would always get the right value (modulo setqs).
Note that in this context, "environment" means the Environment structure (one
per non-let function). At least according to current plans, even when we do
inter-routine register allocation, the different functions will have different
environments: we just "equate" the environments. So the number of live
per-environment TNs is bounded by the size of a "function", and doesn't blow up
in block compilation.
The implementation is simple: per-environment TNs are flagged by the
:Environment kind. :Environment TNs are treated the same as :Normal TNs by
everyone except for lifetime/conflict analysis. An environment's TNs are also
stashed in a list in the IR2-Environment structure. During during the conflict
analysis post-pass, we look at each block's environment, and make all the
environment's TNs always-live in that block.
We can implement the "fixed save location" concept needed for lazy frame
creation by allocating the save TNs as wired TNs at IR2 conversion time. We
would use the new "environment lifetime" concept to specify the lifetimes of
the save locations. There isn't any run-time overhead if we never get around
to using the save TNs. [Pack would also have to notice TNs with pre-allocated
save TNs, packing the original TN in the stack location if its FSC is the
stack.]
We want a standard (recognizable) format for an "escape" frame. We must make
an escape frame whenever we start running another function without the current
function getting a chance to save its registers. This may be due either to a
truly asynchronous event such as a software interrupt, or due to an "escape"
from a miscop. An escape frame marks a brief conversion to a callee-saves
convention.
Whenever a miscop saves registers, it should make an escape frame. This
ensures that the "current" register contents can always be located by the
debugger. In this case, it may be desirable to be able to indicate that only
partial saving has been done. For example, we don't want to have to save all
the FP registers just so that we can use a couple extra general registers.
When when the debugger see an escape frame, it knows that register values are
located in the escape frame's "register save" area, rather than in the normal
save locations.
It would be nice if there was a better solution to this internal error concept.
One problem is that it seems there is a substantial space penalty for emitting
all that error code, especially now that we don't share error code between
errors because we want to preserve the source context in the PC. But this
probably isn't really all that bad when considered as a fraction of the code.
For example, the check part of a type check is 12 bytes, whereas the error part
is usually only 6. In this case, we could never reduce the space overhead for
type checks by more than 1/3, thus the total code size reduction would be
small. This will be made even less important when we do type check
optimizations to reduce the number of type checks.
Probably we should stick to the same general internal error mechanism, but make
it interact with the debugger better by allocating linkage registers and
allowing proceedable errors. We could support shared error calls and
non-proceedable errors when space is more important than debuggability, but
this is probably more complexity than is worthwhile.
We jump or trap to a routine that saves the context (allocating at most the
return PC register). We then encode the error and context in the code
immediately following the jump/trap. (On the MIPS, the error code can be
encoded in the trap itself.) The error arguments would be encoded as
SC-offsets relative to the saved context. This could solve both the
arg-trashing problem and save space, since we could encode the SC-offsets more
tersely than the corresponding move instructions.

18
doc/cmucl/internals/design.tex Normal file
View file

@ -0,0 +1,18 @@
\documentstyle[cmu-titlepage]{report} % -*- Dictionary: design -*-
\title{Design of CMU Common Lisp}
\author{Robert A. MacLachlan (ed)}
\trnumber{CMU-CS-91-???}
\abstract{This report documents internal details of the CMU Common Lisp
compiler and run-time system. CMU Common Lisp is a public domain
implementation of Common Lisp that runs on various Unix workstations.}
\begin{document}
\maketitle
\tableofcontents
\include{architecture}
\include{compiler}
\include{retargeting}
\include{run-time}
\appendix
\include{glossary}
\end{document}

3
doc/cmucl/internals/environment.tex Normal file
View file

@ -0,0 +1,3 @@
\chapter{The Type System}
\chapter{The Info Database}

23
doc/cmucl/internals/errata-object Normal file
View file

@ -0,0 +1,23 @@
Look at primtype.lisp and objdef.lisp (and early-objdef.lisp) for more
up-to-date definitions of various tags. (For example, the simple
string tag has changed since object.tex was written.)
The string format has changed. According to "object.tex", string length is
stored in the 24 bits of the string header. Instead, those 24 bits
are set to zero, and string length is encoded in the same way as the
other specialized simple-array counts, as a fixnum following the
header.
The number of slots for objects has changed since object.tex was
written. The only reliable source for current slot definitions seems
to be the primitive object data maintained by the compiler itself. See
primtype.lisp and objdef.lisp, or look at the genesis code which reads
this data to generate the various slot offsets in the C header file.
The meaning of the function-self slot has changed in the X86 port:
it points directly to the code to be executed.
Nothing about FDEFN objects seems to be documented. FDEFN objects
replace the simple SYMBOL-FUNCTION slot with a much more complicated
mechanism, which I [WHN] dislike and would like to get rid of, but
haven't [yet?].

584
doc/cmucl/internals/fasl.tex Normal file
View file

@ -0,0 +1,584 @@
\chapter{Fasload File Format}% -*- Dictionary: design -*-
\section{General}
The purpose of Fasload files is to allow concise storage and rapid
loading of Lisp data, particularly function definitions. The intent
is that loading a Fasload file has the same effect as loading the
ASCII file from which the Fasload file was compiled, but accomplishes
the tasks more efficiently. One noticeable difference, of course, is
that function definitions may be in compiled form rather than
S-expression form. Another is that Fasload files may specify in what
parts of memory the Lisp data should be allocated. For example,
constant lists used by compiled code may be regarded as read-only.
In some Lisp implementations, Fasload file formats are designed to
allow sharing of code parts of the file, possibly by direct mapping
of pages of the file into the address space of a process. This
technique produces great performance improvements in a paged
time-sharing system. Since the Mach project is to produce a
distributed personal-computer network system rather than a
time-sharing system, efficiencies of this type are explicitly {\it not}
a goal for the CMU Common Lisp Fasload file format.
On the other hand, CMU Common Lisp is intended to be portable, as it will
eventually run on a variety of machines. Therefore an explicit goal
is that Fasload files shall be transportable among various
implementations, to permit efficient distribution of programs in
compiled form. The representations of data objects in Fasload files
shall be relatively independent of such considerations as word
length, number of type bits, and so on. If two implementations
interpret the same macrocode (compiled code format), then Fasload
files should be completely compatible. If they do not, then files
not containing compiled code (so-called "Fasdump" data files) should
still be compatible. While this may lead to a format which is not
maximally efficient for a particular implementation, the sacrifice of
a small amount of performance is deemed a worthwhile price to pay to
achieve portability.
The primary assumption about data format compatibility is that all
implementations can support I/O on finite streams of eight-bit bytes.
By "finite" we mean that a definite end-of-file point can be detected
irrespective of the content of the data stream. A Fasload file will
be regarded as such a byte stream.
\section{Strategy}
A Fasload file may be regarded as a human-readable prefix followed by
code in a funny little language. When interpreted, this code will
cause the construction of the encoded data structures. The virtual
machine which interprets this code has a {\it stack} and a {\it table},
both initially empty. The table may be thought of as an expandable
register file; it is used to remember quantities which are needed
more than once. The elements of both the stack and the table are
Lisp data objects. Operators of the funny language may take as
operands following bytes of the data stream, or items popped from the
stack. Results may be pushed back onto the stack or pushed onto the
table. The table is an indexable stack that is never popped; it is
indexed relative to the base, not the top, so that an item once
pushed always has the same index.
More precisely, a Fasload file has the following macroscopic
organization. It is a sequence of zero or more groups concatenated
together. End-of-file must occur at the end of the last group. Each
group begins with a series of seven-bit ASCII characters terminated
by one or more bytes of all ones \verb|#xFF|; this is called the
{\it header}. Following the bytes which terminate the header is the
{\it body}, a stream of bytes in the funny binary language. The body
of necessity begins with a byte other than \verb|#xFF|. The body is
terminated by the operation {\tt FOP-END-GROUP}.
The first nine characters of the header must be "{\tt FASL FILE}" in
upper-case letters. The rest may be any ASCII text, but by
convention it is formatted in a certain way. The header is divided
into lines, which are grouped into paragraphs. A paragraph begins
with a line which does {\it not} begin with a space or tab character,
and contains all lines up to, but not including, the next such line.
The first word of a paragraph, defined to be all characters up to but
not including the first space, tab, or end-of-line character, is the
{\it name} of the paragraph. A Fasload file header might look something like
this:
\begin{verbatim}
FASL FILE >SteelesPerq>User>Guy>IoHacks>Pretty-Print.Slisp
Package Pretty-Print
Compiled 31-Mar-1988 09:01:32 by some random luser
Compiler Version 1.6, Lisp Version 3.0.
Functions: INITIALIZE DRIVER HACK HACK1 MUNGE MUNGE1 GAZORCH
MINGLE MUDDLE PERTURB OVERDRIVE GOBBLE-KEYBOARD
FRY-USER DROP-DEAD HELP CLEAR-MICROCODE
%AOS-TRIANGLE %HARASS-READTABLE-MAYBE
Macros: PUSH POP FROB TWIDDLE
\end{verbatim}
{\it one or more bytes of \verb|#xFF|}
The particular paragraph names and contents shown here are only intended as
suggestions.
\section{Fasload Language}
Each operation in the binary Fasload language is an eight-bit
(one-byte) opcode. Each has a name beginning with "{\tt FOP-}". In
the following descriptions, the name is followed by operand
descriptors. Each descriptor denotes operands that follow the opcode
in the input stream. A quantity in parentheses indicates the number
of bytes of data from the stream making up the operand. Operands
which implicitly come from the stack are noted in the text. The
notation "$\Rightarrow$ stack" means that the result is pushed onto the
stack; "$\Rightarrow$ table" similarly means that the result is added to the
table. A construction like "{\it n}(1) {\it value}({\it n})" means that
first a single byte {\it n} is read from the input stream, and this
byte specifies how many bytes to read as the operand named {\it value}.
All numeric values are unsigned binary integers unless otherwise
specified. Values described as "signed" are in two's-complement form
unless otherwise specified. When an integer read from the stream
occupies more than one byte, the first byte read is the least
significant byte, and the last byte read is the most significant (and
contains the sign bit as its high-order bit if the entire integer is
signed).
Some of the operations are not necessary, but are rather special
cases of or combinations of others. These are included to reduce the
size of the file or to speed up important cases. As an example,
nearly all strings are less than 256 bytes long, and so a special
form of string operation might take a one-byte length rather than a
four-byte length. As another example, some implementations may
choose to store bits in an array in a left-to-right format within
each word, rather than right-to-left. The Fasload file format may
support both formats, with one being significantly more efficient
than the other for a given implementation. The compiler for any
implementation may generate the more efficient form for that
implementation, and yet compatibility can be maintained by requiring
all implementations to support both formats in Fasload files.
Measurements are to be made to determine which operation codes are
worthwhile; little-used operations may be discarded and new ones
added. After a point the definition will be "frozen", meaning that
existing operations may not be deleted (though new ones may be added;
some operations codes will be reserved for that purpose).
\begin{description}
\item[0:] \hspace{2em} {\tt FOP-NOP} \\
No operation. (This is included because it is recognized
that some implementations may benefit from alignment of operands to some
operations, for example to 32-bit boundaries. This operation can be used
to pad the instruction stream to a desired boundary.)
\item[1:] \hspace{2em} {\tt FOP-POP} \hspace{2em} $\Rightarrow$ \hspace{2em} table \\
One item is popped from the stack and added to the table.
\item[2:] \hspace{2em} {\tt FOP-PUSH} \hspace{2em} {\it index}(4) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
Item number {\it index} of the table is pushed onto the stack.
The first element of the table is item number zero.
\item[3:] \hspace{2em} {\tt FOP-BYTE-PUSH} \hspace{2em} {\it index}(1) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
Item number {\it index} of the table is pushed onto the stack.
The first element of the table is item number zero.
\item[4:] \hspace{2em} {\tt FOP-EMPTY-LIST} \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
The empty list ({\tt ()}) is pushed onto the stack.
\item[5:] \hspace{2em} {\tt FOP-TRUTH} \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
The standard truth value ({\tt T}) is pushed onto the stack.
\item[6:] \hspace{2em} {\tt FOP-SYMBOL-SAVE} \hspace{2em} {\it n}(4) \hspace{2em} {\it name}({\it n})
\hspace{2em} $\Rightarrow$ \hspace{2em} stack \& table\\
The four-byte operand {\it n} specifies the length of the print name
of a symbol. The name follows, one character per byte,
with the first byte of the print name being the first read.
The name is interned in the default package,
and the resulting symbol is both pushed onto the stack and added to the table.
\item[7:] \hspace{2em} {\tt FOP-SMALL-SYMBOL-SAVE} \hspace{2em} {\it n}(1) \hspace{2em} {\it name}({\it n}) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \& table\\
The one-byte operand {\it n} specifies the length of the print name
of a symbol. The name follows, one character per byte,
with the first byte of the print name being the first read.
The name is interned in the default package,
and the resulting symbol is both pushed onto the stack and added to the table.
\item[8:] \hspace{2em} {\tt FOP-SYMBOL-IN-PACKAGE-SAVE} \hspace{2em} {\it index}(4)
\hspace{2em} {\it n}(4) \hspace{2em} {\it name}({\it n})
\hspace{2em} $\Rightarrow$ \hspace{2em} stack \& table\\
The four-byte {\it index} specifies a package stored in the table.
The four-byte operand {\it n} specifies the length of the print name
of a symbol. The name follows, one character per byte,
with the first byte of the print name being the first read.
The name is interned in the specified package,
and the resulting symbol is both pushed onto the stack and added to the table.
\item[9:] \hspace{2em} {\tt FOP-SMALL-SYMBOL-IN-PACKAGE-SAVE} \hspace{2em} {\it index}(4)
\hspace{2em} {\it n}(1) \hspace{2em} {\it name}({\it n}) \hspace{2em}
$\Rightarrow$ \hspace{2em} stack \& table\\
The four-byte {\it index} specifies a package stored in the table.
The one-byte operand {\it n} specifies the length of the print name
of a symbol. The name follows, one character per byte,
with the first byte of the print name being the first read.
The name is interned in the specified package,
and the resulting symbol is both pushed onto the stack and added to the table.
\item[10:] \hspace{2em} {\tt FOP-SYMBOL-IN-BYTE-PACKAGE-SAVE} \hspace{2em} {\it index}(1)
\hspace{2em} {\it n}(4) \hspace{2em} {\it name}({\it n})
\hspace{2em} $\Rightarrow$ \hspace{2em} stack \& table\\
The one-byte {\it index} specifies a package stored in the table.
The four-byte operand {\it n} specifies the length of the print name
of a symbol. The name follows, one character per byte,
with the first byte of the print name being the first read.
The name is interned in the specified package,
and the resulting symbol is both pushed onto the stack and added to the table.
\item[11:]\hspace{2em} {\tt FOP-SMALL-SYMBOL-IN-BYTE-PACKAGE-SAVE} \hspace{2em} {\it index}(1)
\hspace{2em} {\it n}(1) \hspace{2em} {\it name}({\it n}) \hspace{2em}
$\Rightarrow$ \hspace{2em} stack \& table\\
The one-byte {\it index} specifies a package stored in the table.
The one-byte operand {\it n} specifies the length of the print name
of a symbol. The name follows, one character per byte,
with the first byte of the print name being the first read.
The name is interned in the specified package,
and the resulting symbol is both pushed onto the stack and added to the table.
\item[12:] \hspace{2em} {\tt FOP-UNINTERNED-SYMBOL-SAVE} \hspace{2em} {\it n}(4) \hspace{2em} {\it name}({\it n})
\hspace{2em} $\Rightarrow$ \hspace{2em} stack \& table\\
Like {\tt FOP-SYMBOL-SAVE}, except that it creates an uninterned symbol.
\item[13:] \hspace{2em} {\tt FOP-UNINTERNED-SMALL-SYMBOL-SAVE} \hspace{2em} {\it n}(1)
\hspace{2em} {\it name}({\it n}) \hspace{2em} $\Rightarrow$ \hspace{2em} stack
\& table\\
Like {\tt FOP-SMALL-SYMBOL-SAVE}, except that it creates an uninterned symbol.
\item[14:] \hspace{2em} {\tt FOP-PACKAGE} \hspace{2em} $\Rightarrow$ \hspace{2em} table \\
An item is popped from the stack; it must be a symbol. The package of
that name is located and pushed onto the table.
\item[15:] \hspace{2em} {\tt FOP-LIST} \hspace{2em} {\it length}(1) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
The unsigned operand {\it length} specifies a number of
operands to be popped from the stack. These are made into a list
of that length, and the list is pushed onto the stack.
The first item popped from the stack becomes the last element of
the list, and so on. Hence an iterative loop can start with
the empty list and perform "pop an item and cons it onto the list"
{\it length} times.
(Lists of length greater than 255 can be made by using {\tt FOP-LIST*}
repeatedly.)
\item[16:] \hspace{2em} {\tt FOP-LIST*} \hspace{2em} {\it length}(1) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
This is like {\tt FOP-LIST} except that the constructed list is terminated
not by {\tt ()} (the empty list), but by an item popped from the stack
before any others are. Therefore {\it length}+1 items are popped in all.
Hence an iterative loop can start with
a popped item and perform "pop an item and cons it onto the list"
{\it length}+1 times.
\item[17-24:] \hspace{2em} {\tt FOP-LIST-1}, {\tt FOP-LIST-2}, ..., {\tt FOP-LIST-8} \\
{\tt FOP-LIST-{\it k}} is like {\tt FOP-LIST} with a byte containing {\it k}
following it. These exist purely to reduce the size of Fasload files.
Measurements need to be made to determine the useful values of {\it k}.
\item[25-32:] \hspace{2em} {\tt FOP-LIST*-1}, {\tt FOP-LIST*-2}, ..., {\tt FOP-LIST*-8} \\
{\tt FOP-LIST*-{\it k}} is like {\tt FOP-LIST*} with a byte containing {\it k}
following it. These exist purely to reduce the size of Fasload files.
Measurements need to be made to determine the useful values of {\it k}.
\item[33:] \hspace{2em} {\tt FOP-INTEGER} \hspace{2em} {\it n}(4) \hspace{2em} {\it value}({\it n}) \hspace{2em}
$\Rightarrow$ \hspace{2em} stack \\
A four-byte unsigned operand specifies the number of following
bytes. These bytes define the value of a signed integer in two's-complement
form. The first byte of the value is the least significant byte.
\item[34:] \hspace{2em} {\tt FOP-SMALL-INTEGER} \hspace{2em} {\it n}(1) \hspace{2em} {\it value}({\it n})
\hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
A one-byte unsigned operand specifies the number of following
bytes. These bytes define the value of a signed integer in two's-complement
form. The first byte of the value is the least significant byte.
\item[35:] \hspace{2em} {\tt FOP-WORD-INTEGER} \hspace{2em} {\it value}(4) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
A four-byte signed integer (in the range $-2^{31}$ to $2^{31}-1$) follows the
operation code. A LISP integer (fixnum or bignum) with that value
is constructed and pushed onto the stack.
\item[36:] \hspace{2em} {\tt FOP-BYTE-INTEGER} \hspace{2em} {\it value}(1) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
A one-byte signed integer (in the range -128 to 127) follows the
operation code. A LISP integer (fixnum or bignum) with that value
is constructed and pushed onto the stack.
\item[37:] \hspace{2em} {\tt FOP-STRING} \hspace{2em} {\it n}(4) \hspace{2em} {\it name}({\it n})
\hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
The four-byte operand {\it n} specifies the length of a string to
construct. The characters of the string follow, one per byte.
The constructed string is pushed onto the stack.
\item[38:] \hspace{2em} {\tt FOP-SMALL-STRING} \hspace{2em} {\it n}(1) \hspace{2em} {\it name}({\it n}) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
The one-byte operand {\it n} specifies the length of a string to
construct. The characters of the string follow, one per byte.
The constructed string is pushed onto the stack.
\item[39:] \hspace{2em} {\tt FOP-VECTOR} \hspace{2em} {\it n}(4) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
The four-byte operand {\it n} specifies the length of a vector of LISP objects
to construct. The elements of the vector are popped off the stack;
the first one popped becomes the last element of the vector.
The constructed vector is pushed onto the stack.
\item[40:] \hspace{2em} {\tt FOP-SMALL-VECTOR} \hspace{2em} {\it n}(1) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
The one-byte operand {\it n} specifies the length of a vector of LISP objects
to construct. The elements of the vector are popped off the stack;
the first one popped becomes the last element of the vector.
The constructed vector is pushed onto the stack.
\item[41:] \hspace{2em} {\tt FOP-UNIFORM-VECTOR} \hspace{2em} {\it n}(4) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
The four-byte operand {\it n} specifies the length of a vector of LISP objects
to construct. A single item is popped from the stack and used to initialize
all elements of the vector. The constructed vector is pushed onto the stack.
\item[42:] \hspace{2em} {\tt FOP-SMALL-UNIFORM-VECTOR} \hspace{2em} {\it n}(1) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
The one-byte operand {\it n} specifies the length of a vector of LISP objects
to construct. A single item is popped from the stack and used to initialize
all elements of the vector. The constructed vector is pushed onto the stack.
\item[43:] \hspace{2em} {\tt FOP-INT-VECTOR} \hspace{2em} {\it len}(4) \hspace{2em}
{\it size}(1) \hspace{2em} {\it data}($\left\lceil len*count/8\right\rceil$)
\hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
The four-byte operand {\it n} specifies the length of a vector of
unsigned integers to be constructed. Each integer is {\it size}
bits long, and is packed according to the machine's native byte ordering.
{\it size} must be a directly supported i-vector element size. Currently
supported values are 1,2,4,8,16 and 32.
\item[44:] \hspace{2em} {\tt FOP-UNIFORM-INT-VECTOR} \hspace{2em} {\it n}(4) \hspace{2em} {\it size}(1) \hspace{2em}
{\it value}(@ceiling<{\it size}/8>) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
The four-byte operand {\it n} specifies the length of a vector of unsigned
integers to construct.
Each integer is {\it size} bits big, and is initialized to the value
of the operand {\it value}.
The constructed vector is pushed onto the stack.
\item[45:] Unused
\item[46:] \hspace{2em} {\tt FOP-SINGLE-FLOAT} \hspace{2em} {\it data}(4) \hspace{2em}
$\Rightarrow$ \hspace{2em} stack \\
The {\it data} bytes are read as an integer, then turned into an IEEE single
float (as though by {\tt make-single-float}).
\item[47:] \hspace{2em} {\tt FOP-DOUBLE-FLOAT} \hspace{2em} {\it data}(8) \hspace{2em}
$\Rightarrow$ \hspace{2em} stack \\
The {\it data} bytes are read as an integer, then turned into an IEEE double
float (as though by {\tt make-double-float}).
\item[48:] \hspace{2em} {\tt FOP-STRUCT} \hspace{2em} {\it n}(4) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
The four-byte operand {\it n} specifies the length structure to construct. The
elements of the vector are popped off the stack; the first one popped becomes
the last element of the structure. The constructed vector is pushed onto the
stack.
\item[49:] \hspace{2em} {\tt FOP-SMALL-STRUCT} \hspace{2em} {\it n}(1) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
The one-byte operand {\it n} specifies the length structure to construct. The
elements of the vector are popped off the stack; the first one popped becomes
the last element of the structure. The constructed vector is pushed onto the
stack.
\item[50-52:] Unused
\item[53:] \hspace{2em} {\tt FOP-EVAL} \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
Pop an item from the stack and evaluate it (give it to {\tt EVAL}).
Push the result back onto the stack.
\item[54:] \hspace{2em} {\tt FOP-EVAL-FOR-EFFECT} \\
Pop an item from the stack and evaluate it (give it to {\tt EVAL}).
The result is ignored.
\item[55:] \hspace{2em} {\tt FOP-FUNCALL} \hspace{2em} {\it nargs}(1) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
Pop {\it nargs}+1 items from the stack and apply the last one popped
as a function to
all the rest as arguments (the first one popped being the last argument).
Push the result back onto the stack.
\item[56:] \hspace{2em} {\tt FOP-FUNCALL-FOR-EFFECT} \hspace{2em} {\it nargs}(1) \\
Pop {\it nargs}+1 items from the stack and apply the last one popped
as a function to
all the rest as arguments (the first one popped being the last argument).
The result is ignored.
\item[57:] \hspace{2em} {\tt FOP-CODE-FORMAT} \hspace{2em} {\it implementation}(1)
\hspace{2em} {\it version}(1) \\
This FOP specifiers the code format for following code objects. The operations
{\tt FOP-CODE} and its relatives may not occur in a group until after {\tt
FOP-CODE-FORMAT} has appeared; there is no default format. The {\it
implementation} is an integer indicating the target hardware and environment.
See {\tt compiler/generic/vm-macs.lisp} for the currently defined
implementations. {\it version} for an implementation is increased whenever
there is a change that renders old fasl files unusable.
\item[58:] \hspace{2em} {\tt FOP-CODE} \hspace{2em} {\it nitems}(4) \hspace{2em} {\it size}(4) \hspace{2em}
{\it code}({\it size}) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
A compiled function is constructed and pushed onto the stack.
This object is in the format specified by the most recent
occurrence of {\tt FOP-CODE-FORMAT}.
The operand {\it nitems} specifies a number of items to pop off
the stack to use in the "boxed storage" section. The operand {\it code}
is a string of bytes constituting the compiled executable code.
\item[59:] \hspace{2em} {\tt FOP-SMALL-CODE} \hspace{2em} {\it nitems}(1) \hspace{2em} {\it size}(2) \hspace{2em}
{\it code}({\it size}) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
A compiled function is constructed and pushed onto the stack.
This object is in the format specified by the most recent
occurrence of {\tt FOP-CODE-FORMAT}.
The operand {\it nitems} specifies a number of items to pop off
the stack to use in the "boxed storage" section. The operand {\it code}
is a string of bytes constituting the compiled executable code.
\item[60-61:] Unused
\item[62:] \hspace{2em} {\tt FOP-VERIFY-TABLE-SIZE} \hspace{2em} {\it size}(4) \\
If the current size of the table is not equal to {\it size},
then an inconsistency has been detected. This operation
is inserted into a Fasload file purely for error-checking purposes.
It is good practice for a compiler to output this at least at the
end of every group, if not more often.
\item[63:] \hspace{2em} {\tt FOP-VERIFY-EMPTY-STACK} \\
If the stack is not currently empty,
then an inconsistency has been detected. This operation
is inserted into a Fasload file purely for error-checking purposes.
It is good practice for a compiler to output this at least at the
end of every group, if not more often.
\item[64:] \hspace{2em} {\tt FOP-END-GROUP} \\
This is the last operation of a group. If this is not the
last byte of the file, then a new group follows; the next
nine bytes must be "{\tt FASL FILE}".
\item[65:] \hspace{2em} {\tt FOP-POP-FOR-EFFECT} \hspace{2em} stack \hspace{2em} $\Rightarrow$ \hspace{2em} \\
One item is popped from the stack.
\item[66:] \hspace{2em} {\tt FOP-MISC-TRAP} \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
A trap object is pushed onto the stack.
\item[67:] Unused
\item[68:] \hspace{2em} {\tt FOP-CHARACTER} \hspace{2em} {\it character}(3) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
The three bytes are read as an integer then converted to a character. This FOP
is currently rather useless, as extended characters are not supported.
\item[69:] \hspace{2em} {\tt FOP-SHORT-CHARACTER} \hspace{2em} {\it character}(1) \hspace{2em}
$\Rightarrow$ \hspace{2em} stack \\
The one byte specifies the code of a Common Lisp character object. A character
is constructed and pushed onto the stack.
\item[70:] \hspace{2em} {\tt FOP-RATIO} \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
Creates a ratio from two integers popped from the stack.
The denominator is popped first, the numerator second.
\item[71:] \hspace{2em} {\tt FOP-COMPLEX} \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
Creates a complex number from two numbers popped from the stack.
The imaginary part is popped first, the real part second.
\item[72-73:] Unused
\item[74:] \hspace{2em} {\tt FOP-FSET} \hspace{2em} \\
Except in the cold loader (Genesis), this is a no-op with two stack arguments.
In the initial core this is used to make DEFUN functions defined at cold-load
time so that global functions can be called before top-level forms are run
(which normally installs definitions.) Genesis pops the top two things off of
the stack and effectively does (SETF SYMBOL-FUNCTION).
\item[75:] \hspace{2em} {\tt FOP-LISP-SYMBOL-SAVE} \hspace{2em} {\it n}(4) \hspace{2em} {\it name}({\it n})
\hspace{2em} $\Rightarrow$ \hspace{2em} stack \& table\\
Like {\tt FOP-SYMBOL-SAVE}, except that it creates a symbol in the LISP
package.
\item[76:] \hspace{2em} {\tt FOP-LISP-SMALL-SYMBOL-SAVE} \hspace{2em} {\it n}(1)
\hspace{2em} {\it name}({\it n}) \hspace{2em} $\Rightarrow$ \hspace{2em} stack
\& table\\
Like {\tt FOP-SMALL-SYMBOL-SAVE}, except that it creates a symbol in the LISP
package.
\item[77:] \hspace{2em} {\tt FOP-KEYWORD-SYMBOL-SAVE} \hspace{2em} {\it n}(4) \hspace{2em} {\it name}({\it n})
\hspace{2em} $\Rightarrow$ \hspace{2em} stack \& table\\
Like {\tt FOP-SYMBOL-SAVE}, except that it creates a symbol in the
KEYWORD package.
\item[78:] \hspace{2em} {\tt FOP-KEYWORD-SMALL-SYMBOL-SAVE} \hspace{2em} {\it n}(1)
\hspace{2em} {\it name}({\it n}) \hspace{2em} $\Rightarrow$ \hspace{2em} stack
\& table\\
Like {\tt FOP-SMALL-SYMBOL-SAVE}, except that it creates a symbol in the
KEYWORD package.
\item[79-80:] Unused
\item[81:] \hspace{2em} {\tt FOP-NORMAL-LOAD}\\
This FOP is used in conjunction with the cold loader (Genesis) to read
top-level package manipulation forms. These forms are to be read as though by
the normal loaded, so that they can be evaluated at cold load time, instead of
being dumped into the initial core image. A no-op in normal loading.
\item[82:] \hspace{2em} {\tt FOP-MAYBE-COLD-LOAD}\\
Undoes the effect of {\tt FOP-NORMAL-LOAD}.
\item[83:] \hspace{2em} {\tt FOP-ARRAY} \hspace{2em} {\it rank}(4)
\hspace{2em} $\Rightarrow$ \hspace{2em} stack\\
This operation creates a simple array header (used for simple-arrays with rank
/= 1). The data vector is popped off of the stack, and then {\it rank}
dimensions are popped off of the stack (the highest dimensions is on top.)
\item[84-139:] Unused
\item[140:] \hspace{2em} {\tt FOP-ALTER-CODE} \hspace{2em} {\it index}(4)\\
This operation modifies the constants part of a code object (necessary for
creating certain circular function references.) It pops the new value and code
object are off of the stack, storing the new value at the specified index.
\item[141:] \hspace{2em} {\tt FOP-BYTE-ALTER-CODE} \hspace{2em} {\it index}(1)\\
Like {\tt FOP-ALTER-CODE}, but has only a one byte offset.
\item[142:] \hspace{2em} {\tt FOP-FUNCTION-ENTRY} \hspace{2em} {\it index}(4)
\hspace{2em} $\Rightarrow$ \hspace{2em} stack\\
Initializes a function-entry header inside of a pre-existing code object, and
returns the corresponding function descriptor. {\it index} is the byte offset
inside of the code object where the header should be plunked down. The stack
arguments to this operation are the code object, function name, function debug
arglist and function type.
\item[143:] Unused
\item[144:] \hspace{2em} {\tt FOP-ASSEMBLER-CODE} \hspace{2em} {\it length}(4)
\hspace{2em} $\Rightarrow$ \hspace{2em} stack\\
This operation creates a code object holding assembly routines. {\it length}
bytes of code are read and placed in the code object, and the code object
descriptor is pushed on the stack. This FOP is only recognized by the cold
loader (Genesis.)
\item[145:] \hspace{2em} {\tt FOP-ASSEMBLER-ROUTINE} \hspace{2em} {\it offset}(4)
\hspace{2em} $\Rightarrow$ \hspace{2em} stack\\
This operation records an entry point into an assembler code object (for use
with {\tt FOP-ASSEMBLER-FIXUP}). The routine name (a symbol) is on stack top.
The code object is underneath. The entry point is defined at {\it offset}
bytes inside the code area of the code object, and the code object is left on
stack top (allowing multiple uses of this FOP to be chained.) This FOP is only
recognized by the cold loader (Genesis.)
\item[146:] Unused
\item[147:] \hspace{2em} {\tt FOP-FOREIGN-FIXUP} \hspace{2em} {\it len}(1)
\hspace{2em} {\it name}({\it len})
\hspace{2em} {\it offset}(4) \hspace{2em} $\Rightarrow$ \hspace{2em} stack\\
This operation resolves a reference to a foreign (C) symbol. {\it len} bytes
are read and interpreted as the symbol {\it name}. First the {\it kind} and the
code-object to patch are popped from the stack. The kind is a target-dependent
symbol indicating the instruction format of the patch target (at {\it offset}
bytes from the start of the code area.) The code object is left on
stack top (allowing multiple uses of this FOP to be chained.)
\item[148:] \hspace{2em} {\tt FOP-ASSEMBLER-FIXUP} \hspace{2em} {\it offset}(4)
\hspace{2em} $\Rightarrow$ \hspace{2em} stack\\
This operation resolves a reference to an assembler routine. The stack args
are ({\it routine-name}, {\it kind} and {\it code-object}). The kind is a
target-dependent symbol indicating the instruction format of the patch target
(at {\it offset} bytes from the start of the code area.) The code object is
left on stack top (allowing multiple uses of this FOP to be chained.)
\item[149-199:] Unused
\item[200:] \hspace{2em} {\tt FOP-RPLACA} \hspace{2em} {\it table-idx}(4)
\hspace{2em} {\it cdr-offset}(4)\\
\item[201:] \hspace{2em} {\tt FOP-RPLACD} \hspace{2em} {\it table-idx}(4)
\hspace{2em} {\it cdr-offset}(4)\\
These operations destructively modify a list entered in the table. {\it
table-idx} is the table entry holding the list, and {\it cdr-offset} designates
the cons in the list to modify (like the argument to {\tt nthcdr}.) The new
value is popped off of the stack, and stored in the {\tt car} or {\tt cdr},
respectively.
\item[202:] \hspace{2em} {\tt FOP-SVSET} \hspace{2em} {\it table-idx}(4)
\hspace{2em} {\it vector-idx}(4)\\
Destructively modifies a {\tt simple-vector} entered in the table. Pops the
new value off of the stack, and stores it in the {\it vector-idx} element of
the contents of the table entry {\it table-idx.}
\item[203:] \hspace{2em} {\tt FOP-NTHCDR} \hspace{2em} {\it cdr-offset}(4)
\hspace{2em} $\Rightarrow$ \hspace{2em} stack\\
Does {\tt nthcdr} on the top-of stack, leaving the result there.
\item[204:] \hspace{2em} {\tt FOP-STRUCTSET} \hspace{2em} {\it table-idx}(4)
\hspace{2em} {\it vector-idx}(4)\\
Like {\tt FOP-SVSET}, except it alters structure slots.
\item[255:] \hspace{2em} {\tt FOP-END-HEADER} \\ Indicates the end of a group header,
as described above.
\end{description}

943
doc/cmucl/internals/front.tex Normal file
View file

@ -0,0 +1,943 @@
\chapter{ICR conversion} % -*- Dictionary: design -*-
\section{Canonical forms}
\#|
Would be useful to have a Freeze-Type proclamation. Its primary use would to
be say that the indicated type won't acquire any new subtypes in the future.
This allows better open-coding of structure type predicates, since the possible
types that would satisfy the predicate will be constant at compile time, and
thus can be compiled as a skip-chain of EQ tests.
Of course, this is only a big win when the subtypes are few: the most important
case is when there are none. If the closure of the subtypes is much larger
than the average number of supertypes of an inferior, then it is better to grab
the list of superiors out of the object's type, and test for membership in that
list.
Should type-specific numeric equality be done by EQL rather than =? i.e.
should = on two fixnums become EQL and then convert to EQL/FIXNUM?
Currently we transform EQL into =, which is complicated, since we have to prove
the operands are the class of numeric type before we do it. Also, when EQL
sees one operand is a FIXNUM, it transforms to EQ, but the generator for EQ
isn't expecting numbers, so it doesn't use an immediate compare.
Array hackery:
Array type tests are transformed to %array-typep, separation of the
implementation-dependent array-type handling. This way we can transform
STRINGP to:
(or (simple-string-p x)
(and (complex-array-p x)
(= (array-rank x) 1)
(simple-string-p (%array-data x))))
In addition to the similar bit-vector-p, we also handle vectorp and any type
tests on which the dimension isn't wild.
[Note that we will want to expand into frobs compatible with those that
array references expand into so that the same optimizations will work on both.]
These changes combine to convert hairy type checks into hairy typep's, and then
convert hairyp typeps into simple typeps.
Do we really need non-VOP templates? It seems that we could get the desired
effect through implementation-dependent ICR transforms. The main risk would be
of obscuring the type semantics of the code. We could fairly easily retain all
the type information present at the time the tranform is run, but if we
discover new type information, then it won't be propagated unless the VM also
supplies type inference methods for its internal frobs (precluding the use of
%PRIMITIVE, since primitives don't have derive-type methods.)
I guess one possibility would be to have the call still considered "known" even
though it has been transformed. But this doesn't work, since we start doing
LET optimizations that trash the arglist once the call has been transformed
(and indeed we want to.)
Actually, I guess the overhead for providing type inference methods for the
internal frobs isn't that great, since we can usually borrow the inference
method for a Common Lisp function. For example, in our AREF case:
(aref x y)
==>
(let ((\#:len (array-dimension x 0)))
(%unchecked-aref x (%check-in-bounds y \#:len)))
Now in this case, if we made %UNCHECKED-AREF have the same derive-type method
as AREF, then if we discovered something new about X's element type, we could
derive a new type for the entire expression.
Actually, it seems that baring this detail at the ICR level is beneficial,
since it admits the possibly of optimizing away the bounds check using type
information. If we discover X's dimensions, then \#:LEN becomes a constant that
can be substituted. Then %CHECK-IN-BOUNDS can notice that the bound is
constant and check it against the type for Y. If Y is known to be in range,
then we can optimize away the bounds check.
Actually in this particular case, the best thing to do would be if we
discovered the bound is constant, then replace the bounds check with an
implicit type check. This way all the type check optimization mechanisms would
be brought into the act.
So we actually want to do the bounds-check expansion as soon as possible,
rather than later than possible: it should be a source-transform, enabled by
the fast-safe policy.
With multi-dimensional arrays we probably want to explicitly do the index
computation: this way portions of the index computation can become loop
invariants. In a scan in row-major order, the inner loop wouldn't have to do
any multiplication: it would only do an addition. We would use normal
fixnum arithmetic, counting on * to cleverly handle multiplication by a
constant, and appropriate inline expansion.
Note that in a source transform, we can't make any assumptions the type of the
array. If it turns out to be a complex array without declared dimensions, then
the calls to ARRAY-DIMENSION will have to turn into a VOP that can be affected.
But if it is simple, then the VOP is unaffected, and if we know the bounds, it
is constant. Similarly, we would have %ARRAY-DATA and %ARRAY-DISPLACEMENT
operations. %ARRAY-DISPLACEMENT would optimize to 0 if we discover the array
is simple. [This is somewhat inefficient when the array isn't eventually
discovered to be simple, since finding the data and finding the displacement
duplicate each other. We could make %ARRAY-DATA return both as MVs, and then
optimize to (VALUES (%SIMPLE-ARRAY-DATA x) 0), but this would require
optimization of trivial VALUES uses.]
Also need (THE (ARRAY * * * ...) x) to assert correct rank.
|\#
A bunch of functions have source transforms that convert them into the
canonical form that later parts of the compiler want to see. It is not legal
to rely on the canonical form since source transforms can be inhibited by a
Notinline declaration. This shouldn't be a problem, since everyone should keep
their hands off of Notinline calls.
Some transformations:
Endp ==> (NULL (THE LIST ...))
(NOT xxx) or (NULL xxx) => (IF xxx NIL T)
(typep x '<simple type>) => (<simple predicate> x)
(typep x '<complex type>) => ...composition of simpler operations...
TYPEP of AND, OR and NOT types turned into conditionals over multiple TYPEP
calls. This makes hairy TYPEP calls more digestible to type constraint
propagation, and also means that the TYPEP code generators don't have to deal
with these cases. [\#\#\# In the case of union types we may want to do something
to preserve information for type constraint propagation.]
(apply \#'foo a b c)
==>
(multiple-value-call \#'foo (values a) (values b) (values-list c))
This way only MV-CALL needs to know how to do calls with unknown numbers of
arguments. It should be nearly as efficient as a special-case VMR-Convert
method could be.
Make-String => Make-Array
N-arg predicates associated into two-arg versions.
Associate N-arg arithmetic ops.
Expand CxxxR and FIRST...nTH
Zerop, Plusp, Minusp, 1+, 1-, Min, Max, Rem, Mod
(Values x), (Identity x) => (Prog1 x)
All specialized aref functions => (aref (the xxx) ...)
Convert (ldb (byte ...) ...) into internal frob that takes size and position as
separate args. Other byte functions also...
Change for-value primitive predicates into (if <pred> t nil). This isn't
particularly useful during ICR phases, but makes life easy for VMR conversion.
This last can't be a source transformation, since a source transform can't tell
where the form appears. Instead, ICR conversion special-cases calls to known
functions with the Predicate attribute by doing the conversion when the
destination of the result isn't an IF. It isn't critical that this never be
done for predicates that we ultimately discover to deliver their value to an
IF, since IF optimizations will flush unnecessary IFs in a predicate.
\section{Inline functions}
[\#\#\# Inline expansion is especially powerful in the presence of good lisp-level
optimization ("partial evaluation"). Many "optimizations" usually done in Lisp
compilers by special-case source-to-source transforms can be had simply by
making the source of the general case function available for inline expansion.
This is especially helpful in Common Lisp, which has many commonly used
functions with simple special cases but bad general cases (list and sequence
functions, for example.)
Inline expansion of recursive functions is allowed, and is not as silly as it
sounds. When expanded in a specific context, much of the overhead of the
recursive calls may be eliminated (especially if there are many keyword
arguments, etc.)
[Also have MAYBE-INLINE]
]
We only record a function's inline expansion in the global environment when the
function is in the null lexical environment, since it the expansion must be
represented as source.
We do inline expansion of functions locally defined by FLET or LABELS even when
the environment is not null. Since the appearances of the local function must
be nested within the desired environment, it is possible to expand local
functions inline even when they use the environment. We just stash the source
form and environments in the Functional for the local function. When we
convert a call to it, we just reconvert the source in the saved environment.
An interesting alternative to the inline/full-call dichotomy is "semi-inline"
coding. Whenever we have an inline expansion for a function, we can expand it
only once per block compilation, and then use local call to call this copied
version. This should get most of the speed advantage of real inline coding
with much less code bloat. This is especially attractive for simple system
functions such as Read-Char.
The main place where true inline expansion would still be worth doing is where
large amounts of the function could be optimized away by constant folding or
other optimizations that depend on the exact arguments to the call.
\section{Compilation policy}
We want more sophisticated control of compilation safety than is offered in CL,
so that we can emit only those type checks that are likely to discover
something (i.e. external interfaces.)
\#|
\section{Notes}
Generalized back-end notion provides dynamic retargeting? (for byte code)
The current node type annotations seem to be somewhat unsatisfactory, since we
lose information when we do a THE on a continuation that already has uses, or
when we convert a let where the actual result continuation has other uses.
But the case with THE isn't really all that bad, since the test of whether
there are any uses happens before conversion of the argument, thus THE loses
information only when there are uses outside of the declared form. The LET
case may not be a big deal either.
Note also that losing user assertions isn't really all that bad, since it won't
damage system integrity. At worst, it will cause a bug to go undetected. More
likely, it will just cause the error to be signaled in a different place (and
possibly in a less informative way). Of course, there is an efficiency hit for
losing type information, but if it only happens in strange cases, then this
isn't a big deal.
\chapter{Local call analysis}
All calls to local functions (known named functions and LETs) are resolved to
the exact LAMBDA node which is to be called. If the call is syntactically
illegal, then we emit a warning and mark the reference as :notinline, forcing
the call to be a full call. We don't even think about converting APPLY calls;
APPLY is not special-cased at all in ICR. We also take care not to convert
calls in the top-level component, which would join it to normal code. Calls to
functions with rest args and calls with non-constant keywords are also not
converted.
We also convert MV-Calls that look like MULTIPLE-VALUE-BIND to local calls,
since we know that they can be open-coded. We replace the optional dispatch
with a call to the last optional entry point, letting MV-Call magically default
the unsupplied values to NIL.
When ICR optimizations discover a possible new local call, they explicitly
invoke local call analysis on the code that needs to be reanalyzed.
[\#\#\# Let conversion. What is means to be a let. Argument type checking done
by caller. Significance of local call is that all callers are known, so
special call conventions may be used.]
A lambda called in only one place is called a "let" call, since a Let would
turn into one.
In addition to enabling various ICR optimizations, the let/non-let distinction
has important environment significance. We treat the code in function and all
of the lets called by that function as being in the same environment. This
allows exits from lets to be treated as local exits, and makes life easy for
environment analysis.
Since we will let-convert any function with only one call, we must be careful
about cleanups. It is possible that a lexical exit from the let function may
have to clean up dynamic bindings not lexically apparent at the exit point. We
handle this by annotating lets with any cleanup in effect at the call site.
The cleanup for continuations with no immediately enclosing cleanup is the
lambda that the continuation is in. In this case, we look at the lambda to see
if any cleanups need to be done.
Let conversion is disabled for entry-point functions, since otherwise we might
convert the call from the XEP to the entry point into a let. Then later on, we
might want to convert a non-local reference into a local call, and not be able
to, since once a function has been converted to a let, we can't convert it
back.
A function's return node may also be deleted if it is unreachable, which can
happen if the function never returns normally. Such functions are not lets.
\chapter{Find components}
This is a post-pass to ICR conversion that massages the flow graph into the
shape subsequent phases expect. Things done:
Compute the depth-first ordering for the flow graph.
Find the components (disconnected parts) of the flow graph.
This pass need only be redone when newly converted code has been added to the
flow graph. The reanalyze flag in the component structure should be set by
people who mess things up.
We create the initial DFO using a variant of the basic algorithm. The initial
DFO computation breaks the ICR up into components, which are parts that can be
compiled independently. This is done to increase the efficiency of large block
compilations. In addition to improving locality of reference and reducing the
size of flow analysis problems, this allows back-end data structures to be
reclaimed after the compilation of each component.
ICR optimization can change the connectivity of the flow graph by discovering
new calls or eliminating dead code. Initial DFO determination splits up the
flow graph into separate components, but does so conservatively, ensuring that
parts that might become joined (due to local call conversion) are joined from
the start. Initial DFO computation also guarantees that all code which shares
a lexical environment is in the same component so that environment analysis
needs to operate only on a single component at a time.
[This can get a bit hairy, since code seemingly reachable from the
environment entry may be reachable from a NLX into that environment. Also,
function references must be considered as links joining components even though
the flow graph doesn't represent these.]
After initial DFO determination, components are neither split nor joined. The
standard DFO computation doesn't attempt to split components that have been
disconnected.
\chapter{ICR optimize}
{\bf Somewhere describe basic ICR utilities: continuation-type,
constant-continuation-p, etc. Perhaps group by type in ICR description?}
We are conservative about doing variable-for-variable substitution in ICR
optimization, since if we substitute a variable with a less restrictive type,
then we may prevent use of a "good" representation within the scope of the
inner binding.
Note that variable-variable substitutions aren't really crucial in ICR, since
they don't create opportunities for new optimizations (unlike substitution of
constants and functions). A spurious variable-variable binding will show up as
a Move operation in VMR. This can be optimized away by reaching-definitions
and also by targeting. [\#\#\# But actually, some optimizers do see if operands
are the same variable.]
\#|
The IF-IF optimization can be modeled as a value driven optimization, since
adding a use definitely is cause for marking the continuation for
reoptimization. [When do we add uses? Let conversion is the only obvious
time.] I guess IF-IF conversion could also be triggered by a non-immediate use
of the test continuation becoming immediate, but to allow this to happen would
require Delete-Block (or somebody) to mark block-starts as needing to be
reoptimized when a predecessor changes. It's not clear how important it is
that IF-IF conversion happen under all possible circumstances, as long as it
happens to the obvious cases.
[\#\#\# It isn't totally true that code flushing never enables other worthwhile
optimizations. Deleting a functional reference can cause a function to cease
being an XEP, or even trigger let conversion. It seems we still want to flush
code during ICR optimize, but maybe we want to interleave it more intimately
with the optimization pass.
Ref-flushing works just as well forward as backward, so it could be done in the
forward pass. Call flushing doesn't work so well, but we could scan the block
backward looking for any new flushable stuff if we flushed a call on the
forward pass.
When we delete a variable due to lack of references, we leave the variable
in the lambda-list so that positional references still work. The initial value
continuation is flushed, though (replaced with NIL) allowing the initial value
for to be deleted (modulo side-effects.)
Note that we can delete vars with no refs even when they have sets. I guess
when there are no refs, we should also flush all sets, allowing the value
expressions to be flushed as well.
Squeeze out single-reference unset let variables by changing the dest of the
initial value continuation to be the node that receives the ref. This can be
done regardless of what the initial value form is, since we aren't actually
moving the evaluation. Instead, we are in effect using the continuation's
locations in place of the temporary variable.
Doing this is of course, a wild violation of stack discipline, since the ref
might be inside a loop, etc. But with the VMR back-end, we only need to
preserve stack discipline for unknown-value continuations; this ICR
transformation must be already be inhibited when the DEST of the REF is a
multiple-values receiver (EXIT, RETURN or MV-COMBINATION), since we must
preserve the single-value semantics of the let-binding in this case.
The REF and variable must be deleted as part of this operation, since the ICR
would otherwise be left in an inconsistent state; we can't wait for the REF to
be deleted due to bing unused, since we have grabbed the arg continuation and
substituted it into the old DEST.
The big reason for doing this transformation is that in macros such as INCF and
PSETQ, temporaries are squeezed out, and the new value expression is evaluated
directly to the setter, allowing any result type assertion to be applied to the
expression evaluation. Unlike in the case of substitution, there is no point
in inhibiting this transformation when the initial value type is weaker than
the variable type. Instead, we intersect the asserted type for the old REF's
CONT with the type assertion on the initial value continuation. Note that the
variable's type has already been asserted on the initial-value continuation.
Of course, this transformation also simplifies the ICR even when it doesn't
discover interesting type assertions, so it makes sense to do it whenever
possible. This reduces the demands placed on register allocation, etc.
|\#
There are three dead-code flushing rules:
1] Refs with no DEST may be flushed.
2] Known calls with no dest that are flushable may be flushed. We null the
DEST in all the args.
3] If a lambda-var has no refs, then it may be deleted. The flushed argument
continuations have their DEST nulled.
These optimizations all enable one another. We scan blocks backward, looking
for nodes whose CONT has no DEST, then type-dispatching off of the node. If we
delete a ref, then we check to see if it is a lambda-var with no refs. When we
flush an argument, we mark the blocks for all uses of the CONT as needing to be
reoptimized.
\section{Goals for ICR optimizations}
\#|
When an optimization is disabled, code should still be correct and not
ridiculously inefficient. Phases shouldn't be made mandatory when they have
lots of non-required stuff jammed into them.
|\#
This pass is optional, but is desirable if anything is more important than
compilation speed.
This phase is a grab-bag of optimizations that concern themselves with the flow
of values through the code representation. The main things done are type
inference, constant folding and dead expression elimination. This phase can be
understood as a walk of the expression tree that propagates assertions down the
tree and propagates derived information up the tree. The main complication is
that there isn't any expression tree, since ICR is flow-graph based.
We repeat this pass until we don't discover anything new. This is a bit of
feat, since we dispatch to arbitrary functions which may do arbitrary things,
making it hard to tell if anything really happened. Even if we solve this
problem by requiring people to flag when they changed or by checking to see if
they changed something, there are serious efficiency problems due to massive
redundant computation, since in many cases the only way to tell if anything
changed is to recompute the value and see if it is different from the old one.
We solve this problem by requiring that optimizations for a node only depend on
the properties of the CONT and the continuations that have the node as their
DEST. If the continuations haven't changed since the last pass, then we don't
attempt to re-optimize the node, since we know nothing interesting will happen.
We keep track of which continuations have changed by a REOPTIMIZE flag that is
set whenever something about the continuation's value changes.
When doing the bottom up pass, we dispatch to type specific code that knows how
to tell when a node needs to be reoptimized and does the optimization. These
node types are special-cased: COMBINATION, IF, RETURN, EXIT, SET.
The REOPTIMIZE flag in the COMBINATION-FUN is used to detect when the function
information might have changed, so that we know when where are new assertions
that could be propagated from the function type to the arguments.
When we discover something about a leaf, or substitute for leaf, we reoptimize
the CONT for all the REF and SET nodes.
We have flags in each block that indicate when any nodes or continuations in
the block need to be re-optimized, so we don't have to scan blocks where there
is no chance of anything happening.
It is important for efficiency purposes that optimizers never say that they did
something when they didn't, but this by itself doesn't guarantee timely
termination. I believe that with the type system implemented, type inference
will converge in finite time, but as a practical matter, it can take far too
long to discover not much. For this reason, ICR optimization is terminated
after three consecutive passes that don't add or delete code. This premature
termination only happens 2% of the time.
\section{Flow graph simplification}
Things done:
Delete blocks with no predecessors.
Merge blocks that can be merged.
Convert local calls to Let calls.
Eliminate degenerate IFs.
We take care not to merge blocks that are in different functions or have
different cleanups. This guarantees that non-local exits are always at block
ends and that cleanup code never needs to be inserted within a block.
We eliminate IFs with identical consequent and alternative. This would most
likely happen if both the consequent and alternative were optimized away.
[Could also be done if the consequent and alternative were different blocks,
but computed the same value. This could be done by a sort of cross-jumping
optimization that looked at the predecessors for a block and merged code shared
between predecessors. IFs with identical branches would eventually be left
with nothing in their branches.]
We eliminate IF-IF constructs:
(IF (IF A B C) D E) ==>
(IF A (IF B D E) (IF C D E))
In reality, what we do is replicate blocks containing only an IF node where the
predicate continuation is the block start. We make one copy of the IF node for
each use, leaving the consequent and alternative the same. If you look at the
flow graph representation, you will see that this is really the same thing as
the above source to source transformation.
\section{Forward ICR optimizations}
In the forward pass, we scan the code in forward depth-first order. We
examine each call to a known function, and:
\begin{itemize}
\item Eliminate any bindings for unused variables.
\item Do top-down type assertion propagation. In local calls, we propagate
asserted and derived types between the call and the called lambda.
\item
Replace calls of foldable functions with constant arguments with the
result. We don't have to actually delete the call node, since Top-Down
optimize will delete it now that its value is unused.
\item
Run any Optimizer for the current function. The optimizer does arbitrary
transformations by hacking directly on the IR. This is useful primarily
for arithmetic simplification and similar things that may need to examine
and modify calls other than the current call. The optimizer is responsible
for recording any changes that it makes. An optimizer can inhibit further
optimization of the node during the current pass by returning true. This
is useful when deleting the node.
\item
Do ICR transformations, replacing a global function call with equivalent
inline lisp code.
\item
Do bottom-up type propagation/inferencing. For some functions such as
Coerce we will dispatch to a function to find the result type. The
Derive-Type function just returns a type structure, and we check if it is
different from the old type in order to see if there was a change.
\item
Eliminate IFs with predicates known to be true or false.
\item
Substitute the value for unset let variables that are bound to constants,
unset lambda variables or functionals.
\item
Propagate types from local call args to var refs.
\end{itemize}
We use type info from the function continuation to find result types for
functions that don't have a derive-type method.
ICR transformation:
ICR transformation does "source to source" transformations on known global
functions, taking advantage of semantic information such as argument types and
constant arguments. Transformation is optional, but should be done if speed or
space is more important than compilation speed. Transformations which increase
space should pass when space is more important than speed.
A transform is actually an inline function call where the function is computed
at compile time. The transform gets to peek at the continuations for the
arguments, and computes a function using the information gained. Transforms
should be cautious about directly using the values of constant continuations,
since the compiler must preserve eqlness of named constants, and it will have a
hard time if transforms go around randomly copying constants.
The lambda that the transform computes replaces the original function variable
reference as the function for the call. This lets the compiler worry about
evaluating each argument once in the right order. We want to be careful to
preserve type information when we do a transform, since it may be less than
obvious what the transformed code does.
There can be any number of transforms for a function. Each transform is
associated with a function type that the call must be compatible with. A
transform is only invoked if the call has the right type. This provides a way
to deal with the common case of a transform that only applies when the
arguments are of certain types and some arguments are not specified. We always
use the derived type when determining whether a transform is applicable. Type
check is responsible for setting the derived type to the intersection of the
asserted and derived types.
If the code in the expansion has insufficient explicit or implicit argument
type checking, then it should cause checks to be generated by making
declarations.
A transformation may decide to pass if it doesn't like what it sees when it
looks at the args. The Give-Up function unwinds out of the transform and deals
with complaining about inefficiency if speed is more important than brevity.
The format args for the message are arguments to Give-Up. If a transform can't
be done, we just record the message where ICR finalize can find it. note. We
can't complain immediately, since it might get transformed later on.
\section{Backward ICR optimizations}
In the backward pass, we scan each block in reverse order, and
eliminate any effectless nodes with unused values. In ICR this is the
only way that code is deleted other than the elimination of unreachable blocks.
\chapter{Type checking}
[\#\#\# Somehow split this section up into three parts:
-- Conceptual: how we know a check is necessary, and who is responsible for
doing checks.
-- Incremental: intersection of derived and asserted types, checking for
non-subtype relationship.
-- Check generation phase.
]
We need to do a pretty good job of guessing when a type check will ultimately
need to be done. Generic arithmetic, for example: In the absence of
declarations, we will use use the safe variant, but if we don't know this, we
will generate a check for NUMBER anyway. We need to look at the fast-safe
templates and guess if any of them could apply.
We compute a function type from the VOP arguments
and assertions on those arguments. This can be used with Valid-Function-Use
to see which templates do or might apply to a particular call. If we guess
that a safe implementation will be used, then we mark the continuation so as to
force a safe implementation to be chosen. [This will happen if ICR optimize
doesn't run to completion, so the icr optimization after type check generation
can discover new type information. Since we won't redo type check at that
point, there could be a call that has applicable unsafe templates, but isn't
type checkable.]
[\#\#\# A better and more general optimization of structure type checks: in type
check conversion, we look at the *original derived* type of the continuation:
if the difference between the proven type and the asserted type is a simple
type check, then check for the negation of the difference. e.g. if we want a
FOO and we know we've got (OR FOO NULL), then test for (NOT NULL). This is a
very important optimization for linked lists of structures, but can also apply
in other situations.]
If after ICR phases, we have a continuation with check-type set in a context
where it seems likely a check will be emitted, and the type is too
hairy to be easily checked (i.e. no CHECK-xxx VOP), then we do a transformation
on the ICR equivalent to:
(... (the hair <foo>) ...)
==>
(... (funcall \#'(lambda (\#:val)
(if (typep \#:val 'hair)
\#:val
(%type-check-error \#:val 'hair)))
<foo>)
...)
This way, we guarantee that VMR conversion never has to emit type checks for
hairy types.
[Actually, we need to do a MV-bind and several type checks when there is a MV
continuation. And some values types are just too hairy to check. We really
can't check any assertion for a non-fixed number of values, since there isn't
any efficient way to bind arbitrary numbers of values. (could be done with
MV-call of a more-arg function, I guess...)
]
[Perhaps only use CHECK-xxx VOPs for types equivalent to a ptype? Exceptions
for CONS and SYMBOL? Anyway, no point in going to trouble to implement and
emit rarely used CHECK-xxx vops.]
One potential lose in converting a type check to explicit conditionals rather
than to a CHECK-xxx VOP is that VMR code motion optimizations won't be able to
do anything. This shouldn't be much of an issue, though, since type constraint
propagation has already done global optimization of type checks.
This phase is optional, but should be done if anything is more important than
compile speed.
Type check is responsible for reconciling the continuation asserted and derived
types, emitting type checks if appropriate. If the derived type is a subtype
of the asserted type, then we don't need to do anything.
If there is no intersection between the asserted and derived types, then there
is a manifest type error. We print a warning message, indicating that
something is almost surely wrong. This will inhibit any transforms or
generators that care about their argument types, yet also inhibits further
error messages, since NIL is a subtype of every type.
If the intersection is not null, then we set the derived type to the
intersection of the asserted and derived types and set the Type-Check flag in
the continuation. We always set the flag when we can't prove that the type
assertion is satisfied, regardless of whether we will ultimately actually emit
a type check or not. This is so other phases such as type constraint
propagation can use the Type-Check flag to detect an interesting type
assertion, instead of having to duplicate much of the work in this phase.
[\#\#\# 7 extremely random values for CONTINUATION-TYPE-CHECK.]
Type checks are generated on the fly during VMR conversion. When VMR
conversion generates the check, it prints an efficiency note if speed is
important. We don't flame now since type constraint progpagation may decide
that the check is unnecessary. [\#\#\# Not done now, maybe never.]
In local function call, it is the caller that is in effect responsible for
checking argument types. This happens in the same way as any other type check,
since ICR optimize propagates the declared argument types to the type
assertions for the argument continuations in all the calls.
Since the types of arguments to entry points are unknown at compile time, we
want to do runtime checks to ensure that the incoming arguments are of the
correct type. This happens without any special effort on the part of type
check, since the XEP is represented as a local call with unknown type
arguments. These arguments will be marked as needing to be checked.
\chapter{Constraint propagation}
\#|
New lambda-var-slot:
constraints: a list of all the constraints on this var for either X or Y.
How to maintain consistency? Does it really matter if there are constraints
with deleted vars lying around? Note that whatever mechanism we use for
getting the constraints in the first place should tend to keep them up to date.
Probably we would define optimizers for the interesting relations that look at
their CONT's dest and annotate it if it is an IF.
But maybe it is more trouble then it is worth trying to build up the set of
constraints during ICR optimize (maintaining consistency in the process).
Since ICR optimize iterates a bunch of times before it converges, we would be
wasting time recomputing the constraints, when nobody uses them till constraint
propagation runs.
It seems that the only possible win is if we re-ran constraint propagation
(which we might want to do.) In that case, we wouldn't have to recompute all
the constraints from scratch. But it seems that we could do this just as well
by having ICR optimize invalidate the affected parts of the constraint
annotation, rather than trying to keep them up to date. This also fits better
with the optional nature of constraint propagation, since we don't want ICR
optimize to commit to doing a lot of the work of constraint propagation.
For example, we might have a per-block flag indicating that something happened
in that block since the last time constraint propagation ran. We might have
different flags to represent the distinction between discovering a new type
assertion inside the block and discovering something new about an if
predicate, since the latter would be cheaper to update and probably is more
common.
It's fairly easy to see how we can build these sets of restrictions and
propagate them using flow analysis, but actually using this information seems
a bit more ad-hoc.
Probably the biggest thing we do is look at all the refs. If have proven that
the value is EQ (EQL for a number) to some other leaf (constant or lambda-var),
then we can substitute for that reference. In some cases, we will want to do
special stuff depending on the DEST. If the dest is an IF and we proved (not
null), then we can substitute T. And if the dest is some relation on the same
two lambda-vars, then we want to see if we can show that relation is definitely
true or false.
Otherwise, we can do our best to invert the set of restrictions into a type.
Since types hold only constant info, we have to ignore any constraints between
two vars. We can make some use of negated type restrictions by using
TYPE-DIFFERENCE to remove the type from the ref types. If our inferred type is
as good as the type assertion, then the continuation's type-check flag will be
cleared.
It really isn't much of a problem that we don't infer union types on joins,
since union types are relatively easy to derive without using flow information.
The normal bottom-up type inference done by ICR optimize does this for us: it
annotates everything with the union of all of the things it might possibly be.
Then constraint propagation subtracts out those types that can't be in effect
because of predicates or checks.
This phase is optional, but is desirable if anything is more important than
compilation speed. We use an algorithm similar to available expressions to
propagate variable type information that has been discovered by implicit or
explicit type tests, or by type inference.
We must do a pre-pass which locates set closure variables, since we cannot do
flow analysis on such variables. We set a flag in each set closure variable so
that we can quickly tell that it is losing when we see it again. Although this
may seem to be wastefully redundant with environment analysis, the overlap
isn't really that great, and the cost should be small compared to that of the
flow analysis that we are preparing to do. [Or we could punt on set
variables...]
A type constraint is a structure that includes sset-element and has the type
and variable.
[\#\#\# Also a not-p flag indicating whether the sense is negated.]
Each variable has a list of its type constraints. We create a
type constraint when we see a type test or check. If there is already a
constraint for the same variable and type, then we just re-use it. If there is
already a weaker constraint, then we generate both the weak constraints and the
strong constraint so that the weak constraints won't be lost even if the strong
one is unavailable.
We find all the distinct type constraints for each variable during the pre-pass
over the lambda nesting. Each constraint has a list of the weaker constraints
so that we can easily generate them.
Every block generates all the type constraints in it, but a constraint is
available in a successor only if it is available in all predecessors. We
determine the actual type constraint for a variable at a block by intersecting
all the available type constraints for that variable.
This isn't maximally tense when there are constraints that are not
hierarchically related, e.g. (or a b) (or b c). If these constraints were
available from two predecessors, then we could infer that we have an (or a b c)
constraint, but the above algorithm would come up with none. This probably
isn't a big problem.
[\#\#\# Do we want to deal with (if (eq <var> '<foo>) ...) indicating singleton
member type?]
We detect explicit type tests by looking at type test annotation in the IF
node. If there is a type check, the OUT sets are stored in the node, with
different sets for the consequent and alternative. Implicit type checks are
located by finding Ref nodes whose Cont has the Type-Check flag set. We don't
actually represent the GEN sets, we just initialize OUT to it, and then form
the union in place.
When we do the post-pass, we clear the Type-Check flags in the continuations
for Refs when we discover that the available constraints satisfy the asserted
type. Any explicit uses of typep should be cleaned up by the ICR optimizer for
typep. We can also set the derived type for Refs to the intersection of the
available type assertions. If we discover anything, we should consider redoing
ICR optimization, since better type information might enable more
optimizations.
\chapter{ICR finalize} % -*- Dictionary: design -*-
This pass looks for interesting things in the ICR so that we can forget about
them. Used and not defined things are flamed about.
We postpone these checks until now because the ICR optimizations may discover
errors that are not initially obvious. We also emit efficiency notes about
optimizations that we were unable to do. We can't emit the notes immediately,
since we don't know for sure whether a repeated attempt at optimization will
succeed.
We examine all references to unknown global function variables and update the
approximate type accordingly. We also record the names of the unknown
functions so that they can be flamed about if they are never defined. Unknown
normal variables are flamed about on the fly during ICR conversion, so we
ignore them here.
We check each newly defined global function for compatibility with previously
recorded type information. If there is no :defined or :declared type, then we
check for compatibility with any approximate function type inferred from
previous uses.
\chapter{Environment analysis}
\#|
A related change would be to annotate ICR with information about tail-recursion
relations. What we would do is add a slot to the node structure that points to
the corresponding Tail-Info when a node is in a TR position. This annotation
would be made in a final ICR pass that runs after cleanup code is generated
(part of environment analysis). When true, the node is in a true TR position
(modulo return-convention incompatibility). When we determine return
conventions, we null out the tail-p slots in XEP calls or known calls where we
decided not to preserve tail-recursion.
In this phase, we also check for changes in the dynamic binding environment
that require cleanup code to be generated. We just check for changes in the
Continuation-Cleanup on local control transfers. If it changes from
an inner dynamic context to an outer one that is in the same environment, then
we emit code to clean up the dynamic bindings between the old and new
continuation. We represent the result of cleanup detection to the back end by
interposing a new block containing a call to a funny function. Local exits
from CATCH or UNWIND-PROTECT are detected in the same way.
|\#
The primary activity in environment analysis is the annotation of ICR with
environment structures describing where variables are allocated and what values
the environment closes over.
Each lambda points to the environment where its variables are allocated, and
the environments point back. We always allocate the environment at the Bind
node for the sole non-let lambda in the environment, so there is a close
relationship between environments and functions. Each "real function" (i.e.
not a LET) has a corresponding environment.
We attempt to share the same environment among as many lambdas as possible so
that unnecessary environment manipulation is not done. During environment
analysis the only optimization of this sort is realizing that a Let (a lambda
with no Return node) cannot need its own environment, since there is no way
that it can return and discover that its old values have been clobbered.
When the function is called, values from other environments may need to be made
available in the function's environment. These values are said to be "closed
over".
Even if a value is not referenced in a given environment, it may need to be
closed over in that environment so that it can be passed to a called function
that does reference the value. When we discover that a value must be closed
over by a function, we must close over the value in all the environments where
that function is referenced. This applies to all references, not just local
calls, since at other references we must have the values on hand so that we can
build a closure. This propagation must be applied recursively, since the value
must also be available in *those* functions' callers.
If a closure reference is known to be "safe" (not an upward funarg), then the
closure structure may be allocated on the stack.
Closure analysis deals only with closures over values, while Common Lisp
requires closures over variables. The difference only becomes significant when
variables are set. If a variable is not set, then we can freely make copies of
it without keeping track of where they are. When a variable is set, we must
maintain a single value cell, or at least the illusion thereof. We achieve
this by creating a heap-allocated "value cell" structure for each set variable
that is closed over. The pointer to this value cell is passed around as the
"value" corresponding to that variable. References to the variable must
explicitly indirect through the value cell.
When we are scanning over the lambdas in the component, we also check for bound
but not referenced variables.
Environment analysis emits cleanup code for local exits and markers for
non-local exits.
A non-local exit is a control transfer from one environment to another. In a
non-local exit, we must close over the continuation that we transfer to so that
the exiting function can find its way back. We indicate the need to close a
continuation by placing the continuation structure in the closure and also
pushing it on a list in the environment structure for the target of the exit.
[\#\#\# To be safe, we would treat the continuation as a set closure variable so
that we could invalidate it when we leave the dynamic extent of the exit point.
Transferring control to a meaningless stack pointer would be apt to cause
horrible death.]
Each local control transfer may require dynamic state such as special bindings
to be undone. We represent cleanup actions by funny function calls in a new
block linked in as an implicit MV-PROG1.

411
doc/cmucl/internals/glossary.tex Normal file
View file

@ -0,0 +1,411 @@
\chapter{Glossary}% -*- Dictionary: int:design -*-
% Note: in an entry, any word that is also defined should be \it
% should entries have page references as well?
\begin{description}
\item[assert (a type)]
In Python, all type checking is done via a general type assertion
mechanism. Explicit declarations and implicit assertions (e.g. the arg to
+ is a number) are recorded in the front-end (implicit continuation)
representation. Type assertions (and thus type-checking) are "unbundled"
from the operations that are affected by the assertion. This has two major
advantages:
\begin{itemize}
\item Code that implements operations need not concern itself with checking
operand types.
\item Run-time type checks can be eliminated when the compiler can prove that
the assertion will always be satisfied.
\end{itemize}
See also {\it restrict}.
\item[back end] The back end is the part of the compiler that operates on the
{\it virtual machine} intermediate representation. Also included are the
compiler phases involved in the conversion from the {\it front end}
representation (or {\it ICR}).
\item[bind node] This is a node type the that marks the start of a {\it lambda}
body in {\it ICR}. This serves as a placeholder for environment manipulation
code.
\item[IR1] The first intermediate representation, also known as {\it ICR}, or
the Implicit Continuation Represenation.
\item[IR2] The second intermediate representation, also known as {\it VMR}, or
the Virtual Machine Representation.
\item[basic block] A basic block (or simply "block") has the pretty much the
usual meaning of representing a straight-line sequence of code. However, the
code sequence ultimately generated for a block might contain internal branches
that were hidden inside the implementation of a particular operation. The type
of a block is actually {\tt cblock}. The {\tt block-info} slot holds an
{\tt VMR-block} containing backend information.
\item[block compilation] Block compilation is a term commonly used to describe
the compile-time resolution of function names. This enables many
optimizations.
\item[call graph]
Each node in the call graph is a function (represented by a {\it flow graph}.)
The arcs in the call graph represent a possible call from one function to
another. See also {\it tail set}.
\item[cleanup]
A cleanup is the part of the implicit continuation representation that
retains information scoping relationships. For indefinite extent bindings
(variables and functions), we can abandon scoping information after ICR
conversion, recovering the lifetime information using flow analysis. But
dynamic bindings (special values, catch, unwind protect, etc.) must be
removed at a precise time (whenever the scope is exited.) Cleanup
structures form a hierarchy that represents the static nesting of dynamic
binding structures. When the compiler does a control transfer, it can use
the cleanup information to determine what cleanup code needs to be emitted.
\item[closure variable]
A closure variable is any lexical variable that has references outside of
its {\it home environment}. See also {\it indirect value cell}.
\item[closed continuation] A closed continuation represents a {\tt tagbody} tag
or {\tt block} name that is closed over. These two cases are mostly
indistinguishable in {\it ICR}.
\item[home] Home is a term used to describe various back-pointers. A lambda
variable's "home" is the lambda that the variable belongs to. A lambda's "home
environment" is the environment in which that lambda's variables are allocated.
\item[indirect value cell]
Any closure variable that has assignments ({\tt setq}s) will be allocated in an
indirect value cell. This is necessary to ensure that all references to
the variable will see assigned values, since the compiler normally freely
copies values when creating a closure.
\item[set variable] Any variable that is assigned to is called a "set
variable". Several optimizations must special-case set variables, and set
closure variables must have an {\it indirect value cell}.
\item[code generator] The code generator for a {\it VOP} is a potentially
arbitrary list code fragment which is responsible for emitting assembly code to
implement that VOP.
\item[constant pool] The part of a compiled code object that holds pointers to
non-immediate constants.
\item[constant TN]
A constant TN is the {\it VMR} of a compile-time constant value. A
constant may be immediate, or may be allocated in the {\it constant pool}.
\item[constant leaf]
A constant {\it leaf} is the {\it ICR} of a compile-time constant value.
\item[combination]
A combination {\it node} is the {\it ICR} of any fixed-argument function
call (not {\tt apply} or {\tt multiple-value-call}.)
\item[top-level component]
A top-level component is any component whose only entry points are top-level
lambdas.
\item[top-level lambda]
A top-level lambda represents the execution of the outermost form on which
the compiler was invoked. In the case of {\tt compile-file}, this is often a
truly top-level form in the source file, but the compiler can recursively
descend into some forms ({\tt eval-when}, etc.) breaking them into separate
compilations.
\item[component] A component is basically a sequence of blocks. Each component
is compiled into a separate code object. With {\it block compilation} or {\it
local functions}, a component will contain the code for more than one function.
This is called a component because it represents a connected portion of the
call graph. Normally the blocks are in depth-first order ({\it DFO}).
\item[component, initial] During ICR conversion, blocks are temporarily
assigned to initial components. The "flow graph canonicalization" phase
determines the true component structure.
\item[component, head and tail]
The head and tail of a component are dummy blocks that mark the start and
end of the {\it DFO} sequence. The component head and tail double as the root
and finish node of the component's flow graph.
\item[local function (call)]
A local function call is a call to a function known at compile time to be
in the same {\it component}. Local call allows compile time resolution of the
target address and calling conventions. See {\it block compilation}.
\item[conflict (of TNs, set)]
Register allocation terminology. Two TNs conflict if they could ever be
live simultaneously. The conflict set of a TN is all TNs that it conflicts
with.
\item[continuation]
The ICR data structure which represents both:
\begin{itemize}
\item The receiving of a value (or multiple values), and
\item A control location in the flow graph.
\end{itemize}
In the Implicit Continuation Representation, the environment is implicit in the
continuation's BLOCK (hence the name.) The ICR continuation is very similar to
a CPS continuation in its use, but its representation doesn't much resemble (is
not interchangeable with) a lambda.
\item[cont] A slot in the {\it node} holding the {\it continuation} which
receives the node's value(s). Unless the node ends a {\it block}, this also
implicitly indicates which node should be evaluated next.
\item[cost] Approximations of the run-time costs of operations are widely used
in the back end. By convention, the unit is generally machine cycles, but the
values are only used for comparison between alternatives. For example, the
VOP cost is used to determine the preferred order in which to try possible
implementations.
\item[CSP, CFP] See {\it control stack pointer} and {\it control frame
pointer}.
\item[Control stack] The main call stack, which holds function stack frames.
All words on the control stack are tagged {\it descriptors}. In all ports done
so far, the control stack grows from low memory to high memory. The most
recent call frames are considered to be ``on top'' of earlier call frames.
\item[Control stack pointer] The allocation pointer for the {\it control
stack}. Generally this points to the first free word at the top of the stack.
\item[Control frame pointer] The pointer to the base of the {\it control stack}
frame for a particular function invocation. The CFP for the running function
must be in a register.
\item[Number stack] The auxiliary stack used to hold any {\it non-descriptor}
(untagged) objects. This is generally the same as the C call stack, and thus
typically grows down.
\item[Number stack pointer] The allocation pointer for the {\it number stack}.
This is typically the C stack pointer, and is thus kept in a register.
\item[NSP, NFP] See {\it number stack pointer}, {\it number frame pointer}.
\item[Number frame pointer] The pointer to the base of the {\it number stack}
frame for a particular function invocation. Functions that don't use the
number stack won't have an NFP, but if an NFP is allocated, it is always
allocated in a particular register. If there is no variable-size data on the
number stack, then the NFP will generally be identical to the NSP.
\item[Lisp return address] The name of the {\it descriptor} encoding the
"return pc" for a function call.
\item[LRA] See {\it lisp return address}. Also, the name of the register where
the LRA is passed.
\item[Code pointer] A pointer to the header of a code object. The code pointer
for the currently running function is stored in the {\tt code} register.
\item[Interior pointer] A pointer into the inside of some heap-allocated
object. Interior pointers confuse the garbage collector, so their use is
highly constrained. Typically there is a single register dedicated to holding
interior pointers.
\item[dest]
A slot in the {\it continuation} which points the the node that receives this
value. Null if this value is not received by anyone.
\item[DFN, DFO] See {\it Depth First Number}, {\it Depth First Order}.
\item[Depth first number] Blocks are numbered according to their appearance in
the depth-first ordering (the {\tt block-number} slot.) The numbering actually
increases from the component tail, so earlier blocks have larger numbers.
\item[Depth first order] This is a linearization of the flow graph, obtained by
a depth-first walk. Iterative flow analysis algorithms work better when blocks
are processed in DFO (or reverse DFO.)
\item[Object] In low-level design discussions, an object is one of the
following:
\begin{itemize}
\item a single word containing immediate data (characters, fixnums, etc)
\item a single word pointing to an object (structures, conses, etc.)
\end{itemize}
These are tagged with three low-tag bits as described in the section
\ref{tagging} This is synonymous with {\it descriptor}.
In other parts of the documentation, may be used more loosely to refer to a
{\it lisp object}.
\item[Lisp object]
A Lisp object is a high-level object discussed as a data type in the Common
Lisp definition.
\item[Data-block]
A data-block is a dual-word aligned block of memory that either manifests a
Lisp object (vectors, code, symbols, etc.) or helps manage a Lisp object on
the heap (array header, function header, etc.).
\item[Descriptor]
A descriptor is a tagged, single-word object. It either contains immediate
data or a pointer to data. This is synonymous with {\it object}. Storage
locations that must contain descriptors are referred to as descriptor
locations.
\item[Pointer descriptor]
A descriptor that points to a {\it data block} in memory (i.e. not an immediate
object.)
\item[Immediate descriptor]
A descriptor that encodes the object value in the descriptor itself; used for
characters, fixnums, etc.
\item[Word]
A word is a 32-bit quantity.
\item[Non-descriptor]
Any chunk of bits that isn't a valid tagged descriptor. For example, a
double-float on the number stack. Storage locations that are not scanned by
the garbage collector (and thus cannot contain {\it pointer descriptors}) are
called non-descriptor locations. {\it Immediate descriptors} can be stored in
non-descriptor locations.
\item[Entry point] An entry point is a function that may be subject to
``unpredictable'' control transfers. All entry points are linked to the root
of the flow graph (the component head.) The only functions that aren't entry
points are {\it let} functions. When complex lambda-list syntax is used,
multiple entry points may be created for a single lisp-level function.
See {\it external entry point}.
\item[External entry point] A function that serves as a ``trampoline'' to
intercept function calls coming in from outside of the component. The XEP does
argument syntax and type checking, and may also translate the arguments and
return values for a locally specialized calling calling convention.
\item[XEP] An {\it external entry point}.
\item[lexical environment] A lexical environment is a structure that is used
during VMR conversion to represent all lexically scoped bindings (variables,
functions, declarations, etc.) Each {\tt node} is annotated with its lexical
environment, primarily for use by the debugger and other user interfaces. This
structure is also the environment object passed to {\tt macroexpand}.
\item[environment] The environment is part of the ICR, created during
environment analysis. Environment analysis apportions code to disjoint
environments, with all code in the same environment sharing the same stack
frame. Each environment has a ``{\it real}'' function that allocates it, and
some collection {\tt let} functions. Although environment analysis is the
last ICR phase, in earlier phases, code is sometimes said to be ``in the
same/different environment(s)''. This means that the code will definitely be
in the same environment (because it is in the same real function), or that is
might not be in the same environment, because it is not in the same function.
\item[fixup] Some sort of back-patching annotation. The main sort encountered
are load-time {\it assembler fixups}, which are a linkage annotation mechanism.
\item[flow graph] A flow graph is a directed graph of basic blocks, where each
arc represents a possible control transfer. The flow graph is the basic data
structure used to represent code, and provides direct support for data flow
analysis. See component and ICR.
\item[foldable] An attribute of {\it known functions}. A function is foldable
if calls may be constant folded whenever the arguments are compile-time
constant. Generally this means that it is a pure function with no side
effects.
FSC
full call
function attribute
function
"real" (allocates environment)
meaning function-entry
more vague (any lambda?)
funny function
GEN (kill and...)
global TN, conflicts, preference
GTN (number)
IR ICR VMR ICR conversion, VMR conversion (translation)
inline expansion, call
kill (to make dead)
known function
LAMBDA
leaf
let call
lifetime analysis, live (tn, variable)
load tn
LOCS (passing, return locations)
local call
local TN, conflicts, (or just used in one block)
location (selection)
LTN (number)
main entry
mess-up (for cleanup)
more arg (entry)
MV
non-local exit
non-packed SC, TN
non-set variable
operand (to vop)
optimizer (in icr optimize)
optional-dispatch
pack, packing, packed
pass (in a transform)
passing
locations (value)
conventions (known, unknown)
policy (safe, fast, small, ...)
predecessor block
primitive-type
reaching definition
REF
representation
selection
for value
result continuation (for function)
result type assertion (for template) (or is it restriction)
restrict
a TN to finite SBs
a template operand to a primitive type (boxed...)
a tn-ref to particular SCs
return (node, vops)
safe, safety
saving (of registers, costs)
SB
SC (restriction)
semi-inline
side-effect
in ICR
in VMR
sparse set
splitting (of VMR blocks)
SSET
SUBPRIMITIVE
successor block
tail recursion
tail recursive
tail recursive loop
user tail recursion
template
TN
TNBIND
TN-REF
transform (source, ICR)
type
assertion
inference
top-down, bottom-up
assertion propagation
derived, asserted
descriptor, specifier, intersection, union, member type
check
type-check (in continuation)
UNBOXED (boxed) descriptor
unknown values continuation
unset variable
unwind-block, unwinding
used value (dest)
value passing
VAR
VM
VOP
XEP
\end{description}

6
doc/cmucl/internals/interface.tex Normal file
View file

@ -0,0 +1,6 @@
\chapter{User Interface}
\section{Error Message Utilities}
\section{Source Paths}

694
doc/cmucl/internals/internal-design.txt Normal file
View file

@ -0,0 +1,694 @@
;;;; Terminology.
OBJECT
An object is one of the following:
a single word containing immediate data (characters, fixnums, etc)
a single word pointing to an object (structures, conses, etc.)
These are tagged with three low-tag bits as described in the section
"Tagging". This is synonymous with DESCRIPTOR.
LISP OBJECT
A Lisp object is a high-level object discussed as a data type in Common
Lisp: The Language.
DATA-BLOCK
A data-block is a dual-word aligned block of memory that either manifests a
Lisp object (vectors, code, symbols, etc.) or helps manage a Lisp object on
the heap (array header, function header, etc.).
DESCRIPTOR
A descriptor is a tagged, single-word object. It either contains immediate
data or a pointer to data. This is synonymous with OBJECT.
WORD
A word is a 32-bit quantity.
;;;; Tagging.
The following is a key of the three bit low-tagging scheme:
000 even fixnum
001 function pointer
010 other-immediate (header-words, characters, symbol-value trap value, etc.)
011 list pointer
100 odd fixnum
101 structure pointer
110 unused
111 other-pointer to data-blocks (other than conses, structures,
and functions)
This taging scheme forces a dual-word alignment of data-blocks on the heap, but
this can be pretty negligible:
RATIOS and COMPLEX must have a header-word anyway since they are not a
major type. This wastes one word for these infrequent data-blocks since
they require two words for the data.
BIGNUMS must have a header-word and probably contain only one other word
anyway, so we probably don't waste any words here. Most bignums just
barely overflow fixnums, that is by a bit or two.
Single and double FLOATS?
no waste
one word wasted
SYMBOLS are dual-word aligned with the header-word.
Everything else is vector-like including code, so these probably take up
so many words that one extra one doesn't matter.
;;;; GC Comments.
Data-Blocks comprise only descriptors, or they contain immediate data and raw
bits interpreted by the system. GC must skip the latter when scanning the
heap, so it does not look at a word of raw bits and interpret it as a pointer
descriptor. These data-blocks require headers for GC as well as for operations
that need to know how to interpret the raw bits. When GC is scanning, and it
sees a header-word, then it can determine how to skip that data-block if
necessary. Header-Words are tagged as other-immediates. See the sections
"Other-Immediates" and "Data-Blocks and Header-Words" for comments on
distinguishing header-words from other-immediate data. This distinction is
necessary since we scan through data-blocks containing only descriptors just as
we scan through the heap looking for header-words introducing data-blocks.
Data-Blocks containing only descriptors do not require header-words for GC
since the entire data-block can be scanned by GC a word at a time, taking
whatever action is necessary or appropriate for the data in that slot. For
example, a cons is referenced by a descriptor with a specific tag, and the
system always knows the size of this data-block. When GC encounters a pointer
to a cons, it can transport it into the new space, and when scanning, it can
simply scan the two words manifesting the cons interpreting each word as a
descriptor. Actually there is no cons tag, but a list tag, so we make sure the
cons is not nil when appropriate. A header may still be desired if the pointer
to the data-block does not contain enough information to adequately maintain
the data-block. An example of this is a simple-vector containing only
descriptor slots, and we attach a header-word because the descriptor pointing
to the vector lacks necessary information -- the type of the vector's elements,
its length, etc.
There is no need for a major tag for GC forwarding pointers. Since the tag
bits are in the low end of the word, a range check on the start and end of old
space tells you if you need to move the thing. This is all GC overhead.
;;;; Structures.
Structures comprise a word for each slot in the definition in addition to one
word, a type slot which is a pointer descriptor. This points to a structure
describing the data-block as a structure, a defstruct-descriptor object. When
operating on a structure, doing a structure test can be done by simply checking
the tag bits on the pointer descriptor referencing it. As described in section
"GC Comments", data-blocks such as those representing structures may avoid
having a header-word since they are GC-scanable without any problem. This
saves two words for every structure instance.
;;;; Fixnums.
A fixnum has one of the following formats in 32 bits:
-------------------------------------------------------
| 30 bit 2's complement even integer | 0 0 0 |
-------------------------------------------------------
or
-------------------------------------------------------
| 30 bit 2's complement odd integer | 1 0 0 |
-------------------------------------------------------
Effectively, there is one tag for immediate integers, two zeros. This buys one
more bit for fixnums, and now when these numbers index into simple-vectors or
offset into memory, they point to word boundaries on 32-bit, byte-addressable
machines. That is, no shifting need occur to use the number directly as an
offset.
This format has another advantage on byte-addressable machines when fixnums are
offsets into vector-like data-blocks, including structures. Even though we
previously mentioned data-blocks are dual-word aligned, most indexing and slot
accessing is word aligned, and so are fixnums with effectively two tag bits.
Two tags also allow better usage of special instructions on some machines that
can deal with two low-tag bits but not three.
Since the two bits are zeros, we avoid having to mask them off before using the
words for arithmetic, but division and multiplication require special shifting.
;;;; Other-immediates.
An other-immediate has the following format:
----------------------------------------------------------------
| Data (24 bits) | Type (8 bits with low-tag) | 0 1 0 |
----------------------------------------------------------------
The system uses eight bits of type when checking types and defining system
constants. This allows allows for 32 distinct other-immediate objects given
the three low-tag bits tied down.
The system uses this format for characters, SYMBOL-VALUE unbound trap value,
and header-words for data-blocks on the heap. The type codes are laid out to
facilitate range checks for common subtypes; for example, all numbers will have
contiguous type codes which are distinct from the contiguous array type codes.
See section "Data-Blocks and Other-immediates Typing" for details.
;;;; Data-Blocks and Header-Word Format.
Pointers to data-blocks have the following format:
----------------------------------------------------------------
| Dual-word address of data-block (29 bits) | 1 1 1 |
----------------------------------------------------------------
The word pointed to by the above descriptor is a header-word, and it has the
same format as an other-immediate:
----------------------------------------------------------------
| Data (24 bits) | Type (8 bits with low-tag) | 0 1 0 |
----------------------------------------------------------------
This is convenient for scanning the heap when GC'ing, but it does mean that
whenever GC encounters an other-immediate word, it has to do a range check on
the low byte to see if it is a header-word or just a character (for example).
This is easily acceptable performance hit for scanning.
The system interprets the data portion of the header-word for non-vector
data-blocks as the word length excluding the header-word. For example, the
data field of the header for ratio and complex numbers is two, one word each
for the numerator and denominator or for the real and imaginary parts.
For vectors and data-blocks representing Lisp objects stored like vectors, the
system ignores the data portion of the header-word:
----------------------------------------------------------------
| Unused Data (24 bits) | Type (8 bits with low-tag) | 0 1 0 |
----------------------------------------------------------------
| Element Length of Vector (30 bits) | 0 0 |
----------------------------------------------------------------
Using a separate word allows for much larger vectors, and it allows LENGTH to
simply access a single word without masking or shifting. Similarly, the header
for complex arrays and vectors has a second word, following the header-word,
the system uses for the fill pointer, so computing the length of any array is
the same code sequence.
;;;; Data-Blocks and Other-immediates Typing.
These are the other-immediate types. We specify them including all low eight
bits, including the other-immediate tag, so we can think of the type bits as
one type -- not an other-immediate major type and a subtype. Also, fetching a
byte and comparing it against a constant is more efficient than wasting even a
small amount of time shifting out the other-immediate tag to compare against a
five bit constant.
Number (< 30)
00000 010 bignum 10
00000 010 ratio 14
00000 010 single-float 18
00000 010 double-float 22
00000 010 complex 26
Array (>= 30 code 86)
Simple-Array (>= 20 code 70)
00000 010 simple-array 30
Vector (>= 34 code 82)
00000 010 simple-string 34
00000 010 simple-bit-vector 38
00000 010 simple-vector 42
00000 010 (simple-array (unsigned-byte 2) (*)) 46
00000 010 (simple-array (unsigned-byte 4) (*)) 50
00000 010 (simple-array (unsigned-byte 8) (*)) 54
00000 010 (simple-array (unsigned-byte 16) (*)) 58
00000 010 (simple-array (unsigned-byte 32) (*)) 62
00000 010 (simple-array single-float (*)) 66
00000 010 (simple-array double-float (*)) 70
00000 010 complex-string 74
00000 010 complex-bit-vector 78
00000 010 (array * (*)) -- general complex vector. 82
00000 010 complex-array 86
00000 010 code-header-type 90
00000 010 function-header-type 94
00000 010 closure-header-type 98
00000 010 funcallable-instance-header-type 102
00000 010 unused-function-header-1-type 106
00000 010 unused-function-header-2-type 110
00000 010 unused-function-header-3-type 114
00000 010 closure-function-header-type 118
00000 010 return-pc-header-type 122
00000 010 value-cell-header-type 126
00000 010 symbol-header-type 130
00000 010 base-character-type 134
00000 010 system-area-pointer-type (header type) 138
00000 010 unbound-marker 142
00000 010 weak-pointer-type 146
;;;; Strings.
All strings in the system are C-null terminated. This saves copying the bytes
when calling out to C. The only time this wastes memory is when the string
contains a multiple of eight characters, and then the system allocates two more
words (since Lisp objects are dual-word aligned) to hold the C-null byte.
Since the system will make heavy use of C routines for systems calls and
libraries that save reimplementation of higher level operating system
functionality (such as pathname resolution or current directory computation),
saving on copying strings for C should make C call out more efficient.
The length word in a string header, see section "Data-Blocks and Header-Word
Format", counts only the characters truly in the Common Lisp string.
Allocation and GC will have to know to handle the extra C-null byte, and GC
already has to deal with rounding up various objects to dual-word alignment.
;;;; Symbols and NIL.
Symbol data-block has the following format:
-------------------------------------------------------
| 5 (data-block words) | Symbol Type (8 bits) |
-------------------------------------------------------
| Value Descriptor |
-------------------------------------------------------
| Function Pointer |
-------------------------------------------------------
| Raw Function Address |
-------------------------------------------------------
| Setf Function |
-------------------------------------------------------
| Property List |
-------------------------------------------------------
| Print Name |
-------------------------------------------------------
| Package |
-------------------------------------------------------
Most of these slots are self-explanatory given what symbols must do in Common
Lisp, but a couple require comments. We added the Raw Function Address slot to
speed up named call which is the most common calling convention. This is a
non-descriptor slot, but since objects are dual word aligned, the value
inherently has fixnum low-tag bits. The GC method for symbols must know to
update this slot. The Setf Function slot is currently unused, but we had an
extra slot due to adding Raw Function Address since objects must be dual-word
aligned.
The issues with nil are that we want it to act like a symbol, and we need list
operations such as CAR and CDR to be fast on it. CMU Common Lisp solves this
by putting nil as the first object in static space, where other global values
reside, so it has a known address in the system:
------------------------------------------------------- <-- start static
| 0 | space
-------------------------------------------------------
| 5 (data-block words) | Symbol Type (8 bits) |
------------------------------------------------------- <-- nil
| Value/CAR |
-------------------------------------------------------
| Definition/CDR |
-------------------------------------------------------
| Raw Function Address |
-------------------------------------------------------
| Setf Function |
-------------------------------------------------------
| Property List |
-------------------------------------------------------
| Print Name |
-------------------------------------------------------
| Package |
-------------------------------------------------------
| ... |
-------------------------------------------------------
In addition, we make the list typed pointer to nil actually point past the
header word of the nil symbol data-block. This has usefulness explained below.
The value and definition of nil are nil. Therefore, any reference to nil used
as a list has quick list type checking, and CAR and CDR can go right through
the first and second words as if nil were a cons object.
When there is a reference to nil used as a symbol, the system adds offsets to
the address the same as it does for any symbol. This works due to a
combination of nil pointing past the symbol header-word and the chosen list and
other-pointer type tags. The list type tag is four less than the other-pointer
type tag, but nil points four additional bytes into its symbol data-block.
;;;; Array Headers.
The array-header data-block has the following format:
----------------------------------------------------------------
| Header Len (24 bits) = Array Rank +5 | Array Type (8 bits) |
----------------------------------------------------------------
| Fill Pointer (30 bits) | 0 0 |
----------------------------------------------------------------
| Available Elements (30 bits) | 0 0 |
----------------------------------------------------------------
| Data Vector (29 bits) | 1 1 1 |
----------------------------------------------------------------
| Displacement (30 bits) | 0 0 |
----------------------------------------------------------------
| Displacedp (29 bits) -- t or nil | 1 1 1 |
----------------------------------------------------------------
| Range of First Index (30 bits) | 0 0 |
----------------------------------------------------------------
.
.
.
The array type in the header-word is one of the eight-bit patterns from section
"Data-Blocks and Other-immediates Typing", indicating that this is a complex
string, complex vector, complex bit-vector, or a multi-dimensional array. The
data portion of the other-immediate word is the length of the array header
data-block. Due to its format, its length is always five greater than the
array's number of dimensions. The following words have the following
interpretations and types:
Fill Pointer
This is a fixnum indicating the number of elements in the data vector
actually in use. This is the logical length of the array, and it is
typically the same value as the next slot. This is the second word, so
LENGTH of any array, with or without an array header, is just four bytes
off the pointer to it.
Available Elements
This is a fixnum indicating the number of elements for which there is
space in the data vector. This is greater than or equal to the logical
length of the array when it is a vector having a fill pointer.
Data Vector
This is a pointer descriptor referencing the actual data of the array.
This a data-block whose first word is a header-word with an array type as
described in sections "Data-Blocks and Header-Word Format" and
"Data-Blocks and Other-immediates Typing"
Displacement
This is a fixnum added to the computed row-major index for any array.
This is typically zero.
Displacedp
This is either t or nil. This is separate from the displacement slot, so
most array accesses can simply add in the displacement slot. The rare
need to know if an array is displaced costs one extra word in array
headers which probably aren't very frequent anyway.
Range of First Index
This is a fixnum indicating the number of elements in the first dimension
of the array. Legal index values are zero to one less than this number
inclusively. IF the array is zero-dimensional, this slot is
non-existent.
... (remaining slots)
There is an additional slot in the header for each dimension of the
array. These are the same as the Range of First Index slot.
;;;; Bignums.
Bignum data-blocks have the following format:
-------------------------------------------------------
| Length (24 bits) | Bignum Type (8 bits) |
-------------------------------------------------------
| least significant bits |
-------------------------------------------------------
.
.
.
The elements contain the two's complement representation of the integer with
the least significant bits in the first element or closer to the header. The
sign information is in the high end of the last element.
;;;; Code Data-Blocks.
A code data-block is the run-time representation of a "component". A component
is a connected portion of a program's flow graph that is compiled as a single
unit, and it contains code for many functions. Some of these functions are
callable from outside of the component, and these are termed "entry points".
Each entry point has an associated user-visible function data-block (of type
FUNCTION). The full call convention provides for calling an entry point
specified by a function object.
Although all of the function data-blocks for a component's entry points appear
to the user as distinct objects, the system keeps all of the code in a single
code data-block. The user-visible function object is actually a pointer into
the middle of a code data-block. This allows any control transfer within a
component to be done using a relative branch.
Besides a function object, there are other kinds of references into the middle
of a code data-block. Control transfer into a function also occurs at the
return-PC for a call. The system represents a return-PC somewhat similarly to
a function, so GC can also recognize a return-PC as a reference to a code
data-block.
It is incorrect to think of a code data-block as a concatenation of "function
data-blocks". Code for a function is not emitted in any particular order with
respect to that function's function-header (if any). The code following a
function-header may only be a branch to some other location where the
function's "real" definition is.
The following are the three kinds of pointers to code data-blocks:
Code pointer (labeled A below):
A code pointer is a descriptor, with other-pointer low-tag bits, pointing
to the beginning of the code data-block. The code pointer for the
currently running function is always kept in a register (CODE). In
addition to allowing loading of non-immediate constants, this also serves
to represent the currently running function to the debugger.
Return-PC (labeled B below):
The return-PC is a descriptor, with other-pointer low-tag bits, pointing
to a location for a function call. Note that this location contains no
descriptors other than the one word of immediate data, so GC can treat
return-PC locations the same as instructions.
Function (labeled C below):
A function is a descriptor, with function low-tag bits, that is user
callable. When a function header is referenced from a closure or from
the function header's self-pointer, the pointer has other-pointer low-tag
bits, instead of function low-tag bits. This ensures that the internal
function data-block associated with a closure appears to be uncallable
(although users should never see such an object anyway).
Information about functions that is only useful for entry points is kept
in some descriptors following the function's self-pointer descriptor.
All of these together with the function's header-word are known as the
"function header". GC must be able to locate the function header. We
provide for this by chaining together the function headers in a NIL
terminated list kept in a known slot in the code data-block.
A code data-block has the following format:
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ <-- A
| Header-Word count (24 bits) | %Code-Type (8 bits) |
----------------------------------------------------------------
| Number of code words (fixnum tag) |
----------------------------------------------------------------
| Pointer to first function header (other-pointer tag) |
----------------------------------------------------------------
| Debug information (structure tag) |
----------------------------------------------------------------
| First constant (a descriptor) |
----------------------------------------------------------------
| ... |
----------------------------------------------------------------
| Last constant (and last word of code header) |
----------------------------------------------------------------
| Some instructions (non-descriptor) |
----------------------------------------------------------------
| (pad to dual-word boundary if necessary) |
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ <-- B
| Word offset from code header (24) | %Return-PC-Type (8) |
----------------------------------------------------------------
| First instruction after return |
----------------------------------------------------------------
| ... more code and return-PC header-words |
----------------------------------------------------------------
| (pad to dual-word boundary if necessary) |
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ <-- C
| Offset from code header (24) | %Function-Header-Type (8) |
----------------------------------------------------------------
| Self-pointer back to previous word (with other-pointer tag) |
----------------------------------------------------------------
| Pointer to next function (other-pointer low-tag) or NIL |
----------------------------------------------------------------
| Function name (a string or a symbol) |
----------------------------------------------------------------
| Function debug arglist (a string) |
----------------------------------------------------------------
| Function type (a list-style function type specifier) |
----------------------------------------------------------------
| Start of instructions for function (non-descriptor) |
----------------------------------------------------------------
| More function headers and instructions and return PCs, |
| until we reach the total size of header-words + code |
| words. |
----------------------------------------------------------------
The following are detailed slot descriptions:
Code data-block header-word:
The immediate data in the code data-block's header-word is the number of
leading descriptors in the code data-block, the fixed overhead words plus
the number of constants. The first non-descriptor word, some code,
appears at this word offset from the header.
Number of code words:
The total number of non-header-words in the code data-block. The total
word size of the code data-block is the sum of this slot and the
immediate header-word data of the previous slot. The system accesses
this slot with the system constant, %Code-Code-Size-Slot, offset from the
header-word.
Pointer to first function header:
A NIL-terminated list of the function headers for all entry points to
this component. The system accesses this slot with the system constant,
%Code-Entry-Points-Slot, offset from the header-word.
Debug information:
The DEBUG-INFO structure describing this component. All information that
the debugger wants to get from a running function is kept in this
structure. Since there are many functions, the current PC is used to
locate the appropriate debug information. The system keeps the debug
information separate from the function data-block, since the currently
running function may not be an entry point. There is no way to recover
the function object for the currently running function, since this
data-block may not exist. The system accesses this slot with the system
constant, %Code-Debug-Info-Slot, offset from the header-word.
First constant ... last constant:
These are the constants referenced by the component, if there are any.
The system accesses the first constant slot with the system constant,
%Code-Constants-Offset, offset from the header-word.
Return-PC header word:
The immediate header-word data is the word offset from the enclosing code
data-block's header-word to this word. This allows GC and the debugger
to easily recover the code data-block from a return-PC. The code at the
return point restores the current code pointer using a subtract immediate
of the offset, which is known at compile time.
Function entry point header-word:
The immediate header-word data is the word offset from the enclosing code
data-block's header-word to this word. This is the same as for the
retrun-PC header-word.
Self-pointer back to header-word:
In a non-closure function, this self-pointer to the previous header-word
allows the call sequence to always indirect through the second word in a
user callable function. See section "Closure Format". With a closure,
indirecting through the second word gets you a function header-word. The
system ignores this slot in the function header for a closure, since it
has already indirected once, and this slot could be some random thing
that causes an error if you jump to it. This pointer has an
other-pointer tag instead of a function pointer tag, indicating it is not
a user callable Lisp object. The system accesses this slot with the
system constant, %Function-Code-Slot, offset from the function
header-word.
Pointer to next function:
This is the next link in the thread of entry point functions found in
this component. This value is NIL when the current header is the last
entry point in the component. The system accesses this slot with the
system constant, %Function-Header-Next-Slot, offset from the function
header-word.
Function name:
This function's name (for printing). If the user defined this function
with DEFUN, then this is the defined symbol, otherwise it is a
descriptive string. The system accesses this slot with the system
constant, %Function-Header-Name-Slot, offset from the function
header-word.
Function debug arglist:
A printed string representing the function's argument list, for human
readability. If it is a macroexpansion function, then this is the
original DEFMACRO arglist, not the actual expander function arglist. The
system accesses this slot with the system constant,
%Function-Header-Debug-Arglist-Slot, offset from the function
header-word.
Function type:
A list-style function type specifier representing the argument signature
and return types for this function. For example,
(FUNCTION (FIXNUM FIXNUM FIXNUM) FIXNUM)
or
(FUNCTION (STRING &KEY (:START UNSIGNED-BYTE)) STRING)
This information is intended for machine readablilty, such as by the
compiler. The system accesses this slot with the system constant,
%Function-Header-Type-Slot, offset from the function header-word.
;;;; Closure Format.
A closure data-block has the following format:
----------------------------------------------------------------
| Word size (24 bits) | %Closure-Type (8 bits) |
----------------------------------------------------------------
| Pointer to function header (other-pointer low-tag) |
----------------------------------------------------------------
| . |
| Environment information |
| . |
----------------------------------------------------------------
A closure descriptor has function low-tag bits. This means that a descriptor
with function low-tag bits may point to either a function header or to a
closure. The idea is that any callable Lisp object has function low-tag bits.
Insofar as call is concerned, we make the format of closures and non-closure
functions compatible. This is the reason for the self-pointer in a function
header. Whenever you have a callable object, you just jump through the second
word, offset some bytes, and go.
;;;; Function call.
Due to alignment requirements and low-tag codes, it is not possible to use a
hardware call instruction to compute the return-PC. Instead the return-PC
for a call is computed by doing an add-immediate to the start of the code
data-block.
An advantage of using a single data-block to represent both the descriptor and
non-descriptor parts of a function is that both can be represented by a
single pointer. This reduces the number of memory accesses that have to be
done in a full call. For example, since the constant pool is implicit in a
return-PC, a call need only save the return-PC, rather than saving both the
return PC and the constant pool.
;;;; Memory Layout.
CMU Common Lisp has four spaces, read-only, static, dynamic-0, and dynamic-1.
Read-only contains objects that the system never modifies, moves, or reclaims.
Static space contains some global objects necessary for the system's runtime or
performance (since they are located at a known offset at a know address), and
the system never moves or reclaims these. However, GC does need to scan static
space for references to moved objects. Dynamic-0 and dynamic-1 are the two
heap areas for stop-and-copy GC algorithms.
What global objects are at the head of static space???
NIL
eval::*top-of-stack*
lisp::*current-catch-block*
lisp::*current-unwind-protect*
FLAGS (RT only)
BSP (RT only)
HEAP (RT only)
In addition to the above spaces, the system has a control stack, binding stack,
and a number stack. The binding stack contains pairs of descriptors, a symbol
and its previous value. The number stack is the same as the C stack, and the
system uses it for non-Lisp objects such as raw system pointers, saving
non-Lisp registers, parts of bignum computations, etc.
;;;; System Pointers.
The system pointers reference raw allocated memory, data returned by foreign
function calls, etc. The system uses these when you need a pointer to a
non-Lisp block of memory, using an other-pointer. This provides the greatest
flexibility by relieving contraints placed by having more direct references
that require descriptor type tags.
A system area pointer data-block has the following format:
-------------------------------------------------------
| 1 (data-block words) | SAP Type (8 bits) |
-------------------------------------------------------
| system area pointer |
-------------------------------------------------------
"SAP" means "system area pointer", and much of our code contains this naming
scheme. We don't currently restrict system pointers to one area of memory, but
if they do point onto the heap, it is up to the user to prevent being screwed
by GC or whatever.

191
doc/cmucl/internals/interpreter.tex Normal file
View file

@ -0,0 +1,191 @@
% -*- Dictionary: design; Package: C -*-
May be worth having a byte-code representation for interpreted code. This way,
an entire system could be compiled into byte-code for debugging (the
"check-out" compiler?).
Given our current inclination for using a stack machine to interpret IR1, it
would be straightforward to layer a byte-code interpreter on top of this.
Interpreter:
Instead of having no interpreter, or a more-or-less conventional interpreter,
or byte-code interpreter, how about directly executing IR1?
We run through the IR1 passes, possibly skipping optional ones, until we get
through environment analysis. Then we run a post-pass that annotates IR1 with
information about where values are kept, i.e. the stack slot.
We can lazily convert functions by having FUNCTION make an interpreted function
object that holds the code (really a closure over the interpreter). The first
time that we try to call the function, we do the conversion and processing.
Also, we can easily keep track of which interpreted functions we have expanded
macros in, so that macro redefinition automatically invalidates the old
expansion, causing lazy reconversion.
Probably the interpreter will want to represent MVs by a recognizable structure
that is always heap-allocated. This way, we can punt the stack issues involved
in trying to spread MVs. So a continuation value can always be kept in a
single cell.
The compiler can have some special frobs for making the interpreter efficient,
such as a call operation that extracts arguments from the stack
slots designated by a continuation list. Perhaps
(values-mapcar fun . lists)
<==>
(values-list (mapcar fun . lists))
This would be used with MV-CALL.
This scheme seems to provide nearly all of the advantages of both the compiler
and conventional interpretation. The only significant disadvantage with
respect to a conventional interpreter is that there is the one-time overhead of
conversion, but doing this lazily should make this quite acceptable.
With respect to a conventional interpreter, we have major advantages:
+ Full syntax checking: safety comparable to compiled code.
+ Semantics similar to compiled code due to code sharing. Similar diagnostic
messages, etc. Reduction of error-prone code duplication.
+ Potential for full type checking according to declarations (would require
running IR1 optimize?)
+ Simplifies debugger interface, since interpreted code can look more like
compiled code: source paths, edit definition, etc.
For all non-run-time symbol annotations (anything other than SYMBOL-FUNCTION
and SYMBOL-VALUE), we use the compiler's global database. MACRO-FUNCTION will
use INFO, rather than vice-versa.
When doing the IR1 phases for the interpreter, we probably want to suppress
optimizations that change user-visible function calls:
-- Don't do local call conversion of any named functions (even lexical ones).
This is so that a call will appear on the stack that looks like the call in
the original source. The keyword and optional argument transformations
done by local call mangle things quite a bit. Also, note local-call
converting prevents unreferenced arguments from being deleted, which is
another non-obvious transformation.
-- Don't run source-transforms, IR1 transforms and IR1 optimizers. This way,
TRACE and BACKTRACE will show calls with the original arguments, rather
than the "optimized" form, etc. Also, for the interpreter it will
actually be faster to call the original function (which is compiled) than
to "inline expand" it. Also, this allows implementation-dependent
transforms to expand into %PRIMITIVE uses.
There are some problems with stepping, due to our non-syntactic IR1
representation. The source path information is the key that makes this
conceivable. We can skip over the stepping of a subform by quietly evaluating
nodes whose source path lies within the form being skipped.
One problem with determining what value has been returned by a form. With a
function call, it is theoretically possible to precisely determine this, since
if we complete evaluation of the arguments, then we arrive at the Combination
node whose value is synonymous with the value of the form. We can even detect
this case, since the Node-Source will be EQ to the form. And we can also
detect when we unwind out of the evaluation, since we will leave the form
without having ever reached this node.
But with macros and special-forms, there is no node whose value is the value of
the form, and no node whose source is the macro call or special form. We can
still detect when we leave the form, but we can't be sure whether this was a
normal evaluation result or an explicit RETURN-FROM.
But does this really matter? It seems that we can print the value returned (if
any), then just print the next form to step. In the rare case where we did
unwind, the user should be able to figure it out.
[We can look at this as a side-effect of CPS: there isn't any difference
between a "normal" return and a non-local one.]
[Note that in any control transfer (normal or otherwise), the stepper may need
to unwind out of an arbitrary number of levels of stepping. This is because a
form in a TR position may yield its to a node arbitrarily far our.]
Another problem is with deciding what form is being stepped. When we start
evaluating a node, we dive into code that is nested somewhere down inside that
form. So we actually have to do a loop of asking questions before we do any
evaluation. But what do we ask about?
If we ask about the outermost enclosing form that is a subform of the the last
form that the user said to execute, then we might offer a form that isn't
really evaluated, such as a LET binding list.
But once again, is this really a problem? It is certainly different from a
conventional stepper, but a pretty good argument could be made that it is
superior. Haven't you ever wanted to skip the evaluation of all the
LET bindings, but not the body? Wouldn't it be useful to be able to skip the
DO step forms?
All of this assumes that nobody ever wants to step through the guts of a
macroexpansion. This seems reasonable, since steppers are for weenies, and
weenies don't define macros (hence don't debug them). But there are probably
some weenies who don't know that they shouldn't be writing macros.
We could handle this by finding the "source paths" in the expansion of each
macro by sticking some special frob in the source path marking the place where
the expansion happened. When we hit code again that is in the source, then we
revert to the normal source path. Something along these lines might be a good
idea anyway (for compiler error messages, for example).
The source path hack isn't guaranteed to work quite so well in generated code,
though, since macros return stuff that isn't freshly consed. But we could
probably arrange to win as long as any given expansion doesn't return two EQ
forms.
It might be nice to have a command that skipped stepping of the form, but
printed the results of each outermost enclosed evaluated subform, i.e. if you
used this on the DO step-list, it would print the result of each new-value
form. I think this is implementable. I guess what you would do is print each
value delivered to a DEST whose source form is the current or an enclosing
form. Along with the value, you would print the source form for the node that
is computing the value.
The stepper can also have a "back" command that "unskips" or "unsteps". This
would allow the evaluation of forms that are pure (modulo lexical variable
setting) to be undone. This is useful, since in stepping it is common that you
skip a form that you shouldn't have, or get confused and want to restart at
some earlier point.
What we would do is remember the current node and the values of all local
variables. heap before doing each step or skip action. We can then back up
the state of all lexical variables and the "program counter". To make this
work right with set closure variables, we would copy the cell's value, rather
than the value cell itself.
[To be fair, note that this could easily be done with our current interpreter:
the stepper could copy the environment alists.]
We can't back up the "program counter" when a control transfer leaves the
current function, since this state is implicitly represented in the
interpreter's state, and is discarded when we exit. We probably want to ask
for confirmation before leaving the function to give users a chance to "unskip"
the forms in a TR position.
Another question is whether the conventional stepper is really a good thing to
imitate... How about an editor-based mouse-driven interface? Instead of
"skipping" and "stepping", you would just designate the next form that you
wanted to stop at. Instead of displaying return values, you replace the source
text with the printed representation of the value.
It would show the "program counter" by highlighting the *innermost* form that
we are about to evaluate, i.e. the source form for the node that we are stopped
at. It would probably also be useful to display the start of the form that was
used to designate the next stopping point, although I guess this could be
implied by the mouse position.
Such an interface would be a little harder to implement than a dumb stepper,
but it would be much easier to use. [It would be impossible for an evalhook
stepper to do this.]
%PRIMITIVE usage:
Note: %PRIMITIVE can only be used in compiled code. It is a trapdoor into the
compiler, not a general syntax for accessing "sub-primitives". It's main use
is in implementation-dependent compiler transforms. It saves us the effort of
defining a "phony function" (that is not really defined), and also allows
direct communication with the code generator through codegen-info arguments.
Some primitives may be exported from the VM so that %PRIMITIVE can be used to
make it explicit that an escape routine or interpreter stub is assuming an
operation is implemented by the compiler.

10
doc/cmucl/internals/lowlev.tex Normal file
View file

@ -0,0 +1,10 @@
\chapter{Memory Management}
\section{Stacks and Globals}
\section{Heap Layout}
\section{Garbage Collection}
\chapter{Interface to C and Assembler}
\chapter{Low-level debugging}
\chapter{Core File Format}

649
doc/cmucl/internals/middle.tex Normal file
View file

@ -0,0 +1,649 @@
% -*- Dictionary: design -*-
\chapter{Virtual Machine Representation Introduction}
\chapter{Global TN assignment}
[\#\#\# Rename this phase so as not to be confused with the local/global TN
representation.]
The basic mechanism for closing over values is to pass the values as additional
implicit arguments in the function call. This technique is only applicable
when:
-- the calling function knows which values the called function wants to close
over, and
-- the values to be closed over are available in the calling environment.
The first condition is always true of local function calls. Environment
analysis can guarantee that the second condition holds by closing over any
needed values in the calling environment.
If the function that closes over values may be called in an environment where
the closed over values are not available, then we must store the values in a
"closure" so that they are always accessible. Closures are called using the
"full call" convention. When a closure is called, control is transferred to
the "external entry point", which fetches the values out of the closure and
then does a local call to the real function, passing the closure values as
implicit arguments.
In this scheme there is no such thing as a "heap closure variable" in code,
since the closure values are moved into TNs by the external entry point. There
is some potential for pessimization here, since we may end up moving the values
from the closure into a stack memory location, but the advantages are also
substantial. Simplicity is gained by always representing closure values the
same way, and functions with closure references may still be called locally
without allocating a closure. All the TN based VMR optimizations will apply
to closure variables, since closure variables are represented in the same way
as all other variables in VMR. Closure values will be allocated in registers
where appropriate.
Closures are created at the point where the function is referenced, eliminating
the need to be able to close over closures. This lazy creation of closures has
the additional advantage that when a closure reference is conditionally not
done, then the closure consing will never be done at all. The corresponding
disadvantage is that a closure over the same values may be created multiple
times if there are multiple references. Note however, that VMR loop and common
subexpression optimizations can eliminate redundant closure consing. In any
case, multiple closures over the same variables doesn't seem to be that common.
\#|
Having the Tail-Info would also make return convention determination trivial.
We could just look at the type, checking to see if it represents a fixed number
of values. To determine if the standard return convention is necessary to
preserve tail-recursion, we just iterate over the equivalent functions, looking
for XEPs and uses in full calls.
|\#
The Global TN Assignment pass (GTN) can be considered a post-pass to
environment analysis. This phase assigns the TNs used to hold local lexical
variables and pass arguments and return values and determines the value-passing
strategy used in local calls.
To assign return locations, we look at the function's tail-set.
If the result continuation for an entry point is used as the continuation for a
full call, then we may need to constrain the continuation's values passing
convention to the standard one. This is not necessary when the call is known
not to be part of a tail-recursive loop (due to being a known function).
Once we have figured out where we must use the standard value passing strategy,
we can use a more flexible strategy to determine the return locations for local
functions. We determine the possible numbers of return values from each
function by examining the uses of all the result continuations in the
equivalence class of the result continuation.
If the tail-set type is for a fixed number of
values, then we return that fixed number of values from all the functions whose
result continuations are equated. If the number of values is not fixed, then
we must use the unknown-values convention, although we are not forced to use
the standard locations. We assign the result TNs at this time.
We also use the tail-sets to see what convention we want to use. What we do is
use the full convention for any function that has a XEP its tail-set, even if
we aren't required to do so by a tail-recursive full call, as long as there are
no non-tail-recursive local calls in the set. This prevents us from
gratuitously using a non-standard convention when there is no reason to.
\chapter{Local TN assignment}
[Want a different name for this so as not to be confused with the different
local/global TN representations. The really interesting stuff in this phase is
operation selection, values representation selection, return strategy, etc.
Maybe this phase should be conceptually lumped with GTN as "implementation
selection", since GTN determines call strategies and locations.]
\#|
[\#\#\# I guess I believe that it is OK for VMR conversion to dick the ICR flow
graph. An alternative would be to give VMR its very own flow graph, but that
seems like overkill.
In particular, it would be very nice if a TR local call looked exactly like a
jump in VMR. This would allow loop optimizations to be done on loops written
as recursions. In addition to making the call block transfer to the head of
the function rather than to the return, we would also have to do something
about skipping the part of the function prolog that moves arguments from the
passing locations, since in a TR call they are already in the right frame.
In addition to directly indicating whether a call should be coded with a TR
variant, the Tail-P annotation flags non-call nodes that can directly return
the value (an "advanced return"), rather than moving the value to the result
continuation and jumping to the return code. Then (according to policy), we
can decide to advance all possible returns. If all uses of the result are
Tail-P, then LTN can annotate the result continuation as :Unused, inhibiting
emission of the default return code.
[\#\#\# But not really. Now there is a single list of templates, and a given
template has only one policy.]
In LTN, we use the :Safe template as a last resort even when the policy is
unsafe. Note that we don't try :Fast-Safe; if this is also a good unsafe
template, then it should have the unsafe policies explicitly specified.
With a :Fast-Safe template, the result type must be proven to satisfy the
output type assertion. This means that a fast-safe template with a fixnum
output type doesn't need to do fixnum overflow checking. [\#\#\# Not right to
just check against the Node-Derived-Type, since type-check intersects with
this.]
It seems that it would be useful to have a kind of template where the args must
be checked to be fixnum, but the template checks for overflow and signals an
error. In the case where an output assertion is present, this would generate
better code than conditionally branching off to make a bignum, and then doing a
type check on the result.
How do we deal with deciding whether to do a fixnum overflow check? This
is perhaps a more general problem with the interpretation of result type
restrictions in templates. It would be useful to be able to discriminate
between the case where the result has been proven to be a fixnum and where
it has simply been asserted to be so.
The semantics of result type restriction is that the result must be proven
to be of that type *except* for safe generators, which are assumed to
verify the assertion. That way "is-fixnum" case can be a fast-safe
generator and the "should-be-fixnum" case is a safe generator. We could
choose not to have a safe "should-be-fixnum" generator, and let the
unrestricted safe generator handle it. We would then have to do an
explicit type check on the result.
In other words, for all template except Safe, a type restriction on either
an argument or result means "this must be true; if it is not the system may
break." In contrast, in a Safe template, the restriction means "If this is
not true, I will signal an error."
Since the node-derived-type only takes into consideration stuff that can be
proved from the arguments, we can use the node-derived-type to select
fast-safe templates. With unsafe policies, we don't care, since the code
is supposed to be unsafe.
|\#
Local TN assignment (LTN) assigns all the TNs needed to represent the values of
continuations. This pass scans over the code for the component, examining each
continuation and its destination. A number of somewhat unrelated things are
also done at the same time so that multiple passes aren't necessary.
-- Determine the Primitive-Type for each continuation value and assigns TNs
to hold the values.
-- Use policy information to determine the implementation strategy for each
call to a known function.
-- Clear the type-check flags in continuations whose destinations have safe
implementations.
-- Determine the value-passing strategy for each continuation: known or
unknown.
-- Note usage of unknown-values continuations so that stack analysis can tell
when stack values must be discarded.
If safety is more important that speed and space, then we consider generating
type checks on the values of nodes whose CONT has the Type-Check flag set. If
the destinatation for the continuation value is safe, then we don't need to do
a check. We assume that all full calls are safe, and use the template
information to determine whether inline operations are safe.
This phase is where compiler policy switches have most of their effect. The
speed/space/safety tradeoff can determine which of a number of coding
strategies are used. It is important to make the policy choice in VMR
conversion rather than in code generation because the cost and storage
requirement information which drives TNBIND will depend strongly on what actual
VOP is chosen. In the case of +/FIXNUM, there might be three or more
implementations, some optimized for speed, some for space, etc. Some of these
VOPS might be open-coded and some not.
We represent the implementation strategy for a call by either marking it as a
full call or annotating it with a "template" representing the open-coding
strategy. Templates are selected using a two-way dispatch off of operand
primitive-types and policy. The general case of LTN is handled by the
LTN-Annotate function in the function-info, but most functions are handled by a
table-driven mechanism. There are four different translation policies that a
template may have:
\begin{description}
\item[Safe]
The safest implementation; must do argument type checking.
\item[Small]
The (unsafe) smallest implementation.
\item[Fast]
The (unsafe) fastest implementation.
\item[Fast-Safe]
An implementation optimized for speed, but which does any necessary
checks exclusive of argument type checking. Examples are array bounds
checks and fixnum overflow checks.
\end{description}
Usually a function will have only one or two distinct templates. Either or
both of the safe and fast-safe templates may be omitted; if both are specified,
then they should be distinct. If there is no safe template and our policy is
safe, then we do a full call.
We use four different coding strategies, depending on the policy:
\begin{description}
\item[Safe:] safety $>$ space $>$ speed, or
we want to use the fast-safe template, but there isn't one.
\item[Small:] space $>$ (max speed safety)
\item[Fast:] speed $>$ (max space safety)
\item[Fast-Safe (and type check):] safety $>$ speed $>$ space, or we want to use
the safe template, but there isn't one.
\end{description}
``Space'' above is actually the maximum of space and cspeed, under the theory
that less code will take less time to generate and assemble. [\#\#\# This could
lose if the smallest case is out-of-line, and must allocate many linkage
registers.]
\chapter{Control optimization}
In this phase we annotate blocks with drop-throughs. This controls how code
generation linearizes code so that drop-throughs are used most effectively. We
totally linearize the code here, allowing code generation to scan the blocks
in the emit order.
There are basically two aspects to this optimization:
1] Dynamically reducing the number of branches taken v.s. branches not
taken under the assumption that branches not taken are cheaper.
2] Statically minimizing the number of unconditional branches, saving space
and presumably time.
These two goals can conflict, but if they do it seems pretty clear that the
dynamic optimization should get preference. The main dynamic optimization is
changing the sense of a conditional test so that the more commonly taken branch
is the fall-through case. The problem is determining which branch is more
commonly taken.
The most clear-cut case is where one branch leads out of a loop and the other
is within. In this case, clearly the branch within the loop should be
preferred. The only added complication is that at some point in the loop there
has to be a backward branch, and it is preferable for this branch to be
conditional, since an unconditional branch is just a waste of time.
In the absence of such good information, we can attempt to guess which branch
is more popular on the basis of difference in the cost between the two cases.
Min-max strategy suggests that we should choose the cheaper alternative, since
the percentagewise improvement is greater when the branch overhead is
significant with respect to the cost of the code branched to. A tractable
approximation of this is to compare only the costs of the two blocks
immediately branched to, since this would avoid having to do any hairy graph
walking to find all the code for the consequent and the alternative. It might
be worthwhile discriminating against ultra-expensive functions such as ERROR.
For this to work, we have to detect when one of the options is empty. In this
case, the next for one branch is a successor of the other branch, making the
comparison meaningless. We use dominator information to detect this situation.
When a branch is empty, one of the predecessors of the first block in the empty
branch will be dominated by the first block in the other branch. In such a
case we favor the empty branch, since that's about as cheap as you can get.
Statically minimizing branches is really a much more tractable problem, but
what literature there is makes it look hard. Clearly the thing to do is to use
a non-optimal heuristic algorithm.
A good possibility is to use an algorithm based on the depth first ordering.
We can modify the basic DFO algorithm so that it chooses an ordering which
favors any drop-thrus that we may choose for dynamic reasons. When we are
walking the graph, we walk the desired drop-thru arc last, which will place it
immediately after us in the DFO unless the arc is a retreating arc.
We scan through the DFO and whenever we find a block that hasn't been done yet,
we build a straight-line segment by setting the drop-thru to the unreached
successor block which has the lowest DFN greater than that for the block. We
move to the drop-thru block and repeat the process until there is no such
block. We then go back to our original scan through the DFO, looking for the
head of another straight-line segment.
This process will automagically implement all of the dynamic optimizations
described above as long as we favor the appropriate IF branch when creating the
DFO. Using the DFO will prevent us from making the back branch in a loop the
drop-thru, but we need to be clever about favoring IF branches within loops
while computing the DFO. The IF join will be favored without any special
effort, since we follow through the most favored path until we reach the end.
This needs some knowledge about the target machine, since on most machines
non-tail-recursive calls will use some sort of call instruction. In this case,
the call actually wants to drop through to the return point, rather than
dropping through to the beginning of the called function.
\chapter{VMR conversion}
\#|
Single-use let var continuation substitution not really correct, since it can
cause a spurious type error. Maybe we do want stuff to prove that an NLX can't
happen after all. Or go back to the idea of moving a combination arg to the
ref location, and having that use the ref cont (with its output assertion.)
This lossage doesn't seem very likely to actually happen, though.
[\#\#\# must-reach stuff wouldn't work quite as well as combination substitute in
psetq, etc., since it would fail when one of the new values is random code
(might unwind.)]
Is this really a general problem with eager type checking? It seems you could
argue that there was no type error in this code:
(+ :foo (throw 'up nil))
But we would signal an error.
Emit explicit you-lose operation when we do a move between two non-T ptypes,
even when type checking isn't on. Can this really happen? Seems we should
treat continuations like this as though type-check was true. Maybe LTN should
leave type-check true in this case, even when the policy is unsafe. (Do a type
check against NIL?)
At continuation use time, we may in general have to do both a coerce-to-t and a
type check, allocating two temporary TNs to hold the intermediate results.
VMR Control representation:
We represent all control transfer explicitly. In particular, :Conditional VOPs
take a single Target continuation and a Not-P flag indicating whether the sense
of the test is negated. Then an unconditional Branch VOP will be emitted
afterward if the other path isn't a drop-through.
So we linearize the code before VMR-conversion. This isn't a problem,
since there isn't much change in control flow after VMR conversion (none until
loop optimization requires introduction of header blocks.) It does make
cost-based branch prediction a bit ucky, though, since we don't have any cost
information in ICR. Actually, I guess we do have pretty good cost information
after LTN even before VMR conversion, since the most important thing to know is
which functions are open-coded.
|\#
VMR preserves the block structure of ICR, but replaces the nodes with a target
dependent virtual machine (VM) representation. Different implementations may
use different VMs without making major changes in the back end. The two main
components of VMR are Temporary Names (TNs) and Virtual OPerations (VOPs). TNs
represent the locations that hold values, and VOPs represent the operations
performed on the values.
A "primitive type" is a type meaningful at the VM level. Examples are Fixnum,
String-Char, Short-Float. During VMR conversion we use the primitive type of
an expression to determine both where we can store the result of the expression
and which type-specific implementations of an operation can be applied to the
value. [Ptype is a set of SCs == representation choices and representation
specific operations]
The VM specific definitions provide functions that do stuff like find the
primitive type corresponding to a type and test for primitive type subtypep.
Usually primitive types will be disjoint except for T, which represents all
types.
The primitive type T is special-cased. Not only does it overlap with all the
other types, but it implies a descriptor ("boxed" or "pointer") representation.
For efficiency reasons, we sometimes want to use
alternate representations for some objects such as numbers. The majority of
operations cannot exploit alternate representations, and would only be
complicated if they had to be able to convert alternate representations into
descriptors. A template can require an operand to be a descriptor by
constraining the operand to be of type T.
A TN can only represent a single value, so we bare the implementation of MVs at
this point. When we know the number of multiple values being handled, we use
multiple TNs to hold them. When the number of values is actually unknown, we
use a convention that is compatible with full function call.
Everything that is done is done by a VOP in VMR. Calls to simple primitive
functions such as + and CAR are translated to VOP equivalents by a table-driven
mechanism. This translation is specified by the particular VM definition; VMR
conversion makes no assumptions about which operations are primitive or what
operand types are worth special-casing. The default calling mechanisms and
other miscellaneous builtin features are implemented using standard VOPs that
must implemented by each VM.
Type information can be forgotten after VMR conversion, since all type-specific
operation selections have been made.
Simple type checking is explicitly done using CHECK-xxx VOPs. They act like
innocuous effectless/unaffected VOPs which return the checked thing as a
result. This allows loop-invariant optimization and common subexpression
elimination to remove redundant checks. All type checking is done at the time
the continuation is used.
Note that we need only check asserted types, since if type inference works, the
derived types will also be satisfied. We can check whichever is more
convenient, since both should be true.
Constants are turned into special Constant TNs, which are wired down in a SC
that is determined by their type. The VM definition provides a function that
returns constant a TN to represent a Constant Leaf.
Each component has a constant pool. There is a register dedicated to holding
the constant pool for the current component. The back end allocates
non-immediate constants in the constant pool when it discovers them during
translation from ICR.
[\#\#\# Check that we are describing what is actually implemented. But this
really isn't very good in the presence of interesting unboxed
representations...]
Since LTN only deals with values from the viewpoint of the receiver, we must be
prepared during the translation pass to do stuff to the continuation at the
time it is used.
-- If a VOP yields more values than are desired, then we must create TNs to
hold the discarded results. An important special-case is continuations
whose value is discarded. These continuations won't be annotated at all.
In the case of a Ref, we can simply skip evaluation of the reference when
the continuation hasn't been annotated. Although this will eliminate
bogus references that for some reason weren't optimized away, the real
purpose is to handle deferred references.
-- If a VOP yields fewer values than desired, then we must default the extra
values to NIL.
-- If a continuation has its type-check flag set, then we must check the type
of the value before moving it into the result location. In general, this
requires computing the result in a temporary, and having the type-check
operation deliver it in the actual result location.
-- If the template's result type is T, then we must generate a boxed
temporary to compute the result in when the continuation's type isn't T.
We may also need to do stuff to the arguments when we generate code for a
template. If an argument continuation isn't annotated, then it must be a
deferred reference. We use the leaf's TN instead. We may have to do any of
the above use-time actions also. Alternatively, we could avoid hair by not
deferring references that must be type-checked or may need to be boxed.
\section{Stack analysis}
Think of this as a lifetime problem: a values generator is a write and a values
receiver is a read. We want to annotate each VMR-Block with the unknown-values
continuations that are live at that point. If we do a control transfer to a
place where fewer continuations are live, then we must deallocate the newly
dead continuations.
We want to convince ourselves that values deallocation based on lifetime
analysis actually works. In particular, we need to be sure that it doesn't
violate the required stack discipline. It is clear that it is impossible to
deallocate the values before they become dead, since later code may decide to
use them. So the only thing we need to ensure is that the "right" time isn't
later than the time that the continuation becomes dead.
The only reason why we couldn't deallocate continuation A as soon as it becomes
dead would be that there is another continuation B on top of it that isn't dead
(since we can only deallocate the topmost continuation).
The key to understanding why this can't happen is that each continuation has
only one read (receiver). If B is on top of A, then it must be the case that A
is live at the receiver for B. This means that it is impossible for B to be
live without A being live.
The reason that we don't solve this problem using a normal iterative flow
analysis is that we also need to know the ordering of the continuations on the
stack so that we can do deallocation. When it comes time to discard values, we
want to know which discarded continuation is on the bottom so that we can reset
SP to its start.
[I suppose we could also decrement SP by the aggregate size of the discarded
continuations.] Another advantage of knowing the order in which we expect
continuations to be on the stack is that it allows us to do some consistency
checking. Also doing a localized graph walk around the values-receiver is
likely to be much more efficient than doing an iterative flow analysis problem
over all the code in the component (not that big a consideration.)
\#|
Actually, what we do is do a backward graph walk from each unknown-values
receiver. As we go, we mark each walked block with ther ordered list of
continuations we believe are on the stack. Starting with an empty stack, we:
-- When we encounter another unknown-values receiver, we push that
continuation on our simulated stack.
-- When we encounter a receiver (which had better be for the topmost
continuation), we pop that continuation.
-- When we pop all continuations, we terminate our walk.
[\#\#\# not quite right... It seems we may run into "dead values" during the
graph walk too. It seems that we have to check if the pushed continuation is
on stack top, and if not, add it to the ending stack so that the post-pass will
discard it.]
[\#\#\# Also, we can't terminate our walk just because we hit a block previously
walked. We have to compare the the End-Stack with the values received along
the current path: if we have more values on our current walk than on the walk
that last touched the block, then we need to re-walk the subgraph reachable
from from that block, using our larger set of continuations. It seems that our
actual termination condition is reaching a block whose End-Stack is already EQ
to our current stack.]
If at the start, the block containing the values receiver has already been
walked, the we skip the walk for that continuation, since it has already been
handled by an enclosing values receiver. Once a walk has started, we
ignore any signs of a previous walk, clobbering the old result with our own,
since we enclose that continuation, and the previous walk doesn't take into
consideration the fact that our values block underlies its own.
When we are done, we have annotated each block with the stack current both at
the beginning and at the end of that block. Blocks that aren't walked don't
have anything on the stack either place (although they may hack MVs
internally).
We then scan all the blocks in the component, looking for blocks that have
predecessors with a different ending stack than that block's starting stack.
(The starting stack had better be a tail of the predecessor's ending stack.)
We insert a block intervening between all of these predecessors that sets SP to
the end of the values for the continuation that should be on stack top. Of
course, this pass needn't be done if there aren't any global unknown MVs.
Also, if we find any block that wasn't reached during the walk, but that USEs
an outside unknown-values continuation, then we know that the DEST can't be
reached from this point, so the values are unused. We either insert code to
pop the values, or somehow mark the code to prevent the values from ever being
pushed. (We could cause the popping to be done by the normal pass if we
iterated over the pushes beforehand, assigning a correct END-STACK.)
[\#\#\# But I think that we have to be a bit clever within blocks, given the
possibility of blocks being joined. We could collect some unknown MVs in a
block, then do a control transfer out of the receiver, and this control
transfer could be squeezed out by merging blocks. How about:
(tagbody
(return
(multiple-value-prog1 (foo)
(when bar
(go UNWIND))))
UNWIND
(return
(multiple-value-prog1 (baz)
bletch)))
But the problem doesn't happen here (can't happen in general?) since a node
buried within a block can't use a continuation outside of the block. In fact,
no block can have more then one PUSH continuation, and this must always be be
last continuation. So it is trivially (structurally) true that all pops come
before any push.
[\#\#\# But not really: the DEST of an embedded continuation may be outside the
block. There can be multiple pushes, and we must find them by iterating over
the uses of MV receivers in LTN. But it would be hard to get the order right
this way. We could easily get the order right if we added the generators as we
saw the uses, except that we can't guarantee that the continuations will be
annotated at that point. (Actually, I think we only need the order for
consistency checks, but that is probably worthwhile). I guess the thing to do
is when we process the receiver, add the generator blocks to the
Values-Generators, then do a post-pass that re-scans the blocks adding the
pushes.]
I believe that above concern with a dead use getting mashed inside a block
can't happen, since the use inside the block must be the only use, and if the
use isn't reachable from the push, then the use is totally unreachable, and
should have been deleted, which would prevent the prevent it from ever being
annotated.
]
]
|\#
We find the partial ordering of the values globs for unknown values
continuations in each environment. We don't have to scan the code looking for
unknown values continuations since LTN annotates each block with the
continuations that were popped and not pushed or pushed and not popped. This
is all we need to do the inter-block analysis.
After we have found out what stuff is on the stack at each block boundary, we
look for blocks with predecessors that have junk on the stack. For each such
block, we introduce a new block containing code to restore the stack pointer.
Since unknown-values continuations are represented as <start, count>, we can
easily pop a continuation using the Start TN.
Note that there is only doubt about how much stuff is on the control stack,
since only it is used for unknown values. Any special stacks such as number
stacks will always have a fixed allocation.
\section{Non-local exit}
If the starting and ending continuations are not in the same environment, then
the control transfer is a non-local exit. In this case just call Unwind with
the appropriate stack pointer, and let the code at the re-entry point worry
about fixing things up.
It seems like maybe a good way to organize VMR conversion of NLX would be to
have environment analysis insert funny functions in new interposed cleanup
blocks. The thing is that we need some way for VMR conversion to:
1] Get its hands on the returned values.
2] Do weird control shit.
3] Deliver the values to the original continuation destination.
I.e. we need some way to interpose arbitrary code in the path of value
delivery.
What we do is replace the NLX uses of the continuation with another
continuation that is received by a MV-Call to %NLX-VALUES in a cleanup block
that is interposed between the NLX uses and the old continuation's block. The
MV-Call uses the original continuation to deliver it's values to.
[Actually, it's not really important that this be an MV-Call, since it has to
be special-cased by LTN anyway. Or maybe we would want it to be an MV call.
If did normal LTN analysis of an MV call, it would force the returned values
into the unknown values convention, which is probably pretty convenient for use
in NLX.
Then the entry code would have to use some special VOPs to receive the unknown
values. But we probably need special VOPs for NLX entry anyway, and the code
can share with the call VOPs. Also we probably need the technology anyway,
since THROW will use truly unknown values.]
On entry to a dynamic extent that has non-local-exists into it (always at an
ENTRY node), we take a complete snapshot of the dynamic state:
the top pointers for all stacks
current Catch and Unwind-Protect
current special binding (binding stack pointer in shallow binding)
We insert code at the re-entry point which restores the saved dynamic state.
All TNs live at a NLX EP are forced onto the stack, so we don't have to restore
them, and we don't have to worry about getting them saved.

713
doc/cmucl/internals/object.tex Normal file
View file

@ -0,0 +1,713 @@
\chapter{Object Format}
\section{Tagging}
The following is a key of the three bit low-tagging scheme:
\begin{description}
\item[000] even fixnum
\item[001] function pointer
\item[010] even other-immediate (header-words, characters, symbol-value trap value, etc.)
\item[011] list pointer
\item[100] odd fixnum
\item[101] structure pointer
\item[110] odd other immediate
\item[111] other-pointer to data-blocks (other than conses, structures,
and functions)
\end{description}
This tagging scheme forces a dual-word alignment of data-blocks on the heap,
but this can be pretty negligible:
\begin{itemize}
\item RATIOS and COMPLEX must have a header-word anyway since they are not a
major type. This wastes one word for these infrequent data-blocks since
they require two words for the data.
\item BIGNUMS must have a header-word and probably contain only one other word
anyway, so we probably don't waste any words here. Most bignums just
barely overflow fixnums, that is by a bit or two.
\item Single and double FLOATS?
no waste, or
one word wasted
\item SYMBOLS have a pad slot (current called the setf function, but unused.)
\end{itemize}
Everything else is vector-like including code, so these probably take up
so many words that one extra one doesn't matter.
\section{GC Comments}
Data-Blocks comprise only descriptors, or they contain immediate data and raw
bits interpreted by the system. GC must skip the latter when scanning the
heap, so it does not look at a word of raw bits and interpret it as a pointer
descriptor. These data-blocks require headers for GC as well as for operations
that need to know how to interpret the raw bits. When GC is scanning, and it
sees a header-word, then it can determine how to skip that data-block if
necessary. Header-Words are tagged as other-immediates. See the sections
"Other-Immediates" and "Data-Blocks and Header-Words" for comments on
distinguishing header-words from other-immediate data. This distinction is
necessary since we scan through data-blocks containing only descriptors just as
we scan through the heap looking for header-words introducing data-blocks.
Data-Blocks containing only descriptors do not require header-words for GC
since the entire data-block can be scanned by GC a word at a time, taking
whatever action is necessary or appropriate for the data in that slot. For
example, a cons is referenced by a descriptor with a specific tag, and the
system always knows the size of this data-block. When GC encounters a pointer
to a cons, it can transport it into the new space, and when scanning, it can
simply scan the two words manifesting the cons interpreting each word as a
descriptor. Actually there is no cons tag, but a list tag, so we make sure the
cons is not nil when appropriate. A header may still be desired if the pointer
to the data-block does not contain enough information to adequately maintain
the data-block. An example of this is a simple-vector containing only
descriptor slots, and we attach a header-word because the descriptor pointing
to the vector lacks necessary information -- the type of the vector's elements,
its length, etc.
There is no need for a major tag for GC forwarding pointers. Since the tag
bits are in the low end of the word, a range check on the start and end of old
space tells you if you need to move the thing. This is all GC overhead.
\section{Structures}
A structure descriptor has the structure lowtag type code, making
{\tt structurep} a fast operation. A structure
data-block has the following format:
\begin{verbatim}
-------------------------------------------------------
| length (24 bits) | Structure header type (8 bits) |
-------------------------------------------------------
| structure type name (a symbol) |
-------------------------------------------------------
| structure slot 0 |
-------------------------------------------------------
| ... structure slot length - 2 |
-------------------------------------------------------
\end{verbatim}
The header word contains the structure length, which is the number of words
(other than the header word.) The length is always at least one, since the
first word of the structure data is the structure type name.
\section{Fixnums}
A fixnum has one of the following formats in 32 bits:
\begin{verbatim}
-------------------------------------------------------
| 30 bit 2's complement even integer | 0 0 0 |
-------------------------------------------------------
\end{verbatim}
or
\begin{verbatim}
-------------------------------------------------------
| 30 bit 2's complement odd integer | 1 0 0 |
-------------------------------------------------------
\end{verbatim}
Effectively, there is one tag for immediate integers, two zeros. This buys one
more bit for fixnums, and now when these numbers index into simple-vectors or
offset into memory, they point to word boundaries on 32-bit, byte-addressable
machines. That is, no shifting need occur to use the number directly as an
offset.
This format has another advantage on byte-addressable machines when fixnums are
offsets into vector-like data-blocks, including structures. Even though we
previously mentioned data-blocks are dual-word aligned, most indexing and slot
accessing is word aligned, and so are fixnums with effectively two tag bits.
Two tags also allow better usage of special instructions on some machines that
can deal with two low-tag bits but not three.
Since the two bits are zeros, we avoid having to mask them off before using the
words for arithmetic, but division and multiplication require special shifting.
\section{Other-immediates}
As for fixnums, there are two different three-bit lowtag codes for
other-immediate, allowing 64 other-immediate types:
\begin{verbatim}
----------------------------------------------------------------
| Data (24 bits) | Type (8 bits with low-tag) | 1 0 |
----------------------------------------------------------------
\end{verbatim}
The type-code for an other-immediate type is considered to include the two
lowtag bits. This supports the concept of a single "type code" namespace for
all descriptors, since the normal lowtag codes are disjoint from the
other-immediate codes.
For other-pointer objects, the full eight bits of the header type code are used
as the type code for that kind of object. This is why we use two lowtag codes
for other-immediate types: each other-pointer object needs a distinct
other-immediate type to mark its header.
The system uses the other-immediate format for characters,
the {\tt symbol-value} unbound trap value, and header-words for data-blocks on
the heap. The type codes are laid out to facilitate range checks for common
subtypes; for example, all numbers will have contiguous type codes which are
distinct from the contiguous array type codes. See section
\ref{data-blocks-and-o-i} for details.
\section{Data-Blocks and Header-Word Format}
Pointers to data-blocks have the following format:
\begin{verbatim}
----------------------------------------------------------------
| Dual-word address of data-block (29 bits) | 1 1 1 |
----------------------------------------------------------------
\end{verbatim}
The word pointed to by the above descriptor is a header-word, and it has the
same format as an other-immediate:
\begin{verbatim}
----------------------------------------------------------------
| Data (24 bits) | Type (8 bits with low-tag) | 0 1 0 |
----------------------------------------------------------------
\end{verbatim}
This is convenient for scanning the heap when GC'ing, but it does mean that
whenever GC encounters an other-immediate word, it has to do a range check on
the low byte to see if it is a header-word or just a character (for example).
This is easily acceptable performance hit for scanning.
The system interprets the data portion of the header-word for non-vector
data-blocks as the word length excluding the header-word. For example, the
data field of the header for ratio and complex numbers is two, one word each
for the numerator and denominator or for the real and imaginary parts.
For vectors and data-blocks representing Lisp objects stored like vectors, the
system ignores the data portion of the header-word:
\begin{verbatim}
----------------------------------------------------------------
| Unused Data (24 bits) | Type (8 bits with low-tag) | 0 1 0 |
----------------------------------------------------------------
| Element Length of Vector (30 bits) | 0 0 |
----------------------------------------------------------------
\end{verbatim}
Using a separate word allows for much larger vectors, and it allows {\tt
length} to simply access a single word without masking or shifting. Similarly,
the header for complex arrays and vectors has a second word, following the
header-word, the system uses for the fill pointer, so computing the length of
any array is the same code sequence.
\section{Data-Blocks and Other-immediates Typing}
\label{data-blocks-and-o-i}
These are the other-immediate types. We specify them including all low eight
bits, including the other-immediate tag, so we can think of the type bits as
one type -- not an other-immediate major type and a subtype. Also, fetching a
byte and comparing it against a constant is more efficient than wasting even a
small amount of time shifting out the other-immediate tag to compare against a
five bit constant.
\begin{verbatim}
Number (< 30)
bignum 10
ratio 14
single-float 18
double-float 22
complex 26
Array (>= 30 code 86)
Simple-Array (>= 20 code 70)
simple-array 30
Vector (>= 34 code 82)
simple-string 34
simple-bit-vector 38
simple-vector 42
(simple-array (unsigned-byte 2) (*)) 46
(simple-array (unsigned-byte 4) (*)) 50
(simple-array (unsigned-byte 8) (*)) 54
(simple-array (unsigned-byte 16) (*)) 58
(simple-array (unsigned-byte 32) (*)) 62
(simple-array single-float (*)) 66
(simple-array double-float (*)) 70
complex-string 74
complex-bit-vector 78
(array * (*)) -- general complex vector. 82
complex-array 86
code-header-type 90
function-header-type 94
closure-header-type 98
funcallable-instance-header-type 102
unused-function-header-1-type 106
unused-function-header-2-type 110
unused-function-header-3-type 114
closure-function-header-type 118
return-pc-header-type (a.k.a LRA) 122
value-cell-header-type 126
symbol-header-type 130
base-character-type 134
system-area-pointer-type (header type) 138
unbound-marker 142
weak-pointer-type 146
structure-header-type 150
\end{verbatim}
\section{Strings}
All strings in the system are C-null terminated. This saves copying the bytes
when calling out to C. The only time this wastes memory is when the string
contains a multiple of eight characters, and then the system allocates two more
words (since Lisp objects are dual-word aligned) to hold the C-null byte.
Since the system will make heavy use of C routines for systems calls and
libraries that save reimplementation of higher level operating system
functionality (such as pathname resolution or current directory computation),
saving on copying strings for C should make C call out more efficient.
The length word in a string header, see section "Data-Blocks and Header-Word
Format", counts only the characters truly in the Common Lisp string.
Allocation and GC will have to know to handle the extra C-null byte, and GC
already has to deal with rounding up various objects to dual-word alignment.
\section{Symbols and NIL}
Symbol data-block has the following format:
\begin{verbatim}
-------------------------------------------------------
| 7 (data-block words) | Symbol Type (8 bits) |
-------------------------------------------------------
| Value Descriptor |
-------------------------------------------------------
| Function Pointer |
-------------------------------------------------------
| Raw Function Address |
-------------------------------------------------------
| Setf Function |
-------------------------------------------------------
| Property List |
-------------------------------------------------------
| Print Name |
-------------------------------------------------------
| Package |
-------------------------------------------------------
\end{verbatim}
Most of these slots are self-explanatory given what symbols must do in Common
Lisp, but a couple require comments. We added the Raw Function Address slot to
speed up named call which is the most common calling convention. This is a
non-descriptor slot, but since objects are dual word aligned, the value
inherently has fixnum low-tag bits. The GC method for symbols must know to
update this slot. The Setf Function slot is currently unused, but we had an
extra slot due to adding Raw Function Address since objects must be dual-word
aligned.
The issues with nil are that we want it to act like a symbol, and we need list
operations such as CAR and CDR to be fast on it. CMU Common Lisp solves this
by putting nil as the first object in static space, where other global values
reside, so it has a known address in the system:
\begin{verbatim}
------------------------------------------------------- <-- space
| 0 | start
-------------------------------------------------------
| 7 (data-block words) | Symbol Type (8 bits) |
------------------------------------------------------- <-- nil
| Value/CAR |
-------------------------------------------------------
| Definition/CDR |
-------------------------------------------------------
| Raw Function Address |
-------------------------------------------------------
| Setf Function |
-------------------------------------------------------
| Property List |
-------------------------------------------------------
| Print Name |
-------------------------------------------------------
| Package |
-------------------------------------------------------
| ... |
-------------------------------------------------------
\end{verbatim}
In addition, we make the list typed pointer to nil actually point past the
header word of the nil symbol data-block. This has usefulness explained below.
The value and definition of nil are nil. Therefore, any reference to nil used
as a list has quick list type checking, and CAR and CDR can go right through
the first and second words as if nil were a cons object.
When there is a reference to nil used as a symbol, the system adds offsets to
the address the same as it does for any symbol. This works due to a
combination of nil pointing past the symbol header-word and the chosen list and
other-pointer type tags. The list type tag is four less than the other-pointer
type tag, but nil points four additional bytes into its symbol data-block.
;;;; Array Headers.
The array-header data-block has the following format:
\begin{verbatim}
----------------------------------------------------------------
| Header Len (24 bits) = Array Rank +5 | Array Type (8 bits) |
----------------------------------------------------------------
| Fill Pointer (30 bits) | 0 0 |
----------------------------------------------------------------
| Available Elements (30 bits) | 0 0 |
----------------------------------------------------------------
| Data Vector (29 bits) | 1 1 1 |
----------------------------------------------------------------
| Displacement (30 bits) | 0 0 |
----------------------------------------------------------------
| Displacedp (29 bits) -- t or nil | 1 1 1 |
----------------------------------------------------------------
| Range of First Index (30 bits) | 0 0 |
----------------------------------------------------------------
.
.
.
\end{verbatim}
The array type in the header-word is one of the eight-bit patterns from section
"Data-Blocks and Other-immediates Typing", indicating that this is a complex
string, complex vector, complex bit-vector, or a multi-dimensional array. The
data portion of the other-immediate word is the length of the array header
data-block. Due to its format, its length is always five greater than the
array's number of dimensions. The following words have the following
interpretations and types:
\begin{description}
\item[Fill Pointer:]
This is a fixnum indicating the number of elements in the data vector
actually in use. This is the logical length of the array, and it is
typically the same value as the next slot. This is the second word, so
LENGTH of any array, with or without an array header, is just four bytes
off the pointer to it.
\item[Available Elements:]
This is a fixnum indicating the number of elements for which there is
space in the data vector. This is greater than or equal to the logical
length of the array when it is a vector having a fill pointer.
\item[Data Vector:]
This is a pointer descriptor referencing the actual data of the array.
This a data-block whose first word is a header-word with an array type as
described in sections "Data-Blocks and Header-Word Format" and
"Data-Blocks and Other-immediates Typing"
\item[Displacement:]
This is a fixnum added to the computed row-major index for any array.
This is typically zero.
\item[Displacedp:]
This is either t or nil. This is separate from the displacement slot, so
most array accesses can simply add in the displacement slot. The rare
need to know if an array is displaced costs one extra word in array
headers which probably aren't very frequent anyway.
\item[Range of First Index:]
This is a fixnum indicating the number of elements in the first dimension
of the array. Legal index values are zero to one less than this number
inclusively. IF the array is zero-dimensional, this slot is
non-existent.
\item[... (remaining slots):]
There is an additional slot in the header for each dimension of the
array. These are the same as the Range of First Index slot.
\end{description}
\section{Bignums}
Bignum data-blocks have the following format:
\begin{verbatim}
-------------------------------------------------------
| Length (24 bits) | Bignum Type (8 bits) |
-------------------------------------------------------
| least significant bits |
-------------------------------------------------------
.
.
.
\end{verbatim}
The elements contain the two's complement representation of the integer with
the least significant bits in the first element or closer to the header. The
sign information is in the high end of the last element.
\section{Code Data-Blocks}
A code data-block is the run-time representation of a "component". A component
is a connected portion of a program's flow graph that is compiled as a single
unit, and it contains code for many functions. Some of these functions are
callable from outside of the component, and these are termed "entry points".
Each entry point has an associated user-visible function data-block (of type
{\tt function}). The full call convention provides for calling an entry point
specified by a function object.
Although all of the function data-blocks for a component's entry points appear
to the user as distinct objects, the system keeps all of the code in a single
code data-block. The user-visible function object is actually a pointer into
the middle of a code data-block. This allows any control transfer within a
component to be done using a relative branch.
Besides a function object, there are other kinds of references into the middle
of a code data-block. Control transfer into a function also occurs at the
return-PC for a call. The system represents a return-PC somewhat similarly to
a function, so GC can also recognize a return-PC as a reference to a code
data-block. This representation is known as a Lisp Return Address (LRA).
It is incorrect to think of a code data-block as a concatenation of "function
data-blocks". Code for a function is not emitted in any particular order with
respect to that function's function-header (if any). The code following a
function-header may only be a branch to some other location where the
function's "real" definition is.
The following are the three kinds of pointers to code data-blocks:
\begin{description}
\item[Code pointer (labeled A below):]
A code pointer is a descriptor, with other-pointer low-tag bits, pointing
to the beginning of the code data-block. The code pointer for the
currently running function is always kept in a register (CODE). In
addition to allowing loading of non-immediate constants, this also serves
to represent the currently running function to the debugger.
\item[LRA (labeled B below):]
The LRA is a descriptor, with other-pointer low-tag bits, pointing
to a location for a function call. Note that this location contains no
descriptors other than the one word of immediate data, so GC can treat
LRA locations the same as instructions.
\item[Function (labeled C below):]
A function is a descriptor, with function low-tag bits, that is user
callable. When a function header is referenced from a closure or from
the function header's self-pointer, the pointer has other-pointer low-tag
bits, instead of function low-tag bits. This ensures that the internal
function data-block associated with a closure appears to be uncallable
(although users should never see such an object anyway).
Information about functions that is only useful for entry points is kept
in some descriptors following the function's self-pointer descriptor.
All of these together with the function's header-word are known as the
"function header". GC must be able to locate the function header. We
provide for this by chaining together the function headers in a NIL
terminated list kept in a known slot in the code data-block.
\end{description}
A code data-block has the following format:
\begin{verbatim}
A -->
****************************************************************
| Header-Word count (24 bits) | Code-Type (8 bits) |
----------------------------------------------------------------
| Number of code words (fixnum tag) |
----------------------------------------------------------------
| Pointer to first function header (other-pointer tag) |
----------------------------------------------------------------
| Debug information (structure tag) |
----------------------------------------------------------------
| First constant (a descriptor) |
----------------------------------------------------------------
| ... |
----------------------------------------------------------------
| Last constant (and last word of code header) |
----------------------------------------------------------------
| Some instructions (non-descriptor) |
----------------------------------------------------------------
| (pad to dual-word boundary if necessary) |
B -->
****************************************************************
| Word offset from code header (24) | Return-PC-Type (8) |
----------------------------------------------------------------
| First instruction after return |
----------------------------------------------------------------
| ... more code and LRA header-words |
----------------------------------------------------------------
| (pad to dual-word boundary if necessary) |
C -->
****************************************************************
| Offset from code header (24) | Function-Header-Type (8) |
----------------------------------------------------------------
| Self-pointer back to previous word (with other-pointer tag) |
----------------------------------------------------------------
| Pointer to next function (other-pointer low-tag) or NIL |
----------------------------------------------------------------
| Function name (a string or a symbol) |
----------------------------------------------------------------
| Function debug arglist (a string) |
----------------------------------------------------------------
| Function type (a list-style function type specifier) |
----------------------------------------------------------------
| Start of instructions for function (non-descriptor) |
----------------------------------------------------------------
| More function headers and instructions and return PCs, |
| until we reach the total size of header-words + code |
| words. |
----------------------------------------------------------------
\end{verbatim}
The following are detailed slot descriptions:
\begin{description}
\item[Code data-block header-word:]
The immediate data in the code data-block's header-word is the number of
leading descriptors in the code data-block, the fixed overhead words plus
the number of constants. The first non-descriptor word, some code,
appears at this word offset from the header.
\item[Number of code words:]
The total number of non-header-words in the code data-block. The total
word size of the code data-block is the sum of this slot and the
immediate header-word data of the previous slot.
header-word.
\item[Pointer to first function header:]
A NIL-terminated list of the function headers for all entry points to
this component.
\item[Debug information:]
The DEBUG-INFO structure describing this component. All information that
the debugger wants to get from a running function is kept in this
structure. Since there are many functions, the current PC is used to
locate the appropriate debug information. The system keeps the debug
information separate from the function data-block, since the currently
running function may not be an entry point. There is no way to recover
the function object for the currently running function, since this
data-block may not exist.
\item[First constant ... last constant:]
These are the constants referenced by the component, if there are any.
\vspace{1ex}
\item[LRA header word:]
The immediate header-word data is the word offset from the enclosing code
data-block's header-word to this word. This allows GC and the debugger
to easily recover the code data-block from a LRA. The code at the
return point restores the current code pointer using a subtract immediate
of the offset, which is known at compile time.
\vspace{1ex}
\item[Function entry point header-word:]
The immediate header-word data is the word offset from the enclosing code
data-block's header-word to this word. This is the same as for the
retrun-PC header-word.
\item[Self-pointer back to header-word:]
In a non-closure function, this self-pointer to the previous header-word
allows the call sequence to always indirect through the second word in a
user callable function. See section "Closure Format". With a closure,
indirecting through the second word gets you a function header-word. The
system ignores this slot in the function header for a closure, since it
has already indirected once, and this slot could be some random thing
that causes an error if you jump to it. This pointer has an
other-pointer tag instead of a function pointer tag, indicating it is not
a user callable Lisp object.
\item[Pointer to next function:]
This is the next link in the thread of entry point functions found in
this component. This value is NIL when the current header is the last
entry point in the component.
\item[Function name:]
This function's name (for printing). If the user defined this function
with DEFUN, then this is the defined symbol, otherwise it is a
descriptive string.
\item[Function debug arglist:]
A printed string representing the function's argument list, for human
readability. If it is a macroexpansion function, then this is the
original DEFMACRO arglist, not the actual expander function arglist.
\item[Function type:]
A list-style function type specifier representing the argument signature
and return types for this function. For example,
\begin{verbatim}
(function (fixnum fixnum fixnum) fixnum)
\end{verbatim}
or
\begin{verbatim}
(function (string &key (:start unsigned-byte)) string)
\end{verbatim}
This information is intended for machine readablilty, such as by the
compiler.
\end{description}
\section{Closure Format}
A closure data-block has the following format:
\begin{verbatim}
----------------------------------------------------------------
| Word size (24 bits) | Closure-Type (8 bits) |
----------------------------------------------------------------
| Pointer to function header (other-pointer low-tag) |
----------------------------------------------------------------
| . |
| Environment information |
| . |
----------------------------------------------------------------
\end{verbatim}
A closure descriptor has function low-tag bits. This means that a descriptor
with function low-tag bits may point to either a function header or to a
closure. The idea is that any callable Lisp object has function low-tag bits.
Insofar as call is concerned, we make the format of closures and non-closure
functions compatible. This is the reason for the self-pointer in a function
header. Whenever you have a callable object, you just jump through the second
word, offset some bytes, and go.
\section{Function call}
Due to alignment requirements and low-tag codes, it is not possible to use a
hardware call instruction to compute the LRA. Instead the LRA
for a call is computed by doing an add-immediate to the start of the code
data-block.
An advantage of using a single data-block to represent both the descriptor and
non-descriptor parts of a function is that both can be represented by a
single pointer. This reduces the number of memory accesses that have to be
done in a full call. For example, since the constant pool is implicit in a
LRA, a call need only save the LRA, rather than saving both the
return PC and the constant pool.
\section{Memory Layout}
CMU Common Lisp has four spaces, read-only, static, dynamic-0, and dynamic-1.
Read-only contains objects that the system never modifies, moves, or reclaims.
Static space contains some global objects necessary for the system's runtime or
performance (since they are located at a known offset at a know address), and
the system never moves or reclaims these. However, GC does need to scan static
space for references to moved objects. Dynamic-0 and dynamic-1 are the two
heap areas for stop-and-copy GC algorithms.
What global objects are at the head of static space???
\begin{verbatim}
NIL
eval::*top-of-stack*
lisp::*current-catch-block*
lisp::*current-unwind-protect*
FLAGS (RT only)
BSP (RT only)
HEAP (RT only)
\end{verbatim}
In addition to the above spaces, the system has a control stack, binding stack,
and a number stack. The binding stack contains pairs of descriptors, a symbol
and its previous value. The number stack is the same as the C stack, and the
system uses it for non-Lisp objects such as raw system pointers, saving
non-Lisp registers, parts of bignum computations, etc.
\section{System Pointers}
The system pointers reference raw allocated memory, data returned by foreign
function calls, etc. The system uses these when you need a pointer to a
non-Lisp block of memory, using an other-pointer. This provides the greatest
flexibility by relieving contraints placed by having more direct references
that require descriptor type tags.
A system area pointer data-block has the following format:
\begin{verbatim}
-------------------------------------------------------
| 1 (data-block words) | SAP Type (8 bits) |
-------------------------------------------------------
| system area pointer |
-------------------------------------------------------
\end{verbatim}
"SAP" means "system area pointer", and much of our code contains this naming
scheme. We don't currently restrict system pointers to one area of memory, but
if they do point onto the heap, it is up to the user to prevent being screwed
by GC or whatever.

120
doc/cmucl/internals/outline.txt Normal file
View file

@ -0,0 +1,120 @@
Todo:
fasl.tex
In good shape.
object.tex
Fairly good, but should probably be integrated with description of primitives
in vm.tex.
front.tex
Needs updating cleanup scan. Not too bad.
middle.tex
Need VMR overview. New names for GTN/LTN? Needs general cleanup, but not too
bad. NLX and stack are the worst.
back.tex
Pack and assembler need more info. General cleanup.
compiler-overview.tex
Adapt introductory material from /../fred/usr/ram/comp.mss, pap:talk.mss
Division between ICR overview and ICR convert needs work.
debugger.tex
Needs much work. Merge much info from debug-info and debug-int. Duplicating a
fair amount of stuff in the source may make sense where, since this is a part
of the system that is generally interesting. And also, a part that people
building on CMU CL might want to understand.
glossary.tex
Finish, integrate w/ main text?
interpreter.tex
Very sketchy and tentative. Needs to be fleshed out from the code.
retargeting.tex
Very rough. Needs to be merged with parts of vm.tex (call vops). Needs some
additional text. Documentation of assembler, and all other exported
interfaces. (Generate defined VOP descriptions from the core, keyed to files?)
vm.tex
This file should probably cease to exist, going into object, retargeting and
introductory material. [Also other scrap in stuff/]
[VMR and ICR overview also needed...]
architecture.tex
Missing sections on startup code, compiling, building.
environment.tex
Needs to be written: type system and info database interfaces.
interface.tex
Needs to be written: source paths and error message utilities.
lowlev.tex
Needs to be written. All manner of low-level stuff: memory layout and
management, core file format, C interface, low-level debugging (and ldb.)
Several different audiences:
-- Curious compiler implementors (not a big priority. Downplay academic
aspects, i.e. comparisons to other techniques, analysis of limitations,
future work...) Compiler part can be more academic, and include some
justifications of other design decisions.
-- System maintainers.
-- People retargeting the compiler.
-- People bringing up the system in a new environment.
Sys arch part:
Package + file structure [system.txt]
system building [compiling.txt]
bootstrapping & cross compiling
Compiler design:
Overview (mirror structure of rest of the part)
ICR data structure
Front end [front.tex]
Basic VMR data structures (no back-end stuff)
Middle end [middle.tex]
Back end + data structures [back.tex]
Error system interface
Source tracking
Compiler retargeting:
VM definition concepts [porting.txt, mail.txt, retargeting.tex]
SCs, SBs, primitive-types
Defining VOPS
time specification
defining
and using the assembler
Required VOPs [internal.txt, lowlev.txt, vm.mss]
Standard primitives [vm.mss] (broken down by type, parallels object format
section structure.)
Customizing VMR conversion
multiple hardware
constant operands
VM specific transforms
special-case IR2 convert methods
Run-time system:
type system
info database
Data format [object.tex]
Debugger:
Info format [debug.txt]
Stack parsing [debug.txt]
Breakpoints
Internal errors
Signals
Memory management: [William]
heap Layout
stacks
GC
misc implementation stuff: foreign call, assembly routines [lowlev.txt]
LDB and low-level debugging
core file format [William]
fasl format [fasl.tex]

1082
doc/cmucl/internals/retargeting.tex Normal file
View file

File diff suppressed because it is too large Load diff

4150
doc/cmucl/internals/rtguts.mss Normal file
View file

File diff suppressed because it is too large Load diff

7
doc/cmucl/internals/run-time.tex Normal file
View file

@ -0,0 +1,7 @@
\part{Run-Time system}
\include{environment}
\include{interpreter}
\include{debugger}
\include{object}
\include{lowlev}
\include{fasl}

1454
doc/cmucl/internals/vm.tex Normal file
View file

File diff suppressed because it is too large Load diff

1006
doc/compiler.sgml Normal file
View file

File diff suppressed because it is too large Load diff

87
doc/efficiency.sgml Normal file
View file

@ -0,0 +1,87 @@
<chapter id="efficiency"><title>Efficiency</>
<para>FIXME: The material in the &CMUCL; manual about getting good
performance from the compiler should be reviewed, reformatted in
DocBook, lightly edited for &SBCL;, and substituted into this
manual. In the meantime, the original &CMUCL; manual is still 95+%
correct for the &SBCL; version of the &Python; compiler. See the
sections
<itemizedlist>
<listitem><para>Advanced Compiler Use and Efficiency Hints</></>
<listitem><para>Advanced Compiler Introduction</></>
<listitem><para>More About Types in Python</></>
<listitem><para>Type Inference</></>
<listitem><para>Source Optimization</></>
<listitem><para>Tail Recursion</></>
<listitem><para>Local Call</></>
<listitem><para>Block Compilation</></>
<listitem><para>Inline Expansion</></>
<listitem><para>Object Representation</></>
<listitem><para>Numbers</></>
<listitem><para>General Efficiency Hints</></>
<listitem><para>Efficiency Notes</></>
</itemizedlist>
</para>
<para>Besides this information from the &CMUCL; manual, there are a
few other points to keep in mind.
<itemizedlist>
<listitem><para>The &CMUCL; manual doesn't seem to state it explicitly,
but &Python; has a mental block about type inference when
assignment is. involved &Python; is very aggressive and clever
about inferring the types of values bound with <function>let</>,
<function>let*</>, inline function call, and so forth. However,
it's much more passive and dumb about inferring the types of
values assigned with <function>setq</>, <function>setf</>, and
friends. It would be nice to fix this, but in the meantime don't
expect that just because it's very smart about types in most
respects it will be smart about types involved in assignments.
(This doesn't affect its ability to benefit from explicit type
declarations involving the assigned variables, only its ability to
get by without explicit type declarations.)</para></listitem>
<listitem><para>Since the time the &CMUCL; manual was written,
&CMUCL; (and thus &SBCL;) has gotten a generational garbage
collector. This means that there are some efficiency implications
of various patterns of memory usage which aren't discussed in the
&CMUCL; manual. (Some new material should be written about
this.)</para></listitem>
<listitem><para>&SBCL; has some important known efficiency problems.
Perhaps the most important are
<itemizedlist>
<listitem><para>There is no support for the &ANSI;
<parameter>dynamic-extent</> declaration, not even for
closures or <parameter>&amp;rest</> lists.</para></listitem>
<listitem><para>The garbage collector is not particularly
efficient.</para></listitem>
<listitem><para>Various aspects of the PCL implementation
of CLOS are more inefficient than necessary.</para></listitem>
</itemizedlist>
</para></listitem>
</itemizedlist>
</para>
<para>Finally, note that &CommonLisp; defines many constructs which, in
the infamous phrase, <quote>could be compiled efficiently by a
sufficiently smart compiler</quote>. The phrase is infamous because
making a compiler which actually is sufficiently smart to find all
these optimizations systematically is well beyond the state of the art
of current compiler technology. Instead, they're optimized on a
case-by-case basis by hand-written code, or not optimized at all if
the appropriate case hasn't been hand-coded. Some cases where no such
hand-coding has been done as of &SBCL; version 0.6.3 include
<itemizedlist>
<listitem><para><literal>(reduce #'f x)</>
where the type of <varname>x</> is known at compile
time</para></listitem>
<listitem><para>various bit vector operations, e.g.
<literal>(position 0 some-bit-vector)</></para></listitem>
</itemizedlist>
If your system's performance is suffering because of some construct
which could in principle be compiled efficiently, but which the &SBCL;
compiler can't in practice compile efficiently, consider writing a
patch to the compiler and submitting it for inclusion in the main
sources. Such code is often reasonably straightforward to write;
search the sources for the string <quote><function>deftransform</></>
to find many examples (some straightforward, some less so).</para>
</chapter>

32
doc/ffi.sgml Normal file
View file

@ -0,0 +1,32 @@
<chapter id="ffi"><title>The Foreign Function Interface</>
<para>FIXME: The material in the &CMUCL; manual about the foreign
function interface should be reviewed, reformatted in DocBook,
lightly edited for &SBCL;, and substituted into this manual. But in
the meantime, the original &CMUCL; manual is still 95+% correct for
the &SBCL; version of the foreign function interface. (The main
difference is that the package names have changed from
<quote><literal>ALIEN</></> and <quote><literal>C-CALL</></> to
<quote><literal>SB-ALIEN</></> and <quote><literal>SB-C-CALL</></>.)
<!-- FIXME: Oh, and I seem to remember that the CMUCL manual
was out of date about how to test for a null pointer,
there's a builtin operator to do it, you don't need to
do the nasty idiom the manual says you need to do. -->
<!-- FIXME: Also, the CMU CL alien documentation claims you
can just do (DEF-ALIEN-VARIABLE "errno" INT), which fails
with modern multithreading hacks. -->
<!-- FIXME: Also, LOAD-FOREIGN isn't implemented as of sbcl-0.6.7,
but LOAD-1-FOREIGN is. -->
See the sections
<itemizedlist>
<listitem><para>Type Translations</></>
<listitem><para>System Area Pointers</></>
<listitem><para>Alien Objects</></>
<listitem><para>Alien Types</></>
<listitem><para>Alien Operations</></>
<listitem><para>Alien Variables</></>
<listitem><para>Alien Function Calls</></>
</itemizedlist>
</para>
</chapter>

154
doc/intro.sgml Normal file
View file

@ -0,0 +1,154 @@
<chapter id="intro"><title>Introduction</>
<para>&SBCL; is a mostly-conforming implementation of the &ANSI;
&CommonLisp; standard. This manual focuses on behavior which is
specific to &SBCL;, not on behavior which is common to all
implementations of &ANSI; &CommonLisp;.</para>
<sect1><title>More Information on &CommonLisp; in General</>
<para>If you are an experienced programmer in general but need
information on using &CommonLisp; in particular, <emphasis>ANSI Common
Lisp</>, by Paul Graham, is a good place to start. <emphasis>Paradigms
Of Artificial Intelligence Programming</>, by Peter Norvig, also has
some good information on general &CommonLisp; programming, and many
nontrivial examples. For CLOS in particular, <emphasis>Object-Oriented
Programming In Common Lisp</> by Sonya Keene is useful.</para>
<para>Two very useful resources for working with any implementation of
&CommonLisp; are the
<ulink url="http://ilisp.cons.org"><application>ILISP</></ulink>
package for <application>Emacs</> and
<ulink url="http://www.harlequin.com/books/HyperSpec">the &CommonLisp;
HyperSpec</>.</para>
</sect1>
<sect1><title>More Information on SBCL</title>
<para>Besides this manual, some other &SBCL;-specific information is
available:
<itemizedlist>
<listitem><para>There is a Unix <quote>man page</> file
<filename>sbcl.1</> in the &SBCL; distribution,
describing command options and other usage information
for the Unix <function>sbcl</> command which invokes
the &SBCL; system.</para></listitem>
<listitem><para>Documentation for non-&ANSI; extensions for
various commands is available online from the &SBCL; executable
itself. The extensions for functions which have their own
command prompts (e.g. the debugger, and <function>inspect</>)
are documented in text available by typing <userinput>help</>
at their command prompts. The extensions for functions which
don't have their own command prompt (e.g. <function>trace</>)
are described in their documentation strings,
unless your &SBCL was compiled with an option not
to include documentation strings, in which case the doc strings
are only readable in the source code.</para></listitem>
<listitem><para>The <ulink url="http://sbcl.sourceforge.net/">
&SBCL; home page</ulink> has some general
information, plus links to mailing lists devoted to &SBCL;,
and to archives of these mailing lists.</para></listitem>
<listitem><para>Some low-level information describing the
programming details of the conversion from &CMUCL; to &SBCL;
is available in the <filename>doc/FOR-CMUCL-DEVELOPERS</>
file in the &SBCL; distribution.</para></listitem>
</itemizedlist>
</para>
</sect1>
<sect1 id="implementation"><title>System Implementation and History</>
<para>You can work productively with SBCL without understanding
anything about how it was and is implemented, but a little knowledge
can be helpful in order to better understand error messages,
troubleshoot problems, to understand why some parts of the system are
better debugged than others, and to anticipate which known bugs, known
performance problems, and missing extensions are likely to be fixed,
tuned, or added.</para>
<para>&SBCL; is descended from &CMUCL;, which is itself descended from
Spice Lisp. Early implementations for the Mach operating system on the
IBM RT, back in the 1980s. Design decisions from that time are still
reflected in the current implementation:
<itemizedlist>
<listitem><para>The system expects to be loaded into a
fixed-at-compile-time location in virtual memory, and also expects
the location of all of its heap storage to be specified
at compile time.</para></listitem>
<listitem><para>The system overcommits memory, allocating large
amounts of address space from the system (often more than
the amount of virtual memory available) and then failing
if ends up using too much of the allocated storage.</para></listitem>
<listitem><para>A word is a 32-bit quantity. The system has been
ported to many processor architectures without altering this
basic principle. Some hacks allow the system to run on the Alpha
chip (a 64-bit architecture) but the assumption that a word is
32 bits wide is implicit in hundreds of places in the
system.</para></listitem>
<listitem><para>The system is implemented as a C program which is
responsible for supplying low-level services and loading a
Lisp <quote>.core</quote> file.
</para></listitem>
</itemizedlist>
</para>
<para>&SBCL; also inherited some newer architectural features from
&CMUCL;. The most important is that it has a generational garbage
collector (<quote>GC</>), which has various implications (mostly good)
for performance. These are discussed in <link linkend="efficiency">
another chapter</link>.</para>
<para>The direct ancestor of &SBCL; is the X86 port of &CMUCL;.
This port is in some ways the least mature of any in the &CMUCL;
system, and some things (like profiling and backtracing)
do not work particularly well there. &SBCL; should be able
to improve in these areas, but it may take a while.</para>
<para>The &SBCL; GC, like the GC on the X86 port of &CMUCL;, is
<emphasis>conservative</>. This means that it doesn't maintain a
strict separation between tagged and untagged data, instead treating
some untagged data (e.g. raw floating point numbers) as
possibly-tagged data and so not collecting any Lisp objects that they
point to. This has some negative consequences for average time
efficiency (though possibly no worse than the negative consequences of
trying to implement an exact GC on a processor architecture as
register-poor as the X86) and also has potentially unlimited
consequences for worst-case memory efficiency. In practice,
conservative garbage collectors work reasonably well, not getting
anywhere near the worst case. But they can occasionally cause
odd patterns of memory usage.</para>
<para>The fork from &CMUCL; was based on a major rewrite of the system
bootstrap process. &CMUCL; has for many years tolerated a very unusual
<quote>build</> procedure which doesn't actually build the complete
system from scratch, but instead progressively overwrites parts of a
running system with new versions. This quasi-build procedure can cause
various bizarre bootstrapping hangups, especially when a major change
is made to the system. It also makes the connection between the
current source code and the current executable more tenuous than in
any other software system I'm aware of -- it's easy to accidentally
<quote>build</> a &CMUCL; system containing characteristics not
reflected in the current version of the source code.</para>
<para>Other major changes since the fork from &CMUCL; include
<itemizedlist>
<listitem><para>&SBCL; has dropped support for many &CMUCL; extensions,
(e.g. remote procedure call, Unix system interface, and X11
interface).</para></listitem>
<listitem><para>&SBCL; has deleted or deprecated
some nonstandard features and code complexity which helped
efficiency at the price of maintainability. For example, the
&SBCL; compiler no longer implements memory pooling internally
(and so is simpler and more maintainable, but generates more
garbage and runs more slowly), and various block-compilation
efficiency-increasing extensions to the language have been
deleted or are no longer used in the implementation of &SBCL;
itself.</para></listitem>
</itemizedlist>
</para>
</sect1>
</chapter>

5
doc/make-doc.sh Normal file
View file

@ -0,0 +1,5 @@
#!/bin/sh
rm -f book1.htm
jade -t sgml -ihtml -d sbcl-html.dsl\#html user-manual.sgml
ln -sf book1.htm user-manual.html

104
doc/sbcl-html.dsl Normal file
View file

@ -0,0 +1,104 @@
<!DOCTYPE style-sheet PUBLIC "-//James Clark//DTD DSSSL Style Sheet//EN"
--
This is a stylesheet for converting DocBook to HTML, implemented as a
customization layer over Norman Walsh's modular DocBook stylesheets.
It's possible that it could be useful for other documents, or even
that it could be an example of decent DSSSL style, but if so, that's
basically an accident, since it was written based on a superficial
reading of chapter 4 of _DocBook: The Definitive Guide_, by Norman
Walsh and Leonard Muellner, and has only been tested on the SBCL
manual.
This software is part of the SBCL system. See the README file for more
information.
The SBCL system is derived from the CMU CL system, which was written
at Carnegie Mellon University and released into the public domain. The
software is in the public domain and is provided with absolutely no
warranty. See the COPYING and CREDITS files for more information.
--
[<!ENTITY docbook.dsl
SYSTEM
"/usr/lib/sgml/stylesheets/nwalsh-modular/html/docbook.dsl"
CDATA
dsssl>]>
<style-sheet>
<style-specification id="html" use="docbook">
<style-specification-body>
;;; FIXME: It would be nice to have output files have ".html" extensions
;;; instead of ".htm" extensions.
;;; Essentially all the stuff in the "Programming languages and
;;; constructs" section (pp. 40-41 of _DocBook: The Definitive Guide_)
;;; is to be monospaced. The one exception is "replaceable", which
;;; needs to be distinguishable from the others.
;;;
;;; (In the modular stylesheets as of 1.54, some elements like "type"
;;; were typeset in the same font as running text, which led to
;;; horrible confusion in the SBCL manual.)
(element action ($mono-seq$))
(element classname ($mono-seq$))
(element constant ($mono-seq$))
(element errorcode ($mono-seq$))
(element errorname ($mono-seq$))
(element errortype ($mono-seq$))
(element function ($mono-seq$))
(element interface ($mono-seq$))
(element interfacedefinition ($mono-seq$))
(element literal ($mono-seq$))
(element msgtext ($mono-seq$))
(element parameter ($mono-seq$))
(element property ($mono-seq$))
(element replaceable ($italic-seq$))
(element returnvalue ($mono-seq$))
(element structfield ($mono-seq$))
(element structname ($mono-seq$))
(element symbol ($mono-seq$))
(element token ($mono-seq$))
(element type ($mono-seq$))
(element varname ($mono-seq$))
;;; Things in the "Operating systems" and "General purpose"
;;; sections (pp. 41-42 and pp. 42-43
;;; of _DocBook: The Definitive Guide_) are handled on a case
;;; by case basis.
;;;
;;; "Operating systems" section
(element application ($charseq$))
(element command ($mono-seq$))
(element envar ($mono-seq$))
(element filename ($mono-seq$))
(element medialabel ($mono-seq$))
;;; (The "msgtext" element is handled in another section.)
(element option ($mono-seq$))
;;; (The "parameter" element is handled in another section.)
(element prompt ($bold-mono-seq$))
(element systemitem ($mono-seq$))
;;;
;;; "General purpose" section
(element database ($charseq$))
(element email ($mono-seq$))
;;; (The "filename" element is handled in another section.)
(element hardware ($mono-seq$))
(element inlinegraphic ($mono-seq$))
;;; (The "literal" element is handled in another section.)
;;; (The "medialabel" element is handled in another section.)
;;; (The "option" element is handled in another section.)
(element optional ($italic-mono-seq$))
;;; (The "replaceable" element is handled in another section.)
;;; (The "symbol" element is handled in another section.)
;;; (The "token" element is handled in another section.)
;;; (The "type" element is handled in another section.)
</style-specification-body>
</style-specification>
<external-specification id="docbook" document="docbook.dsl">
</style-sheet>

383
doc/sbcl.1 Normal file
View file

@ -0,0 +1,383 @@
.\" -*- Mode: Text -*-
.\"
.\" man page introduction to SBCL
.\"
.\" SBCL, including this man page, is derived from CMU Common Lisp, of
.\" which it was said (ca. 1991)
.\" **********************************************************************
.\" This code was written as part of the CMU Common Lisp project at
.\" Carnegie Mellon University, and has been placed in the public domain.
.\" If you want to use this code or any part of CMU Common Lisp, please
.\" contact Scott Fahlman or slisp-group@cs.cmu.edu.
.\" **********************************************************************
.\"
.\" $Header$
.\" FIXME: The date below should be $Date$.
.TH SBCL 1 "$Date$"
.AT 3
.SH NAME
SBCL -- "Steel Bank Common Lisp"
.SH DESCRIPTION
SBCL is a free Common Lisp programming environment. It is derived from
the free CMU CL programming environment. (The name is intended to
acknowledge the connection: steel and banking are the industries where
Carnegie and Mellon made the big bucks.)
.SH COMMAND LINE SYNTAX
Command line syntax can be considered an advanced topic; for ordinary
interactive use, no command line arguments should be necessary.
In order to understand the command line argument syntax for SBCL, it
is helpful to understand that the SBCL system is implemented as two
components, a low-level runtime environment written in C and a
higher-level system written in Common Lisp itself. Some command line
arguments are processed during the initialization of the low-level
runtime environment, some command line arguments are processed during
the initialization of the Common Lisp system, and any remaining
command line arguments are passed on to user code.
The full, unambiguous syntax for SBCL is
.TP 3
.B sbcl [runtime options] --end-runtime-options [toplevel options] --end-toplevel-options [user options]
.PP
For convenience, the --end-runtime-options and --end-toplevel-options
elements can be omitted. Omitting these elements can be convenient
when you are running the program interactively, and you can see that
no ambiguities are possible with the option values you are using.
Omitting these elements is probably a bad idea for any batch file
where any of the options are under user control, since it makes it
impossible for SBCL to detect erroneous command line input, so that
erroneous command line arguments will be passed on to the user program
even if they was intended for the runtime system or the Lisp system.
Supported runtime options are
.TP 3
.B --core <corefilename>
Run the specified Lisp core file instead of the default. (See the FILES
section.) Note that if the Lisp core file is a user-created core file, it may
run a nonstandard toplevel which does not accept the standard toplevel options.
.TP 3
.B --noinform
Suppress the printing of any banner or other informational message at
startup. (Combined with the --noprint toplevel option, this makes it
straightforward to write Lisp "scripts" which work as Unix pipes.)
.PP
In the future, runtime options may be added to control behavior such
as lazy allocation of memory.
Runtime options, including any --end-runtime-options option,
are stripped out of the command line before the
Lisp toplevel logic gets a chance to see it.
Supported toplevel options for the standard SBCL core are
.TP 3
.B --sysinit <filename>
Load filename instead of the default system-wide
initialization file. (See the FILES section.)
There is no special option to cause
no system-wide initialization file to be read, but on a Unix
system "--sysinit /dev/null" can be used to achieve the same effect.
.TP 3
.B --userinit <filename>
Load filename instead of the default user
initialization file. (See the FILES section.)
There is no special option to cause
no user initialization file to be read, but on a Unix
system "--userinit /dev/null" can be used to achieve the same effect.
.TP 3
.B --eval <command>
After executing any initialization file, but before starting the
read-eval-print loop on standard input,
evaluate the command given. More than
one --eval option can be used, and all will be executed,
in the order they appear on the command line.
.TP 3
.B --noprint
When ordinarily the toplevel "read-eval-print loop" would be
executed, execute a "read-eval loop" instead, i.e. don't print
a prompt and don't echo results. (Combined with the --noinform
runtime option, this makes it straightforward to write Lisp
"scripts" which work as Unix pipe utilities.)
.TP 3
.B --noprogrammer
Ordinarily the system initializes *DEBUG-IO* to *TERMINAL-IO*.
When the --notty option is set, however, *DEBUG-IO* is instead
set to a stream which sends its output to *ERROR-OUTPUT* and
which raises an error on input. As a result, any attempt by the
program to get programmer feedback through the debugger
causes an error which abnormally terminates the entire
Lisp environment. (This can be useful behavior for programs
which are to run without programmer supervision.)
.PP
Regardless of the order in which --sysinit, --userinit, and --eval
options appear on the command line, the sysinit file, if it exists, is
loaded first; then the userinit file, if it exists, is loaded; then
any --eval commands are executed in sequence; then the read-eval-print
loop is started on standard input. At any step, error conditions or
commands such as SB-EXT:QUIT can cause execution to be terminated
before proceeding to subsequent steps.
Note that when running SBCL from a core file created by a user call to
the SB-EXT:SAVE-LISP-AND-DIE, the toplevel options may be under the
control of user code passed as arguments to SB-EXT:SAVE-LISP-AND-DIE.
For this purpose, the --end-toplevel-options option itself can be
considered a toplevel option, i.e. the user core, at its option, may
not support it.
In the standard SBCL startup sequence (i.e. with no user core
involved) toplevel options and any --end-toplevel-options option are
stripped out of the command line argument list before user code gets a
chance to see it.
.SH OVERVIEW
SBCL aims for but has not reached ANSI compliance.
SBCL compiles Lisp to native code, or optionally to more-compact but
much slower byte code.
SBCL's garbage collector is generational and conservative.
SBCL includes a source level debugger, as well as the ANSI TRACE
facility and a rudimentary profiler.
.SH DIFFERENCES FROM CMU CL
SBCL can be built from scratch using a plain vanilla ANSI Common Lisp
system and a C compiler, and all of its properties are specified by
the version of the source code that it was created from. (This clean
bootstrappability was the immediate motivation for forking off of the
CMU CL development tree.)
Many extensions supported by CMU CL, like Motif support,
the Hemlock editor, search paths, the WIRE protocol, various
user-level macros and functions (e.g. LETF, ITERATE, MEMQ,
REQUIRED-ARGUMENT), and many others.
SBCL has retained some extensions of its parent CMU CL. Many
of them are in three categories:
.TP 3
\--
hooks into the low level workings of the system which can be useful
for debugging (e.g. a list of functions to be run whenever GC occurs,
or an operator to cause a particular string to be compiled into a fasl
file)
.TP 3
\--
non-portable performance hacks (e.g. PURIFY, which causes
everything currently in existence to become immune to GC)
.TP 3
\--
things which might be in the new ANSI spec (e.g. weak pointers,
finalization, foreign function interface to C, and Gray streams)
.PP
There are also various retained extensions which don't fall into
any particular category, e.g.
.TP 3
\--
the ability to save running Lisp images as executable files
.PP
Some of the retained extensions have new names and/or different
options than their CMU CL counterparts. For example, the SBCL function
which saves a Lisp image to disk and kills it is called
SAVE-LISP-AND-DIE instead of SAVE-LISP, and it supports fewer keyword
options than CMU CL's SAVE-LISP.
.SH THE COMPILER
SBCL inherits from CMU CL the "Python" native code compiler. This
compiler is very clever about understanding the type system of Common
Lisp and using it to produce efficient code, and about producing notes
to let the user know when the compiler doesn't have enough type
information to produce efficient code. It also tries (almost always
successfully) to follow the unusual but very useful principle that
type declarations should be checked at runtime unless the user
explicitly tells the system that speed is more important than safety.
The CMU CL version of this compiler reportedly produces pretty good
code for modern machines which have lots of registers, but its code
for the X86 is marred by a lot of extra loads and stores to
stack-based temporary variables. Because of this, and because of the
extra levels of indirection in Common Lisp relative to C, we find a
typical performance decrease by a factor of perhaps 2 to 5 for small
programs coded in SBCL instead of GCC.
For more information about the compiler, see the user manual.
.SH DOCUMENTATION
Currently, the documentation for the system is
.TP 3
\--
the user manual
.TP 3
\--
this man page
.TP 3
\--
doc strings and online help built into the SBCL executable
.PP
.SH SYSTEM REQUIREMENTS
Unlike its distinguished ancestor CMU CL, SBCL is currently only
supported on X86. Linux and FreeBSD are currently available. It would
probably be straightforward to port the CMU CL support for Alpha or
SPARC as well, or to OpenBSD or NetBSD, but at the time of this
writing no such efforts are underway.
As of version 0.6.3, SBCL requires on the order of 16Mb to run. In
some future version, this number could shrink significantly, since
large parts of the system are far from execution bottlenecks and could
reasonably be stored in compact byte compiled form. (CMU CL does this
routinely; the only reason SBCL doesn't currently do this is a
combination of bootstrapping technicalities and inertia.)
.SH ENVIRONMENT
.TP 10n
.BR SBCL_HOME
If this variable is set, it overrides the default directories for
files like "sbclrc" and "sbcl.core", so that instead of being searched
for in e.g. /etc/, /usr/local/etc/, /usr/lib/, and /usr/local/lib/, they
are searched for only in the directory named by SBCL_HOME. This is
intended to support users who wish to use their own version of SBCL
instead of the version which is currently installed as the system
default.
.PP
.SH FILES
/usr/lib/sbcl.core and /usr/local/lib/sbcl.core are the standard
locations for the standard SBCL core, unless overridden by the SBCL_HOME
variable.
/etc/sbclrc and /usr/local/etc/sbclrc are the standard locations for
system-wide SBCL initialization files, unless overridden by the
SBCL_HOME variable.
$HOME/.sbclrc is the standard location for a user's SBCL
initialization file.
.SH BUGS
Too numerous to list, alas. This section attempts to list the most
serious known bugs, and a reasonably representative sampling of
others. For more information on bugs, see the BUGS file in the
distribution.
It is possible to get in deep trouble by exhausting
memory. To plagiarize a sadly apt description of a language not
renowned for the production of bulletproof software, "[The current
SBCL implementation of] Common Lisp makes it harder for you to shoot
yourself in the foot, but when you do, the entire universe explodes."
.TP 3
\--
The system doesn't deal well with stack overflow.
.TP 3
\--
The SBCL system overcommits memory at startup. On typical Unix-alikes
like Linux and *BSD, this can cause other processes to be killed
randomly (!) if the SBCL system turns out to use more virtual memory
than the system has available for it.
.PP
The facility for dumping a running Lisp image to disk gets confused
when run without the PURIFY option, and creates an unnecessarily large
core file (apparently representing memory usage up to the previous
high-water mark). Moreover, when the file is loaded, it confuses the
GC, so that thereafter memory usage can never be reduced below that
level.
By default, the compiler is overaggressive about static typing,
assuming that a function's return type never changes. Thus compiling
and loading a file containing
(DEFUN FOO (X) NIL)
(DEFUN BAR (X) (IF (FOO X) 1 2))
(DEFUN FOO (X) (PLUSP X))
then running (FOO 1) gives 2 (because the compiler "knew"
that FOO's return type is NULL).
The compiler's handling of function return values unnecessarily
violates the "declarations are assertions" principle that it otherwise
adheres to. Using PROCLAIM or DECLAIM to specify the return type of a
function causes the compiler to believe you without checking. Thus
compiling a file containing
(DECLAIM (FTYPE (FUNCTION (T) NULL) SOMETIMES))
(DEFUN SOMETIMES (X) (ODDP X))
(DEFUN FOO (X) (IF (SOMETIMES X) 'THIS-TIME 'NOT-THIS-TIME))
then running (FOO 1) gives NOT-THIS-TIME, because the
never compiled code to check the declaration.
The TRACE facility can't be used on some kinds of functions.
The profiler is flaky, e.g. sometimes it fails by throwing a
signal instead of giving you a result.
SYMBOL-FUNCTION is much slower than you'd expect, being implemented
not as a slot access but as a search through the compiler/kernel
"globaldb" database.
CLOS (based on the PCL reference implementation) is quite slow.
The interpreter's pre-processing freezes in the macro definitions in effect at
the time an interpreted function is defined.
There are many nagging pre-ANSIisms, e.g.
.TP 3
\--
CLOS (based on the PCL reference implementation) is incompletely
integrated into the system, so that e.g. SB-PCL::FIND-CLASS is a
different function than CL::FIND-CLASS. (This is less of a problem in
practice than the speed, but it's still distasteful.)
.TP 3
--
The ANSI-recommended idiom for creating a function which is only
sometimes expanded inline,
(DECLAIM (INLINE F))
(DEFUN F ...)
(DECLAIM (NOTINLINE F)),
doesn't do what you'd expect. (Instead, you have to declare the
function as SB-EXT:MAYBE-INLINE to get the desired effect.)
.TP 3
--
Compiling DEFSTRUCT in strange places (e.g. inside a DEFUN) doesn't
do anything like what it should.
.TP 3
\--
The symbol * is the name of a type similar to T. (It's used as part
of the implementation of compound types like (ARRAY * 1).)
.TP 3
\--
The DESCRIBE facility doesn't use CLOS (PRINT-OBJECT, etc.) as it should.
Instead it is based on old hardwired TYPECASEs.
.TP 3
\--
The printer doesn't use CLOS (PRINT-OBJECT, etc.) everywhere it should.
Instead it still uses old hardwired TYPECASEs. (This one is not as
annoying as it sounds, since the printer does use PRINT-OBJECT in the
places where it tends to matter most.)
.PP
.SH SUPPORT
Please send bug reports or other information to
<william.newman@airmail.net>.
.SH DISTRIBUTION
SBCL is a free implementation of Common Lisp derived from CMU CL. Both
sources and executables are freely available; this software is "as
is", and has no warranty of any kind. CMU and the authors assume no
responsibility for the consequences of any use of this software. See
the CREDITS file in the distribution for more information about
history, contributors and permissions.

68
doc/user-manual.sgml Normal file
View file

@ -0,0 +1,68 @@
<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook V3.1//EN" [
<!-- markup for common expressions -->
<!ENTITY ANSI "<acronym>ANSI</>">
<!ENTITY CMUCL "<application>CMU CL</>">
<!ENTITY IEEE "<acronym>IEEE</>">
<!ENTITY Python "<application>Python</>">
<!ENTITY SBCL "<application>SBCL</>">
<!-- common expressions I haven't figured out how to mark up -->
<!-- KLUDGE: There doesn't seem to be any DocBook tag for names of
programming languages. Typesetting Lisp Common Lisp as an
<application> looks funny. Is there a better way?
WHN 20000505 -->
<!ENTITY CommonLisp "Common Lisp">
<!ENTITY Lisp "Lisp">
<!-- common expressions I haven't figured out how to express -->
<!ENTITY mdash "-">
<!-- document components -->
<!ENTITY ch-intro SYSTEM "intro.sgml">
<!ENTITY ch-compiler SYSTEM "compiler.sgml">
<!ENTITY ch-efficiency SYSTEM "efficiency.sgml">
<!ENTITY ch-beyond-ansi SYSTEM "beyond-ansi.sgml">
<!ENTITY ch-ffi SYSTEM "ffi.sgml">
]>
<book>
<bookinfo>
<title>&SBCL; User Manual</title>
<legalnotice>
<para>This manual is part of the &SBCL; software system. See the
<filename>README</> file for more information.</para>
<para>This manual is derived in part from the manual for the &CMUCL;
system, which was produced at Carnegie Mellon University and
later released into the public domain. This manual is in the
public domain and is provided with absolutely no warranty. See the
<filename>COPYING</> and <filename>CREDITS</> files for more
information.</para>
</legalnotice>
</bookinfo>
&ch-intro;
&ch-compiler;
&ch-efficiency;
&ch-beyond-ansi;
&ch-ffi;
<colophon>
<para>This manual is maintained in SGML/DocBook, and automatically
translated into other forms (e.g. HTML or TeX). If you're
<emphasis>reading</> this manual in one of these non-DocBook
translated forms, that's fine, but if you want to <emphasis>modify</>
this manual, you are strongly advised to seek out a DocBook version
and modify that instead of modifying a translated version. Even
better might be to seek out <emphasis>the</> DocBook version
(maintained at the time of this writing as part of
<ulink url="http://sbcl.sourceforge.net/">the &SBCL; project</>)
and submit a patch.</para>
</colophon>
</book>

7
install.sh Normal file
View file

@ -0,0 +1,7 @@
#!/bin/sh
# Install SBCL files into the usual places.
cp src/runtime/sbcl /usr/local/bin/
cp output/sbcl.core /usr/local/lib/
cp doc/sbcl.1 /usr/local/man/man1/

79
make-config.sh Normal file
View file

@ -0,0 +1,79 @@
#!/bin/sh
# The make-config.sh script uses information about the target machine
# to set things up for compilation. It's vaguely like a stripped-down
# version of autoconf. It's intended to be run as part of make.sh. The
# only time you'd want to run it by itself is if you're trying to
# cross-compile the system or if you're doing some kind of
# troubleshooting.
# This software is part of the SBCL system. See the README file for
# more information.
#
# This software is derived from the CMU CL system, which was
# written at Carnegie Mellon University and released into the
# public domain. The software is in the public domain and is
# provided with absolutely no warranty. See the COPYING and CREDITS
# files for more information.
echo //entering make-config.sh
ltf=`pwd`/local-target-features.lisp-expr
echo //initializing $ltf
echo '; This is a machine-generated file and should not be edited by hand.' > $ltf
echo -n '(' >> $ltf
echo '//setting up "target"-named symlinks to designate target architecture'
sbcl_arch=x86 # (the only possibility supported, at least as of sbcl-0.6.7)
echo -n ":x86" >> $ltf # (again, the only possibility supported)
for d in src/compiler src/assembly; do
echo //setting up symlink $d/target
original_dir=`pwd`
cd $d
if [ -L target ] ; then
rm target
elif [ -e target ] ; then
echo "I'm afraid to replace non-symlink $d/target with a symlink."
exit 1
fi
if [ -d $sbcl_arch ] ; then
ln -s $sbcl_arch target
else
echo "missing sbcl_arch directory $PWD/$sbcl_arch"
exit 1
fi
cd $original_dir
done
echo //setting up OS-dependent information
cd src/runtime/
rm -f Config
if [ `uname` = Linux ]; then
echo -n ' :linux' >> $ltf
ln -s Config.x86-linux Config
elif uname | grep BSD; then
if [ `uname` = FreeBSD ]; then
echo -n ' :freebsd' >> $ltf
elif [ `uname` = OpenBSD ]; then
echo -n ' :openbsd' >> $ltf
else
echo unsupported BSD variant: `uname`
exit 1
fi
echo -n ' :bsd' >> $ltf
ln -s Config.x86-bsd Config
else
echo unsupported OS type: `uname`
exit 1
fi
echo //finishing $ltf
echo ')' >> $ltf
# FIXME: The version system should probably be redone along these lines:
#
# echo //setting up version information.
# versionfile=version.txt
# cp base-version.txt $versionfile
# echo " (built `date -u` by `whoami`@`hostname`)" >> $versionfile
# echo 'This is a machine-generated file and should not be edited by hand.' >> $versionfile

42
make-host-1.sh Normal file
View file

@ -0,0 +1,42 @@
#!/bin/sh
# This is a script to be run as part of make.sh. The only time you'd
# want to run it by itself is if you're trying to cross-compile the
# system or if you're doing some kind of troubleshooting.
# This software is part of the SBCL system. See the README file for
# more information.
#
# This software is derived from the CMU CL system, which was
# written at Carnegie Mellon University and released into the
# public domain. The software is in the public domain and is
# provided with absolutely no warranty. See the COPYING and CREDITS
# files for more information.
echo //entering make-host-1.sh
# Compile and load the cross-compiler. (We load it here not because we're
# about to use it, but because it's written under the assumption that each
# file will be loaded before the following file is compiled.)
#
# Also take the opportunity to compile and load genesis, to create the
# header file sbcl.h which will be needed to create the C runtime
# environment.
echo //building cross-compiler, and doing first genesis
$SBCL_XC_HOST <<-'EOF' || exit 1
;; (We want to have some limit on print length and print level
;; during bootstrapping because PRINT-OBJECT only gets set
;; up rather late, and running without PRINT-OBJECT it's easy
;; to fall into printing enormous (or infinitely circular)
;; low-level representations of things.)
(setf *print-level* 5 *print-length* 5)
(load "src/cold/shared.lisp")
(in-package "SB-COLD")
(setf *host-obj-prefix* "obj/from-host/")
(load "src/cold/shared.lisp")
(load "src/cold/set-up-cold-packages.lisp")
(load "src/cold/defun-load-or-cload-xcompiler.lisp")
(load-or-cload-xcompiler #'host-cload-stem)
(host-cload-stem "compiler/generic/genesis")
(sb!vm:genesis :c-header-file-name "src/runtime/sbcl.h")
EOF

142
make-host-2.sh Normal file
View file

@ -0,0 +1,142 @@
#!/bin/sh
# This is a script to be run as part of make.sh. The only time you'd
# want to run it by itself is if you're trying to cross-compile the
# system or if you're doing some kind of troubleshooting.
# This software is part of the SBCL system. See the README file for
# more information.
#
# This software is derived from the CMU CL system, which was
# written at Carnegie Mellon University and released into the
# public domain. The software is in the public domain and is
# provided with absolutely no warranty. See the COPYING and CREDITS
# files for more information.
echo //entering make-host-2.sh
# In a fresh host Lisp invocation, load and run the cross-compiler to
# create the target object files describing the target SBCL.
#
# (There are at least three advantages to running the cross-compiler in a
# fresh host Lisp invocation instead of just using the same Lisp invocation
# that we used to compile it:
# (1) It reduces the chance that the cross-compilation process
# inadvertently comes to depend on some weird compile-time
# side-effect.
# (2) It reduces peak memory demand (because definitions wrapped in
# (EVAL-WHEN (:COMPILE-TOPLEVEL :EXECUTE) ..) aren't defined
# in the fresh image).
# (3) It makes it easier to jump in and retry a step when tweaking
# and experimenting with the bootstrap procedure.
# Admittedly, these don't seem to be enormously important advantages, but
# the only disadvantage seems to be the extra time required to reload
# the fasl files into the new host Lisp, and that doesn't seem to be
# an enormously important disadvantage, either.)
echo //running cross-compiler to create target object files
$SBCL_XC_HOST <<-'EOF' || exit 1
(setf *print-level* 5 *print-length* 5)
(load "src/cold/shared.lisp")
(in-package "SB-COLD")
(setf *host-obj-prefix* "obj/from-host/"
*target-obj-prefix* "obj/from-xc/")
(load "src/cold/set-up-cold-packages.lisp")
(load "src/cold/defun-load-or-cload-xcompiler.lisp")
(load-or-cload-xcompiler #'host-load-stem)
(defun proclaim-target-optimization ()
(let ((debug (if (find :sb-show *shebang-features*) 2 1)))
(sb-xc:proclaim `(optimize (compilation-speed 1)
(debug ,debug)
(sb!ext:inhibit-warnings 2)
(safety 3)
(space 1)
(speed 2)))))
(compile 'proclaim-target-optimization)
(defun in-target-cross-compilation-mode (fn)
"Call FN with everything set up appropriately for cross-compiling
a target file."
(let (;; Life is simpler at genesis/cold-load time if we
;; needn't worry about byte-compiled code.
(sb!ext:*byte-compile-top-level* nil)
;; Let the target know that we're the cross-compiler.
(*features* (cons :sb-xc *features*))
;; We need to tweak the readtable..
(*readtable* (copy-readtable))
;; In order to reduce peak memory usage during GENESIS,
;; it helps to stuff several toplevel forms together
;; into the same function.
(sb!c::*top-level-lambda-max* 10))
;; ..in order to make backquotes expand into target code
;; instead of host code.
;; FIXME: Isn't this now taken care of automatically by
;; toplevel forms in the xcompiler backq.lisp file?
(set-macro-character #\` #'sb!impl::backquote-macro)
(set-macro-character #\, #'sb!impl::comma-macro)
;; Control optimization policy.
(proclaim-target-optimization)
;; Specify where target machinery lives.
(with-additional-nickname ("SB-XC" "SB!XC")
(funcall fn))))
(compile 'in-target-cross-compilation-mode)
(setf *target-compile-file* 'sb-xc:compile-file)
(setf *target-assemble-file* 'sb!c:assemble-file)
(setf *in-target-compilation-mode-fn*
#'in-target-cross-compilation-mode)
(load "src/cold/compile-cold-sbcl.lisp")
(let ((filename "output/object-filenames-for-genesis.lisp-expr"))
(ensure-directories-exist filename :verbose t)
(with-open-file (s filename :direction :output)
(write *target-object-file-names* :stream s :readably t)))
;; If you're experimenting with the system under a
;; cross-compilation host which supports CMU-CL-style SAVE-LISP,
;; this can be a good time to run it,
;; The resulting core isn't used in the normal build, but
;; can be handy for experimenting with the system.
(when (find :sb-show *shebang-features*)
#+cmu (ext:save-lisp "output/after-xc.core" :load-init-file nil)
#+sbcl (sb-ext:save-lisp-and-die "output/after-xc.core"))
EOF
# Run GENESIS again in order to create cold-sbcl.core.
#
# In a fresh host Lisp invocation, load the cross-compiler (in order
# to get various definitions that GENESIS needs, not in order to
# cross-compile GENESIS, compile and load GENESIS, then run GENESIS.
# (We use a fresh host Lisp invocation here for basically the same
# reasons we did before when loading and running the cross-compiler.)
#
# (This second invocation of GENESIS is done because in order to
# create a .core file, as opposed to just a .h file, GENESIS needs
# symbol table data on the C runtime, which we can get only after the
# C runtime has been built.)
echo //loading and running GENESIS to create cold-sbcl.core
$SBCL_XC_HOST <<-'EOF' || exit 1
(setf *print-level* 5 *print-length* 5)
(load "src/cold/shared.lisp")
(in-package "SB-COLD")
(setf *host-obj-prefix* "obj/from-host/"
*target-obj-prefix* "obj/from-xc/")
(load "src/cold/set-up-cold-packages.lisp")
(load "src/cold/defun-load-or-cload-xcompiler.lisp")
(load-or-cload-xcompiler #'host-load-stem)
(defparameter *target-object-file-names*
(with-open-file (s "output/object-filenames-for-genesis.lisp-expr"
:direction :input)
(read s)))
(host-load-stem "compiler/generic/genesis")
(sb!vm:genesis :object-file-names *target-object-file-names*
:c-header-file-name "output/sbcl2.h"
:symbol-table-file-name "src/runtime/sbcl.nm"
:core-file-name "output/cold-sbcl.core"
;; The map file is not needed by the system, but can
;; be very handy when debugging cold init problems.
:map-file-name "output/cold-sbcl.map")
EOF
echo //testing for consistency of first and second GENESIS passes
if cmp src/runtime/sbcl.h output/sbcl2.h; then
echo //sbcl2.h matches sbcl.h -- good.
else
echo error: sbcl2.h does not match sbcl.h.
exit 1
fi

29
make-target-1.sh Normal file
View file

@ -0,0 +1,29 @@
#!/bin/sh
# This is a script to be run as part of make.sh. The only time you'd
# want to run it by itself is if you're trying to cross-compile the
# system or if you're doing some kind of troubleshooting.
# This software is part of the SBCL system. See the README file for
# more information.
#
# This software is derived from the CMU CL system, which was
# written at Carnegie Mellon University and released into the
# public domain. The software is in the public domain and is
# provided with absolutely no warranty. See the COPYING and CREDITS
# files for more information.
echo //entering make-target-1.sh
# Build the runtime system and symbol table (.nm) file.
#
# (This C build has to come after the first genesis in order to get
# the sbcl.h the C build needs, and come before the second genesis in
# order to produce the symbol table file that second genesis needs. It
# could come either before or after running the cross compiler; that
# doesn't matter.)
echo //building runtime system and symbol table file
cd src/runtime
${GNUMAKE:-gmake} clean || exit 1
${GNUMAKE:-gmake} depend || exit 1
${GNUMAKE:-gmake} all || exit 1

42
make-target-2.sh Normal file
View file

@ -0,0 +1,42 @@
#!/bin/sh
# This is a script to be run as part of make.sh. The only time you'd
# want to run it by itself is if you're trying to cross-compile the
# system or if you're doing some kind of troubleshooting.
# This software is part of the SBCL system. See the README file for
# more information.
#
# This software is derived from the CMU CL system, which was
# written at Carnegie Mellon University and released into the
# public domain. The software is in the public domain and is
# provided with absolutely no warranty. See the COPYING and CREDITS
# files for more information.
echo //entering make-host-2.sh
# Do warm init stuff, e.g. building and loading CLOS, and stuff which
# can't be done until CLOS is running.
#
# Note that it's normal for the newborn system to think rather hard at
# the beginning of this process (e.g. using nearly 100Mb of virtual memory
# and >30 seconds of CPU time on a 450MHz CPU), and unless you built the
# system with the :SB-SHOW feature enabled, it does it rather silently,
# without trying to tell you about what it's doing. So unless it hangs
# for much longer than that, don't worry, it's likely to be normal.
echo //doing warm init
./src/runtime/sbcl \
--core output/cold-sbcl.core \
--sysinit /dev/null --userinit /dev/null <<-'EOF' || exit 1
(sb!int:/show "hello, world!")
(let ((*print-length* 5)
(*print-level* 5))
(sb!int:/show "about to LOAD warm.lisp")
(load "src/cold/warm.lisp"))
(sb-int:/show "about to SAVE-LISP-AND-DIE")
;; Even if /SHOW output was wanted during build, it's probably
;; not wanted by default after build is complete. (And if it's
;; wanted, it can easily be turned back on.)
#+sb-show (setf sb-int:*/show* nil)
(sb-ext:save-lisp-and-die "output/sbcl.core" :purify t)
EOF

77
make.sh Executable file
View file

@ -0,0 +1,77 @@
#!/bin/sh
# "When we build software, it's a good idea to have a reliable method
# for getting an executable from it. We want any two reconstructions
# starting from the same source to end up in the same result. That's
# just a basic intellectual premise."
# -- Christian Quinnec, in _Lisp In Small Pieces_, p. 313
# This software is part of the SBCL system. See the README file for
# more information.
#
# This software is derived from the CMU CL system, which was
# written at Carnegie Mellon University and released into the
# public domain. The software is in the public domain and is
# provided with absolutely no warranty. See the COPYING and CREDITS
# files for more information.
# The value of SBCL_XC_HOST should be a command to invoke the
# cross-compilation Lisp system in such a way that it reads commands
# from standard input, and terminates when it reaches end of file on
# standard input. Suitable values are:
# "sbcl" to use an existing SBCL binary as a cross-compilation host
# "sbcl --sysinit /dev/null --userinit /dev/null"
# to use an existing SBCL binary as a cross-compilation host
# even though you have stuff in your initialization files
# which makes it behave in such a non-standard way that
# it keeps the build from working
# "lisp -batch" to use an existing CMU CL binary as a cross-compilation host
# "lisp -noinit -batch"
# to use an existing CMU CL binary as a cross-compilation host
# when you have weird things in your .cmucl-init file
#
# FIXME: Make a more sophisticated command line parser, probably
# accepting "sh make.sh --xc-host foolisp" instead of the
# the present "sh make.sh foolisp".
# FIXME: Tweak this script, and the rest of the system, to support
# a second bootstrapping pass in which the cross-compilation host is
# known to be SBCL itself, so that the cross-compiler can do some
# optimizations (especially specializable arrays) that it doesn't
# know how to implement how in a portable way. (Or maybe that wouldn't
# require a second pass, just testing at build-the-cross-compiler time
# whether the cross-compilation host returns suitable values from
# UPGRADED-ARRAY-ELEMENT-TYPE?)
export SBCL_XC_HOST="${1:-sbcl}"
echo //SBCL_XC_HOST=\"$SBCL_XC_HOST\"
# If you're cross-compiling, you should probably just walk through the
# make-config.sh script by hand doing the right thing on both the host
# and target machines.
sh make-config.sh || exit 1
# The foo-host-bar.sh scripts are run on the cross-compilation host,
# and the foo-target-bar.sh scripts are run on the target machine. In
# ordinary compilation, we just do these phases consecutively on the
# same machine, but if you wanted to cross-compile from one machine
# which supports Common Lisp to another which does not (yet) support
# Lisp, you could do something like this:
# Create copies of the source tree on both host and target.
# Create links from "target" to "x86" in "src/compiler/" and
# in "src/assembly/", on both the host and the target. (That
# would ordinarily be done by the make.sh code above; if we're
# doing make.sh stuff by hand, we need to do this by hand, too.)
# On the host system:
# SBCL_XC_HOST=<whatever> sh make-host-1.sh
# Copy src/runtime/sbcl.h from the host system to the target system.
# On the target system:
# sh make-target-1.sh
# Copy src/runtime/sbcl.nm from the target system to the host system.
# On the host system:
# SBCL_XC_HOST=<whatever> sh make-host-2.sh
# Copy output/cold-sbcl.core from the host system to the target system.
# On the target system:
# sh make-host-2.sh
sh make-host-1.sh || exit 1
sh make-target-1.sh || exit 1
sh make-host-2.sh || exit 1
sh make-target-2.sh || exit 1

1720
package-data-list.lisp-expr Normal file
View file

File diff suppressed because it is too large Load diff

BIN
pubring.pgp Normal file
View file

Binary file not shown.

202
src/assembly/assemfile.lisp Normal file
View file

@ -0,0 +1,202 @@
;;;; the extra code necessary to feed an entire file of assembly code
;;;; to the assembler
;;;; This software is part of the SBCL system. See the README file for
;;;; more information.
;;;;
;;;; This software is derived from the CMU CL system, which was
;;;; written at Carnegie Mellon University and released into the
;;;; public domain. The software is in the public domain and is
;;;; provided with absolutely no warranty. See the COPYING and CREDITS
;;;; files for more information.
(in-package "SB!C")
(file-comment
"$Header$")
(defvar *do-assembly* nil
#!+sb-doc "If non-NIL, emit assembly code. If NIL, emit VOP templates.")
(defvar *lap-output-file* nil
#!+sb-doc "the FASL file currently being output to")
(defvar *entry-points* nil
#!+sb-doc "a list of (name . label) for every entry point")
(defvar *assembly-optimize* t
#!+sb-doc
"Set this to NIL to inhibit assembly-level optimization. For compiler
debugging, rather than policy control.")
;;; Note: You might think from the name that this would act like COMPILE-FILE,
;;; but in fact it's arguably more like LOAD, even down to the return
;;; convention. It LOADs a file, then writes out any assembly code created
;;; by the process.
(defun assemble-file (name
&key
(output-file (make-pathname :defaults name
:type "assem")))
;; FIXME: Consider nuking the filename defaulting logic here.
(let* ((*do-assembly* t)
(name (pathname name))
(*lap-output-file* (open-fasl-file (pathname output-file) name))
(*entry-points* nil)
(won nil)
(*code-segment* nil)
(*elsewhere* nil)
(*assembly-optimize* nil)
(*fixups* nil))
(unwind-protect
(let ((*features* (cons :sb-assembling *features*)))
(init-assembler)
(load (merge-pathnames name (make-pathname :type "lisp")))
(fasl-dump-cold-load-form `(in-package ,(package-name *package*))
*lap-output-file*)
(sb!assem:append-segment *code-segment* *elsewhere*)
(setf *elsewhere* nil)
(let ((length (sb!assem:finalize-segment *code-segment*)))
(dump-assembler-routines *code-segment*
length
*fixups*
*entry-points*
*lap-output-file*))
(setq won t))
(close-fasl-file *lap-output-file* (not won)))
won))
(defstruct reg-spec
(kind :temp :type (member :arg :temp :res))
(name nil :type symbol)
(temp nil :type symbol)
(scs nil :type (or list symbol))
(offset nil))
(def!method print-object ((spec reg-spec) stream)
(print-unreadable-object (spec stream :type t)
(format stream
":KIND ~S :NAME ~S :SCS ~S :OFFSET ~S"
(reg-spec-kind spec)
(reg-spec-name spec)
(reg-spec-scs spec)
(reg-spec-offset spec))))
(defun reg-spec-sc (spec)
(if (atom (reg-spec-scs spec))
(reg-spec-scs spec)
(car (reg-spec-scs spec))))
(defun parse-reg-spec (kind name sc offset)
(let ((reg (make-reg-spec :kind kind :name name :scs sc :offset offset)))
(ecase kind
(:temp)
((:arg :res)
(setf (reg-spec-temp reg) (make-symbol (symbol-name name)))))
reg))
(defun emit-assemble (name options regs code)
(collect ((decls))
(loop
(if (and (consp code) (consp (car code)) (eq (caar code) 'declare))
(decls (pop code))
(return)))
`(let (,@(mapcar
#'(lambda (reg)
`(,(reg-spec-name reg)
(make-random-tn
:kind :normal
:sc (sc-or-lose ',(reg-spec-sc reg))
:offset ,(reg-spec-offset reg))))
regs))
,@(decls)
(sb!assem:assemble (*code-segment* ',name)
,name
(push (cons ',name ,name) *entry-points*)
,@code
,@(generate-return-sequence
(or (cadr (assoc :return-style options)) :raw)))
(when sb!xc:*compile-print*
(format *error-output* "~S assembled~%" ',name)))))
(defun arg-or-res-spec (reg)
`(,(reg-spec-name reg)
:scs ,(if (atom (reg-spec-scs reg))
(list (reg-spec-scs reg))
(reg-spec-scs reg))
,@(unless (eq (reg-spec-kind reg) :res)
`(:target ,(reg-spec-temp reg)))))
(defun emit-vop (name options vars)
(let* ((args (remove :arg vars :key #'reg-spec-kind :test-not #'eq))
(temps (remove :temp vars :key #'reg-spec-kind :test-not #'eq))
(results (remove :res vars :key #'reg-spec-kind :test-not #'eq))
(return-style (or (cadr (assoc :return-style options)) :raw))
(cost (or (cadr (assoc :cost options)) 247))
(vop (make-symbol "VOP")))
(unless (member return-style '(:raw :full-call :none))
(error "unknown return-style for ~S: ~S" name return-style))
(multiple-value-bind
(call-sequence call-temps)
(generate-call-sequence name return-style vop)
`(define-vop ,(if (atom name) (list name) name)
(:args ,@(mapcar #'arg-or-res-spec args))
,@(let ((index -1))
(mapcar #'(lambda (arg)
`(:temporary (:sc ,(reg-spec-sc arg)
:offset ,(reg-spec-offset arg)
:from (:argument ,(incf index))
:to (:eval 2))
,(reg-spec-temp arg)))
args))
,@(mapcar #'(lambda (temp)
`(:temporary (:sc ,(reg-spec-sc temp)
:offset ,(reg-spec-offset temp)
:from (:eval 1)
:to (:eval 3))
,(reg-spec-name temp)))
temps)
,@call-temps
(:vop-var ,vop)
,@(let ((index -1))
(mapcar #'(lambda (res)
`(:temporary (:sc ,(reg-spec-sc res)
:offset ,(reg-spec-offset res)
:from (:eval 2)
:to (:result ,(incf index))
:target ,(reg-spec-name res))
,(reg-spec-temp res)))
results))
(:results ,@(mapcar #'arg-or-res-spec results))
(:ignore ,@(mapcar #'reg-spec-name temps)
,@(apply #'append
(mapcar #'cdr
(remove :ignore call-temps
:test-not #'eq :key #'car))))
,@(remove-if #'(lambda (x)
(member x '(:return-style :cost)))
options
:key #'car)
(:generator ,cost
,@(mapcar #'(lambda (arg)
#!+(or hppa alpha) `(move ,(reg-spec-name arg)
,(reg-spec-temp arg))
#!-(or hppa alpha) `(move ,(reg-spec-temp arg)
,(reg-spec-name arg)))
args)
,@call-sequence
,@(mapcar #'(lambda (res)
#!+(or hppa alpha) `(move ,(reg-spec-temp res)
,(reg-spec-name res))
#!-(or hppa alpha) `(move ,(reg-spec-name res)
,(reg-spec-temp res)))
results))))))
(def!macro define-assembly-routine (name&options vars &body code)
(multiple-value-bind (name options)
(if (atom name&options)
(values name&options nil)
(values (car name&options)
(cdr name&options)))
(let ((regs (mapcar #'(lambda (var) (apply #'parse-reg-spec var)) vars)))
(if *do-assembly*
(emit-assemble name options regs code)
(emit-vop name options regs)))))

70
src/assembly/x86/alloc.lisp Normal file
View file

@ -0,0 +1,70 @@
;;;; allocating simple objects
;;;; This software is part of the SBCL system. See the README file for
;;;; more information.
;;;;
;;;; This software is derived from the CMU CL system, which was
;;;; written at Carnegie Mellon University and released into the
;;;; public domain. The software is in the public domain and is
;;;; provided with absolutely no warranty. See the COPYING and CREDITS
;;;; files for more information.
(in-package "SB!VM")
(file-comment
"$Header$")
;;;; from signed/unsigned
;;; KLUDGE: Why don't we want vops for this one and the next
;;; one? -- WHN 19990916
#+sb-assembling ; We don't want a vop for this one.
(define-assembly-routine
(move-from-signed)
((:temp eax unsigned-reg eax-offset)
(:temp ebx unsigned-reg ebx-offset))
(inst mov ebx eax)
(inst shl ebx 1)
(inst jmp :o bignum)
(inst shl ebx 1)
(inst jmp :o bignum)
(inst ret)
BIGNUM
(with-fixed-allocation (ebx bignum-type (+ bignum-digits-offset 1))
(storew eax ebx bignum-digits-offset other-pointer-type))
(inst ret))
#+sb-assembling ; We don't want a vop for this one either.
(define-assembly-routine
(move-from-unsigned)
((:temp eax unsigned-reg eax-offset)
(:temp ebx unsigned-reg ebx-offset))
(inst test eax #xe0000000)
(inst jmp :nz bignum)
;; Fixnum
(inst mov ebx eax)
(inst shl ebx 2)
(inst ret)
BIGNUM
;;; Note: On the mips port space for a two word bignum is always
;;; allocated and the header size is set to either one or two words
;;; as appropriate. On the mips port this is faster, and smaller
;;; inline, but produces more garbage. The inline x86 version uses
;;; the same approach, but here we save garbage and allocate the
;;; smallest possible bignum.
(inst jmp :ns one-word-bignum)
(inst mov ebx eax)
;; Two word bignum
(with-fixed-allocation (ebx bignum-type (+ bignum-digits-offset 2))
(storew eax ebx bignum-digits-offset other-pointer-type))
(inst ret)
ONE-WORD-BIGNUM
(with-fixed-allocation (ebx bignum-type (+ bignum-digits-offset 1))
(storew eax ebx bignum-digits-offset other-pointer-type))
(inst ret))

423
src/assembly/x86/arith.lisp Normal file
View file

@ -0,0 +1,423 @@
;;;; simple cases for generic arithmetic
;;;; This software is part of the SBCL system. See the README file for
;;;; more information.
;;;;
;;;; This software is derived from the CMU CL system, which was
;;;; written at Carnegie Mellon University and released into the
;;;; public domain. The software is in the public domain and is
;;;; provided with absolutely no warranty. See the COPYING and CREDITS
;;;; files for more information.
(in-package "SB!VM")
(file-comment
"$Header$")
;;;; addition, subtraction, and multiplication
(macrolet ((define-generic-arith-routine ((fun cost) &body body)
`(define-assembly-routine (,(symbolicate "GENERIC-" fun)
(:cost ,cost)
(:return-style :full-call)
(:translate ,fun)
(:policy :safe)
(:save-p t))
((:arg x (descriptor-reg any-reg) edx-offset)
(:arg y (descriptor-reg any-reg)
;; this seems wrong esi-offset -- FIXME: What's it mean?
edi-offset)
(:res res (descriptor-reg any-reg) edx-offset)
(:temp eax unsigned-reg eax-offset)
(:temp ebx unsigned-reg ebx-offset)
(:temp ecx unsigned-reg ecx-offset))
(declare (ignorable ebx))
(inst test x 3) ; fixnum?
(inst jmp :nz DO-STATIC-FUN) ; no - do generic
(inst test y 3) ; fixnum?
(inst jmp :z DO-BODY) ; yes - doit here
DO-STATIC-FUN
(inst pop eax)
(inst push ebp-tn)
(inst lea
ebp-tn
(make-ea :dword :base esp-tn :disp word-bytes))
(inst sub esp-tn (fixnumize 2))
(inst push eax) ; callers return addr
(inst mov ecx (fixnumize 2)) ; arg count
(inst jmp
(make-ea :dword
:disp (+ *nil-value*
(static-function-offset
',(symbolicate "TWO-ARG-" fun)))))
DO-BODY
,@body)))
(define-generic-arith-routine (+ 10)
(move res x)
(inst add res y)
(inst jmp :no OKAY)
(inst rcr res 1) ; carry has correct sign
(inst sar res 1) ; remove type bits
(move ecx res)
(with-fixed-allocation (res bignum-type (1+ bignum-digits-offset))
(storew ecx res bignum-digits-offset other-pointer-type))
OKAY)
(define-generic-arith-routine (- 10)
;; FIXME: This is screwed up.
;;; I can't figure out the flags on subtract. Overflow never gets
;;; set and carry always does. (- 0 most-negative-fixnum) can't be
;;; easily detected so just let the upper level stuff do it.
(inst jmp DO-STATIC-FUN)
(move res x)
(inst sub res y)
(inst jmp :no OKAY)
(inst rcr res 1)
(inst sar res 1) ; remove type bits
(move ecx res)
(with-fixed-allocation (res bignum-type (1+ bignum-digits-offset))
(storew ecx res bignum-digits-offset other-pointer-type))
OKAY)
(define-generic-arith-routine (* 30)
(move eax x) ; must use eax for 64-bit result
(inst sar eax 2) ; remove *4 fixnum bias
(inst imul y) ; result in edx:eax
(inst jmp :no okay) ; still fixnum
;; zzz jrd changed edx to ebx in here, as edx isn't listed as a temp, above
;; pfw says that loses big -- edx is target for arg x and result res
;; note that 'edx' is not defined -- using x
(inst shrd eax x 2) ; high bits from edx
(inst sar x 2) ; now shift edx too
(move ecx x) ; save high bits from cdq
(inst cdq) ; edx:eax <- sign-extend of eax
(inst cmp x ecx)
(inst jmp :e SINGLE-WORD-BIGNUM)
(with-fixed-allocation (res bignum-type (+ bignum-digits-offset 2))
(storew eax res bignum-digits-offset other-pointer-type)
(storew ecx res (1+ bignum-digits-offset) other-pointer-type))
(inst jmp DONE)
SINGLE-WORD-BIGNUM
(with-fixed-allocation (res bignum-type (1+ bignum-digits-offset))
(storew eax res bignum-digits-offset other-pointer-type))
(inst jmp DONE)
OKAY
(move res eax)
DONE))
;;;; negation
(define-assembly-routine (generic-negate
(:cost 10)
(:return-style :full-call)
(:policy :safe)
(:translate %negate)
(:save-p t))
((:arg x (descriptor-reg any-reg) edx-offset)
(:res res (descriptor-reg any-reg) edx-offset)
(:temp eax unsigned-reg eax-offset)
(:temp ecx unsigned-reg ecx-offset))
(inst test x 3)
(inst jmp :z FIXNUM)
(inst pop eax)
(inst push ebp-tn)
(inst lea ebp-tn (make-ea :dword :base esp-tn :disp word-bytes))
(inst sub esp-tn (fixnumize 2))
(inst push eax)
(inst mov ecx (fixnumize 1)) ; arg count
(inst jmp (make-ea :dword
:disp (+ *nil-value* (static-function-offset '%negate))))
FIXNUM
(move res x)
(inst neg res) ; (- most-negative-fixnum) is BIGNUM
(inst jmp :no OKAY)
(inst shr res 2) ; sign bit is data - remove type bits
(move ecx res)
(with-fixed-allocation (res bignum-type (1+ bignum-digits-offset))
(storew ecx res bignum-digits-offset other-pointer-type))
OKAY)
;;;; comparison
(macrolet ((define-cond-assem-rtn (name translate static-fn test)
`(define-assembly-routine (,name
(:cost 10)
(:return-style :full-call)
(:policy :safe)
(:translate ,translate)
(:save-p t))
((:arg x (descriptor-reg any-reg) edx-offset)
(:arg y (descriptor-reg any-reg) edi-offset)
(:res res descriptor-reg edx-offset)
(:temp eax unsigned-reg eax-offset)
(:temp ecx unsigned-reg ecx-offset))
;; KLUDGE: The "3" here is a mask for the bits which will be
;; zero in a fixnum. It should have a symbolic name. (Actually,
;; it might already have a symbolic name which the coder
;; couldn't be bothered to use..) -- WHN 19990917
(inst test x 3)
(inst jmp :nz TAIL-CALL-TO-STATIC-FN)
(inst test y 3)
(inst jmp :z INLINE-FIXNUM-COMPARE)
TAIL-CALL-TO-STATIC-FN
(inst pop eax)
(inst push ebp-tn)
(inst lea ebp-tn (make-ea :dword :base esp-tn :disp word-bytes))
(inst sub esp-tn (fixnumize 2)) ; FIXME: Push 2 words on stack,
; weirdly?
(inst push eax)
(inst mov ecx (fixnumize 2)) ; FIXME: FIXNUMIZE and
; SINGLE-FLOAT-BITS are parallel,
; should be named parallelly.
(inst jmp (make-ea :dword
:disp (+ *nil-value*
(static-function-offset
',static-fn))))
INLINE-FIXNUM-COMPARE
(inst cmp x y)
(inst jmp ,test RETURN-TRUE)
(inst mov res *nil-value*)
;; FIXME: A note explaining this return convention, or a
;; symbolic name for it, would be nice. (It looks as though we
;; should be hand-crafting the same return sequence as would be
;; produced by GENERATE-RETURN-SEQUENCE, but in that case it's
;; not clear why we don't just jump to the end of this function
;; to share the return sequence there.
(inst pop eax)
(inst add eax 2)
(inst jmp eax)
RETURN-TRUE
(load-symbol res t))))
(define-cond-assem-rtn generic-< < two-arg-< :l)
(define-cond-assem-rtn generic-> > two-arg-> :g))
(define-assembly-routine (generic-eql
(:cost 10)
(:return-style :full-call)
(:policy :safe)
(:translate eql)
(:save-p t))
((:arg x (descriptor-reg any-reg) edx-offset)
(:arg y (descriptor-reg any-reg) edi-offset)
(:res res descriptor-reg edx-offset)
(:temp eax unsigned-reg eax-offset)
(:temp ecx unsigned-reg ecx-offset))
(inst cmp x y)
(inst jmp :e RETURN-T)
(inst test x 3)
(inst jmp :z RETURN-NIL)
(inst test y 3)
(inst jmp :nz DO-STATIC-FN)
RETURN-NIL
(inst mov res *nil-value*)
(inst pop eax)
(inst add eax 2)
(inst jmp eax)
DO-STATIC-FN
(inst pop eax)
(inst push ebp-tn)
(inst lea ebp-tn (make-ea :dword :base esp-tn :disp word-bytes))
(inst sub esp-tn (fixnumize 2))
(inst push eax)
(inst mov ecx (fixnumize 2))
(inst jmp (make-ea :dword
:disp (+ *nil-value* (static-function-offset 'eql))))
RETURN-T
(load-symbol res t)
;; FIXME: I don't understand how we return from here..
)
(define-assembly-routine (generic-=
(:cost 10)
(:return-style :full-call)
(:policy :safe)
(:translate =)
(:save-p t))
((:arg x (descriptor-reg any-reg) edx-offset)
(:arg y (descriptor-reg any-reg) edi-offset)
(:res res descriptor-reg edx-offset)
(:temp eax unsigned-reg eax-offset)
(:temp ecx unsigned-reg ecx-offset)
)
(inst test x 3) ; descriptor?
(inst jmp :nz DO-STATIC-FN) ; yes do it here
(inst test y 3) ; descriptor?
(inst jmp :nz DO-STATIC-FN)
(inst cmp x y)
(inst jmp :e RETURN-T) ; ok
(inst mov res *nil-value*)
(inst pop eax)
(inst add eax 2)
(inst jmp eax)
DO-STATIC-FN
(inst pop eax)
(inst push ebp-tn)
(inst lea ebp-tn (make-ea :dword :base esp-tn :disp word-bytes))
(inst sub esp-tn (fixnumize 2))
(inst push eax)
(inst mov ecx (fixnumize 2))
(inst jmp (make-ea :dword
:disp (+ *nil-value* (static-function-offset 'two-arg-=))))
RETURN-T
(load-symbol res t))
;;; Support for the Mersenne Twister, MT19937, random number generator
;;; due to Matsumoto and Nishimura.
;;;
;;; Makoto Matsumoto and T. Nishimura, "Mersenne twister: A
;;; 623-dimensionally equidistributed uniform pseudorandom number
;;; generator.", ACM Transactions on Modeling and Computer Simulation,
;;; 1997, to appear.
;;;
;;; State:
;;; 0-1: Constant matrix A. [0, #x9908b0df] (not used here)
;;; 2: Index; init. to 1.
;;; 3-626: State.
;;; This assembly routine is called from the inline VOP and updates
;;; the state vector with new random numbers. The state vector is
;;; passed in the EAX register.
#+sb-assembling ; We don't want a vop for this one.
(define-assembly-routine
(random-mt19937-update)
((:temp state unsigned-reg eax-offset)
(:temp k unsigned-reg ebx-offset)
(:temp y unsigned-reg ecx-offset)
(:temp tmp unsigned-reg edx-offset))
;; Save the temporary registers.
(inst push k)
(inst push y)
(inst push tmp)
;; Generate a new set of results.
(inst xor k k)
LOOP1
(inst mov y (make-ea :dword :base state :index k :scale 4
:disp (- (* (+ 3 sb!vm:vector-data-offset)
sb!vm:word-bytes)
sb!vm:other-pointer-type)))
(inst mov tmp (make-ea :dword :base state :index k :scale 4
:disp (- (* (+ 1 3 sb!vm:vector-data-offset)
sb!vm:word-bytes)
sb!vm:other-pointer-type)))
(inst and y #x80000000)
(inst and tmp #x7fffffff)
(inst or y tmp)
(inst shr y 1)
(inst jmp :nc skip1)
(inst xor y #x9908b0df)
SKIP1
(inst xor y (make-ea :dword :base state :index k :scale 4
:disp (- (* (+ 397 3 sb!vm:vector-data-offset)
sb!vm:word-bytes)
sb!vm:other-pointer-type)))
(inst mov (make-ea :dword :base state :index k :scale 4
:disp (- (* (+ 3 sb!vm:vector-data-offset)
sb!vm:word-bytes)
sb!vm:other-pointer-type))
y)
(inst inc k)
(inst cmp k (- 624 397))
(inst jmp :b loop1)
LOOP2
(inst mov y (make-ea :dword :base state :index k :scale 4
:disp (- (* (+ 3 sb!vm:vector-data-offset)
sb!vm:word-bytes)
sb!vm:other-pointer-type)))
(inst mov tmp (make-ea :dword :base state :index k :scale 4
:disp (- (* (+ 1 3 sb!vm:vector-data-offset)
sb!vm:word-bytes)
sb!vm:other-pointer-type)))
(inst and y #x80000000)
(inst and tmp #x7fffffff)
(inst or y tmp)
(inst shr y 1)
(inst jmp :nc skip2)
(inst xor y #x9908b0df)
SKIP2
(inst xor y (make-ea :dword :base state :index k :scale 4
:disp (- (* (+ (- 397 624) 3 sb!vm:vector-data-offset)
sb!vm:word-bytes)
sb!vm:other-pointer-type)))
(inst mov (make-ea :dword :base state :index k :scale 4
:disp (- (* (+ 3 sb!vm:vector-data-offset)
sb!vm:word-bytes)
sb!vm:other-pointer-type))
y)
(inst inc k)
(inst cmp k (- 624 1))
(inst jmp :b loop2)
(inst mov y (make-ea :dword :base state
:disp (- (* (+ (- 624 1) 3 sb!vm:vector-data-offset)
sb!vm:word-bytes)
sb!vm:other-pointer-type)))
(inst mov tmp (make-ea :dword :base state
:disp (- (* (+ 0 3 sb!vm:vector-data-offset)
sb!vm:word-bytes)
sb!vm:other-pointer-type)))
(inst and y #x80000000)
(inst and tmp #x7fffffff)
(inst or y tmp)
(inst shr y 1)
(inst jmp :nc skip3)
(inst xor y #x9908b0df)
SKIP3
(inst xor y (make-ea :dword :base state
:disp (- (* (+ (- 397 1) 3 sb!vm:vector-data-offset)
sb!vm:word-bytes)
sb!vm:other-pointer-type)))
(inst mov (make-ea :dword :base state
:disp (- (* (+ (- 624 1) 3 sb!vm:vector-data-offset)
sb!vm:word-bytes)
sb!vm:other-pointer-type))
y)
;; Restore the temporary registers and return.
(inst pop tmp)
(inst pop y)
(inst pop k)
(inst ret))

42
src/assembly/x86/array.lisp Normal file
View file

@ -0,0 +1,42 @@
;;;; various array operations that are too expensive (in space) to do
;;;; inline
;;;; This software is part of the SBCL system. See the README file for
;;;; more information.
;;;;
;;;; This software is derived from the CMU CL system, which was
;;;; written at Carnegie Mellon University and released into the
;;;; public domain. The software is in the public domain and is
;;;; provided with absolutely no warranty. See the COPYING and CREDITS
;;;; files for more information.
(in-package "SB!VM")
(file-comment
"$Header$")
;;;; allocation
(define-assembly-routine (allocate-vector
(:policy :fast-safe)
(:translate allocate-vector)
(:arg-types positive-fixnum
positive-fixnum
positive-fixnum))
((:arg type unsigned-reg eax-offset)
(:arg length any-reg ebx-offset)
(:arg words any-reg ecx-offset)
(:res result descriptor-reg edx-offset))
(inst mov result (+ (1- (ash 1 lowtag-bits))
(* vector-data-offset word-bytes)))
(inst add result words)
(inst and result (lognot sb!vm:lowtag-mask))
(pseudo-atomic
(allocation result result)
(inst lea result (make-ea :byte :base result :disp other-pointer-type))
(storew type result 0 other-pointer-type)
(storew length result vector-length-slot other-pointer-type))
(inst ret))
;;;; Note: CMU CL had assembly language primitives for hashing strings,
;;;; but SBCL doesn't.

261
src/assembly/x86/assem-rtns.lisp Normal file
View file

@ -0,0 +1,261 @@
;;;; the machine specific support routines needed by the file assembler
;;;; This software is part of the SBCL system. See the README file for
;;;; more information.
;;;;
;;;; This software is derived from the CMU CL system, which was
;;;; written at Carnegie Mellon University and released into the
;;;; public domain. The software is in the public domain and is
;;;; provided with absolutely no warranty. See the COPYING and CREDITS
;;;; files for more information.
(in-package "SB!VM")
(file-comment
"$Header$")
;;;; RETURN-MULTIPLE
;;; For RETURN-MULTIPLE, we have to move the results from the end of
;;; the frame for the function that is returning to the end of the
;;; frame for the function being returned to.
#+sb-assembling ;; We don't want a vop for this one.
(define-assembly-routine
(return-multiple (:return-style :none))
(;; These four are really arguments.
(:temp eax unsigned-reg eax-offset)
(:temp ebx unsigned-reg ebx-offset)
(:temp ecx unsigned-reg ecx-offset)
(:temp esi unsigned-reg esi-offset)
;; These we need as temporaries.
(:temp edx unsigned-reg edx-offset)
(:temp edi unsigned-reg edi-offset))
;; Pick off the cases where everything fits in register args.
(inst jecxz zero-values)
(inst cmp ecx (fixnumize 1))
(inst jmp :e one-value)
(inst cmp ecx (fixnumize 2))
(inst jmp :e two-values)
(inst cmp ecx (fixnumize 3))
(inst jmp :e three-values)
;; Save the count, because the loop is going to destroy it.
(inst mov edx ecx)
;; Blit the values down the stack. Note: there might be overlap, so we have
;; to be careful not to clobber values before we've read them. Because the
;; stack builds down, we are coping to a larger address. Therefore, we need
;; to iterate from larger addresses to smaller addresses.
;; pfw-this says copy ecx words from esi to edi counting down.
(inst shr ecx 2) ; fixnum to raw word count
(inst std) ; count down
(inst sub esi 4) ; ?
(inst lea edi (make-ea :dword :base ebx :disp (- word-bytes)))
(inst rep)
(inst movs :dword)
;; Restore the count.
(inst mov ecx edx)
;; Set the stack top to the last result.
(inst lea esp-tn (make-ea :dword :base edi :disp word-bytes))
;; Load the register args.
(loadw edx ebx -1)
(loadw edi ebx -2)
(loadw esi ebx -3)
;; And back we go.
(inst jmp eax)
;; Handle the register arg cases.
ZERO-VALUES
(move esp-tn ebx)
(inst mov edx *nil-value*)
(inst mov edi edx)
(inst mov esi edx)
(inst jmp eax)
ONE-VALUE ; Note: we can get this, because the return-multiple vop
; doesn't check for this case when size > speed.
(loadw edx esi -1)
(inst mov esp-tn ebx)
(inst add eax 2)
(inst jmp eax)
TWO-VALUES
(loadw edx esi -1)
(loadw edi esi -2)
(inst mov esi *nil-value*)
(inst lea esp-tn (make-ea :dword :base ebx :disp (* -2 word-bytes)))
(inst jmp eax)
THREE-VALUES
(loadw edx esi -1)
(loadw edi esi -2)
(loadw esi esi -3)
(inst lea esp-tn (make-ea :dword :base ebx :disp (* -3 word-bytes)))
(inst jmp eax))
;;;; TAIL-CALL-VARIABLE
;;; For tail-call-variable, we have to copy the arguments from the end of our
;;; stack frame (were args are produced) to the start of our stack frame
;;; (were args are expected).
;;;
;;; We take the function to call in EAX and a pointer to the arguments in
;;; ESI. EBP says the same over the jump, and the old frame pointer is
;;; still saved in the first stack slot. The return-pc is saved in
;;; the second stack slot, so we have to push it to make it look like
;;; we actually called. We also have to compute ECX from the difference
;;; between ESI and the stack top.
#+sb-assembling ;; No vop for this one either.
(define-assembly-routine
(tail-call-variable
(:return-style :none))
((:temp eax unsigned-reg eax-offset)
(:temp ebx unsigned-reg ebx-offset)
(:temp ecx unsigned-reg ecx-offset)
(:temp edx unsigned-reg edx-offset)
(:temp edi unsigned-reg edi-offset)
(:temp esi unsigned-reg esi-offset))
;; Calculate NARGS (as a fixnum)
(move ecx esi)
(inst sub ecx esp-tn)
;; Check for all the args fitting the the registers.
(inst cmp ecx (fixnumize 3))
(inst jmp :le REGISTER-ARGS)
;; Save the OLD-FP and RETURN-PC because the blit it going to trash
;; those stack locations. Save the ECX, because the loop is going
;; to trash it.
(pushw ebp-tn -1)
(loadw ebx ebp-tn -2)
(inst push ecx)
;; Do the blit. Because we are coping from smaller addresses to larger
;; addresses, we have to start at the largest pair and work our way down.
(inst shr ecx 2) ; fixnum to raw words
(inst std) ; count down
(inst lea edi (make-ea :dword :base ebp-tn :disp (- word-bytes)))
(inst sub esi (fixnumize 1))
(inst rep)
(inst movs :dword)
;; Load the register arguments carefully.
(loadw edx ebp-tn -1)
;; Restore OLD-FP and ECX.
(inst pop ecx)
(popw ebp-tn -1) ; overwrites a0
;; Blow off the stack above the arguments.
(inst lea esp-tn (make-ea :dword :base edi :disp word-bytes))
;; remaining register args
(loadw edi ebp-tn -2)
(loadw esi ebp-tn -3)
;; Push the (saved) return-pc so it looks like we just called.
(inst push ebx)
;; And jump into the function.
(inst jmp
(make-ea :byte :base eax
:disp (- (* closure-function-slot word-bytes)
function-pointer-type)))
;; All the arguments fit in registers, so load them.
REGISTER-ARGS
(loadw edx esi -1)
(loadw edi esi -2)
(loadw esi esi -3)
;; Clear most of the stack.
(inst lea esp-tn
(make-ea :dword :base ebp-tn :disp (* -3 word-bytes)))
;; Push the return-pc so it looks like we just called.
(pushw ebp-tn -2)
;; And away we go.
(inst jmp (make-ea :byte :base eax
:disp (- (* closure-function-slot word-bytes)
function-pointer-type))))
(define-assembly-routine (throw
(:return-style :none))
((:arg target (descriptor-reg any-reg) edx-offset)
(:arg start any-reg ebx-offset)
(:arg count any-reg ecx-offset)
(:temp catch any-reg eax-offset))
(declare (ignore start count))
(load-symbol-value catch sb!impl::*current-catch-block*)
LOOP
(let ((error (generate-error-code nil unseen-throw-tag-error target)))
(inst or catch catch) ; check for NULL pointer
(inst jmp :z error))
(inst cmp target (make-ea-for-object-slot catch catch-block-tag-slot 0))
(inst jmp :e exit)
(loadw catch catch catch-block-previous-catch-slot)
(inst jmp loop)
EXIT
;; Hear EAX points to catch block containing symbol pointed to by EDX.
(inst jmp (make-fixup 'unwind :assembly-routine)))
;;;; non-local exit noise
(define-assembly-routine (unwind
(:return-style :none)
(:translate %continue-unwind)
(:policy :fast-safe))
((:arg block (any-reg descriptor-reg) eax-offset)
(:arg start (any-reg descriptor-reg) ebx-offset)
(:arg count (any-reg descriptor-reg) ecx-offset)
(:temp uwp unsigned-reg esi-offset))
(declare (ignore start count))
(let ((error (generate-error-code nil invalid-unwind-error)))
(inst or block block) ; check for NULL pointer
(inst jmp :z error))
(load-symbol-value uwp sb!impl::*current-unwind-protect-block*)
;; Does *cuwpb* match value stored in argument cuwp slot?
(inst cmp uwp
(make-ea-for-object-slot block unwind-block-current-uwp-slot 0))
;; If a match, return to context in arg block.
(inst jmp :e do-exit)
;; Not a match - return to *current-unwind-protect-block* context.
;; Important! Must save (and return) the arg 'block' for later use!!
(move edx-tn block)
(move block uwp)
;; Set next unwind protect context.
(loadw uwp uwp unwind-block-current-uwp-slot)
(store-symbol-value uwp sb!impl::*current-unwind-protect-block*)
DO-EXIT
(loadw ebp-tn block unwind-block-current-cont-slot)
;; Uwp-entry expects some things in known locations so that they can
;; be saved on the stack: the block in edx-tn; start in ebx-tn; and
;; count in ecx-tn
(inst jmp (make-ea :byte :base block
:disp (* unwind-block-entry-pc-slot word-bytes))))

15
src/assembly/x86/bit-bash.lisp Normal file
View file

@ -0,0 +1,15 @@
;;;; just a dummy file to maintain parallelism with other VMs
;;;; This software is part of the SBCL system. See the README file for
;;;; more information.
;;;;
;;;; This software is derived from the CMU CL system, which was
;;;; written at Carnegie Mellon University and released into the
;;;; public domain. The software is in the public domain and is
;;;; provided with absolutely no warranty. See the COPYING and CREDITS
;;;; files for more information.
(in-package "SB!VM")
(file-comment
"$Header$")

43
src/assembly/x86/support.lisp Normal file
View file

@ -0,0 +1,43 @@
;;;; This software is part of the SBCL system. See the README file for
;;;; more information.
;;;;
;;;; This software is derived from the CMU CL system, which was
;;;; written at Carnegie Mellon University and released into the
;;;; public domain. The software is in the public domain and is
;;;; provided with absolutely no warranty. See the COPYING and CREDITS
;;;; files for more information.
(in-package "SB!VM")
(file-comment
"$Header$")
(def-vm-support-routine generate-call-sequence (name style vop)
(ecase style
(:raw
(values
`((inst call (make-fixup ',name :assembly-routine)))
nil))
(:full-call
(values
`((note-this-location ,vop :call-site)
(inst call (make-fixup ',name :assembly-routine))
(note-this-location ,vop :single-value-return)
(move esp-tn ebx-tn))
'((:save-p :compute-only))))
(:none
(values
`((inst jmp (make-fixup ',name :assembly-routine)))
nil))))
(def-vm-support-routine generate-return-sequence (style)
(ecase style
(:raw
`(inst ret))
(:full-call
`(
(inst pop eax-tn)
(inst add eax-tn 2)
(inst jmp eax-tn)))
(:none)))

71
src/code/alien-type.lisp Normal file
View file

@ -0,0 +1,71 @@
;;;; ALIEN-related type system stuff, done later
;;;; than other type system stuff because it depends on the definition
;;;; of the ALIEN-VALUE target structure type
;;;; This software is part of the SBCL system. See the README file for
;;;; more information.
;;;;
;;;; This software is derived from the CMU CL system, which was
;;;; written at Carnegie Mellon University and released into the
;;;; public domain. The software is in the public domain and is
;;;; provided with absolutely no warranty. See the COPYING and CREDITS
;;;; files for more information.
(in-package "SB!KERNEL")
(file-comment
"$Header$")
(!begin-collecting-cold-init-forms)
(defstruct (alien-type-type
(:include ctype
(class-info (type-class-or-lose 'alien)))
(:constructor %make-alien-type-type (alien-type)))
(alien-type nil :type alien-type))
(define-type-class alien)
(define-type-method (alien :unparse) (type)
`(alien ,(unparse-alien-type (alien-type-type-alien-type type))))
(define-type-method (alien :simple-subtypep) (type1 type2)
(values (alien-subtype-p (alien-type-type-alien-type type1)
(alien-type-type-alien-type type2))
t))
;;; KLUDGE: This DEFINE-SUPERCLASSES gets executed much later than the others
;;; (toplevel form time instead of cold load init time) because ALIEN-VALUE
;;; itself is a structure which isn't defined until fairly late.
;;;
;;; FIXME: I'm somewhat tempted to just punt ALIEN from the type system.
;;; It's sufficiently unlike the others that it's a bit of a pain, and
;;; it doesn't seem to be put to any good use either in type inference or
;;; in type declarations.
(define-superclasses alien ((alien-value)) progn)
(define-type-method (alien :simple-=) (type1 type2)
(let ((alien-type-1 (alien-type-type-alien-type type1))
(alien-type-2 (alien-type-type-alien-type type2)))
(values (or (eq alien-type-1 alien-type-2)
(alien-type-= alien-type-1 alien-type-2))
t)))
(def-type-translator alien (&optional (alien-type nil))
(typecase alien-type
(null
(make-alien-type-type))
(alien-type
(make-alien-type-type alien-type))
(t
(make-alien-type-type (parse-alien-type alien-type (make-null-lexenv))))))
(defun make-alien-type-type (&optional alien-type)
(if alien-type
(let ((lisp-rep-type (compute-lisp-rep-type alien-type)))
(if lisp-rep-type
(specifier-type lisp-rep-type)
(%make-alien-type-type alien-type)))
*universal-type*))
(!defun-from-collected-cold-init-forms !alien-type-cold-init)

1093
src/code/array.lisp Normal file
View file

File diff suppressed because it is too large Load diff

212
src/code/backq.lisp Normal file
View file

@ -0,0 +1,212 @@
;;;; the backquote reader macro
;;;; This software is part of the SBCL system. See the README file for
;;;; more information.
;;;;
;;;; This software is derived from the CMU CL system, which was
;;;; written at Carnegie Mellon University and released into the
;;;; public domain. The software is in the public domain and is
;;;; provided with absolutely no warranty. See the COPYING and CREDITS
;;;; files for more information.
(in-package "SB!IMPL")
(file-comment
"$Header$")
;;; The flags passed back by BACKQUOTIFY can be interpreted as follows:
;;;
;;; |`,|: [a] => a
;;; NIL: [a] => a ;the NIL flag is used only when a is NIL
;;; T: [a] => a ;the T flag is used when a is self-evaluating
;;; QUOTE: [a] => (QUOTE a)
;;; APPEND: [a] => (APPEND . a)
;;; NCONC: [a] => (NCONC . a)
;;; LIST: [a] => (LIST . a)
;;; LIST*: [a] => (LIST* . a)
;;;
;;; The flags are combined according to the following set of rules:
;;; ([a] means that a should be converted according to the previous table)
;;;
;;; \ car || otherwise | QUOTE or | |`,@| | |`,.|
;;;cdr \ || | T or NIL | |
;;;================================================================================
;;; |`,| || LIST* ([a] [d]) | LIST* ([a] [d]) | APPEND (a [d]) | NCONC (a [d])
;;; NIL || LIST ([a]) | QUOTE (a) | <hair> a | <hair> a
;;;QUOTE or T|| LIST* ([a] [d]) | QUOTE (a . d) | APPEND (a [d]) | NCONC (a [d])
;;; APPEND || LIST* ([a] [d]) | LIST* ([a] [d]) | APPEND (a . d) | NCONC (a [d])
;;; NCONC || LIST* ([a] [d]) | LIST* ([a] [d]) | APPEND (a [d]) | NCONC (a . d)
;;; LIST || LIST ([a] . d) | LIST ([a] . d) | APPEND (a [d]) | NCONC (a [d])
;;; LIST* || LIST* ([a] . d) | LIST* ([a] . d) | APPEND (a [d]) | NCONC (a [d])
;;;
;;;<hair> involves starting over again pretending you had read ".,a)" instead
;;; of ",@a)"
(defvar *backquote-count* 0 #!+sb-doc "how deep we are into backquotes")
(defvar *bq-comma-flag* '(|,|))
(defvar *bq-at-flag* '(|,@|))
(defvar *bq-dot-flag* '(|,.|))
(defvar *bq-vector-flag* '(|bqv|))
;;; the actual character macro
(defun backquote-macro (stream ignore)
(declare (ignore ignore))
(let ((*backquote-count* (1+ *backquote-count*)))
(multiple-value-bind (flag thing)
(backquotify stream (read stream t nil t))
(if (eq flag *bq-at-flag*)
(%reader-error stream ",@ after backquote in ~S" thing))
(if (eq flag *bq-dot-flag*)
(%reader-error stream ",. after backquote in ~S" thing))
(values (backquotify-1 flag thing) 'list))))
(defun comma-macro (stream ignore)
(declare (ignore ignore))
(unless (> *backquote-count* 0)
(when *read-suppress*
(return-from comma-macro nil))
(%reader-error stream "comma not inside a backquote"))
(let ((c (read-char stream))
(*backquote-count* (1- *backquote-count*)))
(values
(cond ((char= c #\@)
(cons *bq-at-flag* (read stream t nil t)))
((char= c #\.)
(cons *bq-dot-flag* (read stream t nil t)))
(t (unread-char c stream)
(cons *bq-comma-flag* (read stream t nil t))))
'list)))
;;; This does the expansion from table 2.
(defun backquotify (stream code)
(cond ((atom code)
(cond ((null code) (values nil nil))
((or (numberp code)
(eq code t))
;; Keywords are self-evaluating. Install after packages.
(values t code))
(t (values 'quote code))))
((or (eq (car code) *bq-at-flag*)
(eq (car code) *bq-dot-flag*))
(values (car code) (cdr code)))
((eq (car code) *bq-comma-flag*)
(comma (cdr code)))
((eq (car code) *bq-vector-flag*)
(multiple-value-bind (dflag d) (backquotify stream (cdr code))
(values 'vector (backquotify-1 dflag d))))
(t (multiple-value-bind (aflag a) (backquotify stream (car code))
(multiple-value-bind (dflag d) (backquotify stream (cdr code))
(if (eq dflag *bq-at-flag*)
;; Get the errors later.
(%reader-error stream ",@ after dot in ~S" code))
(if (eq dflag *bq-dot-flag*)
(%reader-error stream ",. after dot in ~S" code))
(cond
((eq aflag *bq-at-flag*)
(if (null dflag)
(comma a)
(values 'append
(cond ((eq dflag 'append)
(cons a d ))
(t (list a (backquotify-1 dflag d)))))))
((eq aflag *bq-dot-flag*)
(if (null dflag)
(comma a)
(values 'nconc
(cond ((eq dflag 'nconc)
(cons a d))
(t (list a (backquotify-1 dflag d)))))))
((null dflag)
(if (member aflag '(quote t nil))
(values 'quote (list a))
(values 'list (list (backquotify-1 aflag a)))))
((member dflag '(quote t))
(if (member aflag '(quote t nil))
(values 'quote (cons a d ))
(values 'list* (list (backquotify-1 aflag a)
(backquotify-1 dflag d)))))
(t (setq a (backquotify-1 aflag a))
(if (member dflag '(list list*))
(values dflag (cons a d))
(values 'list*
(list a (backquotify-1 dflag d)))))))))))
;;; This handles the <hair> cases.
(defun comma (code)
(cond ((atom code)
(cond ((null code)
(values nil nil))
((or (numberp code) (eq code 't))
(values t code))
(t (values *bq-comma-flag* code))))
((eq (car code) 'quote)
(values (car code) (cadr code)))
((member (car code) '(append list list* nconc))
(values (car code) (cdr code)))
((eq (car code) 'cons)
(values 'list* (cdr code)))
(t (values *bq-comma-flag* code))))
;;; This handles table 1.
(defun backquotify-1 (flag thing)
(cond ((or (eq flag *bq-comma-flag*)
(member flag '(t nil)))
thing)
((eq flag 'quote)
(list 'quote thing))
((eq flag 'list*)
(cond ((null (cddr thing))
(cons 'backq-cons thing))
(t
(cons 'backq-list* thing))))
((eq flag 'vector)
(list 'backq-vector thing))
(t (cons (cdr
(assoc flag
'((cons . backq-cons)
(list . backq-list)
(append . backq-append)
(nconc . backq-nconc))
:test #'equal))
thing))))
;;;; magic BACKQ- versions of builtin functions
;;; Define synonyms for the lisp functions we use, so that by using them, we
;;; backquoted material will be recognizable to the pretty-printer.
(macrolet ((def-frob (b-name name)
(let ((args (gensym "ARGS")))
;; FIXME: This function should be INLINE so that the lists
;; aren't consed twice, but I ran into an optimizer bug the
;; first time I tried to make this work for BACKQ-LIST. See
;; whether there's still an optimizer bug, and fix it if so, and
;; then make these INLINE.
`(defun ,b-name (&rest ,args)
(apply #',name ,args)))))
(def-frob backq-list list)
(def-frob backq-list* list*)
(def-frob backq-append append)
(def-frob backq-nconc nconc)
(def-frob backq-cons cons))
(defun backq-vector (list)
(declare (list list))
(coerce list 'simple-vector))
;;;; initialization
;;; Install BACKQ stuff in the current *READTABLE*.
;;;
;;; In the target Lisp, we have to wait to do this until the readtable has been
;;; created. In the cross-compilation host Lisp, we can do this right away.
;;; (You may ask: In the cross-compilation host, which already has its own
;;; implementation of the backquote readmacro, why do we do this at all?
;;; Because the cross-compilation host might -- as SBCL itself does -- express
;;; the backquote expansion in terms of internal, nonportable functions. By
;;; redefining backquote in terms of functions which are guaranteed to exist on
;;; the target Lisp, we ensure that backquote expansions in code-generating
;;; code work properly.)
(defun !backq-cold-init ()
(set-macro-character #\` #'backquote-macro)
(set-macro-character #\, #'comma-macro))
#+sb-xc-host (!backq-cold-init)

2275
src/code/bignum.lisp Normal file
View file

File diff suppressed because it is too large Load diff

520
src/code/bit-bash.lisp Normal file
View file

@ -0,0 +1,520 @@
;;;; functions to implement bitblt-ish operations
;;;; This software is part of the SBCL system. See the README file for
;;;; more information.
;;;;
;;;; This software is derived from the CMU CL system, which was
;;;; written at Carnegie Mellon University and released into the
;;;; public domain. The software is in the public domain and is
;;;; provided with absolutely no warranty. See the COPYING and CREDITS
;;;; files for more information.
(in-package "SB!VM")
(file-comment
"$Header$")
;;;; constants and types
(eval-when (:compile-toplevel :load-toplevel :execute)
(defconstant unit-bits sb!vm:word-bits
#!+sb-doc
"The number of bits to process at a time.")
(defconstant max-bits (ash most-positive-fixnum -2)
#!+sb-doc
"The maximum number of bits that can be delt with during a single call.")
(deftype unit ()
`(unsigned-byte ,unit-bits))
(deftype offset ()
`(integer 0 ,max-bits))
(deftype bit-offset ()
`(integer 0 (,unit-bits)))
(deftype bit-count ()
`(integer 1 (,unit-bits)))
(deftype word-offset ()
`(integer 0 (,(ceiling max-bits unit-bits))))
) ; EVAL-WHEN
;;;; support routines
;;; A particular implementation must offer either VOPs to translate
;;; these, or DEFTRANSFORMs to convert them into something supported
;;; by the architecture.
(macrolet ((def-frob (name &rest args)
`(defun ,name ,args
(,name ,@args))))
(def-frob 32bit-logical-not x)
(def-frob 32bit-logical-and x y)
(def-frob 32bit-logical-or x y)
(def-frob 32bit-logical-xor x y)
(def-frob 32bit-logical-nor x y)
(def-frob 32bit-logical-eqv x y)
(def-frob 32bit-logical-nand x y)
(def-frob 32bit-logical-andc1 x y)
(def-frob 32bit-logical-andc2 x y)
(def-frob 32bit-logical-orc1 x y)
(def-frob 32bit-logical-orc2 x y))
(defun shift-towards-start (number countoid)
#!+sb-doc
"Shift NUMBER by the low-order bits of COUNTOID, adding zero bits at
the ``end'' and removing bits from the ``start.'' On big-endian
machines this is a left-shift and on little-endian machines this is a
right-shift."
(declare (type unit number) (fixnum countoid))
(let ((count (ldb (byte (1- (integer-length unit-bits)) 0) countoid)))
(declare (type bit-offset count))
(if (zerop count)
number
(ecase sb!c:*backend-byte-order*
(:big-endian
(ash (ldb (byte (- unit-bits count) 0) number) count))
(:little-endian
(ash number (- count)))))))
(defun shift-towards-end (number count)
#!+sb-doc
"Shift NUMBER by COUNT bits, adding zero bits at the ``start'' and removing
bits from the ``end.'' On big-endian machines this is a right-shift and
on little-endian machines this is a left-shift."
(declare (type unit number) (fixnum count))
(let ((count (ldb (byte (1- (integer-length unit-bits)) 0) count)))
(declare (type bit-offset count))
(if (zerop count)
number
(ecase sb!c:*backend-byte-order*
(:big-endian
(ash number (- count)))
(:little-endian
(ash (ldb (byte (- unit-bits count) 0) number) count))))))
#!-sb-fluid (declaim (inline start-mask end-mask fix-sap-and-offset))
(defun start-mask (count)
#!+sb-doc
"Produce a mask that contains 1's for the COUNT ``start'' bits and 0's for
the remaining ``end'' bits. Only the lower 5 bits of COUNT are significant."
(declare (fixnum count))
(shift-towards-start (1- (ash 1 unit-bits)) (- count)))
(defun end-mask (count)
#!+sb-doc
"Produce a mask that contains 1's for the COUNT ``end'' bits and 0's for
the remaining ``start'' bits. Only the lower 5 bits of COUNT are
significant."
(declare (fixnum count))
(shift-towards-end (1- (ash 1 unit-bits)) (- count)))
(defun fix-sap-and-offset (sap offset)
#!+sb-doc
"Align the SAP to a word boundary, and update the offset accordingly."
(declare (type system-area-pointer sap)
(type index offset)
(values system-area-pointer index))
(let ((address (sap-int sap)))
(values (int-sap #!-alpha (32bit-logical-andc2 address 3)
#!+alpha (ash (ash address -2) 2))
(+ (* (logand address 3) byte-bits) offset))))
#!-sb-fluid (declaim (inline word-sap-ref %set-word-sap-ref))
(defun word-sap-ref (sap offset)
(declare (type system-area-pointer sap)
(type index offset)
(values (unsigned-byte 32))
(optimize (speed 3) (safety 0) #-sb-xc-host (inhibit-warnings 3)))
(sap-ref-32 sap (the index (ash offset 2))))
(defun %set-word-sap-ref (sap offset value)
(declare (type system-area-pointer sap)
(type index offset)
(type (unsigned-byte 32) value)
(values (unsigned-byte 32))
(optimize (speed 3) (safety 0) (inhibit-warnings 3)))
(setf (sap-ref-32 sap (the index (ash offset 2))) value))
;;;; DO-CONSTANT-BIT-BASH
#!-sb-fluid (declaim (inline do-constant-bit-bash))
(defun do-constant-bit-bash (dst dst-offset length value dst-ref-fn dst-set-fn)
#!+sb-doc
"Fill DST with VALUE starting at DST-OFFSET and continuing for LENGTH bits."
(declare (type offset dst-offset) (type unit value)
(type function dst-ref-fn dst-set-fn))
(multiple-value-bind (dst-word-offset dst-bit-offset)
(floor dst-offset unit-bits)
(declare (type word-offset dst-word-offset)
(type bit-offset dst-bit-offset))
(multiple-value-bind (words final-bits)
(floor (+ dst-bit-offset length) unit-bits)
(declare (type word-offset words) (type bit-offset final-bits))
(if (zerop words)
(unless (zerop length)
(funcall dst-set-fn dst dst-word-offset
(if (= length unit-bits)
value
(let ((mask (shift-towards-end (start-mask length)
dst-bit-offset)))
(declare (type unit mask))
(32bit-logical-or
(32bit-logical-and value mask)
(32bit-logical-andc2
(funcall dst-ref-fn dst dst-word-offset)
mask))))))
(let ((interior (floor (- length final-bits) unit-bits)))
(unless (zerop dst-bit-offset)
(let ((mask (end-mask (- dst-bit-offset))))
(declare (type unit mask))
(funcall dst-set-fn dst dst-word-offset
(32bit-logical-or
(32bit-logical-and value mask)
(32bit-logical-andc2
(funcall dst-ref-fn dst dst-word-offset)
mask))))
(incf dst-word-offset))
(dotimes (i interior)
(funcall dst-set-fn dst dst-word-offset value)
(incf dst-word-offset))
(unless (zerop final-bits)
(let ((mask (start-mask final-bits)))
(declare (type unit mask))
(funcall dst-set-fn dst dst-word-offset
(32bit-logical-or
(32bit-logical-and value mask)
(32bit-logical-andc2
(funcall dst-ref-fn dst dst-word-offset)
mask)))))))))
(values))
;;;; DO-UNARY-BIT-BASH
#!-sb-fluid (declaim (inline do-unary-bit-bash))
(defun do-unary-bit-bash (src src-offset dst dst-offset length
dst-ref-fn dst-set-fn src-ref-fn)
(declare (type offset src-offset dst-offset length)
(type function dst-ref-fn dst-set-fn src-ref-fn))
(multiple-value-bind (dst-word-offset dst-bit-offset)
(floor dst-offset unit-bits)
(declare (type word-offset dst-word-offset)
(type bit-offset dst-bit-offset))
(multiple-value-bind (src-word-offset src-bit-offset)
(floor src-offset unit-bits)
(declare (type word-offset src-word-offset)
(type bit-offset src-bit-offset))
(cond
((<= (+ dst-bit-offset length) unit-bits)
;; We are only writing one word, so it doesn't matter what order
;; we do it in. But we might be reading from multiple words, so take
;; care.
(cond
((zerop length)
;; Actually, we aren't even writing one word. This is real easy.
)
((= length unit-bits)
;; DST-BIT-OFFSET must be equal to zero, or we would be writing
;; multiple words. If SRC-BIT-OFFSET is also zero, then we
;; just transfer the single word. Otherwise we have to extract bits
;; from two src words.
(funcall dst-set-fn dst dst-word-offset
(if (zerop src-bit-offset)
(funcall src-ref-fn src src-word-offset)
(32bit-logical-or
(shift-towards-start
(funcall src-ref-fn src src-word-offset)
src-bit-offset)
(shift-towards-end
(funcall src-ref-fn src (1+ src-word-offset))
(- src-bit-offset))))))
(t
;; We are only writing some portion of the dst word, so we need to
;; preserve the extra bits. Also, we still don't know whether we need
;; one or two source words.
(let ((mask (shift-towards-end (start-mask length) dst-bit-offset))
(orig (funcall dst-ref-fn dst dst-word-offset))
(value
(if (> src-bit-offset dst-bit-offset)
;; The source starts further into the word than does
;; the dst, so the source could extend into the next
;; word. If it does, we have to merge the two words,
;; and if not, we can just shift the first word.
(let ((src-bit-shift (- src-bit-offset dst-bit-offset)))
(if (> (+ src-bit-offset length) unit-bits)
(32bit-logical-or
(shift-towards-start
(funcall src-ref-fn src src-word-offset)
src-bit-shift)
(shift-towards-end
(funcall src-ref-fn src (1+ src-word-offset))
(- src-bit-shift)))
(shift-towards-start
(funcall src-ref-fn src src-word-offset)
src-bit-shift)))
;; The dst starts further into the word than does the
;; source, so we know the source can not extend into
;; a second word (or else the dst would too, and we
;; wouldn't be in this branch.
(shift-towards-end
(funcall src-ref-fn src src-word-offset)
(- dst-bit-offset src-bit-offset)))))
(declare (type unit mask orig value))
;; Replace the dst word.
(funcall dst-set-fn dst dst-word-offset
(32bit-logical-or
(32bit-logical-and value mask)
(32bit-logical-andc2 orig mask)))))))
((= src-bit-offset dst-bit-offset)
;; The source and dst are aligned, so we don't need to shift
;; anything. But we have to pick the direction of the loop
;; in case the source and dst are really the same thing.
(multiple-value-bind (words final-bits)
(floor (+ dst-bit-offset length) unit-bits)
(declare (type word-offset words) (type bit-offset final-bits))
(let ((interior (floor (- length final-bits) unit-bits)))
(declare (type word-offset interior))
(cond
((<= dst-offset src-offset)
;; We need to loop from left to right
(unless (zerop dst-bit-offset)
;; We are only writing part of the first word, so mask off the
;; bits we want to preserve.
(let ((mask (end-mask (- dst-bit-offset)))
(orig (funcall dst-ref-fn dst dst-word-offset))
(value (funcall src-ref-fn src src-word-offset)))
(declare (type unit mask orig value))
(funcall dst-set-fn dst dst-word-offset
(32bit-logical-or (32bit-logical-and value mask)
(32bit-logical-andc2 orig mask))))
(incf src-word-offset)
(incf dst-word-offset))
;; Just copy the interior words.
(dotimes (i interior)
(funcall dst-set-fn dst dst-word-offset
(funcall src-ref-fn src src-word-offset))
(incf src-word-offset)
(incf dst-word-offset))
(unless (zerop final-bits)
;; We are only writing part of the last word.
(let ((mask (start-mask final-bits))
(orig (funcall dst-ref-fn dst dst-word-offset))
(value (funcall src-ref-fn src src-word-offset)))
(declare (type unit mask orig value))
(funcall dst-set-fn dst dst-word-offset
(32bit-logical-or
(32bit-logical-and value mask)
(32bit-logical-andc2 orig mask))))))
(t
;; We need to loop from right to left.
(incf dst-word-offset words)
(incf src-word-offset words)
(unless (zerop final-bits)
(let ((mask (start-mask final-bits))
(orig (funcall dst-ref-fn dst dst-word-offset))
(value (funcall src-ref-fn src src-word-offset)))
(declare (type unit mask orig value))
(funcall dst-set-fn dst dst-word-offset
(32bit-logical-or
(32bit-logical-and value mask)
(32bit-logical-andc2 orig mask)))))
(dotimes (i interior)
(decf src-word-offset)
(decf dst-word-offset)
(funcall dst-set-fn dst dst-word-offset
(funcall src-ref-fn src src-word-offset)))
(unless (zerop dst-bit-offset)
(decf src-word-offset)
(decf dst-word-offset)
(let ((mask (end-mask (- dst-bit-offset)))
(orig (funcall dst-ref-fn dst dst-word-offset))
(value (funcall src-ref-fn src src-word-offset)))
(declare (type unit mask orig value))
(funcall dst-set-fn dst dst-word-offset
(32bit-logical-or
(32bit-logical-and value mask)
(32bit-logical-andc2 orig mask))))))))))
(t
;; They aren't aligned.
(multiple-value-bind (words final-bits)
(floor (+ dst-bit-offset length) unit-bits)
(declare (type word-offset words) (type bit-offset final-bits))
(let ((src-shift (mod (- src-bit-offset dst-bit-offset) unit-bits))
(interior (floor (- length final-bits) unit-bits)))
(declare (type bit-offset src-shift)
(type word-offset interior))
(cond
((<= dst-offset src-offset)
;; We need to loop from left to right
(let ((prev 0)
(next (funcall src-ref-fn src src-word-offset)))
(declare (type unit prev next))
(flet ((get-next-src ()
(setf prev next)
(setf next (funcall src-ref-fn src
(incf src-word-offset)))))
(declare (inline get-next-src))
(unless (zerop dst-bit-offset)
(when (> src-bit-offset dst-bit-offset)
(get-next-src))
(let ((mask (end-mask (- dst-bit-offset)))
(orig (funcall dst-ref-fn dst dst-word-offset))
(value (32bit-logical-or
(shift-towards-start prev src-shift)
(shift-towards-end next (- src-shift)))))
(declare (type unit mask orig value))
(funcall dst-set-fn dst dst-word-offset
(32bit-logical-or
(32bit-logical-and value mask)
(32bit-logical-andc2 orig mask)))
(incf dst-word-offset)))
(dotimes (i interior)
(get-next-src)
(let ((value (32bit-logical-or
(shift-towards-end next (- src-shift))
(shift-towards-start prev src-shift))))
(declare (type unit value))
(funcall dst-set-fn dst dst-word-offset value)
(incf dst-word-offset)))
(unless (zerop final-bits)
(let ((value
(if (> (+ final-bits src-shift) unit-bits)
(progn
(get-next-src)
(32bit-logical-or
(shift-towards-end next (- src-shift))
(shift-towards-start prev src-shift)))
(shift-towards-start next src-shift)))
(mask (start-mask final-bits))
(orig (funcall dst-ref-fn dst dst-word-offset)))
(declare (type unit mask orig value))
(funcall dst-set-fn dst dst-word-offset
(32bit-logical-or
(32bit-logical-and value mask)
(32bit-logical-andc2 orig mask))))))))
(t
;; We need to loop from right to left.
(incf dst-word-offset words)
(incf src-word-offset
(1- (ceiling (+ src-bit-offset length) unit-bits)))
(let ((next 0)
(prev (funcall src-ref-fn src src-word-offset)))
(declare (type unit prev next))
(flet ((get-next-src ()
(setf next prev)
(setf prev (funcall src-ref-fn src
(decf src-word-offset)))))
(declare (inline get-next-src))
(unless (zerop final-bits)
(when (> final-bits (- unit-bits src-shift))
(get-next-src))
(let ((value (32bit-logical-or
(shift-towards-end next (- src-shift))
(shift-towards-start prev src-shift)))
(mask (start-mask final-bits))
(orig (funcall dst-ref-fn dst dst-word-offset)))
(declare (type unit mask orig value))
(funcall dst-set-fn dst dst-word-offset
(32bit-logical-or
(32bit-logical-and value mask)
(32bit-logical-andc2 orig mask)))))
(decf dst-word-offset)
(dotimes (i interior)
(get-next-src)
(let ((value (32bit-logical-or
(shift-towards-end next (- src-shift))
(shift-towards-start prev src-shift))))
(declare (type unit value))
(funcall dst-set-fn dst dst-word-offset value)
(decf dst-word-offset)))
(unless (zerop dst-bit-offset)
(if (> src-bit-offset dst-bit-offset)
(get-next-src)
(setf next prev prev 0))
(let ((mask (end-mask (- dst-bit-offset)))
(orig (funcall dst-ref-fn dst dst-word-offset))
(value (32bit-logical-or
(shift-towards-start prev src-shift)
(shift-towards-end next (- src-shift)))))
(declare (type unit mask orig value))
(funcall dst-set-fn dst dst-word-offset
(32bit-logical-or
(32bit-logical-and value mask)
(32bit-logical-andc2 orig mask)))))))))))))))
(values))
;;;; the actual bashers
(defun bit-bash-fill (value dst dst-offset length)
(declare (type unit value) (type offset dst-offset length))
(locally
(declare (optimize (speed 3) (safety 0)))
(do-constant-bit-bash dst dst-offset length value
#'%raw-bits #'%set-raw-bits)))
(defun system-area-fill (value dst dst-offset length)
(declare (type unit value) (type offset dst-offset length))
(locally
(declare (optimize (speed 3) (safety 0)))
(multiple-value-bind (dst dst-offset) (fix-sap-and-offset dst dst-offset)
(do-constant-bit-bash dst dst-offset length value
#'word-sap-ref #'%set-word-sap-ref))))
(defun bit-bash-copy (src src-offset dst dst-offset length)
(declare (type offset src-offset dst-offset length))
(locally
(declare (optimize (speed 3) (safety 0))
(inline do-unary-bit-bash))
(do-unary-bit-bash src src-offset dst dst-offset length
#'%raw-bits #'%set-raw-bits #'%raw-bits)))
(defun system-area-copy (src src-offset dst dst-offset length)
(declare (type offset src-offset dst-offset length))
(locally
(declare (optimize (speed 3) (safety 0)))
(multiple-value-bind (src src-offset) (fix-sap-and-offset src src-offset)
(declare (type system-area-pointer src))
(multiple-value-bind (dst dst-offset) (fix-sap-and-offset dst dst-offset)
(declare (type system-area-pointer dst))
(do-unary-bit-bash src src-offset dst dst-offset length
#'word-sap-ref #'%set-word-sap-ref
#'word-sap-ref)))))
(defun copy-to-system-area (src src-offset dst dst-offset length)
(declare (type offset src-offset dst-offset length))
(locally
(declare (optimize (speed 3) (safety 0)))
(multiple-value-bind (dst dst-offset) (fix-sap-and-offset dst dst-offset)
(do-unary-bit-bash src src-offset dst dst-offset length
#'word-sap-ref #'%set-word-sap-ref #'%raw-bits))))
(defun copy-from-system-area (src src-offset dst dst-offset length)
(declare (type offset src-offset dst-offset length))
(locally
(declare (optimize (speed 3) (safety 0)))
(multiple-value-bind (src src-offset) (fix-sap-and-offset src src-offset)
(do-unary-bit-bash src src-offset dst dst-offset length
#'%raw-bits #'%set-raw-bits #'word-sap-ref))))
;;; a common idiom for calling COPY-TO-SYSTEM-AREA
;;;
;;; Copy the entire contents of the vector V to memory starting at SAP+OFFSET.
(defun copy-byte-vector-to-system-area (bv sap &optional (offset 0))
;; FIXME: There should be a type like SB!VM:BYTE so that we can write this
;; type as (SIMPLE-ARRAY SB!VM:BYTE 1). Except BYTE is an external symbol of
;; package CL; so maybe SB!VM:VM-BYTE?
(declare (type (simple-array (unsigned-byte 8) 1) bv))
(declare (type sap sap))
(declare (type fixnum offset))
;; FIXME: Actually it looks as though this, and most other calls
;; to COPY-TO-SYSTEM-AREA, could be written more concisely with BYTE-BLT.
;; Except that the DST-END-DST-START convention for the length is confusing.
;; Perhaps I could rename BYTE-BLT to BYTE-BLIT and replace the
;; DST-END argument with an N-BYTES argument?
(copy-to-system-area bv
(* sb!vm:vector-data-offset sb!vm:word-bits)
sap
offset
(* (length bv) sb!vm:byte-bits)))

205
src/code/boot-extensions.lisp Normal file
View file

@ -0,0 +1,205 @@
;;;; extensions which are needed in order to (cross-)compile target-only code
;;;; This software is part of the SBCL system. See the README file for
;;;; more information.
;;;;
;;;; This software is derived from the CMU CL system, which was
;;;; written at Carnegie Mellon University and released into the
;;;; public domain. The software is in the public domain and is
;;;; provided with absolutely no warranty. See the COPYING and CREDITS
;;;; files for more information.
(in-package "SB!EXT")
(file-comment
"$Header$")
;;; Lots of code wants to get to the KEYWORD package or the COMMON-LISP package
;;; without a lot of fuss, so we cache them in variables. TO DO: How much
;;; does this actually buy us? It sounds sensible, but I don't know for sure
;;; that it saves space or time.. -- WHN 19990521
(declaim (type package *cl-package* *keyword-package*))
(defvar *cl-package* (find-package "COMMON-LISP"))
(defvar *keyword-package* (find-package "KEYWORD"))
;;;; the COLLECT macro
;;; helper functions for COLLECT, which become the expanders of the MACROLET
;;; definitions created by COLLECT
;;;
;;; COLLECT-NORMAL-EXPANDER handles normal collection macros.
;;;
;;; COLLECT-LIST-EXPANDER handles the list collection case. N-TAIL
;;; is the pointer to the current tail of the list, or NIL if the list
;;; is empty.
(defun collect-normal-expander (n-value fun forms)
`(progn
,@(mapcar #'(lambda (form) `(setq ,n-value (,fun ,form ,n-value))) forms)
,n-value))
(defun collect-list-expander (n-value n-tail forms)
(let ((n-res (gensym)))
`(progn
,@(mapcar #'(lambda (form)
`(let ((,n-res (cons ,form nil)))
(cond (,n-tail
(setf (cdr ,n-tail) ,n-res)
(setq ,n-tail ,n-res))
(t
(setq ,n-tail ,n-res ,n-value ,n-res)))))
forms)
,n-value)))
;;; the ultimate collection macro...
(defmacro collect (collections &body body)
#!+sb-doc
"Collect ({(Name [Initial-Value] [Function])}*) {Form}*
Collect some values somehow. Each of the collections specifies a bunch of
things which collected during the evaluation of the body of the form. The
name of the collection is used to define a local macro, a la MACROLET.
Within the body, this macro will evaluate each of its arguments and collect
the result, returning the current value after the collection is done. The
body is evaluated as a PROGN; to get the final values when you are done, just
call the collection macro with no arguments.
INITIAL-VALUE is the value that the collection starts out with, which
defaults to NIL. FUNCTION is the function which does the collection. It is
a function which will accept two arguments: the value to be collected and the
current collection. The result of the function is made the new value for the
collection. As a totally magical special-case, FUNCTION may be COLLECT,
which tells us to build a list in forward order; this is the default. If an
INITIAL-VALUE is supplied for Collect, the stuff will be RPLACD'd onto the
end. Note that FUNCTION may be anything that can appear in the functional
position, including macros and lambdas."
(let ((macros ())
(binds ()))
(dolist (spec collections)
(unless (proper-list-of-length-p spec 1 3)
(error "Malformed collection specifier: ~S." spec))
(let* ((name (first spec))
(default (second spec))
(kind (or (third spec) 'collect))
(n-value (gensym (concatenate 'string
(symbol-name name)
"-N-VALUE-"))))
(push `(,n-value ,default) binds)
(if (eq kind 'collect)
(let ((n-tail (gensym (concatenate 'string
(symbol-name name)
"-N-TAIL-"))))
(if default
(push `(,n-tail (last ,n-value)) binds)
(push n-tail binds))
(push `(,name (&rest args)
(collect-list-expander ',n-value ',n-tail args))
macros))
(push `(,name (&rest args)
(collect-normal-expander ',n-value ',kind args))
macros))))
`(macrolet ,macros (let* ,(nreverse binds) ,@body))))
(declaim (ftype (function () nil) required-argument))
(defun required-argument ()
#!+sb-doc
"This function can be used as the default value for keyword arguments that
must be always be supplied. Since it is known by the compiler to never
return, it will avoid any compile-time type warnings that would result from a
default value inconsistent with the declared type. When this function is
called, it signals an error indicating that a required keyword argument was
not supplied. This function is also useful for DEFSTRUCT slot defaults
corresponding to required arguments."
(/show0 "entering REQUIRED-ARGUMENT")
(error "A required keyword argument was not supplied."))
;;; "the ultimate iteration macro"
;;;
;;; note for Schemers: This seems to be identical to Scheme's "named LET".
(defmacro iterate (name binds &body body)
#!+sb-doc
"Iterate Name ({(Var Initial-Value)}*) Declaration* Form*
This is syntactic sugar for Labels. It creates a local function Name with
the specified Vars as its arguments and the Declarations and Forms as its
body. This function is then called with the Initial-Values, and the result
of the call is returned from the macro."
(dolist (x binds)
(unless (proper-list-of-length-p x 2)
(error "Malformed ITERATE variable spec: ~S." x)))
`(labels ((,name ,(mapcar #'first binds) ,@body))
(,name ,@(mapcar #'second binds))))
;;; Once-Only is a utility useful in writing source transforms and macros.
;;; It provides an easy way to wrap a LET around some code to ensure that some
;;; forms are only evaluated once.
(defmacro once-only (specs &body body)
#!+sb-doc
"Once-Only ({(Var Value-Expression)}*) Form*
Create a Let* which evaluates each Value-Expression, binding a temporary
variable to the result, and wrapping the Let* around the result of the
evaluation of Body. Within the body, each Var is bound to the corresponding
temporary variable."
(iterate frob
((specs specs)
(body body))
(if (null specs)
`(progn ,@body)
(let ((spec (first specs)))
;; FIXME: should just be DESTRUCTURING-BIND of SPEC
(unless (proper-list-of-length-p spec 2)
(error "malformed ONCE-ONLY binding spec: ~S" spec))
(let* ((name (first spec))
(exp-temp (gensym (symbol-name name))))
`(let ((,exp-temp ,(second spec))
(,name (gensym "OO-")))
`(let ((,,name ,,exp-temp))
,,(frob (rest specs) body))))))))
;;;; some old-fashioned functions. (They're not just for old-fashioned
;;;; code, they're also used as optimized forms of the corresponding
;;;; general functions when the compiler can prove that they're
;;;; equivalent.)
;;; like (MEMBER ITEM LIST :TEST #'EQ)
(defun memq (item list)
#!+sb-doc
"Returns tail of LIST beginning with first element EQ to ITEM."
;; KLUDGE: These could be and probably should be defined as
;; (MEMBER ITEM LIST :TEST #'EQ)),
;; but when I try to cross-compile that, I get an error from
;; LTN-ANALYZE-KNOWN-CALL, "Recursive known function definition". The
;; comments for that error say it "is probably a botched interpreter stub".
;; Rather than try to figure that out, I just rewrote this function from
;; scratch. -- WHN 19990512
(do ((i list (cdr i)))
((null i))
(when (eq (car i) item)
(return i))))
;;; like (ASSOC ITEM ALIST :TEST #'EQ)
(defun assq (item alist)
#!+sb-doc
"Return the first pair of ALIST where ITEM is EQ to the key of the pair."
;; KLUDGE: CMU CL defined this with
;; (DECLARE (INLINE ASSOC))
;; (ASSOC ITEM ALIST :TEST #'EQ))
;; which is pretty, but which would have required adding awkward
;; build order constraints on SBCL (or figuring out some way to make
;; inline definitions installable at build-the-cross-compiler time,
;; which was too ambitious for now). Rather than mess with that,
;; we just define ASSQ explicitly in terms of more primitive operations:
(dolist (pair alist)
(when (eq (car pair) item)
(return pair))))
(defun delq (item list)
#!+sb-doc
"Delete all LIST entries EQ to ITEM (destructively modifying LIST), and
return the modified LIST."
(let ((list list))
(do ((x list (cdr x))
(splice '()))
((endp x) list)
(cond ((eq item (car x))
(if (null splice)
(setq list (cdr x))
(rplacd splice (cdr x))))
(t (setq splice x)))))) ; Move splice along to include element.

60
src/code/bsd-os.lisp Normal file
View file

@ -0,0 +1,60 @@
;;;; OS interface functions for CMU CL under BSD Unix.
;;;; This code was written as part of the CMU Common Lisp project at
;;;; Carnegie Mellon University, and has been placed in the public
;;;; domain.
(sb!int:file-comment
"$Header$")
(in-package "SB!SYS")
;;;; Check that target machine features are set up consistently with
;;;; this file.
#!-bsd (eval-when (:compile-toplevel :load-toplevel :execute)
(error "The :BSD feature is missing, we shouldn't be doing this code."))
(defun software-type ()
#!+sb-doc
"Return a string describing the supporting software."
(the string ; (to force error in case of unsupported BSD variant)
#!+FreeBSD "FreeBSD"
#!+OpenBSD "OpenBSD"))
(defun software-version ()
#!+sb-doc
"Return a string describing version of the supporting software, or NIL
if not available."
#+nil ; won't work until we support RUN-PROGRAM..
(unless *software-version*
(setf *software-version*
(string-trim '(#\newline)
(with-output-to-string (stream)
(run-program "/usr/bin/uname"
'("-r")
:output stream)))))
nil)
;;; OS-COLD-INIT-OR-REINIT initializes our operating-system interface.
;;; It sets the values of the global port variables to what they
;;; should be and calls the functions that set up the argument blocks
;;; for the server interfaces.
(defun os-cold-init-or-reinit ()
(setf *software-version* nil))
;;; Return system time, user time and number of page faults.
(defun get-system-info ()
(multiple-value-bind (err? utime stime maxrss ixrss idrss
isrss minflt majflt)
(sb!unix:unix-getrusage sb!unix:rusage_self)
(declare (ignore maxrss ixrss idrss isrss minflt))
(unless err?
(error "Unix system call getrusage failed: ~A."
(sb!unix:get-unix-error-msg utime)))
(values utime stime majflt)))
;;; Return the system page size.
(defun get-page-size ()
;; FIXME: probably should call getpagesize()
4096)

1339
src/code/byte-interp.lisp Normal file
View file

File diff suppressed because it is too large Load diff

110
src/code/byte-types.lisp Normal file
View file

@ -0,0 +1,110 @@
;;;; types which are needed to implement byte-compiled functions
;;;; This software is part of the SBCL system. See the README file for
;;;; more information.
;;;;
;;;; This software is derived from the CMU CL system, which was
;;;; written at Carnegie Mellon University and released into the
;;;; public domain. The software is in the public domain and is
;;;; provided with absolutely no warranty. See the COPYING and CREDITS
;;;; files for more information.
(in-package "SB!C")
(file-comment
"$Header$")
;;;; types
(deftype stack-pointer ()
`(integer 0 ,(1- most-positive-fixnum)))
;;; KLUDGE: bare numbers, no documentation, ick.. -- WHN 19990701
(eval-when (:compile-toplevel :load-toplevel :execute)
(defconstant max-pc (1- (ash 1 24))))
(deftype pc ()
`(integer 0 ,max-pc))
(deftype return-pc ()
`(integer ,(- max-pc) ,max-pc))
;;;; byte functions
;;; This abstract class represents any type of byte-compiled function.
(defstruct (byte-function-or-closure
(:alternate-metaclass funcallable-instance
funcallable-structure-class
make-funcallable-structure-class)
(:type funcallable-structure)
(:constructor nil)
(:copier nil)))
;;; a byte-compiled closure
(defstruct (byte-closure
(:include byte-function-or-closure)
(:constructor make-byte-closure (function data))
(:type funcallable-structure)
(:print-object
(lambda (x stream)
(print-unreadable-object (x stream :type t :identity t)
(prin1 (byte-function-name (byte-closure-function x))
stream)))))
;; the byte function that we call
(function (required-argument) :type byte-function)
;; the closure data vector
(data (required-argument) :type simple-vector))
;;; any non-closure byte function (including the hidden function
;;; object for a closure)
(defstruct (byte-function (:include byte-function-or-closure)
(:type funcallable-structure)
(:constructor nil))
;; The component that this XEP is an entry point into. NIL until
;; LOAD or MAKE-CORE-BYTE-COMPONENT fills it in. They count on this
;; being the first slot.
(component nil :type (or null code-component))
;; Debug name of this function.
(name nil))
(def!method print-object ((x byte-function) stream)
;; FIXME: I think functions should probably print either as
;; #<FUNCTION ..> or as #<COMPILED-FUNCTION ..>, since those are
;; their user-visible types. (And this should be true for
;; BYTE-CLOSURE objects too.)
(print-unreadable-object (x stream :identity t)
(format stream "byte function ~S" (byte-function-name x))))
;;; fixed-argument byte function
(defstruct (simple-byte-function (:include byte-function)
(:type funcallable-structure))
;; The number of arguments expected.
(num-args 0 :type (integer 0 #.call-arguments-limit))
;; The start of the function.
(entry-point 0 :type index))
;;; variable-arg-count byte function
(defstruct (hairy-byte-function (:include byte-function)
(:type funcallable-structure))
;; The minimum and maximum number of args, ignoring &REST and &KEY.
(min-args 0 :type (integer 0 #.call-arguments-limit))
(max-args 0 :type (integer 0 #.call-arguments-limit))
;; List of the entry points for min-args, min-args+1, ... max-args.
(entry-points nil :type list)
;; The entry point to use when there are more than max-args. Only
;; filled in where okay. In other words, only when &REST or &KEY is
;; specified.
(more-args-entry-point nil :type (or null (unsigned-byte 24)))
;; The number of ``more-arg'' args.
(num-more-args 0 :type (integer 0 #.call-arguments-limit))
;; True if there is a rest-arg.
(rest-arg-p nil :type (member t nil))
;; True if there are keywords. Note: keywords might still be NIL
;; because having &KEY with no keywords is valid and should result
;; in allow-other-keys processing. If :allow-others, then allow
;; other keys.
(keywords-p nil :type (member t nil :allow-others))
;; List of keyword arguments. Each element is a list of:
;; key, default, supplied-p.
(keywords nil :type list))
#!-sb-fluid (declaim (freeze-type byte-function-or-closure))

394
src/code/char.lisp Normal file
View file

@ -0,0 +1,394 @@
;;;; character functions
;;;;
;;;; This file assumes the use of ASCII codes and the specific
;;;; character formats used in SBCL (and its ancestor, CMU CL). It is
;;;; optimized for performance rather than for portability and
;;;; elegance, and may have to be rewritten if the character
;;;; representation is changed.
;;;;
;;;; FIXME: should perhaps be renamed ascii.lisp since it's an
;;;; unportable ASCII-dependent implementation
;;;; This software is part of the SBCL system. See the README file for
;;;; more information.
;;;;
;;;; This software is derived from the CMU CL system, which was
;;;; written at Carnegie Mellon University and released into the
;;;; public domain. The software is in the public domain and is
;;;; provided with absolutely no warranty. See the COPYING and CREDITS
;;;; files for more information.
(in-package "SB!IMPL")
(file-comment
"$Header$")
;;; We compile some trivial character operations via inline expansion.
#!-sb-fluid
(declaim (inline standard-char-p graphic-char-p alpha-char-p
upper-case-p lower-case-p both-case-p alphanumericp
char-int))
(declaim (maybe-inline digit-char-p digit-weight))
(defconstant char-code-limit 256
#!+sb-doc
"the upper exclusive bound on values produced by CHAR-CODE")
(deftype char-code ()
`(integer 0 (,char-code-limit)))
(macrolet ((frob (char-names-list)
(collect ((results))
(dolist (code char-names-list)
(destructuring-bind (ccode names) code
(dolist (name names)
(results (cons name (code-char ccode))))))
`(defparameter *char-name-alist* ',(results)
#!+sb-doc
"This is the alist of (character-name . character) for characters with
long names. The first name in this list for a given character is used
on typeout and is the preferred form for input."))))
(frob ((#x00 ("Null" "^@" "Nul"))
(#x01 ("^a" "Soh"))
(#x02 ("^b" "Stx"))
(#x03 ("^c" "Etx"))
(#x04 ("^d" "Eot"))
(#x05 ("^e" "Enq"))
(#x06 ("^f" "Ack"))
(#x07 ("Bell" "^g" "Bel"))
(#x08 ("Backspace" "^h" "Bs"))
(#x09 ("Tab" "^i" "Ht"))
(#x0A ("Newline" "Linefeed" "^j" "Lf" "Nl" ))
(#x0B ("Vt" "^k"))
(#x0C ("Page" "^l" "Form" "Formfeed" "Ff" "Np"))
(#x0D ("Return" "^m" "Cr"))
(#x0E ("^n" "So"))
(#x0F ("^o" "Si"))
(#x10 ("^p" "Dle"))
(#x11 ("^q" "Dc1"))
(#x12 ("^r" "Dc2"))
(#x13 ("^s" "Dc3"))
(#x14 ("^t" "Dc4"))
(#x15 ("^u" "Nak"))
(#x16 ("^v" "Syn"))
(#x17 ("^w" "Etb"))
(#x18 ("^x" "Can"))
(#x19 ("^y" "Em"))
(#x1A ("^z" "Sub"))
(#x1B ("Escape" "^[" "Altmode" "Esc" "Alt"))
(#x1C ("^\\" "Fs"))
(#x1D ("^]" "Gs"))
(#x1E ("^^" "Rs"))
(#x1F ("^_" "Us"))
(#x20 ("Space" "Sp"))
(#x7f ("Rubout" "Delete" "Del")))))
;;;; accessor functions
(defun char-code (char)
#!+sb-doc
"Returns the integer code of CHAR."
(etypecase char
(base-char (char-code (truly-the base-char char)))))
(defun char-int (char)
#!+sb-doc
"Returns the integer code of CHAR. This is the same as char-code, as
CMU Common Lisp does not implement character bits or fonts."
(char-code char))
(defun code-char (code)
#!+sb-doc
"Returns the character with the code CODE."
(declare (type char-code code))
(code-char code))
(defun character (object)
#!+sb-doc
"Coerces its argument into a character object if possible. Accepts
characters, strings and symbols of length 1."
(flet ((do-error (control args)
(error 'simple-type-error
:datum object
;;?? how to express "symbol with name of length 1"?
:expected-type '(or character (string 1))
:format-control control
:format-arguments args)))
(typecase object
(character object)
(string (if (= 1 (length (the string object)))
(char object 0)
(do-error
"String is not of length one: ~S" (list object))))
(symbol (if (= 1 (length (symbol-name object)))
(schar (symbol-name object) 0)
(do-error
"Symbol name is not of length one: ~S" (list object))))
(t (do-error "~S cannot be coerced to a character." (list object))))))
(defun char-name (char)
#!+sb-doc
"Given a character object, char-name returns the name for that
object (a symbol)."
(car (rassoc char *char-name-alist*)))
(defun name-char (name)
#!+sb-doc
"Given an argument acceptable to string, name-char returns a character
object whose name is that symbol, if one exists. Otherwise, () is returned."
(cdr (assoc (string name) *char-name-alist* :test #'string-equal)))
;;;; predicates
(defun standard-char-p (char)
#!+sb-doc
"The argument must be a character object. Standard-char-p returns T if the
argument is a standard character -- one of the 95 ASCII printing characters
or <return>."
(declare (character char))
(and (typep char 'base-char)
(let ((n (char-code (the base-char char))))
(or (< 31 n 127)
(= n 10)))))
(defun %standard-char-p (thing)
#!+sb-doc
"Return T if and only if THING is a standard-char. Differs from
standard-char-p in that THING doesn't have to be a character."
(and (characterp thing) (standard-char-p thing)))
(defun graphic-char-p (char)
#!+sb-doc
"The argument must be a character object. Graphic-char-p returns T if the
argument is a printing character (space through ~ in ASCII), otherwise
returns ()."
(declare (character char))
(and (typep char 'base-char)
(< 31
(char-code (the base-char char))
127)))
(defun alpha-char-p (char)
#!+sb-doc
"The argument must be a character object. Alpha-char-p returns T if the
argument is an alphabetic character, A-Z or a-z; otherwise ()."
(declare (character char))
(let ((m (char-code char)))
(or (< 64 m 91) (< 96 m 123))))
(defun upper-case-p (char)
#!+sb-doc
"The argument must be a character object; upper-case-p returns T if the
argument is an upper-case character, () otherwise."
(declare (character char))
(< 64
(char-code char)
91))
(defun lower-case-p (char)
#!+sb-doc
"The argument must be a character object; lower-case-p returns T if the
argument is a lower-case character, () otherwise."
(declare (character char))
(< 96
(char-code char)
123))
(defun both-case-p (char)
#!+sb-doc
"The argument must be a character object. Both-case-p returns T if the
argument is an alphabetic character and if the character exists in
both upper and lower case. For ASCII, this is the same as Alpha-char-p."
(declare (character char))
(let ((m (char-code char)))
(or (< 64 m 91) (< 96 m 123))))
(defun digit-char-p (char &optional (radix 10.))
#!+sb-doc
"If char is a digit in the specified radix, returns the fixnum for
which that digit stands, else returns NIL. Radix defaults to 10
(decimal)."
(declare (character char) (type (integer 2 36) radix))
(let ((m (- (char-code char) 48)))
(declare (fixnum m))
(cond ((<= radix 10.)
;; Special-case decimal and smaller radices.
(if (and (>= m 0) (< m radix)) m nil))
;; Digits 0 - 9 are used as is, since radix is larger.
((and (>= m 0) (< m 10)) m)
;; Check for upper case A - Z.
((and (>= (setq m (- m 7)) 10) (< m radix)) m)
;; Also check lower case a - z.
((and (>= (setq m (- m 32)) 10) (< m radix)) m)
;; Else, fail.
(t nil))))
(defun alphanumericp (char)
#!+sb-doc
"Given a character-object argument, alphanumericp returns T if the
argument is either numeric or alphabetic."
(declare (character char))
(let ((m (char-code char)))
(or (< 47 m 58) (< 64 m 91) (< 96 m 123))))
(defun char= (character &rest more-characters)
#!+sb-doc
"Returns T if all of its arguments are the same character."
(do ((clist more-characters (cdr clist)))
((atom clist) T)
(unless (eq (car clist) character) (return nil))))
(defun char/= (character &rest more-characters)
#!+sb-doc
"Returns T if no two of its arguments are the same character."
(do* ((head character (car list))
(list more-characters (cdr list)))
((atom list) T)
(unless (do* ((l list (cdr l))) ;inner loop returns T
((atom l) T) ; iff head /= rest.
(if (eq head (car l)) (return nil)))
(return nil))))
(defun char< (character &rest more-characters)
#!+sb-doc
"Returns T if its arguments are in strictly increasing alphabetic order."
(do* ((c character (car list))
(list more-characters (cdr list)))
((atom list) T)
(unless (< (char-int c)
(char-int (car list)))
(return nil))))
(defun char> (character &rest more-characters)
#!+sb-doc
"Returns T if its arguments are in strictly decreasing alphabetic order."
(do* ((c character (car list))
(list more-characters (cdr list)))
((atom list) T)
(unless (> (char-int c)
(char-int (car list)))
(return nil))))
(defun char<= (character &rest more-characters)
#!+sb-doc
"Returns T if its arguments are in strictly non-decreasing alphabetic order."
(do* ((c character (car list))
(list more-characters (cdr list)))
((atom list) T)
(unless (<= (char-int c)
(char-int (car list)))
(return nil))))
(defun char>= (character &rest more-characters)
#!+sb-doc
"Returns T if its arguments are in strictly non-increasing alphabetic order."
(do* ((c character (car list))
(list more-characters (cdr list)))
((atom list) T)
(unless (>= (char-int c)
(char-int (car list)))
(return nil))))
;;; Equal-Char-Code is used by the following functions as a version of char-int
;;; which loses font, bits, and case info.
(defmacro equal-char-code (character)
`(let ((ch (char-code ,character)))
(if (< 96 ch 123) (- ch 32) ch)))
(defun char-equal (character &rest more-characters)
#!+sb-doc
"Returns T if all of its arguments are the same character.
Font, bits, and case are ignored."
(do ((clist more-characters (cdr clist)))
((atom clist) T)
(unless (= (equal-char-code (car clist))
(equal-char-code character))
(return nil))))
(defun char-not-equal (character &rest more-characters)
#!+sb-doc
"Returns T if no two of its arguments are the same character.
Font, bits, and case are ignored."
(do* ((head character (car list))
(list more-characters (cdr list)))
((atom list) T)
(unless (do* ((l list (cdr l)))
((atom l) T)
(if (= (equal-char-code head)
(equal-char-code (car l)))
(return nil)))
(return nil))))
(defun char-lessp (character &rest more-characters)
#!+sb-doc
"Returns T if its arguments are in strictly increasing alphabetic order.
Font, bits, and case are ignored."
(do* ((c character (car list))
(list more-characters (cdr list)))
((atom list) T)
(unless (< (equal-char-code c)
(equal-char-code (car list)))
(return nil))))
(defun char-greaterp (character &rest more-characters)
#!+sb-doc
"Returns T if its arguments are in strictly decreasing alphabetic order.
Font, bits, and case are ignored."
(do* ((c character (car list))
(list more-characters (cdr list)))
((atom list) T)
(unless (> (equal-char-code c)
(equal-char-code (car list)))
(return nil))))
(defun char-not-greaterp (character &rest more-characters)
#!+sb-doc
"Returns T if its arguments are in strictly non-decreasing alphabetic order.
Font, bits, and case are ignored."
(do* ((c character (car list))
(list more-characters (cdr list)))
((atom list) T)
(unless (<= (equal-char-code c)
(equal-char-code (car list)))
(return nil))))
(defun char-not-lessp (character &rest more-characters)
#!+sb-doc
"Returns T if its arguments are in strictly non-increasing alphabetic order.
Font, bits, and case are ignored."
(do* ((c character (car list))
(list more-characters (cdr list)))
((atom list) T)
(unless (>= (equal-char-code c)
(equal-char-code (car list)))
(return nil))))
;;;; miscellaneous functions
(defun char-upcase (char)
#!+sb-doc
"Returns CHAR converted to upper-case if that is possible."
(declare (character char))
(if (lower-case-p char)
(code-char (- (char-code char) 32))
char))
(defun char-downcase (char)
#!+sb-doc
"Returns CHAR converted to lower-case if that is possible."
(declare (character char))
(if (upper-case-p char)
(code-char (+ (char-code char) 32))
char))
(defun digit-char (weight &optional (radix 10))
#!+sb-doc
"All arguments must be integers. Returns a character object that
represents a digit of the given weight in the specified radix. Returns
NIL if no such character exists. The character will have the specified
font attributes."
(declare (type (integer 2 36) radix) (type unsigned-byte weight))
(and (typep weight 'fixnum)
(>= weight 0) (< weight radix) (< weight 36)
(code-char (if (< weight 10) (+ 48 weight) (+ 55 weight)))))

71
src/code/cl-specials.lisp Normal file
View file

@ -0,0 +1,71 @@
;;;; We proclaim all the special variables in the COMMON-LISP package
;;;; here, in one go, just to try to make sure we don't miss any.
;;;; This software is part of the SBCL system. See the README file for
;;;; more information.
;;;;
;;;; This software is derived from the CMU CL system, which was
;;;; written at Carnegie Mellon University and released into the
;;;; public domain. The software is in the public domain and is
;;;; provided with absolutely no warranty. See the COPYING and CREDITS
;;;; files for more information.
(in-package "COMMON-LISP")
(sb!int:file-comment
"$Header$")
(sb!xc:proclaim '(special cl:*
cl:**
cl:***
cl:*break-on-signals*
cl:*compile-file-pathname*
cl:*compile-file-truename*
cl:*compile-print*
cl:*compile-verbose*
cl:*debug-io*
cl:*debugger-hook*
cl:*default-pathname-defaults*
cl:*error-output*
cl:*features*
cl:*gensym-counter*
cl:*load-pathname*
cl:*load-print*
cl:*load-truename*
cl:*load-verbose*
cl:*macroexpand-hook*
cl:*modules*
cl:*package*
cl:*print-array*
cl:*print-base*
cl:*print-case*
cl:*print-circle*
cl:*print-escape*
cl:*print-gensym*
cl:*print-length*
cl:*print-level*
cl:*print-lines*
cl:*print-miser-width*
cl:*print-pprint-dispatch*
cl:*print-pretty*
cl:*print-radix*
cl:*print-readably*
cl:*print-right-margin*
cl:*query-io*
cl:*random-state*
cl:*read-base*
cl:*read-default-float-format*
cl:*read-eval*
cl:*read-suppress*
cl:*readtable*
cl:*standard-input*
cl:*standard-output*
cl:*terminal-io*
cl:*trace-output*
cl:+
cl:++
cl:+++
cl:-
cl:/
cl://
cl:///))

1228
src/code/class.lisp Normal file
View file

File diff suppressed because it is too large Load diff

318
src/code/coerce.lisp Normal file
View file

@ -0,0 +1,318 @@
;;;; COERCE and related code
;;;; This software is part of the SBCL system. See the README file for
;;;; more information.
;;;;
;;;; This software is derived from the CMU CL system, which was
;;;; written at Carnegie Mellon University and released into the
;;;; public domain. The software is in the public domain and is
;;;; provided with absolutely no warranty. See the COPYING and CREDITS
;;;; files for more information.
(in-package "SB!IMPL")
(file-comment
"$Header$")
(macrolet ((def-frob (name result access src-type &optional typep)
`(defun ,name (object ,@(if typep '(type) ()))
(do* ((index 0 (1+ index))
(length (length (the ,(ecase src-type
(:list 'list)
(:vector 'vector))
object)))
(result ,result)
(in-object object))
((= index length) result)
(declare (fixnum length index))
(setf (,access result index)
,(ecase src-type
(:list '(pop in-object))
(:vector '(aref in-object index))))))))
(def-frob list-to-simple-string* (make-string length) schar :list)
(def-frob list-to-bit-vector* (make-array length :element-type '(mod 2))
sbit :list)
(def-frob list-to-vector* (make-sequence-of-type type length)
aref :list t)
(def-frob vector-to-vector* (make-sequence-of-type type length)
aref :vector t)
(def-frob vector-to-simple-string* (make-string length) schar :vector)
(def-frob vector-to-bit-vector* (make-array length :element-type '(mod 2))
sbit :vector))
(defun vector-to-list* (object)
(let ((result (list nil))
(length (length object)))
(declare (fixnum length))
(do ((index 0 (1+ index))
(splice result (cdr splice)))
((= index length) (cdr result))
(declare (fixnum index))
(rplacd splice (list (aref object index))))))
(defun string-to-simple-string* (object)
(if (simple-string-p object)
object
(with-array-data ((data object)
(start)
(end (length object)))
(declare (simple-string data))
(subseq data start end))))
(defun bit-vector-to-simple-bit-vector* (object)
(if (simple-bit-vector-p object)
object
(with-array-data ((data object)
(start)
(end (length object)))
(declare (simple-bit-vector data))
(subseq data start end))))
(defvar *offending-datum*); FIXME: Remove after debugging COERCE.
;;; These are used both by the full DEFUN function and by various
;;; optimization transforms in the constant-OUTPUT-TYPE-SPEC case.
;;;
;;; Most of them are INLINE so that they can be optimized when the
;;; argument type is known. It might be better to do this with
;;; DEFTRANSFORMs, though.
(declaim (inline coerce-to-list))
(declaim (inline coerce-to-simple-string coerce-to-bit-vector coerce-to-vector))
(defun coerce-to-function (object)
;; (Unlike the other COERCE-TO-FOOs, this one isn't inline, because
;; it's so big and because optimizing away the outer ETYPECASE
;; doesn't seem to buy us that much anyway.)
(etypecase object
(symbol
;; ANSI lets us return ordinary errors (non-TYPE-ERRORs) here.
(cond ((macro-function object)
(error "~S names a macro." object))
((special-operator-p object)
(error "~S is a special operator." object))
(t (fdefinition object))))
(list
(case (first object)
((setf)
(fdefinition object))
((lambda instance-lambda)
;; FIXME: If we go to a compiler-only implementation, this can
;; become COMPILE instead of EVAL, which seems nicer to me.
(eval `(function ,object)))
(t
(error 'simple-type-error
:datum object
:expected-type '(or symbol
;; KLUDGE: ANSI wants us to
;; return a TYPE-ERROR here, and
;; a TYPE-ERROR is supposed to
;; describe the expected type,
;; but it's not obvious how to
;; describe the coerceable cons
;; types, so we punt and just say
;; CONS. -- WHN 20000503
cons)
:format-control "~S can't be coerced to a function."
:format-arguments (list object)))))))
(defun coerce-to-list (object)
(etypecase object
(vector (vector-to-list* object))))
(defun coerce-to-simple-string (object)
(etypecase object
(list (list-to-simple-string* object))
(string (string-to-simple-string* object))
(vector (vector-to-simple-string* object))))
(defun coerce-to-bit-vector (object)
(etypecase object
(list (list-to-bit-vector* object))
(vector (vector-to-bit-vector* object))))
(defun coerce-to-vector (object output-type-spec)
(etypecase object
(list (list-to-vector* object output-type-spec))
(vector (vector-to-vector* object output-type-spec))))
;;; old working version
(defun coerce (object output-type-spec)
#!+sb-doc
"Coerces the Object to an object of type Output-Type-Spec."
(flet ((coerce-error ()
(/show0 "entering COERCE-ERROR")
(error 'simple-type-error
:format-control "~S can't be converted to type ~S."
:format-arguments (list object output-type-spec)))
(check-result (result)
#!+high-security
(check-type-var result output-type-spec)
result))
(let ((type (specifier-type output-type-spec)))
(cond
((%typep object output-type-spec)
object)
((eq type *empty-type*)
(coerce-error))
((csubtypep type (specifier-type 'character))
(character object))
((csubtypep type (specifier-type 'function))
#!+high-security
(when (and (or (symbolp object)
(and (listp object)
(= (length object) 2)
(eq (car object) 'setf)))
(not (fboundp object)))
(error 'simple-type-error
:datum object
:expected-type '(satisfies fboundp)
:format-control "~S isn't fbound."
:format-arguments (list object)))
#!+high-security
(when (and (symbolp object)
(sb!xc:macro-function object))
(error 'simple-type-error
:datum object
:expected-type '(not (satisfies sb!xc:macro-function))
:format-control "~S is a macro."
:format-arguments (list object)))
#!+high-security
(when (and (symbolp object)
(special-operator-p object))
(error 'simple-type-error
:datum object
:expected-type '(not (satisfies special-operator-p))
:format-control "~S is a special operator."
:format-arguments (list object)))
(eval `#',object))
((numberp object)
(let ((res
(cond
((csubtypep type (specifier-type 'single-float))
(%single-float object))
((csubtypep type (specifier-type 'double-float))
(%double-float object))
#!+long-float
((csubtypep type (specifier-type 'long-float))
(%long-float object))
((csubtypep type (specifier-type 'float))
(%single-float object))
((csubtypep type (specifier-type '(complex single-float)))
(complex (%single-float (realpart object))
(%single-float (imagpart object))))
((csubtypep type (specifier-type '(complex double-float)))
(complex (%double-float (realpart object))
(%double-float (imagpart object))))
#!+long-float
((csubtypep type (specifier-type '(complex long-float)))
(complex (%long-float (realpart object))
(%long-float (imagpart object))))
((csubtypep type (specifier-type 'complex))
(complex object))
(t
(coerce-error)))))
;; If RES has the wrong type, that means that rule of canonical
;; representation for complex rationals was invoked. According to
;; the Hyperspec, (coerce 7/2 'complex) returns 7/2. Thus, if the
;; object was a rational, there is no error here.
(unless (or (typep res output-type-spec) (rationalp object))
(coerce-error))
res))
((csubtypep type (specifier-type 'list))
(if (vectorp object)
(vector-to-list* object)
(coerce-error)))
((csubtypep type (specifier-type 'string))
(check-result
(typecase object
(list (list-to-simple-string* object))
(string (string-to-simple-string* object))
(vector (vector-to-simple-string* object))
(t
(coerce-error)))))
((csubtypep type (specifier-type 'bit-vector))
(check-result
(typecase object
(list (list-to-bit-vector* object))
(vector (vector-to-bit-vector* object))
(t
(coerce-error)))))
((csubtypep type (specifier-type 'vector))
(check-result
(typecase object
(list (list-to-vector* object output-type-spec))
(vector (vector-to-vector* object output-type-spec))
(t
(coerce-error)))))
(t
(coerce-error))))))
;;; new version, which seems as though it should be better, but which
;;; does not yet work
#+nil
(defun coerce (object output-type-spec)
#!+sb-doc
"Coerces the Object to an object of type Output-Type-Spec."
(flet ((coerce-error ()
(error 'simple-type-error
:format-control "~S can't be converted to type ~S."
:format-arguments (list object output-type-spec)))
(check-result (result)
#!+high-security
(check-type-var result output-type-spec)
result))
(let ((type (specifier-type output-type-spec)))
(cond
((%typep object output-type-spec)
object)
((eq type *empty-type*)
(coerce-error))
((csubtypep type (specifier-type 'character))
(character object))
((csubtypep type (specifier-type 'function))
(coerce-to-function object))
((numberp object)
(let ((res
(cond
((csubtypep type (specifier-type 'single-float))
(%single-float object))
((csubtypep type (specifier-type 'double-float))
(%double-float object))
#!+long-float
((csubtypep type (specifier-type 'long-float))
(%long-float object))
((csubtypep type (specifier-type 'float))
(%single-float object))
((csubtypep type (specifier-type '(complex single-float)))
(complex (%single-float (realpart object))
(%single-float (imagpart object))))
((csubtypep type (specifier-type '(complex double-float)))
(complex (%double-float (realpart object))
(%double-float (imagpart object))))
#!+long-float
((csubtypep type (specifier-type '(complex long-float)))
(complex (%long-float (realpart object))
(%long-float (imagpart object))))
((csubtypep type (specifier-type 'complex))
(complex object))
(t
(coerce-error)))))
;; If RES has the wrong type, that means that rule of
;; canonical representation for complex rationals was
;; invoked. According to the ANSI spec, (COERCE 7/2
;; 'COMPLEX) returns 7/2. Thus, if the object was a
;; rational, there is no error here.
(unless (or (typep res output-type-spec) (rationalp object))
(coerce-error))
res))
((csubtypep type (specifier-type 'list))
(coerce-to-list object))
((csubtypep type (specifier-type 'string))
(check-result (coerce-to-simple-string object)))
((csubtypep type (specifier-type 'bit-vector))
(check-result (coerce-to-bit-vector object)))
((csubtypep type (specifier-type 'vector))
(check-result (coerce-to-vector object output-type-spec)))
(t
(coerce-error))))))

169
src/code/cold-error.lisp Normal file
View file

@ -0,0 +1,169 @@
;;;; miscellaneous stuff that needs to be in the cold load which would
;;;; otherwise be byte-compiled
;;;; This software is part of the SBCL system. See the README file for
;;;; more information.
;;;;
;;;; This software is derived from the CMU CL system, which was
;;;; written at Carnegie Mellon University and released into the
;;;; public domain. The software is in the public domain and is
;;;; provided with absolutely no warranty. See the COPYING and CREDITS
;;;; files for more information.
(in-package "SB!CONDITIONS")
(sb!int:file-comment
"$Header$")
(defvar *break-on-signals* nil
#!+sb-doc
"When (TYPEP condition *BREAK-ON-SIGNALS*) is true, then calls to SIGNAL will
enter the debugger prior to signalling that condition.")
(defun signal (datum &rest arguments)
#!+sb-doc
"Invokes the signal facility on a condition formed from DATUM and
ARGUMENTS. If the condition is not handled, NIL is returned. If
(TYPEP condition *BREAK-ON-SIGNALS*) is true, the debugger is invoked
before any signalling is done."
(let ((condition (coerce-to-condition datum
arguments
'simple-condition
'signal))
(*handler-clusters* *handler-clusters*))
(let ((old-bos *break-on-signals*)
(*break-on-signals* nil))
(when (typep condition old-bos)
(break "~A~%BREAK was entered because of *BREAK-ON-SIGNALS* (now NIL)."
condition)))
(loop
(unless *handler-clusters* (return))
(let ((cluster (pop *handler-clusters*)))
(dolist (handler cluster)
(when (typep condition (car handler))
(funcall (cdr handler) condition)))))
nil))
;;; COERCE-TO-CONDITION is used in SIGNAL, ERROR, CERROR, WARN, and
;;; INVOKE-DEBUGGER for parsing the hairy argument conventions into a single
;;; argument that's directly usable by all the other routines.
(defun coerce-to-condition (datum arguments default-type function-name)
(cond ((typep datum 'condition)
(if arguments
(cerror "Ignore the additional arguments."
'simple-type-error
:datum arguments
:expected-type 'null
:format-control "You may not supply additional arguments ~
when giving ~S to ~S."
:format-arguments (list datum function-name)))
datum)
((symbolp datum) ; roughly, (SUBTYPEP DATUM 'CONDITION)
(apply #'make-condition datum arguments))
((or (stringp datum) (functionp datum))
(make-condition default-type
:format-control datum
:format-arguments arguments))
(t
(error 'simple-type-error
:datum datum
:expected-type '(or symbol string)
:format-control "bad argument to ~S: ~S"
:format-arguments (list function-name datum)))))
(defun error (datum &rest arguments)
#!+sb-doc
"Invoke the signal facility on a condition formed from datum and arguments.
If the condition is not handled, the debugger is invoked."
(/show0 "entering ERROR")
#!+sb-show
(unless *cold-init-complete-p*
(/show0 "ERROR in cold init, arguments=..")
#!+sb-show (dolist (argument arguments)
(sb!impl::cold-print argument)))
(sb!kernel:infinite-error-protect
(let ((condition (coerce-to-condition datum arguments
'simple-error 'error))
;; FIXME: Why is *STACK-TOP-HINT* in SB-DEBUG instead of SB-DI?
;; SB-DEBUG should probably be only for true interface stuff.
(sb!debug:*stack-top-hint* sb!debug:*stack-top-hint*))
(unless (and (condition-function-name condition)
sb!debug:*stack-top-hint*)
(multiple-value-bind (name frame) (sb!kernel:find-caller-name)
(unless (condition-function-name condition)
(setf (condition-function-name condition) name))
(unless sb!debug:*stack-top-hint*
(setf sb!debug:*stack-top-hint* frame))))
(let ((sb!debug:*stack-top-hint* nil))
(signal condition))
(invoke-debugger condition))))
(defun cerror (continue-string datum &rest arguments)
(sb!kernel:infinite-error-protect
(with-simple-restart
(continue "~A" (apply #'format nil continue-string arguments))
(let ((condition (if (typep datum 'condition)
datum
(coerce-to-condition datum
arguments
'simple-error
'error)))
(sb!debug:*stack-top-hint* sb!debug:*stack-top-hint*))
(unless (and (condition-function-name condition)
sb!debug:*stack-top-hint*)
(multiple-value-bind (name frame) (sb!kernel:find-caller-name)
(unless (condition-function-name condition)
(setf (condition-function-name condition) name))
(unless sb!debug:*stack-top-hint*
(setf sb!debug:*stack-top-hint* frame))))
(with-condition-restarts condition (list (find-restart 'continue))
(let ((sb!debug:*stack-top-hint* nil))
(signal condition))
(invoke-debugger condition)))))
nil)
(defun break (&optional (datum "break") &rest arguments)
#!+sb-doc
"Print a message and invoke the debugger without allowing any possibility
of condition handling occurring."
(sb!kernel:infinite-error-protect
(with-simple-restart (continue "Return from BREAK.")
(let ((sb!debug:*stack-top-hint*
(or sb!debug:*stack-top-hint*
(nth-value 1 (sb!kernel:find-caller-name)))))
(invoke-debugger
(coerce-to-condition datum arguments 'simple-condition 'break)))))
nil)
(defun warn (datum &rest arguments)
#!+sb-doc
"Warn about a situation by signalling a condition formed by DATUM and
ARGUMENTS. While the condition is being signaled, a MUFFLE-WARNING restart
exists that causes WARN to immediately return NIL."
(/noshow0 "entering WARN")
;; KLUDGE: The current cold load initialization logic causes several calls
;; to WARN, so we need to be able to handle them without dying. (And calling
;; FORMAT or even PRINC in cold load is a good way to die.) Of course, the
;; ideal would be to clean up cold load so that it doesn't call WARN..
;; -- WHN 19991009
(if (not *cold-init-complete-p*)
(progn
(/show0 "ignoring WARN in cold init, arguments=..")
#!+sb-show (dolist (argument arguments)
(sb!impl::cold-print argument)))
(sb!kernel:infinite-error-protect
(let ((condition (coerce-to-condition datum arguments
'simple-warning 'warn)))
(check-type condition warning "a warning condition")
(restart-case (signal condition)
(muffle-warning ()
:report "Skip warning."
(return-from warn nil)))
(let ((badness (etypecase condition
(style-warning 'style-warning)
(warning 'warning))))
(format *error-output*
"~&~@<~S: ~3i~:_~A~:>~%"
badness
condition)))))
nil)

67
src/code/cold-init-helper-macros.lisp Normal file
View file

@ -0,0 +1,67 @@
;;;; This file contains machinery for collecting forms that, in the
;;;; target Lisp, must happen before top level forms are run. The
;;;; forms are stuffed into named functions which will be explicitly
;;;; called in the appropriate order by !COLD-INIT.
;;;; This software is part of the SBCL system. See the README file for
;;;; more information.
;;;;
;;;; This software is derived from the CMU CL system, which was
;;;; written at Carnegie Mellon University and released into the
;;;; public domain. The software is in the public domain and is
;;;; provided with absolutely no warranty. See the COPYING and CREDITS
;;;; files for more information.
(in-package "SB!KERNEL")
;;; FIXME: Perhaps this belongs in the %SYS package like some other
;;; cold load stuff.
(file-comment
"$Header$")
(eval-when (:compile-toplevel :load-toplevel :execute)
(defvar *cold-init-forms*))
(defmacro !begin-collecting-cold-init-forms ()
#-sb-xc-host '(eval-when (:compile-toplevel :execute)
(when (boundp '*cold-init-forms*)
(warn "discarding old *COLD-INIT-FORMS* value"))
(setf *cold-init-forms* nil))
#+sb-xc-host nil)
;;; Note: Unlike the analogous COLD-INIT macro in CMU CL, this macro
;;; makes no attempt to simulate a top-level situation by treating
;;; EVAL-WHEN forms specially.
(defmacro !cold-init-forms (&rest forms)
;; In the target Lisp, stuff the forms into a named function which
;; will presumably be executed at the appropriate stage of cold load
;; (i.e. basically as soon as possible).
#-sb-xc-host (progn
(setf *cold-init-forms*
(nconc *cold-init-forms* (copy-list forms)))
nil)
;; In the cross-compilation host Lisp, cold load might not be a
;; meaningful concept and in any case would have happened long ago,
;; so just execute the forms at load time (i.e. basically as soon as
;; possible).
#+sb-xc-host `(let () ,@forms))
(defmacro !defun-from-collected-cold-init-forms (name)
#-sb-xc-host `(progn
(defun ,name ()
,@*cold-init-forms*
(values))
(eval-when (:compile-toplevel :execute)
(makunbound '*cold-init-forms*)))
#+sb-xc-host (declare (ignore name)))
;;; FIXME: These macros should be byte-compiled.
;;; FIXME: Consider renaming this file asap.lisp,
;;; and the renaming the various things
;;; *ASAP-FORMS* or *REVERSED-ASAP-FORMS*
;;; WITH-ASAP-FORMS
;;; ASAP or EVAL-WHEN-COLD-LOAD
;;; DEFUN-FROM-ASAP-FORMS
;;; If so, add a comment explaining that ASAP is colloquial English for "as
;;; soon as possible", and has nothing to do with "system area pointer".

339
src/code/cold-init.lisp Normal file
View file

@ -0,0 +1,339 @@
;;;; cold initialization stuff, plus some other miscellaneous stuff
;;;; that we don't have any better place for
;;;; This software is part of the SBCL system. See the README file for
;;;; more information.
;;;;
;;;; This software is derived from the CMU CL system, which was
;;;; written at Carnegie Mellon University and released into the
;;;; public domain. The software is in the public domain and is
;;;; provided with absolutely no warranty. See the COPYING and CREDITS
;;;; files for more information.
(in-package "SB!IMPL")
(file-comment
"$Header$")
;;;; burning our ships behind us
;;; There's a fair amount of machinery which is needed only at cold
;;; init time, and should be discarded before freezing the final
;;; system. We discard it by uninterning the associated symbols.
;;; Rather than using a special table of symbols to be uninterned,
;;; which might be tedious to maintain, instead we use a hack:
;;; anything whose name matches a magic character pattern is
;;; uninterned.
(defun !unintern-init-only-stuff ()
(do ((any-changes? nil nil))
(nil)
(dolist (package (list-all-packages))
(do-symbols (symbol package)
(let ((name (symbol-name symbol)))
(when (or (string= name "!" :end1 1 :end2 1)
(and (>= (length name) 2)
(string= name "*!" :end1 2 :end2 2)))
(/show0 "uninterning cold-init-only symbol..")
#!+sb-show (%primitive print name)
(unintern symbol package)
(setf any-changes? t)))))
(unless any-changes?
(return))))
;;;; !COLD-INIT
;;; a list of toplevel things set by GENESIS
(defvar *!reversed-cold-toplevels*)
;;; a SIMPLE-VECTOR set by genesis
(defvar *!load-time-values*)
#!+gengc
(defun do-load-time-value-fixup (object offset index)
(declare (type index offset))
(macrolet ((lose (msg)
`(progn
(%primitive print ,msg)
(%halt))))
(let ((value (svref *!load-time-values* index)))
(typecase object
(list
(case offset
(0 (setf (car object) value))
(1 (setf (cdr object) value))
(t (lose "bogus offset in cons cell"))))
(instance
(setf (%instance-ref object (- offset sb!vm:instance-slots-offset))
value))
(code-component
(setf (code-header-ref object offset) value))
(simple-vector
(setf (svref object (- offset sb!vm:vector-data-offset)) value))
(t
(lose "unknown kind of object for load-time-value fixup"))))))
(eval-when (:compile-toplevel :execute)
;; FIXME: Perhaps we should make SHOW-AND-CALL-AND-FMAKUNBOUND, too,
;; and use it for most of the cold-init functions. (Just be careful
;; not to use it for the COLD-INIT-OR-REINIT functions.)
(sb!xc:defmacro show-and-call (name)
`(progn
#!+sb-show (%primitive print ,(symbol-name name))
(,name))))
;;; called when a cold system starts up
(defun !cold-init ()
#!+sb-doc "Give the world a shove and hope it spins."
(/show0 "entering !COLD-INIT")
;; FIXME: It'd probably be cleaner to have most of the stuff here
;; handled by calls a la !GC-COLD-INIT, !ERROR-COLD-INIT, and
;; !UNIX-COLD-INIT. And *TYPE-SYSTEM-INITIALIZED* could be changed to
;; *TYPE-SYSTEM-INITIALIZED-WHEN-BOUND* so that it doesn't need to
;; be explicitly set in order to be meaningful.
(setf *gc-verbose* nil)
(setf *gc-notify-stream* nil)
(setf *before-gc-hooks* nil)
(setf *after-gc-hooks* nil)
#!+gengc (setf sb!conditions::*handler-clusters* nil)
#!-gengc (setf *already-maybe-gcing* t
*gc-inhibit* t
*need-to-collect-garbage* nil
sb!unix::*interrupts-enabled* t
sb!unix::*interrupt-pending* nil)
(setf *break-on-signals* nil)
(setf *maximum-error-depth* 10)
(setf *current-error-depth* 0)
(setf *cold-init-complete-p* nil)
(setf *type-system-initialized* nil)
;; Anyone might call RANDOM to initialize a hash value or something;
;; and there's nothing which needs to be initialized in order for
;; this to be initialized, so we initialize it right away.
(show-and-call !random-cold-init)
;; All sorts of things need INFO and/or (SETF INFO).
(show-and-call !globaldb-cold-init)
;; This needs to be done early, but needs to be after INFO is
;; initialized.
(show-and-call !fdefn-cold-init)
;; Various toplevel forms call MAKE-ARRAY, which calls SUBTYPEP, so
;; the basic type machinery needs to be initialized before toplevel
;; forms run.
(show-and-call !type-class-cold-init)
(show-and-call !typedefs-cold-init)
(show-and-call !classes-cold-init)
(show-and-call !early-type-cold-init)
(show-and-call !late-type-cold-init)
(show-and-call !alien-type-cold-init)
(show-and-call !target-type-cold-init)
(show-and-call !vm-type-cold-init)
;; FIXME: It would be tidy to make sure that that these cold init
;; functions are called in the same relative order as the toplevel
;; forms of the corresponding source files.
(show-and-call !package-cold-init)
;; Set sane values for our toplevel forms.
(show-and-call !set-sane-cookie-defaults)
;; KLUDGE: Why are fixups mixed up with toplevel forms? Couldn't
;; fixups be done separately? Wouldn't that be clearer and better?
;; -- WHN 19991204
(/show0 "doing cold toplevel forms and fixups")
(/show0 "(LENGTH *!REVERSED-COLD-TOPLEVELS*)=..")
#!+sb-show (%primitive print
(sb!impl::hexstr (length *!reversed-cold-toplevels*)))
(let (#!+sb-show (index-in-cold-toplevels 0)
#!+sb-show (filename-in-cold-toplevels nil))
#!+sb-show (declare (type fixnum index-in-cold-toplevels))
(dolist (toplevel-thing (prog1
(nreverse *!reversed-cold-toplevels*)
;; (Now that we've NREVERSEd it, it's
;; somewhat scrambled, so keep anyone
;; else from trying to get at it.)
(makunbound '*!reversed-cold-toplevels*)))
#!+sb-show
(when (zerop (mod index-in-cold-toplevels 1024))
(/show0 "INDEX-IN-COLD-TOPLEVELS=..")
(%primitive print (sb!impl::hexstr index-in-cold-toplevels)))
#!+sb-show
(setf index-in-cold-toplevels
(the fixnum (1+ index-in-cold-toplevels)))
(typecase toplevel-thing
(function
(funcall toplevel-thing))
(cons
(case (first toplevel-thing)
(:load-time-value
(setf (svref *!load-time-values* (third toplevel-thing))
(funcall (second toplevel-thing))))
(:load-time-value-fixup
#!-gengc
(setf (sap-ref-32 (second toplevel-thing) 0)
(get-lisp-obj-address
(svref *!load-time-values* (third toplevel-thing))))
#!+gengc
(do-load-time-value-fixup (second toplevel-thing)
(third toplevel-thing)
(fourth toplevel-thing)))
#!+(and x86 gencgc)
(:load-time-code-fixup
(sb!vm::do-load-time-code-fixup (second toplevel-thing)
(third toplevel-thing)
(fourth toplevel-thing)
(fifth toplevel-thing)))
(t
(%primitive print
"bogus fixup code in *!REVERSED-COLD-TOPLEVELS*")
(%halt))))
(t
(%primitive print "bogus function in *!REVERSED-COLD-TOPLEVELS*")
(%halt)))))
(/show0 "done with loop over cold toplevel forms and fixups")
;; Set sane values again, so that the user sees sane values instead of
;; whatever is left over from the last DECLAIM.
(show-and-call !set-sane-cookie-defaults)
;; Only do this after top level forms have run, 'cause that's where
;; DEFTYPEs are.
(setf *type-system-initialized* t)
(show-and-call os-cold-init-or-reinit)
(show-and-call !filesys-cold-init)
(show-and-call stream-cold-init-or-reset)
(show-and-call !loader-cold-init)
(show-and-call signal-cold-init-or-reinit)
(setf (sb!alien:extern-alien "internal_errors_enabled" boolean) t)
;; FIXME: This list of modes should be defined in one place and
;; explicitly shared between here and REINIT.
(set-floating-point-modes :traps '(:overflow
#!-x86 :underflow
:invalid
:divide-by-zero))
(show-and-call !class-finalize)
;; The reader and printer are initialized very late, so that they
;; can even do hairy things like invoking the compiler as part of
;; their initialization.
(show-and-call !reader-cold-init)
(let ((*readtable* *standard-readtable*))
(show-and-call !sharpm-cold-init)
(show-and-call !backq-cold-init))
(setf *readtable* (copy-readtable *standard-readtable*))
(setf sb!debug:*debug-readtable* (copy-readtable *standard-readtable*))
(sb!pretty:!pprint-cold-init)
;; the ANSI-specified initial value of *PACKAGE*
(setf *package* (find-package "COMMON-LISP-USER"))
;; FIXME: I'm not sure where it should be done, but CL-USER really
;; ought to USE-PACKAGE publicly accessible packages like SB-DEBUG
;; (for ARG and VAR), SB-EXT, SB-EXT-C-CALL, and SB-EXT-ALIEN so
;; that the user has a hint about which symbols we consider public.
;; (Perhaps SB-DEBUG wouldn't need to be in the list if ARG and VAR
;; could be typed directly, with no parentheses, at the debug prompt
;; the way that e.g. F or BACKTRACE can be?)
(/show0 "done initializing")
(setf *cold-init-complete-p* t)
;; Unintern no-longer-needed stuff before we GC.
#!-sb-fluid
(!unintern-init-only-stuff)
;; The system is finally ready for GC.
#!-gengc (setf *already-maybe-gcing* nil)
(/show0 "enabling GC")
(gc-on)
(/show0 "doing first GC")
(gc :full t)
(/show0 "back from first GC")
;; The show is on.
(terpri)
(/show0 "going into toplevel loop")
(let ((wot (catch '%end-of-the-world
(/show0 "inside CATCH '%END-OF-THE-WORLD")
(toplevel))))
(flush-standard-output-streams)
(sb!unix:unix-exit wot)))
(defun quit (&key recklessly-p (unix-code 0))
#!+sb-doc
"Terminate the current Lisp. Things are cleaned up (with UNWIND-PROTECT
and so forth) unless RECKLESSLY-P is non-NIL. On UNIX-like systems,
UNIX-CODE is used as the status code."
(declare (type (signed-byte 32) unix-code))
(if recklessly-p
(sb!unix:unix-exit unix-code)
(throw '%end-of-the-world unix-code)))
;;;; initialization functions
(defun reinit ()
(without-interrupts
(without-gcing
(os-cold-init-or-reinit)
(stream-reinit)
(signal-cold-init-or-reinit)
(gc-cold-init-or-reinit)
(setf (sb!alien:extern-alien "internal_errors_enabled" boolean) t)
(set-floating-point-modes :traps
;; PRINT seems to not like x86 NPX denormal
;; floats like LEAST-NEGATIVE-SINGLE-FLOAT, so
;; the :UNDERFLOW exceptions are disabled by
;; default. Joe User can explicitly enable them
;; if desired.
'(:overflow #!-x86 :underflow :invalid
:divide-by-zero))
;; Clear pseudo atomic in case this core wasn't compiled with
;; support.
;;
;; FIXME: In SBCL our cores are always compiled with support. So
;; we don't need to do this, do we? At least not for this
;; reason.. (Perhaps we should do it anyway in case someone
;; manages to save an image from within a pseudo-atomic-atomic
;; operation?)
#!+x86 (setf sb!impl::*pseudo-atomic-atomic* 0))
(gc-on)))
;;;; some support for any hapless wretches who end up debugging cold
;;;; init code
;;; Decode THING into hex using only machinery available early in cold
;;; init.
#!+sb-show
(defun hexstr (thing)
(let ((addr (sb!kernel:get-lisp-obj-address thing))
(str (make-string 10)))
(setf (char str 0) #\0
(char str 1) #\x)
(dotimes (i 8)
(let* ((nibble (ldb (byte 4 0) addr))
(chr (char "0123456789abcdef" nibble)))
(declare (type (unsigned-byte 4) nibble)
(base-char chr))
(setf (char str (- 9 i)) chr
addr (ash addr -4))))
str))
#!+sb-show
(defun cold-print (x)
(typecase x
(simple-string (sb!sys:%primitive print x))
(symbol (sb!sys:%primitive print (symbol-name x)))
(list (let ((count 0))
(sb!sys:%primitive print "list:")
(dolist (i x)
(when (>= (incf count) 4)
(sb!sys:%primitive print "...")
(return))
(cold-print i))))
(t (sb!sys:%primitive print (hexstr x)))))

208
src/code/cross-float.lisp Normal file
View file

@ -0,0 +1,208 @@
;;;; portable implementations or stubs for nonportable floating point
;;;; things, useful for building Python as a cross-compiler when
;;;; running under an ordinary ANSI Common Lisp implementation
;;;; This software is part of the SBCL system. See the README file for
;;;; more information.
;;;;
;;;; This software is derived from the CMU CL system, which was
;;;; written at Carnegie Mellon University and released into the
;;;; public domain. The software is in the public domain and is
;;;; provided with absolutely no warranty. See the COPYING and CREDITS
;;;; files for more information.
(in-package "SB!IMPL")
(file-comment
"$Header$")
;;; There seems to be no portable way to mask float traps, but we shouldn't
;;; encounter any float traps when cross-compiling SBCL itself, anyway, so we
;;; just make this a no-op.
(defmacro sb!vm::with-float-traps-masked (traps &body body)
(declare (ignore traps))
;; FIXME: should become STYLE-WARNING?
(format *error-output*
"~&(can't portably mask float traps, proceeding anyway)~%")
`(progn ,@body))
;;; a helper function for DOUBLE-FLOAT-FOO-BITS functions
;;;
;;; Return the low N bits of X as a signed N-bit value.
(defun mask-and-sign-extend (x n)
(assert (plusp n))
(let* ((high-bit (ash 1 (1- n)))
(mask (1- (ash high-bit 1)))
(uresult (logand mask x)))
(if (zerop (logand uresult high-bit))
uresult
(logior uresult
(logand -1 (lognot mask))))))
;;; portable implementations of SINGLE-FLOAT-BITS, DOUBLE-FLOAT-LOW-BITS, and
;;; DOUBLE-FLOAT-HIGH-BITS
;;;
;;; KLUDGE: These will fail if the target's floating point isn't IEEE, and so
;;; I'd be more comfortable if there were an assertion "target's floating point
;;; is IEEE" in the code, but I can't see how to express that.
;;;
;;; KLUDGE: It's sort of weird that these functions return signed 32-bit values
;;; instead of unsigned 32-bit values. This is the way that the CMU CL
;;; machine-dependent functions behaved, and I've copied that behavior, but it
;;; seems to me that it'd be more idiomatic to return unsigned 32-bit values.
;;; Maybe someday the machine-dependent functions could be tweaked to return
;;; unsigned 32-bit values?
(defun single-float-bits (x)
(declare (type single-float x))
(assert (= (float-radix x) 2))
(if (zerop x)
0 ; known property of IEEE floating point: 0.0 is represented as 0.
(multiple-value-bind (lisp-significand lisp-exponent lisp-sign)
(integer-decode-float x)
(assert (plusp lisp-significand))
;; Calculate IEEE-style fields from Common-Lisp-style fields.
;;
;; KLUDGE: This code was written from my foggy memory of what IEEE
;; format looks like, augmented by some experiments with
;; the existing implementation of SINGLE-FLOAT-BITS, and what
;; I found floating around on the net at
;; <http://www.scri.fsu.edu/~jac/MAD3401/Backgrnd/ieee.html>,
;; <http://rodin.cs.uh.edu/~johnson2/ieee.html>,
;; and
;; <http://www.ttu.ee/sidu/cas/IEEE_Floating.htm>.
;; And beyond the probable sheer flakiness of the code, all the bare
;; numbers floating around here are sort of ugly, too. -- WHN 19990711
(let* ((significand lisp-significand)
(exponent (+ lisp-exponent 23 127))
(unsigned-result
(if (plusp exponent) ; if not obviously denormalized
(do ()
(nil)
(cond (;; ordinary termination case
(>= significand (expt 2 23))
(assert (< 0 significand (expt 2 24)))
;; Exponent 0 is reserved for denormalized numbers,
;; and 255 is reserved for specials a la NaN.
(assert (< 0 exponent 255))
(return (logior (ash exponent 23)
(logand significand
(1- (ash 1 23))))))
(;; special termination case, denormalized float number
(zerop exponent)
;; Denormalized numbers have exponent one greater than
;; the exponent field.
(return (ash significand -1)))
(t
;; Shift as necessary to set bit 24 of significand.
(setf significand (ash significand 1)
exponent (1- exponent)))))
(do ()
((zerop exponent)
;; Denormalized numbers have exponent one greater than the
;; exponent field.
(ash significand -1))
(unless (zerop (logand significand 1))
(warn "denormalized SINGLE-FLOAT-BITS ~S losing bits" x))
(setf significand (ash significand -1)
exponent (1+ exponent))))))
(ecase lisp-sign
(1 unsigned-result)
(-1 (logior unsigned-result (- (expt 2 31)))))))))
(defun double-float-bits (x)
(declare (type double-float x))
(assert (= (float-radix x) 2))
(if (zerop x)
0 ; known property of IEEE floating point: 0.0d0 is represented as 0.
;; KLUDGE: As per comments in SINGLE-FLOAT-BITS, above.
(multiple-value-bind (lisp-significand lisp-exponent lisp-sign)
(integer-decode-float x)
(assert (plusp lisp-significand))
(let* ((significand lisp-significand)
(exponent (+ lisp-exponent 52 1023))
(unsigned-result
(if (plusp exponent) ; if not obviously denormalized
(do ()
(nil)
(cond (;; ordinary termination case
(>= significand (expt 2 52))
(assert (< 0 significand (expt 2 53)))
;; Exponent 0 is reserved for denormalized numbers,
;; and 2047 is reserved for specials a la NaN.
(assert (< 0 exponent 2047))
(return (logior (ash exponent 52)
(logand significand
(1- (ash 1 52))))))
(;; special termination case, denormalized float number
(zerop exponent)
;; Denormalized numbers have exponent one greater than
;; the exponent field.
(return (ash significand -1)))
(t
;; Shift as necessary to set bit 53 of significand.
(setf significand (ash significand 1)
exponent (1- exponent)))))
(do ()
((zerop exponent)
;; Denormalized numbers have exponent one greater than the
;; exponent field.
(ash significand -1))
(unless (zerop (logand significand 1))
(warn "denormalized SINGLE-FLOAT-BITS ~S losing bits" x))
(setf significand (ash significand -1)
exponent (1+ exponent))))))
(ecase lisp-sign
(1 unsigned-result)
(-1 (logior unsigned-result (- (expt 2 63)))))))))
(defun double-float-low-bits (x)
(declare (type double-float x))
(if (zerop x)
0
;; Unlike DOUBLE-FLOAT-HIGH-BITS or SINGLE-FLOAT-BITS, the CMU CL
;; DOUBLE-FLOAT-LOW-BITS seems to return a unsigned value, not a signed
;; value.
(logand #xffffffff (double-float-bits x))))
(defun double-float-high-bits (x)
(declare (type double-float x))
(if (zerop x)
0
(mask-and-sign-extend (ash (double-float-bits x) -32) 32)))
;;; KLUDGE: These functions will blow up on any cross-compilation
;;; host Lisp which has less floating point precision than the target
;;; Lisp. In practice, this may not be a major problem: IEEE
;;; floating point arithmetic is so common these days that most
;;; cross-compilation host Lisps are likely to have exactly the same
;;; floating point precision as the target Lisp. If it turns out to be
;;; a problem, there are possible workarounds involving portable
;;; representations for target floating point numbers, a la
;;; (DEFSTRUCT TARGET-SINGLE-FLOAT
;;; (SIGN (REQUIRED-ARGUMENT) :TYPE BIT)
;;; (EXPONENT (REQUIRED-ARGUMENT) :TYPE UNSIGNED-BYTE)
;;; (MANTISSA (REQUIRED-ARGUMENT) :TYPE UNSIGNED-BYTE))
;;; with some sort of MAKE-LOAD-FORM-ish magic to cause them to be
;;; written out in the appropriate target format. (And yes, those
;;; workarounds *do* look messy to me, which is why I just went
;;; with this quick kludge instead.) -- WHN 19990711
(defun make-single-float (bits)
(if (zerop bits) ; IEEE float special case
0.0
(let ((sign (ecase (ldb (byte 1 31) bits)
(0 1.0)
(1 -1.0)))
(expt (- (ldb (byte 8 23) bits) 127))
(mant (* (logior (ldb (byte 23 0) bits)
(ash 1 23))
(expt 0.5 23))))
(* sign (expt 2.0 expt) mant))))
(defun make-double-float (hi lo)
(if (and (zerop hi) (zerop lo)) ; IEEE float special case
0.0d0
(let* ((bits (logior (ash hi 32) lo))
(sign (ecase (ldb (byte 1 63) bits)
(0 1.0d0)
(1 -1.0d0)))
(expt (- (ldb (byte 11 52) bits) 1023))
(mant (* (logior (ldb (byte 52 0) bits)
(ash 1 52))
(expt 0.5d0 52))))
(* sign (expt 2.0d0 expt) mant))))

33
src/code/cross-io.lisp Normal file
View file

@ -0,0 +1,33 @@
;;;; cross-compiler-only versions of I/O-related stuff
;;;; This software is part of the SBCL system. See the README file for
;;;; more information.
;;;;
;;;; This software is derived from the CMU CL system, which was
;;;; written at Carnegie Mellon University and released into the
;;;; public domain. The software is in the public domain and is
;;;; provided with absolutely no warranty. See the COPYING and CREDITS
;;;; files for more information.
(in-package "SB!IMPL")
(file-comment
"$Header$")
;;;; fast-read operations
;;;;
;;;; (Portable versions of these are needed at cross-compile time because
;;;; genesis implements some of its cold fops by cloning ordinary fop
;;;; implementations, and the ordinary fop implementations are defined in terms
;;;; of fast-read operations.)
(defmacro prepare-for-fast-read-byte (stream &body forms)
`(let ((%frc-stream% ,stream))
,@forms))
(defmacro fast-read-byte (&optional (eof-error-p t) (eof-value nil) any-type)
(declare (ignore any-type))
`(read-byte %frc-stream% ,eof-error-p ,eof-value))
(defmacro done-with-fast-read-byte ()
`(values))

129
src/code/cross-misc.lisp Normal file
View file

@ -0,0 +1,129 @@
;;;; cross-compile-time-only replacements for miscellaneous unportable
;;;; stuff
;;;; This software is part of the SBCL system. See the README file for
;;;; more information.
;;;;
;;;; This software is derived from the CMU CL system, which was
;;;; written at Carnegie Mellon University and released into the
;;;; public domain. The software is in the public domain and is
;;;; provided with absolutely no warranty. See the COPYING and CREDITS
;;;; files for more information.
(in-package "SB!IMPL")
(file-comment
"$Header$")
;;; In correct code, TRULY-THE has only a performance impact and can be
;;; safely degraded to ordinary THE.
(defmacro truly-the (type expr)
`(the ,type ,expr))
;;; MAYBE-INLINE and FREEZE-TYPE declarations can be safely ignored
;;; (possibly at some cost in efficiency).
(declaim (declaration freeze-type maybe-inline))
;;; INHIBIT-WARNINGS declarations can be safely ignored (although we may then
;;; have to wade through some irrelevant warnings).
(declaim (declaration inhibit-warnings))
;;; Interrupt control isn't an issue in the cross-compiler: we don't use
;;; address-dependent (and thus GC-dependent) hashes, and we only have a single
;;; thread of control.
(defmacro without-interrupts (&rest forms)
`(progn ,@forms))
;;; When we're running as a cross-compiler in an arbitrary host ANSI Lisp, we
;;; don't have any hooks available to manipulate the debugging name and
;;; debugging argument list of an interpreted function object (and don't care
;;; much about getting debugging name and debugging argument list right
;;; anyway).
(defun try-to-rename-interpreted-function-as-macro (f name lambda-list)
(declare (ignore f name lambda-list))
(values))
;;; When we're running as a cross-compiler in an arbitrary host ANSI Lisp, we
;;; shouldn't be doing anything which is sensitive to GC. KLUDGE: I (WHN
;;; 19990131) think the proper long-term solution would be to remove any
;;; operations from cross-compiler source files (putting them in target-only
;;; source files) if they refer to these hooks. This is a short-term hack.
(defvar *before-gc-hooks* nil)
(defvar *after-gc-hooks* nil)
;;; The GENESIS function works with fasl code which would, in the target SBCL,
;;; work on LISP-STREAMs. A true LISP-STREAM doesn't seem to be a meaningful
;;; concept in ANSI Common Lisp, but we can fake it acceptably well using a
;;; standard STREAM.
(deftype lisp-stream () 'stream)
;;; In the target SBCL, the INSTANCE type refers to a base implementation
;;; for compound types. There's no way to express exactly that concept
;;; portably, but we can get essentially the same effect by testing for
;;; any of the standard types which would, in the target SBCL, be derived
;;; from INSTANCE:
(deftype sb!kernel:instance ()
'(or condition standard-object structure-object))
;;; There aren't any FUNCALLABLE-INSTANCEs in the cross-compilation
;;; host Common Lisp.
(defun funcallable-instance-p (x)
(if (typep x 'generic-function)
;; In the target SBCL, FUNCALLABLE-INSTANCEs are used to implement generic
;; functions, so any case which tests for this might in fact be trying to
;; test for generic functions. My (WHN 19990313) expectation is that this
;; case won't arise in the cross-compiler, but if it does, it deserves a
;; little thought, rather than reflexively returning NIL.
(error "not clear how to handle GENERIC-FUNCTION")
nil))
;;; This seems to be the portable Common Lisp type test which corresponds
;;; to the effect of the target SBCL implementation test..
(defun sb!kernel:array-header-p (x)
(and (typep x 'simple-array)
(= 1 (array-rank x))))
;;; Genesis needs these at cross-compile time. The target implementation of
;;; these is reasonably efficient by virtue of its ability to peek into the
;;; internals of the package implementation; this reimplementation is portable
;;; but slow.
(defun package-internal-symbol-count (package)
(let ((result 0))
(declare (type fixnum result))
(do-symbols (i package)
;; KLUDGE: The ANSI Common Lisp specification warns that DO-SYMBOLS may
;; execute its body more than once for symbols that are inherited from
;; multiple packages, and we currently make no attempt to correct for
;; that here. (The current uses of this function at cross-compile time
;; don't really care if the count is a little too high.) -- WHN 19990826
(multiple-value-bind (symbol status)
(find-symbol (symbol-name i) package)
(declare (ignore symbol))
(when (member status '(:internal :inherited))
(incf result))))
result))
(defun package-external-symbol-count (package)
(let ((result 0))
(declare (type fixnum result))
(do-external-symbols (i package)
(declare (ignore i))
(incf result))
result))
;;; In the target Lisp, INTERN* is the primitive and INTERN is implemented in
;;; terms of it. This increases efficiency by letting us reuse a fixed-size
;;; buffer; the alternative would be particularly painful because we don't
;;; implement DYNAMIC-EXTENT. In the host Lisp, this is only used at
;;; cold load time, and we don't care as much about efficiency, so it's fine
;;; to treat the host Lisp's INTERN as primitive and implement INTERN* in
;;; terms of it.
(defun intern* (nameoid length package)
(intern (replace (make-string length) nameoid :end2 length) package))
;;; In the target Lisp this is implemented by reading a fixed slot in the
;;; symbol. In portable ANSI Common Lisp the same criteria can be met (more
;;; slowly, and with the extra property of repeatability between runs) by just
;;; calling SXHASH.
(defun symbol-hash (symbol)
(declare (type symbol symbol))
(sxhash symbol))

63
src/code/cross-sap.lisp Normal file
View file

@ -0,0 +1,63 @@
;;;; support and placeholders for System Area Pointers (SAPs) in the host
;;;; Common Lisp at cross-compile time
;;;; This software is part of the SBCL system. See the README file for
;;;; more information.
;;;;
;;;; This software is derived from the CMU CL system, which was
;;;; written at Carnegie Mellon University and released into the
;;;; public domain. The software is in the public domain and is
;;;; provided with absolutely no warranty. See the COPYING and CREDITS
;;;; files for more information.
(in-package "SB!SYS")
(file-comment
"$Header$")
;;; SYSTEM-AREA-POINTER is not a primitive type in ANSI Common Lisp, so we
;;; need a compound type to represent it in the host Common Lisp at
;;; cross-compile time:
(defstruct (system-area-pointer (:constructor make-sap) (:conc-name "SAP-"))
;; the integer representation of the address
(int (error "missing SAP-INT argument") :type sap-int-type :read-only t))
;;; cross-compilation-host analogues of target-CMU CL primitive SAP operations
(defun int-sap (int)
(make-sap :int int))
(defun sap+ (sap offset)
(declare (type system-area-pointer sap) (type sap-int-type offset))
(make-sap :int (+ (sap-int sap) offset)))
#.`(progn
,@(mapcar (lambda (info)
(destructuring-bind (sap-fun int-fun) info
`(defun ,sap-fun (x y)
(,int-fun (sap-int x) (sap-int y)))))
'((sap< <) (sap<= <=) (sap= =) (sap>= >=) (sap> >) (sap- -))))
;;; dummies, defined so that we can declare they never return and thereby
;;; eliminate a thundering herd of optimization notes a la "can't optimize this
;;; expression because we don't know the return type of SAP-REF-8"
(defun sap-ref-stub (name)
(error "~S doesn't make sense on cross-compilation host." name))
#.`(progn
,@(mapcan (lambda (name)
`((declaim (ftype (function (system-area-pointer fixnum) nil)
,name))
(defun ,name (sap offset)
(declare (ignore sap offset))
(sap-ref-stub ',name))
,@(let ((setter-stub (gensym "SAP-SETTER-STUB-")))
`((defun ,setter-stub (foo sap offset)
(declare (ignore foo sap offset))
(sap-ref-stub '(setf ,name)))
(defsetf ,name ,setter-stub)))))
'(sap-ref-8
sap-ref-16
sap-ref-32
sap-ref-sap
sap-ref-single
sap-ref-double
signed-sap-ref-8
signed-sap-ref-16
signed-sap-ref-32)))

337
src/code/cross-type.lisp Normal file
View file

@ -0,0 +1,337 @@
;;;; cross-compiler-only versions of TYPEP, TYPE-OF, and related functions
;;;; This software is part of the SBCL system. See the README file for
;;;; more information.
;;;;
;;;; This software is derived from the CMU CL system, which was
;;;; written at Carnegie Mellon University and released into the
;;;; public domain. The software is in the public domain and is
;;;; provided with absolutely no warranty. See the COPYING and CREDITS
;;;; files for more information.
(in-package "SB!IMPL")
(file-comment
"$Header$")
;;; (This was a useful warning when trying to get bootstrapping
;;; to work, but it's mostly irrelevant noise now that the system
;;; works.)
(define-condition cross-type-style-warning (style-warning)
((call :initarg :call
:reader cross-type-style-warning-call)
(message :reader cross-type-style-warning-message
#+cmu :initarg #+cmu :message ; to stop bogus non-STYLE WARNING
))
(:report (lambda (c s)
(format
s
"cross-compilation-time type ambiguity (should be OK) in ~S:~%~A"
(cross-type-style-warning-call c)
(cross-type-style-warning-message c)))))
;;; This warning is issued when giving up on a type calculation where a
;;; conservative answer is acceptable. Since a conservative answer is
;;; acceptable, the only downside is lost optimization opportunities.
(define-condition cross-type-giving-up-conservatively
(cross-type-style-warning)
((message :initform "giving up conservatively"
#+cmu :reader #+cmu #.(gensym) ; (to stop bogus non-STYLE WARNING)
)))
;;; This warning refers to the flexibility in the ANSI spec with regard to
;;; run-time distinctions between floating point types. (E.g. the
;;; cross-compilation host might not even distinguish between SINGLE-FLOAT and
;;; DOUBLE-FLOAT, so a DOUBLE-FLOAT number would test positive as
;;; SINGLE-FLOAT.) If the target SBCL does make this distinction, then
;;; information is lost. It's not too hard to contrive situations where this
;;; would be a problem. In practice we don't tend to run into them because all
;;; widely used Common Lisp environments do recognize the distinction between
;;; SINGLE-FLOAT and DOUBLE-FLOAT, and we don't really need the other
;;; distinctions (e.g. between SHORT-FLOAT and SINGLE-FLOAT), so we call
;;; WARN-POSSIBLE-CROSS-TYPE-FLOAT-INFO-LOSS to test at runtime whether
;;; we need to worry about this at all, and not warn unless we do. If we *do*
;;; have to worry about this at runtime, my (WHN 19990808) guess is that
;;; the system will break in multiple places, so this is a real
;;; WARNING, not just a STYLE-WARNING.
;;;
;;; KLUDGE: If we ever try to support LONG-FLOAT or SHORT-FLOAT, this
;;; situation will get a lot more complicated.
(defun warn-possible-cross-type-float-info-loss (call)
(when (or (subtypep 'single-float 'double-float)
(subtypep 'double-float 'single-float))
(warn "possible floating point information loss in ~S" call)))
(defun sb!xc:type-of (object)
(labels (;; FIXME: This function is a no-op now that we no longer have a
;; distinct package T%CL to translate for-the-target-Lisp CL symbols
;; to, and should go away completely.
(translate (expr) expr))
(let ((raw-result (type-of object)))
(cond ((or (subtypep raw-result 'float)
(subtypep raw-result 'complex))
(warn-possible-cross-type-float-info-loss
`(sb!xc:type-of ,object))
(translate raw-result))
((subtypep raw-result 'integer)
(cond ((<= 0 object 1)
'bit)
((target-fixnump object)
'fixnum)
(t
'integer)))
((some (lambda (type) (subtypep raw-result type))
'(array character list symbol))
(translate raw-result))
(t
(error "can't handle TYPE-OF ~S in cross-compilation"))))))
;;; Like TYPEP, but asks whether HOST-OBJECT would be of TARGET-TYPE when
;;; instantiated on the target SBCL. Since this is hard to decide in some
;;; cases, and since in other cases we just haven't bothered to try, it
;;; needs to return two values, just like SUBTYPEP: the first value for
;;; its conservative opinion (never T unless it's certain) and the second
;;; value to tell whether it's certain.
(defun cross-typep (host-object target-type)
(flet ((warn-and-give-up ()
;; We don't have to keep track of this as long as system performance
;; is acceptable, since giving up conservatively is a safe way out.
#+nil
(warn 'cross-type-giving-up-conservatively
:call `(cross-typep ,host-object ,target-type))
(values nil nil))
(warn-about-possible-float-info-loss ()
(warn-possible-cross-type-float-info-loss
`(cross-typep ,host-object ,target-type))))
(cond (;; Handle various SBCL-specific types which can't exist on the
;; ANSI cross-compilation host. KLUDGE: This code will need to be
;; tweaked by hand if the names of these types ever change, ugh!
(if (consp target-type)
(member (car target-type)
'(sb!alien:alien))
(member target-type
'(system-area-pointer
funcallable-instance
sb!alien-internals:alien-value)))
(values nil t))
((typep target-type 'sb!xc::structure-class)
;; SBCL-specific types which have an analogue specially created
;; on the host system
(if (sb!xc:subtypep (sb!xc:class-name target-type)
'sb!kernel::structure!object)
(values (typep host-object (sb!xc:class-name target-type)) t)
(values nil t)))
((and (symbolp target-type)
(find-class target-type nil)
(subtypep target-type 'sb!kernel::structure!object))
(values (typep host-object target-type) t))
((and (symbolp target-type)
(sb!xc:find-class target-type nil)
(sb!xc:subtypep target-type 'cl:structure-object)
(typep host-object '(or symbol number list character)))
(values nil t))
((and (not (unknown-type-p (values-specifier-type target-type)))
(sb!xc:subtypep target-type 'cl:array))
(if (arrayp host-object)
(warn-and-give-up) ; general case of arrays being way too hard
(values nil t))) ; but "obviously not an array" being easy
((consp target-type)
(let ((first (first target-type))
(rest (rest target-type)))
(case first
;; Many complex types are guaranteed to correspond exactly
;; between any host ANSI Common Lisp and the target SBCL.
((integer member mod rational real signed-byte unsigned-byte)
(values (typep host-object target-type) t))
;; Floating point types are guaranteed to correspond, too, but
;; less exactly.
((single-float double-float)
(cond ((floatp host-object)
(warn-about-possible-float-info-loss)
(values (typep host-object target-type) t))
(t
(values nil t))))
;; Some complex types have translations that are less trivial.
(and
;; Note: This could be implemented as a real test, just the way
;; that OR is; I just haven't bothered. -- WHN 19990706
(warn-and-give-up))
(or (let ((opinion nil)
(certain-p t))
(dolist (i rest)
(multiple-value-bind (sub-opinion sub-certain-p)
(cross-typep host-object i)
(cond (sub-opinion (setf opinion t
certain-p t)
(return))
((not sub-certain-p) (setf certain-p nil))))
(if certain-p
(values opinion t)
(warn-and-give-up)))))
;; Some complex types are too hard to handle in the positive
;; case, but at least we can be confident in a large fraction of
;; the negative cases..
((base-string simple-base-string simple-string)
(if (stringp host-object)
(warn-and-give-up)
(values nil t)))
((array simple-array simple-vector vector)
(if (arrayp host-object)
(warn-and-give-up)
(values nil t)))
(function
(if (functionp host-object)
(warn-and-give-up)
(values nil t)))
;; And the Common Lisp type system is complicated, and we don't
;; try to implement everything.
(otherwise (warn-and-give-up)))))
(t
(case target-type
((*)
;; KLUDGE: SBCL has * as an explicit wild type. While this is
;; sort of logical (because (e.g. (ARRAY * 1)) is a valid type)
;; it's not ANSI: looking at the ANSI definitions of complex
;; types like like ARRAY shows that they consider * different
;; from other type names. Someday we should probably get rid of
;; this non-ANSIism in base SBCL, but until we do, we might as
;; well here in the cross compiler. And in order to make sure
;; that we don't continue doing it after we someday patch SBCL's
;; type system so that * is no longer a type, we make this
;; assertion:
(assert (typep (specifier-type '*) 'named-type))
(values t t))
;; Many simple types are guaranteed to correspond exactly between
;; any host ANSI Common Lisp and the target Common Lisp.
((array bit character complex cons float function integer list
nil null number rational real signed-byte string symbol t
unsigned-byte vector)
(values (typep host-object target-type) t))
;; Floating point types are guaranteed to correspond, too, but
;; less exactly.
((single-float double-float)
(cond ((floatp host-object)
(warn-about-possible-float-info-loss)
(values (typep host-object target-type) t))
(t
(values nil t))))
;; Some types require translation between the cross-compilation
;; host Common Lisp and the target SBCL.
(sb!xc:class (values (typep host-object 'sb!xc:class) t))
(fixnum (values (target-fixnump host-object) t))
;; Some types are too hard to handle in the positive case, but at
;; least we can be confident in a large fraction of the negative
;; cases..
((base-string simple-base-string simple-string)
(if (stringp host-object)
(warn-and-give-up)
(values nil t)))
((character base-char)
(cond ((typep host-object 'standard-char)
(values t t))
((not (characterp host-object))
(values nil t))
(t
(warn-and-give-up))))
((stream instance)
;; Neither target CL:STREAM nor target SB!KERNEL:INSTANCE is
;; implemented as a STRUCTURE-OBJECT, so they'll fall through the
;; tests above. We don't want to assume too much about them here,
;; but at least we know enough about them to say that neither T
;; nor NIL nor indeed any other symbol in the cross-compilation
;; host is one. That knowledge suffices to answer so many of the
;; questions that the cross-compiler asks that it's well worth
;; special-casing it here.
(if (symbolp host-object)
(values nil t)
(warn-and-give-up)))
;; And the Common Lisp type system is complicated, and we don't
;; try to implement everything.
(otherwise (warn-and-give-up)))))))
;;; An incomplete TYPEP which runs at cross-compile time to tell whether OBJECT
;;; is the host Lisp representation of a target SBCL type specified by
;;; TARGET-TYPE-SPEC. It need make no pretense to completeness, since it
;;; need only handle the cases which arise when building SBCL itself, e.g.
;;; testing that range limits FOO and BAR in (INTEGER FOO BAR) are INTEGERs.
(defun sb!xc:typep (host-object target-type-spec &optional (env nil env-p))
(declare (ignore env))
(assert (null env-p)) ; 'cause we're too lazy to think about it
(multiple-value-bind (opinion certain-p)
(cross-typep host-object target-type-spec)
;; A program that calls TYPEP doesn't want uncertainty and probably
;; can't handle it.
(if certain-p
opinion
(error "uncertain in SB!XC:TYPEP ~S ~S"
host-object
target-type-spec))))
;;; This implementation is an incomplete, portable version for use at
;;; cross-compile time only.
(defun ctypep (obj ctype)
(check-type ctype ctype)
(let (;; the Common Lisp type specifier corresponding to CTYPE
(type (type-specifier ctype)))
(check-type type (or symbol cons))
(cross-typep obj type)))
(defparameter *universal-function-type*
(make-function-type :wild-args t
:returns *wild-type*))
(defun ctype-of (x)
(typecase x
(function
(if (typep x 'generic-function)
;; Since at cross-compile time we build a CLOS-free bootstrap version of
;; SBCL, it's unclear how to explain to it what a generic function is.
(error "not implemented: cross CTYPE-OF generic function")
;; There's no ANSI way to find out what the function is declared to
;; be, so we just return the CTYPE for the most-general function.
*universal-function-type*))
(symbol
(make-member-type :members (list x)))
(number
(let* ((num (if (complexp x) (realpart x) x))
(res (make-numeric-type
:class (etypecase num
(integer 'integer)
(rational 'rational)
(float 'float))
:format (if (floatp num)
(float-format-name num)
nil))))
(cond ((complexp x)
(setf (numeric-type-complexp res) :complex)
(let ((imag (imagpart x)))
(setf (numeric-type-low res) (min num imag))
(setf (numeric-type-high res) (max num imag))))
(t
(setf (numeric-type-low res) num)
(setf (numeric-type-high res) num)))
res))
(array
(let ((etype (specifier-type (array-element-type x))))
(make-array-type :dimensions (array-dimensions x)
:complexp (not (typep x 'simple-array))
:element-type etype
:specialized-element-type etype)))
(cons (sb!xc:find-class 'cons))
(character
(cond ((typep x 'standard-char)
;; (Note that SBCL doesn't distinguish between BASE-CHAR and
;; CHARACTER.)
(sb!xc:find-class 'base-char))
((not (characterp x))
nil)
(t
;; Beyond this, there seems to be no portable correspondence.
(error "can't map host Lisp CHARACTER ~S to target Lisp" x))))
(structure!object
(sb!xc:find-class (uncross (class-name (class-of x)))))
(t
;; There might be more cases which we could handle with sufficient effort;
;; since all we *need* to handle are enough cases for bootstrapping, we
;; don't try to be complete here. -- WHN 19990512
(error "can't handle ~S in cross CTYPE-OF" x))))

318
src/code/debug-info.lisp Normal file
View file

@ -0,0 +1,318 @@
;;;; structures used for recording debugger information
;;;; This software is part of the SBCL system. See the README file for
;;;; more information.
;;;;
;;;; This software is derived from the CMU CL system, which was
;;;; written at Carnegie Mellon University and released into the
;;;; public domain. The software is in the public domain and is
;;;; provided with absolutely no warranty. See the COPYING and CREDITS
;;;; files for more information.
(in-package "SB!C")
(file-comment
"$Header$")
;;;; SC-OFFSETs
;;;;
;;;; We represent the place where some value is stored with a SC-OFFSET,
;;;; which is the SC number and offset encoded as an integer.
(defconstant sc-offset-scn-byte (byte 5 0))
(defconstant sc-offset-offset-byte (byte 22 5))
(def!type sc-offset () '(unsigned-byte 27))
(defmacro make-sc-offset (scn offset)
`(dpb ,scn sc-offset-scn-byte
(dpb ,offset sc-offset-offset-byte 0)))
(defmacro sc-offset-scn (sco) `(ldb sc-offset-scn-byte ,sco))
(defmacro sc-offset-offset (sco) `(ldb sc-offset-offset-byte ,sco))
;;;; flags for compiled debug variables
;;; FIXME: old CMU CL representation follows:
;;; Compiled debug variables are in a packed binary representation in the
;;; DEBUG-FUNCTION-VARIABLES:
;;; single byte of boolean flags:
;;; uninterned name
;;; packaged name
;;; environment-live
;;; has distinct save location
;;; has ID (name not unique in this fun)
;;; minimal debug-info argument (name generated as ARG-0, ...)
;;; deleted: placeholder for unused minimal argument
;;; [name length in bytes (as var-length integer), if not minimal]
;;; [...name bytes..., if not minimal]
;;; [if packaged, var-length integer that is package name length]
;;; ...package name bytes...]
;;; [If has ID, ID as var-length integer]
;;; SC-Offset of primary location (as var-length integer)
;;; [If has save SC, SC-Offset of save location (as var-length integer)]
;;; FIXME: The first two are no longer used in SBCL.
;;;(defconstant compiled-debug-var-uninterned #b00000001)
;;;(defconstant compiled-debug-var-packaged #b00000010)
(defconstant compiled-debug-var-environment-live #b00000100)
(defconstant compiled-debug-var-save-loc-p #b00001000)
(defconstant compiled-debug-var-id-p #b00010000)
(defconstant compiled-debug-var-minimal-p #b00100000)
(defconstant compiled-debug-var-deleted-p #b01000000)
;;;; compiled debug blocks
;;;;
;;;; Compiled debug blocks are in a packed binary representation in the
;;;; DEBUG-FUNCTION-BLOCKS:
;;;; number of successors + bit flags (single byte)
;;;; elsewhere-p
;;;; ...ordinal number of each successor in the function's blocks vector...
;;;; number of locations in this block
;;;; kind of first location (single byte)
;;;; delta from previous PC (or from 0 if first location in function.)
;;;; [offset of first top-level form, if no function TLF-NUMBER]
;;;; form number of first source form
;;;; first live mask (length in bytes determined by number of VARIABLES)
;;;; ...more <kind, delta, top-level form offset, form-number, live-set>
;;;; tuples...
(defconstant compiled-debug-block-nsucc-byte (byte 2 0))
(defconstant compiled-debug-block-elsewhere-p #b00000100)
(defconstant compiled-code-location-kind-byte (byte 3 0))
(defconstant compiled-code-location-kinds
'#(:unknown-return :known-return :internal-error :non-local-exit
:block-start :call-site :single-value-return :non-local-entry))
;;;; DEBUG-FUNCTION objects
(def!struct (debug-function (:constructor nil)))
(def!struct (compiled-debug-function (:include debug-function)
#-sb-xc-host (:pure t))
;; The name of this function. If from a DEFUN, etc., then this is the
;; function name, otherwise it is a descriptive string.
(name (required-argument) :type (or simple-string cons symbol))
;; The kind of function (same as FUNCTIONAL-KIND):
(kind nil :type (member nil :optional :external :top-level :cleanup))
;; a description of variable locations for this function, in alphabetical
;; order by name; or NIL if no information is available
;;
;; The variable entries are alphabetically ordered. This ordering is used in
;; lifetime info to refer to variables: the first entry is 0, the second
;; entry is 1, etc. Variable numbers are *not* the byte index at which the
;; representation of the location starts.
;;
;; Each entry is:
;; * a FLAGS value, which is a FIXNUM with various
;; COMPILED-DEBUG-FUNCTION-FOO bits set
;; * the symbol which names this variable, unless debug info is minimal
;; * the variable ID, when it has one
;; * SC-offset of primary location, if it has one
;; * SC-offset of save location, if it has one
(variables nil :type (or simple-vector null))
;; A vector of the packed binary representation of the COMPILED-DEBUG-BLOCKs
;; in this function, in the order that the blocks were emitted. The first
;; block is the start of the function. This slot may be NIL to save space.
;;
;; FIXME: The "packed binary representation" description in the comment
;; above is the same as the description of the old representation of
;; VARIABLES which doesn't work properly in SBCL (because it doesn't
;; transform correctly under package renaming). Check whether this slot's
;; data might have the same problem that that slot's data did.
(blocks nil :type (or (simple-array (unsigned-byte 8) (*)) null))
;; If all code locations in this function are in the same top-level form,
;; then this is the number of that form, otherwise NIL. If NIL, then each
;; code location represented in the BLOCKS specifies the TLF number.
(tlf-number nil :type (or index null))
;; A vector describing the variables that the argument values are stored in
;; within this function. The locations are represented by the ordinal number
;; of the entry in the VARIABLES slot value. The locations are in the order
;; that the arguments are actually passed in, but special marker symbols can
;; be interspersed to indicate the original call syntax:
;;
;; DELETED
;; There was an argument to the function in this position, but it was
;; deleted due to lack of references. The value cannot be recovered.
;;
;; SUPPLIED-P
;; The following location is the supplied-p value for the preceding
;; keyword or optional.
;;
;; OPTIONAL-ARGS
;; Indicates that following unqualified args are optionals, not required.
;;
;; REST-ARG
;; The following location holds the list of rest args.
;;
;; MORE-ARG
;; The following two locations are the more arg context and count.
;;
;; <any other symbol>
;; The following location is the value of the keyword argument with the
;; specified name.
;;
;; This may be NIL to save space. If no symbols are present, then this will
;; be represented with an I-vector with sufficiently large element type. If
;; this is :MINIMAL, then this means that the VARIABLES are all required
;; arguments, and are in the order they appear in the VARIABLES vector. In
;; other words, :MINIMAL stands in for a vector where every element holds its
;; index.
(arguments nil :type (or (simple-array * (*)) (member :minimal nil)))
;; There are three alternatives for this slot:
;;
;; A vector
;; A vector of SC-OFFSETS describing the return locations. The
;; vector element type is chosen to hold the largest element.
;;
;; :Standard
;; The function returns using the standard unknown-values convention.
;;
;; :Fixed
;; The function returns using the fixed-values convention, but
;; in order to save space, we elected not to store a vector.
(returns :fixed :type (or (simple-array * (*)) (member :standard :fixed)))
;; SC-Offsets describing where the return PC and return FP are kept.
(return-pc (required-argument) :type sc-offset)
(old-fp (required-argument) :type sc-offset)
;; SC-Offset for the number stack FP in this function, or NIL if no NFP
;; allocated.
(nfp nil :type (or sc-offset null))
;; The earliest PC in this function at which the environment is properly
;; initialized (arguments moved from passing locations, etc.)
(start-pc (required-argument) :type index)
;; The start of elsewhere code for this function (if any.)
(elsewhere-pc (required-argument) :type index))
;;;; minimal debug function
;;; The minimal debug info format compactly represents debug-info for some
;;; cases where the other debug info (variables, blocks) is small enough so
;;; that the per-function overhead becomes relatively large. The minimal
;;; debug-info format can represent any function at level 0, and any fixed-arg
;;; function at level 1.
;;;
;;; In the minimal format, the debug functions and function map are packed into
;;; a single byte-vector which is placed in the
;;; COMPILED-DEBUG-INFO-FUNCTION-MAP. Because of this, all functions in a
;;; component must be representable in minimal format for any function to
;;; actually be dumped in minimal format. The vector is a sequence of records
;;; in this format:
;;; name representation + kind + return convention (single byte)
;;; bit flags (single byte)
;;; setf, nfp, variables
;;; [package name length (as var-length int), if name is packaged]
;;; [...package name bytes, if name is packaged]
;;; [name length (as var-length int), if there is a name]
;;; [...name bytes, if there is a name]
;;; [variables length (as var-length int), if variables flag]
;;; [...bytes holding variable descriptions]
;;; If variables are dumped (level 1), then the variables are all
;;; arguments (in order) with the minimal-arg bit set.
;;; [If returns is specified, then the number of return values]
;;; [...sequence of var-length ints holding sc-offsets of the return
;;; value locations, if fixed return values are specified.]
;;; return-pc location sc-offset (as var-length int)
;;; old-fp location sc-offset (as var-length int)
;;; [nfp location sc-offset (as var-length int), if nfp flag]
;;; code-start-pc (as a var-length int)
;;; This field implicitly encodes start of this function's code in the
;;; function map, as a delta from the previous function's code start.
;;; If the first function in the component, then this is the delta from
;;; 0 (i.e. the absolute offset.)
;;; start-pc (as a var-length int)
;;; This encodes the environment start PC as an offset from the
;;; code-start PC.
;;; elsewhere-pc
;;; This encodes the elsewhere code start for this function, as a delta
;;; from the previous function's elsewhere code start. (i.e. the
;;; encoding is the same as for code-start-pc.)
#|
### For functions with XEPs, name could be represented more simply and
compactly as some sort of info about with how to find the function-entry that
this is a function for. Actually, you really hardly need any info. You can
just chain through the functions in the component until you find the right one.
Well, I guess you need to at least know which function is an XEP for the real
function (which would be useful info anyway).
|#
;;; Following are definitions of bit-fields in the first byte of the minimal
;;; debug function:
(defconstant minimal-debug-function-name-symbol 0)
(defconstant minimal-debug-function-name-packaged 1)
(defconstant minimal-debug-function-name-uninterned 2)
(defconstant minimal-debug-function-name-component 3)
(defconstant minimal-debug-function-name-style-byte (byte 2 0))
(defconstant minimal-debug-function-kind-byte (byte 3 2))
(defconstant minimal-debug-function-kinds
'#(nil :optional :external :top-level :cleanup))
(defconstant minimal-debug-function-returns-standard 0)
(defconstant minimal-debug-function-returns-specified 1)
(defconstant minimal-debug-function-returns-fixed 2)
(defconstant minimal-debug-function-returns-byte (byte 2 5))
;;; The following are bit-flags in the second byte of the minimal debug
;;; function:
;;; If true, wrap (SETF ...) around the name.
(defconstant minimal-debug-function-setf-bit (ash 1 0))
;;; If true, there is a NFP.
(defconstant minimal-debug-function-nfp-bit (ash 1 1))
;;; If true, variables (hence arguments) have been dumped.
(defconstant minimal-debug-function-variables-bit (ash 1 2))
;;;; debug source
(def!struct (debug-source #-sb-xc-host (:pure t))
;; This slot indicates where the definition came from:
;; :File - from a file (Compile-File)
;; :Lisp - from Lisp (Compile)
(from (required-argument) :type (member :file :lisp))
;; If :File, the file name, if :Lisp or :Stream, then a vector of the
;; top-level forms. When from COMPILE, form 0 is #'(LAMBDA ...).
(name nil)
;; File comment for this file, if any.
(comment nil :type (or simple-string null))
;; The universal time that the source was written, or NIL if unavailable.
(created nil :type (or unsigned-byte null))
;; The universal time that the source was compiled.
(compiled (required-argument) :type unsigned-byte)
;; The source path root number of the first form read from this source (i.e.
;; the total number of forms converted previously in this compilation.)
(source-root 0 :type index)
;; The file-positions of each truly top-level form read from this file (if
;; applicable). The vector element type will be chosen to hold the largest
;; element. May be null to save space.
(start-positions nil :type (or (simple-array * (*)) null))
;; If from :LISP, this is the function whose source is form 0.
(info nil))
;;;; DEBUG-INFO structures
(def!struct debug-info
;; Some string describing something about the code in this component.
(name (required-argument) :type simple-string)
;; A list of DEBUG-SOURCE structures describing where the code for this
;; component came from, in the order that they were read.
;;
;; *** NOTE: the offset of this slot is wired into the fasl dumper so that it
;; *** can backpatch the source info when compilation is complete.
(source nil :type list))
(def!struct (compiled-debug-info
(:include debug-info)
#-sb-xc-host (:pure t))
;; a simple-vector of alternating DEBUG-FUNCTION objects and fixnum PCs,
;; used to map PCs to functions, so that we can figure out what function we
;; were running in. Each function is valid between the PC before it
;; (inclusive) and the PC after it (exclusive). The PCs are in sorted order,
;; to allow binary search. We omit the first and last PC, since their values
;; are 0 and the length of the code vector.
;;
;; KLUDGE: PC's can't always be represented by FIXNUMs, unless we're always
;; careful to put our code in low memory. Is that how it works? Would this
;; break if we used a more general memory map? -- WHN 20000120
(function-map (required-argument) :type simple-vector :read-only t))

3694
src/code/debug-int.lisp Normal file
View file

File diff suppressed because it is too large Load diff

95
src/code/debug-var-io.lisp Normal file
View file

@ -0,0 +1,95 @@
;;;; variable-length encoding and other i/o tricks for the debugger
;;;; This software is part of the SBCL system. See the README file for
;;;; more information.
;;;;
;;;; This software is derived from the CMU CL system, which was
;;;; written at Carnegie Mellon University and released into the
;;;; public domain. The software is in the public domain and is
;;;; provided with absolutely no warranty. See the COPYING and CREDITS
;;;; files for more information.
(in-package "SB!C")
(file-comment
"$Header$")
;;;; reading variable length integers
;;;;
;;;; The debug info representation makes extensive use of integers
;;;; encoded in a byte vector using a variable number of bytes:
;;;; 0..253 => the integer
;;;; 254 => read next two bytes for integer
;;;; 255 => read next four bytes for integer
;;; Given a byte vector Vec and an index variable Index, read a variable
;;; length integer and advance index.
(defmacro read-var-integer (vec index)
(once-only ((val `(aref ,vec ,index)))
`(cond ((<= ,val 253)
(incf ,index)
,val)
((= ,val 254)
(prog1
(logior (aref ,vec (+ ,index 1))
(ash (aref ,vec (+ ,index 2)) 8))
(incf ,index 3)))
(t
(prog1
(logior (aref ,vec (+ ,index 1))
(ash (aref ,vec (+ ,index 2)) 8)
(ash (aref ,vec (+ ,index 3)) 16)
(ash (aref ,vec (+ ,index 4)) 24))
(incf ,index 5))))))
;;; Takes an adjustable vector Vec with a fill pointer and pushes the
;;; variable length representation of Int on the end.
(defun write-var-integer (int vec)
(declare (type (unsigned-byte 32) int))
(cond ((<= int 253)
(vector-push-extend int vec))
(t
(let ((32-p (> int #xFFFF)))
(vector-push-extend (if 32-p 255 254) vec)
(vector-push-extend (ldb (byte 8 0) int) vec)
(vector-push-extend (ldb (byte 8 8) int) vec)
(when 32-p
(vector-push-extend (ldb (byte 8 16) int) vec)
(vector-push-extend (ldb (byte 8 24) int) vec)))))
(values))
;;;; packed strings
;;;;
;;;; A packed string is a variable length integer length followed by the
;;;; character codes.
;;; Read a packed string from Vec starting at Index, advancing Index.
(defmacro read-var-string (vec index)
(once-only ((len `(read-var-integer ,vec ,index)))
(once-only ((res `(make-string ,len)))
`(progn
(%primitive byte-blt ,vec ,index ,res 0 ,len)
(incf ,index ,len)
,res))))
;;; Write String into Vec (adjustable, fill-pointer) represented as the
;;; length (in a var-length integer) followed by the codes of the characters.
(defun write-var-string (string vec)
(declare (simple-string string))
(let ((len (length string)))
(write-var-integer len vec)
(dotimes (i len)
(vector-push-extend (char-code (schar string i)) vec)))
(values))
;;;; packed bit vectors
;;; Read the specified number of Bytes out of Vec at Index and convert them
;;; to a bit-vector. Index is incremented.
(defmacro read-packed-bit-vector (bytes vec index)
(once-only ((n-bytes bytes))
(once-only ((n-res `(make-array (* ,n-bytes 8) :element-type 'bit)))
`(progn
(%primitive byte-blt ,vec ,index ,n-res 0 ,n-bytes)
(incf ,index ,n-bytes)
,n-res))))

60
src/code/debug-vm.lisp Normal file
View file

@ -0,0 +1,60 @@
;;;; This is some very low-level support for debugger :FUNCTION-END
;;;; breakpoints.
;;;; This software is part of the SBCL system. See the README file for
;;;; more information.
;;;;
;;;; This software is derived from the CMU CL system, which was
;;;; written at Carnegie Mellon University and released into the
;;;; public domain. The software is in the public domain and is
;;;; provided with absolutely no warranty. See the COPYING and CREDITS
;;;; files for more information.
(in-package "SB!VM")
(file-comment
"$Header$")
(defconstant bogus-lra-constants 2)
(defconstant real-lra-slot (+ code-constants-offset 0))
(defconstant known-return-p-slot (+ code-constants-offset 1))
(defun make-bogus-lra (real-lra &optional known-return-p)
#!+sb-doc
"Make a bogus LRA object that signals a breakpoint trap when returned to. If
the breakpoint trap handler returns to the fake component, the fake code
template returns to real-lra. This returns three values: the bogus LRA
object, the code component it points to, and the pc-offset for the trap
instruction."
(without-gcing
(let* ((src-start (truly-the system-area-pointer
(%primitive foreign-symbol-address
"function_end_breakpoint_guts")))
(src-end (truly-the system-area-pointer
(%primitive foreign-symbol-address
"function_end_breakpoint_end")))
(trap-loc (truly-the system-area-pointer
(%primitive foreign-symbol-address
"function_end_breakpoint_trap")))
(length (sap- src-end src-start))
(code-object (%primitive allocate-code-object
(1+ bogus-lra-constants)
length))
(dst-start (code-instructions code-object)))
(declare (type system-area-pointer src-start src-end dst-start trap-loc)
(type index length))
(setf (code-header-ref code-object code-debug-info-slot) nil)
(setf (code-header-ref code-object code-trace-table-offset-slot) length)
(setf (code-header-ref code-object real-lra-slot) real-lra)
(setf (code-header-ref code-object known-return-p-slot) known-return-p)
(system-area-copy src-start 0 dst-start 0 (* length byte-bits))
(let ((new-lra
(make-lisp-obj (+ (sap-int dst-start) other-pointer-type))))
(sb!kernel:set-header-data new-lra
(logandc2 (+ code-constants-offset
bogus-lra-constants
1)
1))
(values new-lra
code-object
(sap- trap-loc src-start))))))

1518
src/code/debug.lisp Normal file
View file

File diff suppressed because it is too large Load diff

82
src/code/defbangmacro.lisp Normal file
View file

@ -0,0 +1,82 @@
;;;; DEF!MACRO = cold DEFMACRO, a version of DEFMACRO which at
;;;; build-the-cross-compiler time defines its macro both in the
;;;; cross-compilation host Lisp and in the target Lisp. Basically,
;;;; DEF!MACRO does something like
;;;; (DEFMACRO SB!XC:FOO (,@ARGS) (FOO-EXPANDER ,@ARGS))
;;;; #+SB-XC-HOST (SB!XC:DEFMACRO FOO (,@ARGS) (FOO-EXPANDER ,@ARGS))
;;;; an idiom which would otherwise be handwritten repeatedly.
;;;; This software is part of the SBCL system. See the README file for
;;;; more information.
;;;;
;;;; This software is derived from the CMU CL system, which was
;;;; written at Carnegie Mellon University and released into the
;;;; public domain. The software is in the public domain and is
;;;; provided with absolutely no warranty. See the COPYING and CREDITS
;;;; files for more information.
(in-package "SB!IMPL")
(file-comment
"$Header$")
#+sb-xc-host
(progn
;; a description of the DEF!MACRO call to be stored until we get enough
;; of the system running to finish processing it
(defstruct delayed-def!macro
(args (required-argument) :type cons)
(package *package* :type package))
;; a list of DELAYED-DEF!MACROs stored until we get DEF!MACRO working fully
;; so that we can apply it to them. After DEF!MACRO is made to work, this
;; list is processed, and then should no longer be used; it's made unbound in
;; hopes of discouraging any attempt to pushing anything more onto it.
;; (DEF!MACRO knows about this behavior, and uses the unboundness of
;; *DELAYED-DEF!MACROS* as a way to decide to just call SB!XC:DEFMACRO
;; instead of pushing onto *DELAYED-DEF!MACROS*.)
(defvar *delayed-def!macros* nil))
;;; KLUDGE: This is unfortunately somewhat tricky. (A lot of the
;;; cross-compilation-unfriendliness of Common Lisp comes home to roost here.)
(defmacro def!macro (name &rest rest)
#-(or sb-xc-host sb-xc) `(defmacro ,name ,@rest)
#+sb-xc-host `(progn
(defmacro ,name ,@rest)
,(let ((uncrossed-args `(,(uncross name) ,@rest)))
(if (boundp '*delayed-def!macros*)
`(push (make-delayed-def!macro :args ',uncrossed-args)
*delayed-def!macros*)
`(sb!xc:defmacro ,@uncrossed-args))))
;; When cross-compiling, we don't want the DEF!MACRO to have any
;; effect at compile time, because (1) we already defined the macro
;; when building the cross-compiler, so at best it would be redundant
;; and inefficient to replace the current compiled macro body with
;; an interpreted macro body, and (2) because of the various games
;; with SB!XC vs. CL which are played when cross-compiling, we'd
;; be at risk of making an incorrect definition, with something which
;; should be e.g. calling SB!XC:TYPEP instead calling CL:TYPEP
;; and getting all confused. Using an ordinary assignment (and not
;; any special forms like DEFMACRO) guarantees that there are no
;; effects at compile time.
#+sb-xc `(defmacro-mundanely ,name ,@rest))
#+sb-xc-host
(defun force-delayed-def!macros ()
(if (boundp '*delayed-def!macros*)
(progn
(mapcar (lambda (x)
(let ((*package* (delayed-def!macro-package x)))
(eval `(sb!xc:defmacro ,@(delayed-def!macro-args x)))))
(reverse *delayed-def!macros*))
;; We shouldn't need this list any more. Making it unbound serves as a
;; signal to DEF!MACRO that it needn't delayed DEF!MACROs any more.
;; It is also generally a good thing for other reasons: it frees
;; garbage, and it discourages anyone else from pushing anything else
;; onto the list later.
(makunbound '*delayed-def!macros*))
;; This condition is probably harmless if it comes up when
;; interactively experimenting with the system by loading a source
;; file into it more than once. But it's worth warning about it
;; because it definitely shouldn't come up in an ordinary build
;; process.
(warn "*DELAYED-DEF!MACROS* is already unbound.")))

305
src/code/defbangstruct.lisp Normal file
View file

@ -0,0 +1,305 @@
;;;; DEF!STRUCT = bootstrap DEFSTRUCT, a wrapper around DEFSTRUCT which
;;;; provides special features to help at bootstrap time:
;;;; 1. Layout information, inheritance information, and so forth is
;;;; retained in such a way that we can get to it even on vanilla
;;;; ANSI Common Lisp at cross-compiler build time.
;;;; 2. MAKE-LOAD-FORM information is stored in such a way that we can
;;;; get to it at bootstrap time before CLOS is built.
;;;; This software is part of the SBCL system. See the README file for
;;;; more information.
;;;;
;;;; This software is derived from the CMU CL system, which was
;;;; written at Carnegie Mellon University and released into the
;;;; public domain. The software is in the public domain and is
;;;; provided with absolutely no warranty. See the COPYING and CREDITS
;;;; files for more information.
(in-package "SB!KERNEL")
(file-comment
"$Header$")
;;; A bootstrap MAKE-LOAD-FORM method can be a function or the name
;;; of a function.
(deftype def!struct-type-make-load-form-fun () '(or function symbol))
;;; a little single-inheritance system to keep track of MAKE-LOAD-FORM
;;; information for DEF!STRUCT-defined types
(eval-when (#-sb-xc :compile-toplevel :load-toplevel :execute)
;; FIXME: All this could be byte compiled. (Perhaps most of the rest
;; of the file could be, too.)
;; (DEF!STRUCT-SUPERTYPE TYPE) is the DEF!STRUCT-defined type that
;; TYPE inherits from, or NIL if none.
(defvar *def!struct-supertype* (make-hash-table))
(defun def!struct-supertype (type)
(multiple-value-bind (value value-p) (gethash type *def!struct-supertype*)
(unless value-p
(error "~S is not a DEF!STRUCT-defined type." type))
value))
(defun (setf def!struct-supertype) (value type)
(when (and value #-sb-xc-host *type-system-initialized*)
(assert (subtypep value 'structure!object))
(assert (subtypep type value)))
(setf (gethash type *def!struct-supertype*) value))
;; (DEF!STRUCT-TYPE-MAKE-LOAD-FORM-FUN TYPE) is the load form
;; generator associated with the DEF!STRUCT-defined structure named
;; TYPE, stored in a way which works independently of CLOS. The
;; *DEF!STRUCT-TYPE-MAKE-LOAD-FORM-FUN* table is used to store the
;; values. All types defined by DEF!STRUCT have an entry in the
;; table; those with no MAKE-LOAD-FORM function have an explicit NIL
;; entry.
(defvar *def!struct-type-make-load-form-fun* (make-hash-table))
(defun def!struct-type-make-load-form-fun (type)
(do ((supertype type))
(nil)
(multiple-value-bind (value value-p)
(gethash supertype *def!struct-type-make-load-form-fun*)
(unless value-p
(error "~S (supertype of ~S) is not a DEF!STRUCT-defined type."
supertype
type))
(when value
(return value))
(setf supertype (def!struct-supertype supertype))
(unless supertype
(error "There is no MAKE-LOAD-FORM function for bootstrap type ~S."
type)))))
(defun (setf def!struct-type-make-load-form-fun) (new-value type)
(when #+sb-xc-host t #-sb-xc-host *type-system-initialized*
(assert (subtypep type 'structure!object))
(check-type new-value def!struct-type-make-load-form-fun))
(setf (gethash type *def!struct-type-make-load-form-fun*) new-value)))
;;; the simplest, most vanilla MAKE-LOAD-FORM function for DEF!STRUCT
;;; objects
(defun just-dump-it-normally (object &optional (env nil env-p))
(declare (type structure!object object))
(if env-p
(make-load-form-saving-slots object :environment env)
(make-load-form-saving-slots object)))
;;; a MAKE-LOAD-FORM function for objects which don't use the load
;;; form system. This is used for LAYOUT objects because the special
;;; dumping requirements of LAYOUT objects are met by using special
;;; VOPs which bypass the load form system. It's also used for various
;;; compiler internal structures like nodes and VOP-INFO (FIXME:
;;; Why?).
(defun ignore-it (object &optional env)
(declare (type structure!object object))
(declare (ignore object env))
;; This magic tag is handled specially by the compiler downstream.
:ignore-it)
;;; machinery used in the implementation of DEF!STRUCT
#+sb-xc-host
(eval-when (:compile-toplevel :load-toplevel :execute)
;; a description of a DEF!STRUCT call to be stored until we get
;; enough of the system running to finish processing it
(defstruct delayed-def!struct
(args (required-argument) :type cons)
(package *package* :type package))
;; a list of DELAYED-DEF!STRUCTs stored until we get DEF!STRUCT
;; working fully so that we can apply it to them then. After
;; DEF!STRUCT is made to work fully, this list is processed, then
;; made unbound, and should no longer be used.
(defvar *delayed-def!structs* nil))
(eval-when (:compile-toplevel :load-toplevel :execute)
;; Parse the arguments for a DEF!STRUCT call, and return
;; (VALUES NAME DEFSTRUCT-ARGS MAKE-LOAD-FORM-FUN DEF!STRUCT-SUPERTYPE),
;; where NAME is the name of the new type, DEFSTRUCT-ARGS is the
;; munged result suitable for passing on to DEFSTRUCT,
;; MAKE-LOAD-FORM-FUN is the make load form function, or NIL if
;; there's none, and DEF!STRUCT-SUPERTYPE is the direct supertype of
;; the type if it is another DEF!STRUCT-defined type, or NIL
;; otherwise.
(defun parse-def!struct-args (nameoid &rest rest)
(multiple-value-bind (name options) ; Note: OPTIONS can change below.
(if (consp nameoid)
(values (first nameoid) (rest nameoid))
(values nameoid nil))
(let* ((include-clause (find :include options :key #'first))
(def!struct-supertype nil) ; may change below
(mlff-clause (find :make-load-form-fun options :key #'first))
(mlff (and mlff-clause (second mlff-clause))))
(when (find :type options :key #'first)
(error "can't use :TYPE option in DEF!STRUCT"))
(when mlff-clause
(setf options (remove mlff-clause options)))
(when include-clause
(setf def!struct-supertype (second include-clause)))
(if (eq name 'structure!object) ; if root of hierarchy
(assert (not include-clause))
(unless include-clause
(setf def!struct-supertype 'structure!object)
(push `(:include ,def!struct-supertype) options)))
(values name `((,name ,@options) ,@rest) mlff def!struct-supertype)))))
;;; Part of the raison d'etre for DEF!STRUCT is to be able to emulate
;;; these low-level CMU CL functions in a vanilla ANSI Common Lisp
;;; cross compilation host. (The emulation doesn't need to be
;;; efficient, since it's needed for things like dumping objects, not
;;; inner loops.)
#+sb-xc-host
(progn
(defun %instance-length (instance)
(check-type instance structure!object)
(layout-length (class-layout (sb!xc:find-class (type-of instance)))))
(defun %instance-ref (instance index)
(check-type instance structure!object)
(let* ((class (sb!xc:find-class (type-of instance)))
(layout (class-layout class)))
(if (zerop index)
layout
(let* ((dd (layout-info layout))
(dsd (elt (dd-slots dd) (1- index)))
(accessor (dsd-accessor dsd)))
(declare (type symbol accessor))
(funcall accessor instance)))))
(defun %instance-set (instance index new-value)
(check-type instance structure!object)
(let* ((class (sb!xc:find-class (type-of instance)))
(layout (class-layout class)))
(if (zerop index)
(error "can't set %INSTANCE-REF FOO 0 in cross-compilation host")
(let* ((dd (layout-info layout))
(dsd (elt (dd-slots dd) (1- index)))
(accessor (dsd-accessor dsd)))
(declare (type symbol accessor))
(funcall (fdefinition `(setf ,accessor)) new-value instance))))))
;;; a helper function for DEF!STRUCT in the #+SB-XC-HOST case: Return
;;; DEFSTRUCT-style arguments with any class names in the SB!XC
;;; package (i.e. the name of the class being defined, and/or the
;;; names of classes in :INCLUDE clauses) converted from SB!XC::FOO to
;;; CL::FOO.
#+sb-xc-host
(eval-when (:compile-toplevel :load-toplevel :execute)
(defun uncross-defstruct-args (defstruct-args)
(destructuring-bind (name-and-options &rest slots-and-doc) defstruct-args
(multiple-value-bind (name options)
(if (symbolp name-and-options)
(values name-and-options nil)
(values (first name-and-options)
(rest name-and-options)))
(flet ((uncross-option (option)
(if (eq (first option) :include)
(destructuring-bind
(include-keyword included-name &rest rest)
option
`(,include-keyword
,(uncross included-name)
,@rest))
option)))
`((,(uncross name)
,@(mapcar #'uncross-option options))
,@slots-and-doc))))))
;;; DEF!STRUCT's arguments are like DEFSTRUCT's arguments, except that
;;; DEF!STRUCT accepts an extra optional :MAKE-LOAD-FORM-FUN clause.
;;; DEF!STRUCT also does some magic to ensure that anything it defines
;;; includes STRUCTURE!OBJECT, so that when CLOS is/becomes available,
;;; we can hook the DEF!STRUCT system into
;;; (DEFMETHOD MAKE-LOAD-FORM ((X STRUCTURE!OBJECT) &OPTIONAL ENV) ..)
;;; and everything will continue to work.
(defmacro def!struct (&rest args)
(multiple-value-bind (name defstruct-args mlff def!struct-supertype)
(apply #'parse-def!struct-args args)
`(progn
;; (Putting the DEFSTRUCT here, outside the EVAL-WHEN, seems to
;; be necessary in order to cross-compile the hash table
;; implementation. -- WHN 19990809)
(defstruct ,@defstruct-args)
;; (Putting this SETF here, outside the EVAL-WHEN, seems to be
;; necessary in order to allow us to put the DEFSTRUCT outside
;; the EVAL-WHEN.)
(setf (def!struct-type-make-load-form-fun ',name)
,(if (symbolp mlff)
`',mlff
mlff)
(def!struct-supertype ',name)
',def!struct-supertype)
;; This bit of commented-out code hasn't been needed for quite
;; some time, but the comments here about why not might still
;; be useful to me until I finally get the system to work. When
;; I do remove all this, I should be sure also to remove the
;; "outside the EVAL-WHEN" comments above, since they will no
;; longer make sense. -- WHN 19990803
;;(eval-when (:compile-toplevel :load-toplevel :execute)
;; ;; (The DEFSTRUCT used to be in here, but that failed when trying
;; ;; to cross-compile the hash table implementation.)
;; ;;(defstruct ,@defstruct-args)
;; ;; The (SETF (DEF!STRUCT-TYPE-MAKE-LOAD-FORM-FUN ..) ..) used to
;; ;; be in here too, but that failed an assertion in the SETF
;; ;; definition once we moved the DEFSTRUCT outside.)
;; )
#+sb-xc-host ,(let ((u (uncross-defstruct-args defstruct-args)))
(if (boundp '*delayed-def!structs*)
`(push (make-delayed-def!struct :args ',u)
*delayed-def!structs*)
`(sb!xc:defstruct ,@u)))
',name)))
;;; When building the cross-compiler, this function has to be called
;;; some time after SB!XC:DEFSTRUCT is set up, in order to take care
;;; of any processing which had to be delayed until then.
#+sb-xc-host
(defun force-delayed-def!structs ()
(if (boundp '*delayed-def!structs*)
(progn
(mapcar (lambda (x)
(let ((*package* (delayed-def!struct-package x)))
;; KLUDGE(?): EVAL is almost always the wrong thing.
;; However, since we have to map DEFSTRUCT over the
;; list, and since ANSI declined to specify any
;; functional primitives corresponding to the
;; DEFSTRUCT macro, it seems to me that EVAL is
;; required in there somewhere..
(eval `(sb!xc:defstruct ,@(delayed-def!struct-args x)))))
(reverse *delayed-def!structs*))
;; We shouldn't need this list any more. Making it unbound
;; serves as a signal to DEF!STRUCT that it needn't delay
;; DEF!STRUCTs any more. It is also generally a good thing for
;; other reasons: it frees garbage, and it discourages anyone
;; else from pushing anything else onto the list later.
(makunbound '*delayed-def!structs*))
;; This condition is probably harmless if it comes up when
;; interactively experimenting with the system by loading a source
;; file into it more than once. But it's worth warning about it
;; because it definitely shouldn't come up in an ordinary build
;; process.
(warn "*DELAYED-DEF!STRUCTS* is already unbound.")))
;;; The STRUCTURE!OBJECT abstract class is the base of the type
;;; hierarchy for objects which use DEF!STRUCT functionality.
(def!struct (structure!object (:constructor nil)))
;;;; hooking this all into the standard MAKE-LOAD-FORM system
(defun structure!object-make-load-form (object &optional env)
#!+sb-doc
"MAKE-LOAD-FORM for DEF!STRUCT-defined types"
(declare (ignore env))
(funcall (def!struct-type-make-load-form-fun (type-of object))
object))
;;; Do the right thing at cold load time.
;;;
;;; (Eventually this MAKE-LOAD-FORM function be overwritten by CLOS's
;;; generic MAKE-LOAD-FORM, at which time a STRUCTURE!OBJECT method
;;; should be added to call STRUCTURE!OBJECT-MAKE-LOAD-FORM.)
(setf (symbol-function 'sb!xc:make-load-form)
#'structure!object-make-load-form)
;;; Do the right thing in the vanilla ANSI CLOS of the
;;; cross-compilation host. (Something similar will have to be done in
;;; our CLOS, too, but later, some time long after the toplevel forms
;;; of this file have run.)
#+sb-xc-host
(defmethod make-load-form ((obj structure!object) &optional (env nil env-p))
(if env-p
(structure!object-make-load-form obj env)
(structure!object-make-load-form obj)))

59
src/code/defbangtype.lisp Normal file
View file

@ -0,0 +1,59 @@
;;;; This software is part of the SBCL system. See the README file for
;;;; more information.
;;;;
;;;; This software is derived from the CMU CL system, which was
;;;; written at Carnegie Mellon University and released into the
;;;; public domain. The software is in the public domain and is
;;;; provided with absolutely no warranty. See the COPYING and CREDITS
;;;; files for more information.
(in-package "SB!KERNEL")
(file-comment
"$Header$")
;;;; the DEF!TYPE macro
;;; DEF!MACRO = cold DEFTYPE, a version of DEFTYPE which at
;;; build-the-cross-compiler time defines its macro both in the
;;; cross-compilation host Lisp and in the target Lisp. Basically,
;;; DEF!TYPE does something like
;;; (DEFTYPE SB!XC:FOO ..)
;;; #+SB-XC-HOST (SB!XC:DEFTYPE FOO ..)
;;; except that it also automatically delays the SB!XC:DEFTYPE call,
;;; if necessary, until the cross-compiler's DEFTYPE machinery has been
;;; set up.
;;; FIXME: This code was created by cut-and-paste from the
;;; corresponding code for DEF!MACRO. DEF!TYPE and DEF!MACRO are
;;; currently very parallel, and if we ever manage to rationalize the
;;; use of UNCROSS in the cross-compiler, they should become
;;; completely parallel, at which time they should be merged to
;;; eliminate the duplicate code.
(defmacro def!type (&rest rest)
`(progn
(deftype ,@rest)
#+sb-xc-host
,(let ((form `(sb!xc:deftype ,@(uncross rest))))
(if (boundp '*delayed-def!types*)
`(push ',form *delayed-def!types*)
form))))
;;; machinery to implement DEF!TYPE delays
#+sb-xc-host
(progn
(/show "binding *DELAYED-DEF!TYPES*")
(defvar *delayed-def!types* nil)
(/show "done binding *DELAYED-DEF!TYPES*")
(defun force-delayed-def!types ()
(if (boundp '*delayed-def!types*)
(progn
(mapc #'eval *delayed-def!types*)
(makunbound '*delayed-def!types*))
;; This condition is probably harmless if it comes up when
;; interactively experimenting with the system by loading a
;; source file into it more than once. But it's worth warning
;; about it because it definitely shouldn't come up in an
;; ordinary build process.
(warn "*DELAYED-DEF!TYPES* is already unbound."))))

338
src/code/defboot.lisp Normal file
View file

@ -0,0 +1,338 @@
;;;; bootstrapping fundamental machinery (e.g. DEFUN, DEFCONSTANT,
;;;; DEFVAR) from special forms and primitive functions
;;;;
;;;; KLUDGE: The bootstrapping aspect of this is now obsolete. It was
;;;; originally intended that this file file would be loaded into a
;;;; Lisp image which had Common Lisp primitives defined, and DEFMACRO
;;;; defined, and little else. Since then that approach has been
;;;; dropped and this file has been modified somewhat to make it work
;;;; more cleanly when used to predefine macros at
;;;; build-the-cross-compiler time.
;;;; This software is part of the SBCL system. See the README file for
;;;; more information.
;;;;
;;;; This software is derived from the CMU CL system, which was
;;;; written at Carnegie Mellon University and released into the
;;;; public domain. The software is in the public domain and is
;;;; provided with absolutely no warranty. See the COPYING and CREDITS
;;;; files for more information.
(in-package "SB!IMPL")
(file-comment
"$Header$")
;;;; IN-PACKAGE
(defmacro-mundanely in-package (package-designator)
`(eval-when (:compile-toplevel :load-toplevel :execute)
(setq *package* (find-undeleted-package-or-lose ',package-designator))))
;;; MULTIPLE-VALUE-FOO
(defun list-of-symbols-p (x)
(and (listp x)
(every #'symbolp x)))
(defmacro-mundanely multiple-value-bind (vars value-form &body body)
(if (list-of-symbols-p vars)
;; It's unclear why it would be important to special-case the LENGTH=1 case
;; at this level, but the CMU CL code did it, so.. -- WHN 19990411
(if (= (length vars) 1)
`(let ((,(car vars) ,value-form))
,@body)
(let ((ignore (gensym)))
`(multiple-value-call #'(lambda (&optional ,@vars &rest ,ignore)
(declare (ignore ,ignore))
,@body)
,value-form)))
(error "Vars is not a list of symbols: ~S" vars)))
(defmacro-mundanely multiple-value-setq (vars value-form)
(cond ((null vars)
;; The ANSI spec says that the primary value of VALUE-FORM must be
;; returned. The general-case-handling code below doesn't do this
;; correctly in the special case when there are no vars bound, so we
;; handle this special case separately here.
(let ((g (gensym)))
`(multiple-value-bind (,g) ,value-form
,g)))
((list-of-symbols-p vars)
(let ((temps (mapcar #'(lambda (x)
(declare (ignore x))
(gensym)) vars)))
`(multiple-value-bind ,temps ,value-form
,@(mapcar #'(lambda (var temp)
`(setq ,var ,temp))
vars temps)
,(car temps))))
(t (error "Vars is not a list of symbols: ~S" vars))))
(defmacro-mundanely multiple-value-list (value-form)
`(multiple-value-call #'list ,value-form))
;;;; various conditional constructs
;;; COND defined in terms of IF
(defmacro-mundanely cond (&rest clauses)
(if (endp clauses)
nil
(let ((clause (first clauses)))
(if (atom clause)
(error "Cond clause is not a list: ~S" clause)
(let ((test (first clause))
(forms (rest clause)))
(if (endp forms)
(let ((n-result (gensym)))
`(let ((,n-result ,test))
(if ,n-result
,n-result
(cond ,@(rest clauses)))))
`(if ,test
(progn ,@forms)
(cond ,@(rest clauses)))))))))
;;; other things defined in terms of COND
(defmacro-mundanely when (test &body forms)
#!+sb-doc
"First arg is a predicate. If it is non-null, the rest of the forms are
evaluated as a PROGN."
`(cond (,test nil ,@forms)))
(defmacro-mundanely unless (test &body forms)
#!+sb-doc
"First arg is a predicate. If it is null, the rest of the forms are
evaluated as a PROGN."
`(cond ((not ,test) nil ,@forms)))
(defmacro-mundanely and (&rest forms)
(cond ((endp forms) t)
((endp (rest forms)) (first forms))
(t
`(if ,(first forms)
(and ,@(rest forms))
nil))))
(defmacro-mundanely or (&rest forms)
(cond ((endp forms) nil)
((endp (rest forms)) (first forms))
(t
(let ((n-result (gensym)))
`(let ((,n-result ,(first forms)))
(if ,n-result
,n-result
(or ,@(rest forms))))))))
;;;; various sequencing constructs
(defmacro-mundanely prog (varlist &body body-decls)
(multiple-value-bind (body decls) (parse-body body-decls nil)
`(block nil
(let ,varlist
,@decls
(tagbody ,@body)))))
(defmacro-mundanely prog* (varlist &body body-decls)
(multiple-value-bind (body decls) (parse-body body-decls nil)
`(block nil
(let* ,varlist
,@decls
(tagbody ,@body)))))
(defmacro-mundanely prog1 (result &body body)
(let ((n-result (gensym)))
`(let ((,n-result ,result))
,@body
,n-result)))
(defmacro-mundanely prog2 (form1 result &body body)
`(prog1 (progn ,form1 ,result) ,@body))
;;; Now that we have the definition of MULTIPLE-VALUE-BIND, we can make a
;;; reasonably readable definition of DEFUN.
;;;
;;; DEFUN expands into %DEFUN which is a function that is treated
;;; magically by the compiler (through an IR1 transform) in order to
;;; handle stuff like inlining. After the compiler has gotten the
;;; information it wants out of macro definition, it compiles a call
;;; to %%DEFUN which happens at load time.
(defmacro-mundanely defun (&whole whole name args &body body)
(multiple-value-bind (forms decls doc) (parse-body body)
(let ((def `(lambda ,args
,@decls
(block ,(function-name-block-name name)
,@forms))))
`(sb!c::%defun ',name #',def ,doc ',whole))))
#+sb-xc-host (/show "before PROCLAIM" (sb!c::info :function :kind 'sb!c::%%defun))
#+sb-xc-host (sb!xc:proclaim '(ftype function sb!c::%%defun)) ; to avoid
; undefined function warnings
#+sb-xc-host (/show "after PROCLAIM" (sb!c::info :function :kind 'sb!c::%%defun))
(defun sb!c::%%defun (name def doc &optional inline-expansion)
(when (fboundp name)
(style-warn "redefining ~S in DEFUN" name))
(setf (sb!xc:fdefinition name) def)
(when doc
;; FIXME: This should use shared SETF-name parsing logic.
(if (and (consp name) (eq (first name) 'setf))
(setf (fdocumentation (second name) 'setf) doc)
(setf (fdocumentation name 'function) doc)))
(sb!c::proclaim-as-function-name name)
(if (eq (info :function :where-from name) :assumed)
(progn
(setf (info :function :where-from name) :defined)
(if (info :function :assumed-type name)
(setf (info :function :assumed-type name) nil))))
(when (or inline-expansion
(info :function :inline-expansion name))
(setf (info :function :inline-expansion name)
inline-expansion))
name)
;;; Ordinarily this definition of SB!C:%DEFUN as an ordinary function is not
;;; used: the parallel (but different) definition as an IR1 transform takes
;;; precedence. However, it's still good to define this in order to keep the
;;; interpreter happy. We define it here (instead of alongside the parallel
;;; IR1 transform) because while the IR1 transform is needed and appropriate
;;; in the cross-compiler running in the host Common Lisp, this parallel
;;; ordinary function definition is only appropriate in the target Lisp.
(defun sb!c::%defun (name def doc source)
(declare (ignore source))
(setf (sb!eval:interpreted-function-name def) name)
(sb!c::%%defun name def doc))
;;;; DEFVAR and DEFPARAMETER
(defmacro-mundanely defvar (var &optional (val nil valp) (doc nil docp))
#!+sb-doc
"For defining global variables at top level. Declares the variable
SPECIAL and, optionally, initializes it. If the variable already has a
value, the old value is not clobbered. The third argument is an optional
documentation string for the variable."
`(progn
(declaim (special ,var))
,@(when valp
`((unless (boundp ',var)
(setq ,var ,val))))
,@(when docp
`((funcall #'(setf fdocumentation) ',doc ',var 'variable)))
',var))
(defmacro-mundanely defparameter (var val &optional (doc nil docp))
#!+sb-doc
"Defines a parameter that is not normally changed by the program,
but that may be changed without causing an error. Declares the
variable special and sets its value to VAL. The third argument is
an optional documentation string for the parameter."
`(progn
(declaim (special ,var))
(setq ,var ,val)
,@(when docp
;; FIXME: The various FUNCALL #'(SETF FDOCUMENTATION) and
;; other FUNCALL #'(SETF FOO) forms in the code should
;; unbogobootstrapized back to ordinary SETF forms.
`((funcall #'(setf fdocumentation) ',doc ',var 'variable)))
',var))
;;;; iteration constructs
;;; (These macros are defined in terms of a function DO-DO-BODY which is also
;;; used by SB!INT:DO-ANONYMOUS. Since these macros should not be loaded
;;; on the cross-compilation host, but SB!INT:DO-ANONYMOUS and DO-DO-BODY
;;; should be, these macros can't conveniently be in the same file as
;;; DO-DO-BODY.)
(defmacro-mundanely do (varlist endlist &body body)
#!+sb-doc
"DO ({(Var [Init] [Step])}*) (Test Exit-Form*) Declaration* Form*
Iteration construct. Each Var is initialized in parallel to the value of the
specified Init form. On subsequent iterations, the Vars are assigned the
value of the Step form (if any) in parallel. The Test is evaluated before
each evaluation of the body Forms. When the Test is true, the Exit-Forms
are evaluated as a PROGN, with the result being the value of the DO. A block
named NIL is established around the entire expansion, allowing RETURN to be
used as an alternate exit mechanism."
(do-do-body varlist endlist body 'let 'psetq 'do nil))
(defmacro-mundanely do* (varlist endlist &body body)
#!+sb-doc
"DO* ({(Var [Init] [Step])}*) (Test Exit-Form*) Declaration* Form*
Iteration construct. Each Var is initialized sequentially (like LET*) to the
value of the specified Init form. On subsequent iterations, the Vars are
sequentially assigned the value of the Step form (if any). The Test is
evaluated before each evaluation of the body Forms. When the Test is true,
the Exit-Forms are evaluated as a PROGN, with the result being the value
of the DO. A block named NIL is established around the entire expansion,
allowing RETURN to be used as an laternate exit mechanism."
(do-do-body varlist endlist body 'let* 'setq 'do* nil))
;;; DOTIMES and DOLIST could be defined more concisely using destructuring
;;; macro lambda lists or DESTRUCTURING-BIND, but then it'd be tricky to use
;;; them before those things were defined. They're used enough times before
;;; destructuring mechanisms are defined that it looks as though it's worth
;;; just implementing them ASAP, at the cost of being unable to use the
;;; standard destructuring mechanisms.
(defmacro-mundanely dotimes (var-count-result &body body)
(multiple-value-bind ; to roll our own destructuring
(var count result)
(apply (lambda (var count &optional (result nil))
(values var count result))
var-count-result)
(cond ((numberp count)
`(do ((,var 0 (1+ ,var)))
((>= ,var ,count) ,result)
(declare (type unsigned-byte ,var))
,@body))
(t (let ((v1 (gensym)))
`(do ((,var 0 (1+ ,var)) (,v1 ,count))
((>= ,var ,v1) ,result)
(declare (type unsigned-byte ,var))
,@body))))))
(defmacro-mundanely dolist (var-list-result &body body)
(multiple-value-bind ; to roll our own destructuring
(var list result)
(apply (lambda (var list &optional (result nil))
(values var list result))
var-list-result)
;; We repeatedly bind the var instead of setting it so that we never have
;; to give the var an arbitrary value such as NIL (which might conflict
;; with a declaration). If there is a result form, we introduce a
;; gratuitous binding of the variable to NIL w/o the declarations, then
;; evaluate the result form in that environment. We spuriously reference
;; the gratuitous variable, since we don't want to use IGNORABLE on what
;; might be a special var.
(let ((n-list (gensym)))
`(do ((,n-list ,list (cdr ,n-list)))
((endp ,n-list)
,@(if result
`((let ((,var nil))
,var
,result))
'(nil)))
(let ((,var (car ,n-list)))
,@body)))))
;;;; miscellaneous
(defmacro-mundanely return (&optional (value nil))
`(return-from nil ,value))
(defmacro-mundanely psetq (&rest pairs)
#!+sb-doc
"SETQ {var value}*
Set the variables to the values, like SETQ, except that assignments
happen in parallel, i.e. no assignments take place until all the
forms have been evaluated."
;; (This macro is used in the definition of DO, so we can't use DO in the
;; definition of this macro without getting into confusing bootstrap issues.)
(prog ((lets nil)
(setqs nil)
(pairs pairs))
:again
(when (atom (cdr pairs))
(return `(let ,(nreverse lets)
(setq ,@(nreverse setqs))
nil)))
(let ((gen (gensym)))
(setq lets (cons `(,gen ,(cadr pairs)) lets)
setqs (list* gen (car pairs) setqs)
pairs (cddr pairs)))
(go :again)))
(defmacro-mundanely lambda (&whole whole args &body body)
(declare (ignore args body))
`#',whole)

Some files were not shown because too many files have changed in this diff Show more