common-lisp/sbcl.sbcl
1
0
Fork 0
mirror of git://git.code.sf.net/p/sbcl/sbcl synced 2026-09-10 07:26:40 -04:00
Code Issues Actions 8 Packages Projects Releases Wiki Activity

Remove inapplicable documentation

Browse source 
It was explicitly added in 60f84eda5c, which caused me some skepticism
as I saw no merit to keeping deceptive information in-tree. i.e. some of the
internals are not right for SBCL (and possibly not CMUCL any more) such as:

- SBCL executables don't produce three ELF files, there is no linker.sh etc.
- object.tex is more wrong than right. The widetags are completely wrong;
  as are nearly all the descriptions of various headered object types.
  (functions don't contain a pointer to "next function", and they don't embed
  their FTYPE in the raw data portion, etc etc)
- fasl.tex is sufficiently wrong as to be of little utility - the entire point
  of our LEB128-encoding is to remove bifurcation of all fops as it alludes to.
  Not to mention there's no fop-uniform-vector, fop-code-format,
  fop-pop-for-effect, and others.
- foreign-linkage-symbols and linkage-table-data aren't the same at all.
- our INFO database uses <category,kind> for the taxonomy, but CMUCL
  named them <class,type> and said that they are compared via STRING=.
  Neither is more obviously right, so this is just ridiculously confusing.
- The interpreter documentation is flat-out 100% wrong as pertains to SBCL.

All files files removed in this change are obsolete relative to SBCL, but to
revise them in-place would be worse - anyone would wonder whether they document
the current state of SBCL or CMUCL. The right fix would have been to take out
dangling links from our docs - someone can still do that! -  and not to add
explictly incorrect text that increases the amount of stuff you have to
mentally weed out when doing "git grep".

Some "internals" docs are more prosasic and theoretical than detail-oriented,
so it's hard to say whether they're right or wrong. (Is it wrong to postulate
counterfactuals like "it would be nice if x,y,z held ..."?)
I didn't know what to do with Makefile, so I removed it, as well as skeletal
table-of-contents files.
This commit is contained in:
Douglas Katzman 2022-08-27 10:38:51 -04:00
parent 345f14501b
commit e0d0a4579c
11 changed files with 0 additions and 2126 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

10
doc/internals/cmu/.gitignore vendored
View file

@ -1,10 +0,0 @@
/architecture.aux
/compiler.aux
/design.aux
/design.log
/design.out
/design.pdf
/design.toc
/glossary.aux
/retargeting.aux
/run-time.aux

83
doc/internals/cmu/Makefile
View file

@ -1,83 +0,0 @@
LATEX=latex
PDFLATEX=pdflatex
DVIPS=dvips
DVIPSRES=1200
DVIPSOPTS=-t a4 -D $(DVIPSRES)
RM=rm
CP=cp
MV=mv
HTML_SPLIT_LEVEL=2
MASTERFILE=design
FILES=architecture.tex back.tex compiler-overview.tex compiler.tex debugger.tex \
design.tex environment.tex fasl.tex front.tex glossary.tex interface.tex \
interpreter.tex lowlev.tex middle.tex object.tex retargeting.tex \
run-time.tex vm.tex
FIGURES=
all : $(MASTERFILE).pdf
$(MASTERFILE).dvi : $(FILES) $(FIGURES)
$(LATEX) $(MASTERFILE).tex
# $(MASTERFILE).aux $(MASTERFILE).toc : $(FILES) $(FIGURES)
# $(LATEX) $(MASTERFILE).tex
# $(RM) $(MASTERFILE).dvi
# $(MASTERFILE).dvi : $(FILES) $(MASTERFILE).aux $(MASTERFILE).toc $(FIGURES)
# $(LATEX) $(MASTERFILE).tex
# Three times should be enough for everyone(tm).
.PHONY: firsttime
firsttime: $(MASTERFILE).dvi
$(LATEX) $(MASTERFILE).tex
$(LATEX) $(MASTERFILE).tex
.PRECIOUS: %.aux %.toc %.tex
%.dvi : %.tex
$(LATEX) $<
@while ( grep Rerun $*.log > /dev/null ); do \
$(LATEX) $<; \
done
$(LATEX) $<
%.eps : %.fig
fig2dev -L ps $^ $@
%.pdf : %.tex
$(PDFLATEX) $^
@while ( grep 'Rerun to get cross' $*.log > /dev/null ); do \
$(PDFLATEX) $<; \
done
$(PDFLATEX) $<
%.ps : %.dvi
$(DVIPS) $(DVIPSOPTS) -o $@ $*
.PHONY: html
html : $(MASTERFILE).dvi $(FILES) $(FIGURES)
latex2html -local_icons -antialias_text -antialias -scalable_fonts \
-split $(HTML_SPLIT_LEVEL) -transparent -white $(MASTERFILE).tex
clean distclean cvsclean :
-$(RM) -f *.log *.toc *.aux *.dvi *~ *.bak *.ps *.pdf core *.eps
-$(RM) -rf design
.PHONY : allformats
allformats: $(FILES) $(FIGURES)
$(MAKE) firsttime LATEX=pslatex
$(MAKE) design.ps ; $(MV) design.ps design-psfonts.ps
$(MAKE) firsttime
$(MAKE) design.ps DVIPSRES=1200 ; $(MV) design.ps design-1200.ps
$(MAKE) design.ps DVIPSRES=600 ; $(MV) design.ps design-600.ps
$(MAKE) design.ps DVIPSRES=300 ; $(MV) design.ps design-300.ps
$(MAKE) design.pdf
$(MAKE) html HTML_SPLIT_LEVEL=3 ; $(MV) design design-split3
$(MAKE) html HTML_SPLIT_LEVEL=2 ; $(MV) design design-split2
$(MAKE) html HTML_SPLIT_LEVEL=1 ; $(MV) design design-split1

205
doc/internals/cmu/architecture.tex
View file

@ -1,205 +0,0 @@
\part{System Architecture}% -*- Dictionary: int:design -*-
\chapter{Package and File Structure}
\section{Source Tree Structure}
The \cmucl{} source tree has subdirectories for each major subsystem:
\begin{description}
\item[{\tt assembly/}] Holds the CMU CL source-file assembler, and has machine
specific subdirectories holding assembly code for that architecture.
\item[{\tt clx/}] The CLX interface to the X11 window system.
\item[{\tt code/}] The Lisp code for the runtime system and standard CL
utilities.
\item[{\tt compiler/}] The Python compiler. Has architecture-specific
subdirectories which hold backends for different machines. The {\tt generic}
subdirectory holds code that is shared across most backends.
\item[{\tt hemlock/}] The Hemlock editor.
\item[{\tt lisp/}] The C runtime system code and low-level Lisp debugger.
\item[{\tt pcl/}] \cmucl{} version of the PCL implementation of CLOS.
\item[{\tt tools/}] System building command files and source management tools.
\end{description}
\section{Package structure}
Goals: with the single exception of LISP, we want to be able to export from the
package that the code lives in.
\begin{description}
\item[Mach, CLX...] --- These Implementation-dependent system-interface
packages provide direct access to specific features available in the operating
system environment, but hide details of how OS communication is done.
\item[system] contains code that must know about the operating system
environment: I/O, etc. Hides the operating system environment. Provides OS
interface extensions such as {\tt print-directory}, etc.
\item[kernel] hides state and types used for system integration: package
system, error system, streams (?), reader, printer. Also, hides the VM, in
that we don't export anything that reveals the VM interface. Contains code
that needs to use the VM and SYSTEM interface, but is independent of OS and VM
details. This code shouldn't need to be changed in any port of CMU CL, but
won't work when plopped into an arbitrary CL. Uses SYSTEM, VM, EXTENSIONS. We
export "hidden" symbols related to implementation of CL: setf-inverses,
possibly some global variables.
The boundary between KERNEL and VM is fuzzy, but this fuzziness reflects the
fuzziness in the definition of the VM. We can make the VM large, and bring
everything inside, or we can make it small. Obviously, we want the VM to be
as small as possible, subject to efficiency constraints. Pretty much all of
the code in KERNEL could be put in VM. The issue is more what VM hides from
KERNEL: VM knows about everything.
\item[lisp] Originally, this package had all the system code in it. The
current ideal is that this package should have {\it no} code in it, and only
exist to export the standard interface. Note that the name has been changed by
x3j13 to common-lisp.
\item[extensions] contains code that any random user could have written: list
operations, syntactic sugar macros. Uses only LISP, so code in EXTENSIONS is
pure CL. Exports everything defined within that is useful elsewhere. This
package doesn't hide much, so it is relatively safe for users to use
EXTENSIONS, since they aren't getting anything they couldn't have written
themselves. Contrast this to KERNEL, which exports additional operations on
CL's primitive data structures: PACKAGE-INTERNAL-SYMBOL-COUNT, etc. Although
some of the functionality exported from KERNEL could have been defined in CL,
the kernel implementation is much more efficient because it knows about
implementation internals. Currently this package contains only extensions to
CL, but in the ideal scheme of things, it should contain the implementations of
all CL functions that are in KERNEL (the library.)
\item[VM] hides information about the hardware and data structure
representations. Contains all code that knows about this sort of thing: parts
of the compiler, GC, etc. The bulk of the code is the compiler back-end.
Exports useful things that are meaningful across all implementations, such as
operations for examining compiled functions, system constants. Uses COMPILER
and whatever else it wants. Actually, there are different {\it machine}{\tt
-VM} packages for each target implementation. VM is a nickname for whatever
implementation we are currently targeting for.
\item[compiler] hides the algorithms used to map Lisp semantics onto the
operations supplied by the VM. Exports the mechanisms used for defining the
VM. All the VM-independent code in the compiler, partially hiding the compiler
intermediate representations. Uses KERNEL.
\item[eval] holds code that does direct execution of the compiler's ICR. Uses
KERNEL, COMPILER. Exports debugger interface to interpreted code.
\item[debug-internals] presents a reasonable, unified interface to
manipulation of the state of both compiled and interpreted code. (could be in
KERNEL) Uses VM, INTERPRETER, EVAL, KERNEL.
\item[debug] holds the standard debugger, and exports the debugger
\end{description}
\chapter{System Building}
It's actually rather easy to build a CMU CL core with exactly what you want in
it. But to do this you need two things: the source and a working CMU CL.
Basically, you use the working copy of CMU CL to compile the sources,
then run a process call ``genesis'' which builds a ``kernel'' core.
You then load whatever you want into this kernel core, and save it.
In the \verb|tools/| directory in the sources there are several files that
compile everything, and build cores, etc. The first step is to compile the C
startup code.
{\bf Note:} {\it the various scripts mentioned below have hard-wired paths in
them set up for our directory layout here at CMU. Anyone anywhere else will
have to edit them before they will work.}
\section{Compiling the C Startup Code}
There is a circular dependancy between lisp/internals.h and lisp/lisp.map that
causes bootstrapping problems. The easiest way to get around this problem
is to make a fake lisp.nm file that has nothing in it but a version number:
\begin{verbatim}
% echo "Map file for lisp version 0" > lisp.nm
\end{verbatim}
and then run genesis with NIL for the list of files:
\begin{verbatim}
* (load ".../compiler/generic/new-genesis") ; compile before loading
* (lisp::genesis nil ".../lisp/lisp.nm" "/dev/null"
".../lisp/lisp.map" ".../lisp/lisp.h")
\end{verbatim}
It will generate
a whole bunch of warnings about things being undefined, but ignore
that, because it will also generate a correct lisp.h. You can then
compile lisp producing a correct lisp.map:
\begin{verbatim}
% make
\end{verbatim}
and then use \verb|tools/do-worldbuild| and \verb|tools/mk-lisp| to build
\verb|kernel.core| and \verb|lisp.core| (see section \ref{building-cores}.)
\section{Compiling the Lisp Code}
The \verb|tools| directory contains various lisp and C-shell utilities for
building CMU CL:
\begin{description}
\item[compile-all*] Will compile lisp files and build a kernel core. It has
numerous command-line options to control what to compile and how. Try -help to
see a description. It runs a separate Lisp process to compile each
subsystem. Error output is generated in files with ``{\tt .log}'' extension in
the root of the build area.
\item[setup.lisp] Some lisp utilities used for compiling changed files in batch
mode and collecting the error output. Sort of a crude defsystem. Loads into the
``user'' package. See {\tt with-compiler-log-file} and {\tt comf}.
\item[{\it foo}com.lisp] Each system has a ``\verb|.lisp|'' file in
\verb|tools/| which compiles that system.
\end{description}
\section{Building Core Images}
\label{building-cores}
Both the kernel and final core build are normally done using shell script
drivers:
\begin{description}
\item[do-worldbuild*] Builds a kernel core for the current machine. The
version to build is indicated by an optional argument, which defaults to
``alpha''. The \verb|kernel.core| file is written either in the \verb|lisp/|
directory in the build area, or in \verb|/usr/tmp/|. The directory which
already contains \verb|kernel.core| is chosen. You can create a dummy version
with e.g. ``touch'' to select the initial build location.
\item[mk-lisp*] Builds a full core, with conditional loading of subsystems.
The version is the first argument, which defaults to ``alpha''. Any additional
arguments are added to the \verb|*features*| list, which controls system
loading (among other things.) The \verb|lisp.core| file is written in the
current working directory.
\end{description}
These scripts load Lisp command files. When \verb|tools/worldbuild.lisp| is
loaded, it calls genesis with the correct arguments to build a kernel core.
Similarly, \verb|worldload.lisp|
builds a full core. Adding certain symbols to \verb|*features*| before
loading worldload.lisp suppresses loading of different parts of the
system. These symbols are:
\begin{description}
\item[:no-compiler] don't load the compiler.
\item[:no-clx] don't load CLX.
\item[:no-clm] don't load CLM.
\item[:no-hemlock] don't load Hemlock.
\item[:no-pcl] don't load PCL.
\item[:runtime] build a runtime code, implies all of the above, and then some.
\end{description}
Note: if you don't load the compiler, you can't (successfully) load the
pretty-printer or pcl. And if you compiled hemlock with CLX loaded, you can't
load it without CLX also being loaded.
These features are only used during the worldload process; they are
not propagated to the generated \verb|lisp.core| file.

23
doc/internals/cmu/environment.tex
View file

@ -1,23 +0,0 @@
\chapter{The Type System}
\chapter{The Info Database}
The info database provides a functional interface to global
information about named things in \cmucl{}. Information is considered to
be global if it must persist between invocations of the compiler. The
use of a functional interface eliminates the need for the compiler to
worry about the details of the representation. The info database also
handles the need to multiple ``global'' environments, which makes it
possible to change something in the compiler without trashing the
running Lisp environment.
The info database contains arbitrary lisp values, addressed by a
combination of name, class and type. The Name is an EQUAL-thing which
is the name of the thing that we are recording information about.
Class is the kind of object involved: typical classes are Function,
Variable, Type. A type names a particular piece of information within
a given class. Class and Type are symbols, but are compared with
STRING=.

100
doc/internals/cmu/executable.tex
View file

@ -1,100 +0,0 @@
\chapter{Saving Executable Lisp Images}
The :executable flag to SAVE-LISP allows saving the Lisp core sections into an ELF binary that
is executable. To do this, the ELF format is adapted to emulate the old core format.
\section{Saving}
The first part of the save process is almost the same as saving a core-file image except that
instead of a single core image, three ELF object files are created. These are temporary files
called CORRO.o, CORSTA.o and CORDYN.o containing the read-only, static and dynamic spaces.
After that, a script called linker.sh is invoked to use the system linker to create the
executable file. The linker links a file called lisp.a with the three object files. When
SAVE-LISP calls linker.sh, it passes the address of the initial function to be run when lisp is
restarted.
The linker.sh script does several things.
\begin{enumerate}
\item It locates the directory where the C runtime object files are kept.
\item It passes the name of the linker directive script to the linker.
\item It uses the linker to define two symbols: builtin_image_flag and initial_function_addr.
The first symbol has to be the address of some location in memory with non-zero contents
because it is used to tell the system that it is an executable rather than the traditional
runtime loader with core file. This is definitely a kludge but something like this is needed
because it gets around a bootstrapping issue. The second symbol is the startup function;
ordinarily this would be saved in the core file header. It would be possible to save this
information in the ELF file but since we have to use the linker to define builtin_image_flag
anyway, it is simpler to just define initial_function_addr the same way.
\item Finally, the linker script links the executable with the correct set of libraries and
object files. One of these files is lisp.a, containing the object code of the runtime
loader program, which is no longer a separate file.
\end{enumerate}
\section{Running}
The startup procedure for an executable image attempts to emulate the startup procedure for a
core image file by using information saved in the ELF section headers.
When an executable image starts up, it sees the builtin_image_flag that tells it that it is an
executable image. It reads the ELF header of the executable file. It finds the string section
of the file. It then reads ELF section headers, looking for sections with the names CORRO,
CORSTA or CORDYN (it uses the string section to look up the names). Once it finds them, it
mmaps them into the memory space indicated by the parameters in the ELF section header.
Finally it sets variables indicating the starts of the three spaces. It also sets a variable
to indicate how big the dynamic space is.
\section{Porting the Linker Directive Script}
The following describes how to create a linker directive script. It assumes that the system is
based on GCC and the GCC compiler toolchain (in particular, GNU ld).
The linker directive script is used to set up the memory layout for the system. It is a
modified version of the normal linker directive script used by the system.
To obtain a copy of this script (for porting to another operating system), the command ``ld
--verbose'' must be run. This will dump the linker script to standard output. Save this to a
file and edit it as follows.
First, add a ``PHDRS'' part after the SEARCH_DIR part and before the SECTIONS part. The PHDRS
part should look like this:
PHDRS
{
headers PT_PHDR PHDRS ;
interp PT_INTERP ;
text PT_LOAD FILEHDR PHDRS ;
data PT_LOAD ;
dynamic PT_DYNAMIC ;
note PT_NOTE ;
/* Add core sections for Lisp. */
CORRO PT_NOTE ;
CORSTA PT_NOTE ;
CORDYN PT_NOTE ;
}
Note the three CORXXX sections. They are given type PT_NOTE so the system will not try to
process them. An attempt was made to use the PT_LOAD type (and thereby taking advantage of the
ELF loader automatically mmapping the sections into place) but this caused systems to get
confused about the memory layout so that any use of malloc would fail.
Right after the place where the file has something like this:
_end = .;
PROVIDE (end = .);
. = DATA_SEGMENT_END (.);
add the following lines (modified for your system):
/* Lisp core sections. */
CORRO 0x10000000 : { CORRO.o (CORRO) } :CORRO
CORSTA 0x28f00000 : { CORSTA.o (CORSTA) } :CORSTA
CORDYN 0x48000000 : { CORDYN.o (CORDYN) } :CORDYN
Note that the addresses (the second field) must correspond to the addresses of the read-only,
static and dynamic sections given in src/lisp/xxx-validate.h.
This linker directive script will reside in the library: directory along with the linker.sh
script and the lisp.a file created by rebuild-lisp.sh. It should be given a descriptive name
that indicates the system it is used for.

640
doc/internals/cmu/fasl.tex
View file

@ -1,640 +0,0 @@
\chapter{Fasload File Format}% -*- Dictionary: design -*-
\section{General}
The purpose of Fasload files is to allow concise storage and rapid
loading of Lisp data, particularly function definitions. The intent
is that loading a Fasload file has the same effect as loading the
source file from which the Fasload file was compiled, but accomplishes
the tasks more efficiently. One noticeable difference, of course, is
that function definitions may be in compiled form rather than
S-expression form. Another is that Fasload files may specify in what
parts of memory the Lisp data should be allocated. For example,
constant lists used by compiled code may be regarded as read-only.
In some Lisp implementations, Fasload file formats are designed to
allow sharing of code parts of the file, possibly by direct mapping
of pages of the file into the address space of a process. This
technique produces great performance improvements in a paged
time-sharing system. Since the Mach project is to produce a
distributed personal-computer network system rather than a
time-sharing system, efficiencies of this type are explicitly {\it not}
a goal for the CMU Common Lisp Fasload file format.
On the other hand, CMU Common Lisp is intended to be portable, as it will
eventually run on a variety of machines. Therefore an explicit goal
is that Fasload files shall be transportable among various
implementations, to permit efficient distribution of programs in
compiled form. The representations of data objects in Fasload files
shall be relatively independent of such considerations as word
length, number of type bits, and so on. If two implementations
interpret the same macrocode (compiled code format), then Fasload
files should be completely compatible. If they do not, then files
not containing compiled code (so-called ``Fasdump'' data files) should
still be compatible. While this may lead to a format which is not
maximally efficient for a particular implementation, the sacrifice of
a small amount of performance is deemed a worthwhile price to pay to
achieve portability.
The primary assumption about data format compatibility is that all
implementations can support I/O on finite streams of eight-bit bytes.
By ``finite'' we mean that a definite end-of-file point can be detected
irrespective of the content of the data stream. A Fasload file will
be regarded as such a byte stream.
\section{Strategy}
A Fasload file may be regarded as a human-readable prefix followed by
code in a funny little language. When interpreted, this code will
cause the construction of the encoded data structures. The virtual
machine which interprets this code has a {\it stack} and a {\it table},
both initially empty. The table may be thought of as an expandable
register file; it is used to remember quantities which are needed
more than once. The elements of both the stack and the table are
Lisp data objects. Operators of the funny language may take as
operands following bytes of the data stream, or items popped from the
stack. Results may be pushed back onto the stack or pushed onto the
table. The table is an indexable stack that is never popped; it is
indexed relative to the base, not the top, so that an item once
pushed always has the same index.
More precisely, a Fasload file has the following macroscopic
organization. It is a sequence of zero or more groups concatenated
together. End-of-file must occur at the end of the last group. Each
group begins with a series of seven-bit ASCII characters terminated
by one or more bytes of all ones \verb|#xFF|; this is called the
{\it header}. Following the bytes which terminate the header is the
{\it body}, a stream of bytes in the funny binary language. The body
of necessity begins with a byte other than \verb|#xFF|. The body is
terminated by the operation {\tt FOP-END-GROUP}.
The first nine characters of the header must be \verb|FASL FILE| in
upper-case letters. The rest may be any ASCII text, but by
convention it is formatted in a certain way. The header is divided
into lines, which are grouped into paragraphs. A paragraph begins
with a line which does {\it not} begin with a space or tab character,
and contains all lines up to, but not including, the next such line.
The first word of a paragraph, defined to be all characters up to but
not including the first space, tab, or end-of-line character, is the
{\it name} of the paragraph. A Fasload file header might look something like
this:
\begin{verbatim}
FASL FILE >SteelesPerq>User>Guy>IoHacks>Pretty-Print.Slisp
Package Pretty-Print
Compiled 31-Mar-1988 09:01:32 by some random luser
Compiler Version 1.6, Lisp Version 3.0.
Functions: INITIALIZE DRIVER HACK HACK1 MUNGE MUNGE1 GAZORCH
MINGLE MUDDLE PERTURB OVERDRIVE GOBBLE-KEYBOARD
FRY-USER DROP-DEAD HELP CLEAR-MICROCODE
%AOS-TRIANGLE %HARASS-READTABLE-MAYBE
Macros: PUSH POP FROB TWIDDLE
\end{verbatim}
{\it one or more bytes of \verb|#xFF|}
The particular paragraph names and contents shown here are only intended as
suggestions.
\section{Fasload Language}
Each operation in the binary Fasload language is an eight-bit
(one-byte) opcode. Each has a name beginning with ``{\tt FOP-}''. In
the following descriptions, the name is followed by operand
descriptors. Each descriptor denotes operands that follow the opcode
in the input stream. A quantity in parentheses indicates the number
of bytes of data from the stream making up the operand. Operands
which implicitly come from the stack are noted in the text. The
notation ``$\Rightarrow$ stack'' means that the result is pushed onto the
stack; ``$\Rightarrow$ table'' similarly means that the result is added to the
table. A construction like ``{\it n}(1) {\it value}({\it n})'' means that
first a single byte {\it n} is read from the input stream, and this
byte specifies how many bytes to read as the operand named {\it value}.
All numeric values are unsigned binary integers unless otherwise
specified. Values described as ``signed'' are in two's-complement form
unless otherwise specified. When an integer read from the stream
occupies more than one byte, the first byte read is the least
significant byte, and the last byte read is the most significant (and
contains the sign bit as its high-order bit if the entire integer is
signed).
Some of the operations are not necessary, but are rather special
cases of or combinations of others. These are included to reduce the
size of the file or to speed up important cases. As an example,
nearly all strings are less than 256 bytes long, and so a special
form of string operation might take a one-byte length rather than a
four-byte length. As another example, some implementations may
choose to store bits in an array in a left-to-right format within
each word, rather than right-to-left. The Fasload file format may
support both formats, with one being significantly more efficient
than the other for a given implementation. The compiler for any
implementation may generate the more efficient form for that
implementation, and yet compatibility can be maintained by requiring
all implementations to support both formats in Fasload files.
Measurements are to be made to determine which operation codes are
worthwhile; little-used operations may be discarded and new ones
added. After a point the definition will be ``frozen'', meaning that
existing operations may not be deleted (though new ones may be added;
some operations codes will be reserved for that purpose).
\begin{description}
\item[0:] \hspace{2em} {\tt FOP-NOP} \\
No operation. (This is included because it is recognized
that some implementations may benefit from alignment of operands to some
operations, for example to 32-bit boundaries. This operation can be used
to pad the instruction stream to a desired boundary.)
\item[1:] \hspace{2em} {\tt FOP-POP} \hspace{2em} $\Rightarrow$ \hspace{2em} table \\
One item is popped from the stack and added to the table.
\item[2:] \hspace{2em} {\tt FOP-PUSH} \hspace{2em} {\it index}(4) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
Item number {\it index} of the table is pushed onto the stack.
The first element of the table is item number zero.
\item[3:] \hspace{2em} {\tt FOP-BYTE-PUSH} \hspace{2em} {\it index}(1) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
Item number {\it index} of the table is pushed onto the stack.
The first element of the table is item number zero.
\item[4:] \hspace{2em} {\tt FOP-EMPTY-LIST} \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
The empty list ({\tt ()}) is pushed onto the stack.
\item[5:] \hspace{2em} {\tt FOP-TRUTH} \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
The standard truth value ({\tt T}) is pushed onto the stack.
\item[6:] \hspace{2em} {\tt FOP-SYMBOL-SAVE} \hspace{2em} {\it n}(4) \hspace{2em} {\it name}({\it n})
\hspace{2em} $\Rightarrow$ \hspace{2em} stack \& table\\
The four-byte operand {\it n} specifies the length of the print name
of a symbol. The name follows, one character per byte,
with the first byte of the print name being the first read.
The name is interned in the default package,
and the resulting symbol is both pushed onto the stack and added to the table.
\item[7:] \hspace{2em} {\tt FOP-SMALL-SYMBOL-SAVE} \hspace{2em} {\it n}(1) \hspace{2em} {\it name}({\it n}) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \& table\\
The one-byte operand {\it n} specifies the length of the print name
of a symbol. The name follows, one character per byte,
with the first byte of the print name being the first read.
The name is interned in the default package,
and the resulting symbol is both pushed onto the stack and added to the table.
\item[8:] \hspace{2em} {\tt FOP-SYMBOL-IN-PACKAGE-SAVE} \hspace{2em} {\it index}(4)
\hspace{2em} {\it n}(4) \hspace{2em} {\it name}({\it n})
\hspace{2em} $\Rightarrow$ \hspace{2em} stack \& table\\
The four-byte {\it index} specifies a package stored in the table.
The four-byte operand {\it n} specifies the length of the print name
of a symbol. The name follows, one character per byte,
with the first byte of the print name being the first read.
The name is interned in the specified package,
and the resulting symbol is both pushed onto the stack and added to the table.
\item[9:] \hspace{2em} {\tt FOP-SMALL-SYMBOL-IN-PACKAGE-SAVE} \hspace{2em} {\it index}(4)
\hspace{2em} {\it n}(1) \hspace{2em} {\it name}({\it n}) \hspace{2em}
$\Rightarrow$ \hspace{2em} stack \& table\\
The four-byte {\it index} specifies a package stored in the table.
The one-byte operand {\it n} specifies the length of the print name
of a symbol. The name follows, one character per byte,
with the first byte of the print name being the first read.
The name is interned in the specified package,
and the resulting symbol is both pushed onto the stack and added to the table.
\item[10:] \hspace{2em} {\tt FOP-SYMBOL-IN-BYTE-PACKAGE-SAVE} \hspace{2em} {\it index}(1)
\hspace{2em} {\it n}(4) \hspace{2em} {\it name}({\it n})
\hspace{2em} $\Rightarrow$ \hspace{2em} stack \& table\\
The one-byte {\it index} specifies a package stored in the table.
The four-byte operand {\it n} specifies the length of the print name
of a symbol. The name follows, one character per byte,
with the first byte of the print name being the first read.
The name is interned in the specified package,
and the resulting symbol is both pushed onto the stack and added to the table.
\item[11:]\hspace{2em} {\tt FOP-SMALL-SYMBOL-IN-BYTE-PACKAGE-SAVE} \hspace{2em} {\it index}(1)
\hspace{2em} {\it n}(1) \hspace{2em} {\it name}({\it n}) \hspace{2em}
$\Rightarrow$ \hspace{2em} stack \& table\\
The one-byte {\it index} specifies a package stored in the table.
The one-byte operand {\it n} specifies the length of the print name
of a symbol. The name follows, one character per byte,
with the first byte of the print name being the first read.
The name is interned in the specified package,
and the resulting symbol is both pushed onto the stack and added to the table.
\item[12:] \hspace{2em} {\tt FOP-UNINTERNED-SYMBOL-SAVE} \hspace{2em} {\it n}(4) \hspace{2em} {\it name}({\it n})
\hspace{2em} $\Rightarrow$ \hspace{2em} stack \& table\\
Like {\tt FOP-SYMBOL-SAVE}, except that it creates an uninterned symbol.
\item[13:] \hspace{2em} {\tt FOP-UNINTERNED-SMALL-SYMBOL-SAVE} \hspace{2em} {\it n}(1)
\hspace{2em} {\it name}({\it n}) \hspace{2em} $\Rightarrow$ \hspace{2em} stack
\& table\\
Like {\tt FOP-SMALL-SYMBOL-SAVE}, except that it creates an uninterned symbol.
\item[14:] \hspace{2em} {\tt FOP-PACKAGE} \hspace{2em} $\Rightarrow$ \hspace{2em} table \\
An item is popped from the stack; it must be a symbol. The package of
that name is located and pushed onto the table.
\item[15:] \hspace{2em} {\tt FOP-LIST} \hspace{2em} {\it length}(1) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
The unsigned operand {\it length} specifies a number of
operands to be popped from the stack. These are made into a list
of that length, and the list is pushed onto the stack.
The first item popped from the stack becomes the last element of
the list, and so on. Hence an iterative loop can start with
the empty list and perform ``pop an item and cons it onto the list''
{\it length} times.
(Lists of length greater than 255 can be made by using {\tt FOP-LIST*}
repeatedly.)
\item[16:] \hspace{2em} {\tt FOP-LIST*} \hspace{2em} {\it length}(1) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
This is like {\tt FOP-LIST} except that the constructed list is terminated
not by {\tt ()} (the empty list), but by an item popped from the stack
before any others are. Therefore {\it length}+1 items are popped in all.
Hence an iterative loop can start with
a popped item and perform ``pop an item and cons it onto the list''
{\it length}+1 times.
\item[17-24:] \hspace{2em} {\tt FOP-LIST-1}, {\tt FOP-LIST-2}, ..., {\tt FOP-LIST-8} \\
{\tt FOP-LIST-{\it k}} is like {\tt FOP-LIST} with a byte containing {\it k}
following it. These exist purely to reduce the size of Fasload files.
Measurements need to be made to determine the useful values of {\it k}.
\item[25-32:] \hspace{2em} {\tt FOP-LIST*-1}, {\tt FOP-LIST*-2}, ..., {\tt FOP-LIST*-8} \\
{\tt FOP-LIST*-{\it k}} is like {\tt FOP-LIST*} with a byte containing {\it k}
following it. These exist purely to reduce the size of Fasload files.
Measurements need to be made to determine the useful values of {\it k}.
\item[33:] \hspace{2em} {\tt FOP-INTEGER} \hspace{2em} {\it n}(4) \hspace{2em} {\it value}({\it n}) \hspace{2em}
$\Rightarrow$ \hspace{2em} stack \\
A four-byte unsigned operand specifies the number of following
bytes. These bytes define the value of a signed integer in two's-complement
form. The first byte of the value is the least significant byte.
\item[34:] \hspace{2em} {\tt FOP-SMALL-INTEGER} \hspace{2em} {\it n}(1) \hspace{2em} {\it value}({\it n})
\hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
A one-byte unsigned operand specifies the number of following
bytes. These bytes define the value of a signed integer in two's-complement
form. The first byte of the value is the least significant byte.
\item[35:] \hspace{2em} {\tt FOP-WORD-INTEGER} \hspace{2em} {\it value}(4) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
A four-byte signed integer (in the range $-2^{31}$ to $2^{31}-1$) follows the
operation code. A LISP integer (fixnum or bignum) with that value
is constructed and pushed onto the stack.
\item[36:] \hspace{2em} {\tt FOP-BYTE-INTEGER} \hspace{2em} {\it value}(1) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
A one-byte signed integer (in the range -128 to 127) follows the
operation code. A LISP integer (fixnum or bignum) with that value
is constructed and pushed onto the stack.
\item[37:] \hspace{2em} {\tt FOP-STRING} \hspace{2em} {\it n}(4) \hspace{2em} {\it name}({\it n})
\hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
The four-byte operand {\it n} specifies the length of a string to
construct. The characters of the string follow, one per byte.
The constructed string is pushed onto the stack.
\item[38:] \hspace{2em} {\tt FOP-SMALL-STRING} \hspace{2em} {\it n}(1) \hspace{2em} {\it name}({\it n}) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
The one-byte operand {\it n} specifies the length of a string to
construct. The characters of the string follow, one per byte.
The constructed string is pushed onto the stack.
\item[39:] \hspace{2em} {\tt FOP-VECTOR} \hspace{2em} {\it n}(4) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
The four-byte operand {\it n} specifies the length of a vector of LISP objects
to construct. The elements of the vector are popped off the stack;
the first one popped becomes the last element of the vector.
The constructed vector is pushed onto the stack.
\item[40:] \hspace{2em} {\tt FOP-SMALL-VECTOR} \hspace{2em} {\it n}(1) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
The one-byte operand {\it n} specifies the length of a vector of LISP objects
to construct. The elements of the vector are popped off the stack;
the first one popped becomes the last element of the vector.
The constructed vector is pushed onto the stack.
\item[41:] \hspace{2em} {\tt FOP-UNIFORM-VECTOR} \hspace{2em} {\it n}(4) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
The four-byte operand {\it n} specifies the length of a vector of LISP objects
to construct. A single item is popped from the stack and used to initialize
all elements of the vector. The constructed vector is pushed onto the stack.
\item[42:] \hspace{2em} {\tt FOP-SMALL-UNIFORM-VECTOR} \hspace{2em} {\it n}(1) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
The one-byte operand {\it n} specifies the length of a vector of LISP objects
to construct. A single item is popped from the stack and used to initialize
all elements of the vector. The constructed vector is pushed onto the stack.
\item[43:] \hspace{2em} {\tt FOP-INT-VECTOR} \hspace{2em} {\it len}(4) \hspace{2em}
{\it size}(1) \hspace{2em} {\it data}($\left\lceil len*count/8\right\rceil$)
\hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
The four-byte operand {\it n} specifies the length of a vector of
unsigned integers to be constructed. Each integer is {\it size}
bits long, and is packed according to the machine's native byte ordering.
{\it size} must be a directly supported i-vector element size. Currently
supported values are 1,2,4,8,16 and 32.
\item[44:] \hspace{2em} {\tt FOP-UNIFORM-INT-VECTOR} \hspace{2em} {\it n}(4) \hspace{2em} {\it size}(1) \hspace{2em}
{\it value}(@ceiling$<${\it size}/8$>$) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
The four-byte operand {\it n} specifies the length of a vector of unsigned
integers to construct.
Each integer is {\it size} bits big, and is initialized to the value
of the operand {\it value}.
The constructed vector is pushed onto the stack.
\item[45:] \hspace{2em} {\tt FOP-LAYOUT} \hspace{2em} \\
Pops the stack four times to get the name, length, inheritance and depth for a layout object.
\item[46:] \hspace{2em} {\tt FOP-SINGLE-FLOAT} \hspace{2em} {\it data}(4) \hspace{2em}
$\Rightarrow$ \hspace{2em} stack \\
The {\it data} bytes are read as an integer, then turned into an IEEE single
float (as though by {\tt make-single-float}).
\item[47:] \hspace{2em} {\tt FOP-DOUBLE-FLOAT} \hspace{2em} {\it data}(8) \hspace{2em}
$\Rightarrow$ \hspace{2em} stack \\
The {\it data} bytes are read as an integer, then turned into an IEEE double
float (as though by {\tt make-double-float}).
\item[48:] \hspace{2em} {\tt FOP-STRUCT} \hspace{2em} {\it n}(4) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
The four-byte operand {\it n} specifies the length structure to construct. The
elements of the vector are popped off the stack; the first one popped becomes
the last element of the structure. The constructed vector is pushed onto the
stack.
\item[49:] \hspace{2em} {\tt FOP-SMALL-STRUCT} \hspace{2em} {\it n}(1) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
The one-byte operand {\it n} specifies the length structure to construct. The
elements of the vector are popped off the stack; the first one popped becomes
the last element of the structure. The constructed vector is pushed onto the
stack.
\item[50-52:] Unused
\item[53:] \hspace{2em} {\tt FOP-EVAL} \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
Pop an item from the stack and evaluate it (give it to {\tt EVAL}).
Push the result back onto the stack.
\item[54:] \hspace{2em} {\tt FOP-EVAL-FOR-EFFECT} \\
Pop an item from the stack and evaluate it (give it to {\tt EVAL}).
The result is ignored.
\item[55:] \hspace{2em} {\tt FOP-FUNCALL} \hspace{2em} {\it nargs}(1) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
Pop {\it nargs}+1 items from the stack and apply the last one popped
as a function to
all the rest as arguments (the first one popped being the last argument).
Push the result back onto the stack.
\item[56:] \hspace{2em} {\tt FOP-FUNCALL-FOR-EFFECT} \hspace{2em} {\it nargs}(1) \\
Pop {\it nargs}+1 items from the stack and apply the last one popped
as a function to
all the rest as arguments (the first one popped being the last argument).
The result is ignored.
\item[57:] \hspace{2em} {\tt FOP-CODE-FORMAT} \hspace{2em} {\it implementation}(1)
\hspace{2em} {\it version}(1) \\
This FOP specifiers the code format for following code objects. The operations
{\tt FOP-CODE} and its relatives may not occur in a group until after {\tt
FOP-CODE-FORMAT} has appeared; there is no default format. The {\it
implementation} is an integer indicating the target hardware and environment.
See {\tt compiler/generic/vm-macs.lisp} for the currently defined
implementations. {\it version} for an implementation is increased whenever
there is a change that renders old fasl files unusable.
\item[58:] \hspace{2em} {\tt FOP-CODE} \hspace{2em} {\it nitems}(4) \hspace{2em} {\it size}(4) \hspace{2em}
{\it code}({\it size}) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
A compiled function is constructed and pushed onto the stack.
This object is in the format specified by the most recent
occurrence of {\tt FOP-CODE-FORMAT}.
The operand {\it nitems} specifies a number of items to pop off
the stack to use in the ``boxed storage'' section. The operand {\it code}
is a string of bytes constituting the compiled executable code.
\item[59:] \hspace{2em} {\tt FOP-SMALL-CODE} \hspace{2em} {\it nitems}(1) \hspace{2em} {\it size}(2) \hspace{2em}
{\it code}({\it size}) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
A compiled function is constructed and pushed onto the stack.
This object is in the format specified by the most recent
occurrence of {\tt FOP-CODE-FORMAT}.
The operand {\it nitems} specifies a number of items to pop off
the stack to use in the ``boxed storage'' section. The operand {\it code}
is a string of bytes constituting the compiled executable code.
\item[60] \hspace{2em} {\tt FOP-FDEFINITION} \hspace{2em} \\
Pops the stack to get an fdefinition.
\item[61] \hspace{2em} {\tt FOP-SANCTIFY-FOR-EXECUTION} \hspace{2em} \\
A code component is popped from the stack, and the necessary magic is applied
to the code so that it can be executed.
\item[62:] \hspace{2em} {\tt FOP-VERIFY-TABLE-SIZE} \hspace{2em} {\it size}(4) \\
If the current size of the table is not equal to {\it size},
then an inconsistency has been detected. This operation
is inserted into a Fasload file purely for error-checking purposes.
It is good practice for a compiler to output this at least at the
end of every group, if not more often.
\item[63:] \hspace{2em} {\tt FOP-VERIFY-EMPTY-STACK} \\
If the stack is not currently empty,
then an inconsistency has been detected. This operation
is inserted into a Fasload file purely for error-checking purposes.
It is good practice for a compiler to output this at least at the
end of every group, if not more often.
\item[64:] \hspace{2em} {\tt FOP-END-GROUP} \\
This is the last operation of a group. If this is not the
last byte of the file, then a new group follows; the next
nine bytes must be ``{\tt FASL FILE}''.
\item[65:] \hspace{2em} {\tt FOP-POP-FOR-EFFECT} \hspace{2em} stack \hspace{2em} $\Rightarrow$ \hspace{2em} \\
One item is popped from the stack.
\item[66:] \hspace{2em} {\tt FOP-MISC-TRAP} \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
A trap object is pushed onto the stack.
\item[67:] \hspace{2em} {\tt FOP-DOUBLE-DOUBLE-FLOAT} \hspace{2em} {\it double-double-float}(8) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
The next 8 bytes are read, and a double-double-float number is constructed.
\item[68:] \hspace{2em} {\tt FOP-CHARACTER} \hspace{2em} {\it character}(3) \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
The three bytes are read as an integer then converted to a character. This FOP
is currently rather useless, as extended characters are not supported.
\item[69:] \hspace{2em} {\tt FOP-SHORT-CHARACTER} \hspace{2em} {\it character}(1) \hspace{2em}
$\Rightarrow$ \hspace{2em} stack \\
The one byte specifies the code of a Common Lisp character object. A character
is constructed and pushed onto the stack.
\item[70:] \hspace{2em} {\tt FOP-RATIO} \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
Creates a ratio from two integers popped from the stack.
The denominator is popped first, the numerator second.
\item[71:] \hspace{2em} {\tt FOP-COMPLEX} \hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
Creates a complex number from two numbers popped from the stack.
The imaginary part is popped first, the real part second.
\item[72] \hspace{2em} {\tt FOP-COMPLEX-SINGLE-FLOAT} {\it real(4)} {\it imag(4)}\hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
Creates a complex single-float number from the following 8 bytes.
\item[73] \hspace{2em} {\tt FOP-COMPLEX-DOUBLE-FLOAT} {\it real(8)} {\it imag(8)}\hspace{2em} $\Rightarrow$ \hspace{2em} stack \\
Creates a complex double-float number from the following 16 bytes.
\item[74:] \hspace{2em} {\tt FOP-FSET} \hspace{2em} \\
Except in the cold loader (Genesis), this is a no-op with two stack arguments.
In the initial core this is used to make DEFUN functions defined at cold-load
time so that global functions can be called before top-level forms are run
(which normally installs definitions.) Genesis pops the top two things off of
the stack and effectively does (SETF SYMBOL-FUNCTION).
\item[75:] \hspace{2em} {\tt FOP-LISP-SYMBOL-SAVE} \hspace{2em} {\it n}(4) \hspace{2em} {\it name}({\it n})
\hspace{2em} $\Rightarrow$ \hspace{2em} stack \& table\\
Like {\tt FOP-SYMBOL-SAVE}, except that it creates a symbol in the LISP
package.
\item[76:] \hspace{2em} {\tt FOP-LISP-SMALL-SYMBOL-SAVE} \hspace{2em} {\it n}(1)
\hspace{2em} {\it name}({\it n}) \hspace{2em} $\Rightarrow$ \hspace{2em} stack
\& table\\
Like {\tt FOP-SMALL-SYMBOL-SAVE}, except that it creates a symbol in the LISP
package.
\item[77:] \hspace{2em} {\tt FOP-KEYWORD-SYMBOL-SAVE} \hspace{2em} {\it n}(4) \hspace{2em} {\it name}({\it n})
\hspace{2em} $\Rightarrow$ \hspace{2em} stack \& table\\
Like {\tt FOP-SYMBOL-SAVE}, except that it creates a symbol in the
KEYWORD package.
\item[78:] \hspace{2em} {\tt FOP-KEYWORD-SMALL-SYMBOL-SAVE} \hspace{2em} {\it n}(1)
\hspace{2em} {\it name}({\it n}) \hspace{2em} $\Rightarrow$ \hspace{2em} stack
\& table\\
Like {\tt FOP-SMALL-SYMBOL-SAVE}, except that it creates a symbol in the
KEYWORD package.
\item[79-80:] Unused
\item[81:] \hspace{2em} {\tt FOP-NORMAL-LOAD}\\
This FOP is used in conjunction with the cold loader (Genesis) to read
top-level package manipulation forms. These forms are to be read as though by
the normal loaded, so that they can be evaluated at cold load time, instead of
being dumped into the initial core image. A no-op in normal loading.
\item[82:] \hspace{2em} {\tt FOP-MAYBE-COLD-LOAD}\\
Undoes the effect of {\tt FOP-NORMAL-LOAD}.
\item[83:] \hspace{2em} {\tt FOP-ARRAY} \hspace{2em} {\it rank}(4)
\hspace{2em} $\Rightarrow$ \hspace{2em} stack\\
This operation creates a simple array header (used for simple-arrays with rank
/= 1). The data vector is popped off of the stack, and then {\it rank}
dimensions are popped off of the stack (the highest dimensions is on top.)
\item[84:] \hspace{2em} {\tt FOP-SINGLE-FLOAT-VECTOR} \hspace{2em} {\it length}(4) {\it data}(n)
\hspace{2em} $\Rightarrow$ \hspace{2em} stack\\
Creates a {\it (simple-array single-float (*))} object. The number of single-floats is {\it length}.
\item[85:] \hspace{2em} {\tt FOP-DOUBLE-FLOAT-VECTOR} \hspace{2em} {\it length}(4) {\it data}(n)
\hspace{2em} $\Rightarrow$ \hspace{2em} stack\\
Creates a {\it (simple-array double-float (*))} object. The number of double-floats is {\it length}.
\item[86:] \hspace{2em} {\tt FOP-COMPLEX-SINGLE-FLOAT-VECTOR} \hspace{2em} {\it length}(4) {\it data}(n)
\hspace{2em} $\Rightarrow$ \hspace{2em} stack\\
Creates a {\it (simple-array (complex single-float) (*))} object. The number of complex single-floats is {\it length}.
\item[87:] \hspace{2em} {\tt FOP-COMPLEX-DOUBLE-FLOAT-VECTOR} \hspace{2em} {\it length}(4) {\it data}(n)
\hspace{2em} $\Rightarrow$ \hspace{2em} stack\\
Creates a {\it (simple-array (complex double-float) (*))} object. The number of complex double-floats is {\it length}.
\item[88:] \hspace{2em} {\tt FOP-DOUBLE-DOUBLE-FLOAT-VECTOR} \hspace{2em} {\it length}(4) {\it data}(n)
\hspace{2em} $\Rightarrow$ \hspace{2em} stack\\
Creates a {\it (simple-array double-double-float (*))} object. The number of double-double-floats is {\it length}.
\item[89:] \hspace{2em} {\tt FOP-COMPLEX-DOUBLE-DOUBLE-FLOAT} \hspace{2em} {\it data}(32)
\hspace{2em} $\Rightarrow$ \hspace{2em} stack\\
Creates a {\it (complex double-double-float)} object from the following 32 bytes of data.
\item[90:] \hspace{2em} {\tt FOP-COMPLEX-DOUBLE-DOUBLE-FLOAT-VECTOR} \hspace{2em} {\it length}(4) {\it data}(n)
\hspace{2em} $\Rightarrow$ \hspace{2em} stack\\
Creates a {\it (simple-arra (complex double-double-float) (*))} object. The number of complex double-double-floats is {\it length}.
\item[91-139:] Unused
\item[140:] \hspace{2em} {\tt FOP-ALTER-CODE} \hspace{2em} {\it index}(4)\\
This operation modifies the constants part of a code object (necessary for
creating certain circular function references.) It pops the new value and code
object are off of the stack, storing the new value at the specified index.
\item[141:] \hspace{2em} {\tt FOP-BYTE-ALTER-CODE} \hspace{2em} {\it index}(1)\\
Like {\tt FOP-ALTER-CODE}, but has only a one byte offset.
\item[142:] \hspace{2em} {\tt FOP-FUNCTION-ENTRY} \hspace{2em} {\it index}(4)
\hspace{2em} $\Rightarrow$ \hspace{2em} stack\\
Initializes a function-entry header inside of a pre-existing code object, and
returns the corresponding function descriptor. {\it index} is the byte offset
inside of the code object where the header should be plunked down. The stack
arguments to this operation are the code object, function name, function debug
arglist and function type.
\item[143:] \hspace{2em} {\tt FOP-MAKE-BYTE-COMPILED-FUNCTION} \hspace{2em} {\it size}(1) \hspace{2em} $\Rightarrow$ \hspace{2em} stack\\
Create a byte-compiled function. {\it FIXME:} describe what's on the stack.
\item[144:] \hspace{2em} {\tt FOP-ASSEMBLER-CODE} \hspace{2em} {\it length}(4)
\hspace{2em} $\Rightarrow$ \hspace{2em} stack\\
This operation creates a code object holding assembly routines. {\it length}
bytes of code are read and placed in the code object, and the code object
descriptor is pushed on the stack. This FOP is only recognized by the cold
loader (Genesis.)
\item[145:] \hspace{2em} {\tt FOP-ASSEMBLER-ROUTINE} \hspace{2em} {\it offset}(4)
\hspace{2em} $\Rightarrow$ \hspace{2em} stack\\
This operation records an entry point into an assembler code object (for use
with {\tt FOP-ASSEMBLER-FIXUP}). The routine name (a symbol) is on stack top.
The code object is underneath. The entry point is defined at {\it offset}
bytes inside the code area of the code object, and the code object is left on
stack top (allowing multiple uses of this FOP to be chained.) This FOP is only
recognized by the cold loader (Genesis.)
\item[146:] Unused
\item[147:] \hspace{2em} {\tt FOP-FOREIGN-FIXUP} \hspace{2em} {\it len}(1)
\hspace{2em} {\it name}({\it len})
\hspace{2em} {\it offset}(4) \hspace{2em} $\Rightarrow$ \hspace{2em} stack\\
This operation resolves a reference to a foreign (C) symbol. {\it len} bytes
are read and interpreted as the symbol {\it name}. First the {\it kind} and the
code-object to patch are popped from the stack. The kind is a target-dependent
symbol indicating the instruction format of the patch target (at {\it offset}
bytes from the start of the code area.) The code object is left on
stack top (allowing multiple uses of this FOP to be chained.)
\item[148:] \hspace{2em} {\tt FOP-ASSEMBLER-FIXUP} \hspace{2em} {\it offset}(4)
\hspace{2em} $\Rightarrow$ \hspace{2em} stack\\
This operation resolves a reference to an assembler routine. The stack args
are ({\it routine-name}, {\it kind} and {\it code-object}). The kind is a
target-dependent symbol indicating the instruction format of the patch target
(at {\it offset} bytes from the start of the code area.) The code object is
left on stack top (allowing multiple uses of this FOP to be chained.)
\item[149:] \hspace{2em} {\tt FOP-CODE-OBJECT-FIXUP}
\hspace{2em} $\Rightarrow$ \hspace{2em} stack\\
{\it FIXME:} Describe what this does!
\item[150:] \hspace{2em} {\tt FOP-FOREIGN-DATA-FIXUP}
\hspace{2em} $\Rightarrow$ \hspace{2em} stack\\
{\it FIXME:} Describe what this does!
\item[151-156:] Unused
\item[157:] \hspace{2em} {\tt FOP-LONG-CODE-FORMAT} \hspace{2em} {\it implementation}(1)
\hspace{2em} {\it version}(4) \\
Like FOP-CODE-FORMAT, except that the version is 32 bits long.
\item[158-199:] Unused
\item[200:] \hspace{2em} {\tt FOP-RPLACA} \hspace{2em} {\it table-idx}(4)
\hspace{2em} {\it cdr-offset}(4)\\
\item[201:] \hspace{2em} {\tt FOP-RPLACD} \hspace{2em} {\it table-idx}(4)
\hspace{2em} {\it cdr-offset}(4)\\
These operations destructively modify a list entered in the table. {\it
table-idx} is the table entry holding the list, and {\it cdr-offset} designates
the cons in the list to modify (like the argument to {\tt nthcdr}.) The new
value is popped off of the stack, and stored in the {\tt car} or {\tt cdr},
respectively.
\item[202:] \hspace{2em} {\tt FOP-SVSET} \hspace{2em} {\it table-idx}(4)
\hspace{2em} {\it vector-idx}(4)\\
Destructively modifies a {\tt simple-vector} entered in the table. Pops the
new value off of the stack, and stores it in the {\it vector-idx} element of
the contents of the table entry {\it table-idx.}
\item[203:] \hspace{2em} {\tt FOP-NTHCDR} \hspace{2em} {\it cdr-offset}(4)
\hspace{2em} $\Rightarrow$ \hspace{2em} stack\\
Does {\tt nthcdr} on the top-of stack, leaving the result there.
\item[204:] \hspace{2em} {\tt FOP-STRUCTSET} \hspace{2em} {\it table-idx}(4)
\hspace{2em} {\it vector-idx}(4)\\
Like {\tt FOP-SVSET}, except it alters structure slots.
\item[205-254:] Unused
\item[255:] \hspace{2em} {\tt FOP-END-HEADER} \\ Indicates the end of a group header,
as described above.
\end{description}

6
doc/internals/cmu/interface.tex
View file

@ -1,6 +0,0 @@
\chapter{User Interface of the Compiler}
\section{Error Message Utilities}
\section{Source Paths}
\label{source-paths}

196
doc/internals/cmu/interpreter.tex
View file

@ -1,196 +0,0 @@
% -*- Dictionary: design; Package: C -*-
\chapter{The IR1 Interpreter}
May be worth having a byte-code representation for interpreted code. This way,
an entire system could be compiled into byte-code for debugging (the
``check-out'' compiler?).
Given our current inclination for using a stack machine to interpret IR1, it
would be straightforward to layer a byte-code interpreter on top of this.
Instead of having no interpreter, or a more-or-less conventional interpreter,
or byte-code interpreter, how about directly executing IR1?
We run through the IR1 passes, possibly skipping optional ones, until we get
through environment analysis. Then we run a post-pass that annotates IR1 with
information about where values are kept, i.e. the stack slot.
We can lazily convert functions by having FUNCTION make an interpreted function
object that holds the code (really a closure over the interpreter). The first
time that we try to call the function, we do the conversion and processing.
Also, we can easily keep track of which interpreted functions we have expanded
macros in, so that macro redefinition automatically invalidates the old
expansion, causing lazy reconversion.
Probably the interpreter will want to represent MVs by a recognizable structure
that is always heap-allocated. This way, we can punt the stack issues involved
in trying to spread MVs. So a continuation value can always be kept in a
single cell.
The compiler can have some special frobs for making the interpreter efficient,
such as a call operation that extracts arguments from the stack
slots designated by a continuation list. Perhaps
\begin{verbatim}
(values-mapcar fun . lists)
<==>
(values-list (mapcar fun . lists))
\end{verbatim}
This would be used with MV-CALL.
This scheme seems to provide nearly all of the advantages of both the compiler
and conventional interpretation. The only significant disadvantage with
respect to a conventional interpreter is that there is the one-time overhead of
conversion, but doing this lazily should make this quite acceptable.
With respect to a conventional interpreter, we have major advantages:
+ Full syntax checking: safety comparable to compiled code.
+ Semantics similar to compiled code due to code sharing. Similar diagnostic
messages, etc. Reduction of error-prone code duplication.
+ Potential for full type checking according to declarations (would require
running IR1 optimize?)
+ Simplifies debugger interface, since interpreted code can look more like
compiled code: source paths, edit definition, etc.
For all non-run-time symbol annotations (anything other than SYMBOL-FUNCTION
and SYMBOL-VALUE), we use the compiler's global database. MACRO-FUNCTION will
use INFO, rather than vice-versa.
When doing the IR1 phases for the interpreter, we probably want to suppress
optimizations that change user-visible function calls:
-- Don't do local call conversion of any named functions (even lexical ones).
This is so that a call will appear on the stack that looks like the call in
the original source. The keyword and optional argument transformations
done by local call mangle things quite a bit. Also, note local-call
converting prevents unreferenced arguments from being deleted, which is
another non-obvious transformation.
-- Don't run source-transforms, IR1 transforms and IR1 optimizers. This way,
TRACE and BACKTRACE will show calls with the original arguments, rather
than the ``optimized'' form, etc. Also, for the interpreter it will
actually be faster to call the original function (which is compiled) than
to ``inline expand'' it. Also, this allows implementation-dependent
transforms to expand into %PRIMITIVE uses.
There are some problems with stepping, due to our non-syntactic IR1
representation. The source path information is the key that makes this
conceivable. We can skip over the stepping of a subform by quietly evaluating
nodes whose source path lies within the form being skipped.
One problem with determining what value has been returned by a form. With a
function call, it is theoretically possible to precisely determine this, since
if we complete evaluation of the arguments, then we arrive at the Combination
node whose value is synonymous with the value of the form. We can even detect
this case, since the Node-Source will be EQ to the form. And we can also
detect when we unwind out of the evaluation, since we will leave the form
without having ever reached this node.
But with macros and special-forms, there is no node whose value is the value of
the form, and no node whose source is the macro call or special form. We can
still detect when we leave the form, but we can't be sure whether this was a
normal evaluation result or an explicit RETURN-FROM.
But does this really matter? It seems that we can print the value returned (if
any), then just print the next form to step. In the rare case where we did
unwind, the user should be able to figure it out.
[We can look at this as a side-effect of CPS: there isn't any difference
between a ``normal'' return and a non-local one.]
[Note that in any control transfer (normal or otherwise), the stepper may need
to unwind out of an arbitrary number of levels of stepping. This is because a
form in a TR position may yield its to a node arbitrarily far out.]
Another problem is with deciding what form is being stepped. When we start
evaluating a node, we dive into code that is nested somewhere down inside that
form. So we actually have to do a loop of asking questions before we do any
evaluation. But what do we ask about?
If we ask about the outermost enclosing form that is a subform of the last
form that the user said to execute, then we might offer a form that isn't
really evaluated, such as a LET binding list.
But once again, is this really a problem? It is certainly different from a
conventional stepper, but a pretty good argument could be made that it is
superior. Haven't you ever wanted to skip the evaluation of all the
LET bindings, but not the body? Wouldn't it be useful to be able to skip the
DO step forms?
All of this assumes that nobody ever wants to step through the guts of a
macroexpansion. This seems reasonable, since steppers are for weenies, and
weenies don't define macros (hence don't debug them). But there are probably
some weenies who don't know that they shouldn't be writing macros.
We could handle this by finding the ``source paths'' in the expansion of each
macro by sticking some special frob in the source path marking the place where
the expansion happened. When we hit code again that is in the source, then we
revert to the normal source path. Something along these lines might be a good
idea anyway (for compiler error messages, for example).
The source path hack isn't guaranteed to work quite so well in generated code,
though, since macros return stuff that isn't freshly consed. But we could
probably arrange to win as long as any given expansion doesn't return two EQ
forms.
It might be nice to have a command that skipped stepping of the form, but
printed the results of each outermost enclosed evaluated subform, i.e. if you
used this on the DO step-list, it would print the result of each new-value
form. I think this is implementable. I guess what you would do is print each
value delivered to a DEST whose source form is the current or an enclosing
form. Along with the value, you would print the source form for the node that
is computing the value.
The stepper can also have a ``back'' command that ``unskips'' or ``unsteps''. This
would allow the evaluation of forms that are pure (modulo lexical variable
setting) to be undone. This is useful, since in stepping it is common that you
skip a form that you shouldn't have, or get confused and want to restart at
some earlier point.
What we would do is remember the current node and the values of all local
variables. heap before doing each step or skip action. We can then back up
the state of all lexical variables and the ``program counter''. To make this
work right with set closure variables, we would copy the cell's value, rather
than the value cell itself.
[To be fair, note that this could easily be done with our current interpreter:
the stepper could copy the environment alists.]
We can't back up the ``program counter'' when a control transfer leaves the
current function, since this state is implicitly represented in the
interpreter's state, and is discarded when we exit. We probably want to ask
for confirmation before leaving the function to give users a chance to ``unskip''
the forms in a TR position.
Another question is whether the conventional stepper is really a good thing to
imitate... How about an editor-based mouse-driven interface? Instead of
``skipping'' and ``stepping'', you would just designate the next form that you
wanted to stop at. Instead of displaying return values, you replace the source
text with the printed representation of the value.
It would show the ``program counter'' by highlighting the *innermost* form that
we are about to evaluate, i.e. the source form for the node that we are stopped
at. It would probably also be useful to display the start of the form that was
used to designate the next stopping point, although I guess this could be
implied by the mouse position.
Such an interface would be a little harder to implement than a dumb stepper,
but it would be much easier to use. [It would be impossible for an evalhook
stepper to do this.]
\section{Use of \%PRIMITIVE}
Note: \verb|%PRIMITIVE| can only be used in compiled code. It is a
trapdoor into the compiler, not a general syntax for accessing
``sub-primitives''. It's main use is in implementation-dependent
compiler transforms. It saves us the effort of defining a ``phony
function'' (that is not really defined), and also allows direct
communication with the code generator through codegen-info arguments.
Some primitives may be exported from the VM so that \verb|%PRIMITIVE|
can be used to make it explicit that an escape routine or interpreter
stub is assuming an operation is implemented by the compiler.

85
doc/internals/cmu/lowlev.tex
View file

@ -1,85 +0,0 @@
\chapter{Memory Management}
\section{Stacks and Globals}
\section{Heap Layout}
\section{Garbage Collection}
\chapter{Interface to C and Assembler}
\section{Linkage Table}
The linkage table feature is based on how dynamic libraries dispatch.
A table of functions is used which is filled in with the appropriate
code to jump to the correct address.
For \cmucl{}, this table is stored at
\code{target-foreign-linkage-space-start}. Each entry is
\code{target-foreign-linkage-entry-size} bytes long.
At startup, the table is initialized with default values in
\code{os\_foreign\_linkage\_init}. On x86 platforms, the first entry is
code to call the routine \code{resolve\_linkage\_tramp}. All other
entries jump to the first entry. The function
\code{resolve\_linkage\_tramp} looks at where it was called from to
figure out which entry in the table was used. It calls
\code{lazy\_resolve\_linkage} with the address of the linkage entry.
This routine then fills in the appropriate linkage entry with code to
jump to where the real routine is located, and returns the address of
the entry. On return, \code{resolve\_linkage\_tramp} then just jumps to
the returned address to call the desired function. On all subsequent
calls, the entry no longer points to \code{resolve\_linkage\_tramp} but
to the real function.
This describes how function calls are made. For foreign data,
\code{lazy\_resolve\_linkage} stuffs the address of the actual foreign
data into the linkage table. The lisp code then just loads the address
from there to get the actual address of the foreign data.
For sparc, the linkage table is slightly different. The first entry is
the entry for \code{call\_into\_c} so we never have to look this up. All
other entries are for \code{resolve\_linkage\_tramp}. This has the
advantage that \code{resolve\_linkage\_tramp} can be much simpler since
all calls to foreign code go through \code{call\_into\_c} anyway, and
that means all live Lisp registers have already been saved. Also, to
make life simpler, we lie about \code{closure\_tramp} and
\code{undefined\_tramp} in the Lisp code. These are really functions,
but we treat them as foreign data since these two routines are only
used as addresses in the Lisp code to stuff into a lisp function
header.
On the Lisp side, there are two supporting data structures for the
linkage table: \code{*linkage-table-data*} and
\code{*foreign-linkage-symbols*}. The latter is a hash table whose key
is the foreign symbol (a string) and whose value is an index into
\code{*linkage-table-data*}.
\code{*linkage-table-data*} is a vector with an unlispy layout. Each
entry has 3 parts:
\begin{itemize}
\item symbol name
\item type, a fixnum, 1 = code, 2 = data
\item library list - the library list at the time the symbol is registered.
\end{itemize}
Whenever a new foreign symbol is defined, a new
\code{*linkage-table-data*} entry is created.
\code{*foreign-linkage-symbols*} is updated with the symbol and the
entry number into \code{*linkage-table-data*}.
The \code{*linkage-table-data*} is accessed from C (hence the unlispy
layout), to figure out the symbol name and the type so that the
address of the symbol can be determined. The type tells the C code
how to fill in the entry in the linkage-table itself.
% (Should say something about genesis too, but I don't know how that
% works other than the initial table is setup with the appropriate first
% entry.)
\chapter{Low-level debugging}
\chapter{Core File Format}

770
doc/internals/cmu/object.tex
View file

@ -1,770 +0,0 @@
\chapter{Object Format}
\label{sec:tagging}
\section{Tagging}
The following is a key of the three bit low-tagging scheme:
\begin{description}
\item[000] even fixnum
\item[001] function pointer
\item[010] even other-immediate (header-words, characters, symbol-value trap value, etc.)
\item[011] list pointer
\item[100] odd fixnum
\item[101] structure pointer
\item[110] odd other immediate
\item[111] other-pointer to data-blocks (other than conses, structures,
and functions)
\end{description}
This tagging scheme forces a dual-word alignment of data-blocks on the heap,
but this can be pretty negligible:
\begin{itemize}
\item RATIOS and COMPLEX must have a header-word anyway since they are not a
major type. This wastes one word for these infrequent data-blocks since
they require two words for the data.
\item BIGNUMS must have a header-word and probably contain only one other word
anyway, so we probably don't waste any words here. Most bignums just
barely overflow fixnums, that is by a bit or two.
\item Single and double FLOATS?
no waste, or
one word wasted
\item SYMBOLS have a pad slot (current called the setf function, but unused.)
\end{itemize}
Everything else is vector-like including code, so these probably take up
so many words that one extra one doesn't matter.
\section{GC Comments}
Data-Blocks comprise only descriptors, or they contain immediate data and raw
bits interpreted by the system. GC must skip the latter when scanning the
heap, so it does not look at a word of raw bits and interpret it as a pointer
descriptor. These data-blocks require headers for GC as well as for operations
that need to know how to interpret the raw bits. When GC is scanning, and it
sees a header-word, then it can determine how to skip that data-block if
necessary. Header-Words are tagged as other-immediates. See
``Other-Immediates'', section~\ref{sec:other-immediates} and
``Data-Blocks and Header-Words'', section~\ref{sec:data-blocks-and-header} for comments on
distinguishing header-words from other-immediate data. This distinction is
necessary since we scan through data-blocks containing only descriptors just as
we scan through the heap looking for header-words introducing data-blocks.
Data-Blocks containing only descriptors do not require header-words for GC
since the entire data-block can be scanned by GC a word at a time, taking
whatever action is necessary or appropriate for the data in that slot. For
example, a cons is referenced by a descriptor with a specific tag, and the
system always knows the size of this data-block. When GC encounters a pointer
to a cons, it can transport it into the new space, and when scanning, it can
simply scan the two words manifesting the cons interpreting each word as a
descriptor. Actually there is no cons tag, but a list tag, so we make sure the
cons is not nil when appropriate. A header may still be desired if the pointer
to the data-block does not contain enough information to adequately maintain
the data-block. An example of this is a simple-vector containing only
descriptor slots, and we attach a header-word because the descriptor pointing
to the vector lacks necessary information -- the type of the vector's elements,
its length, etc.
There is no need for a major tag for GC forwarding pointers. Since the tag
bits are in the low end of the word, a range check on the start and end of old
space tells you if you need to move the thing. This is all GC overhead.
\section{Structures}
A structure descriptor has the structure lowtag type code, making
{\tt structurep} a fast operation. A structure
data-block has the following format:
\begin{verbatim}
-------------------------------------------------------
| length (24 bits) | Structure header type (8 bits) |
-------------------------------------------------------
| structure type name (a symbol) |
-------------------------------------------------------
| structure slot 0 |
-------------------------------------------------------
| ... structure slot length - 2 |
-------------------------------------------------------
\end{verbatim}
The header word contains the structure length, which is the number of words
(other than the header word.) The length is always at least one, since the
first word of the structure data is the structure type name.
\section{Fixnums}
A fixnum has one of the following formats in 32 bits:
\begin{verbatim}
-------------------------------------------------------
| 30 bit 2's complement even integer | 0 0 0 |
-------------------------------------------------------
\end{verbatim}
or
\begin{verbatim}
-------------------------------------------------------
| 30 bit 2's complement odd integer | 1 0 0 |
-------------------------------------------------------
\end{verbatim}
Effectively, there is one tag for immediate integers, two zeros. This buys one
more bit for fixnums, and now when these numbers index into simple-vectors or
offset into memory, they point to word boundaries on 32-bit, byte-addressable
machines. That is, no shifting need occur to use the number directly as an
offset.
This format has another advantage on byte-addressable machines when fixnums are
offsets into vector-like data-blocks, including structures. Even though we
previously mentioned data-blocks are dual-word aligned, most indexing and slot
accessing is word aligned, and so are fixnums with effectively two tag bits.
Two tags also allow better usage of special instructions on some machines that
can deal with two low-tag bits but not three.
Since the two bits are zeros, we avoid having to mask them off before using the
words for arithmetic, but division and multiplication require special shifting.
\section{Other-immediates}
\label{sec:other-immediates}
As for fixnums, there are two different three-bit lowtag codes for
other-immediate, allowing 64 other-immediate types:
\begin{verbatim}
----------------------------------------------------------------
| Data (24 bits) | Type (8 bits with low-tag) | 1 0 |
----------------------------------------------------------------
\end{verbatim}
The type-code for an other-immediate type is considered to include the two
lowtag bits. This supports the concept of a single ``type code'' namespace for
all descriptors, since the normal lowtag codes are disjoint from the
other-immediate codes.
For other-pointer objects, the full eight bits of the header type code are used
as the type code for that kind of object. This is why we use two lowtag codes
for other-immediate types: each other-pointer object needs a distinct
other-immediate type to mark its header.
The system uses the other-immediate format for characters,
the {\tt symbol-value} unbound trap value, and header-words for data-blocks on
the heap. The type codes are laid out to facilitate range checks for common
subtypes; for example, all numbers will have contiguous type codes which are
distinct from the contiguous array type codes. See
section~\ref{sec:data-blocks-and-o-i}
for details.
\section{Data-Blocks and Header-Word Format}
\label{sec:data-blocks-and-header}
Pointers to data-blocks have the following format:
\begin{verbatim}
----------------------------------------------------------------
| Dual-word address of data-block (29 bits) | 1 1 1 |
----------------------------------------------------------------
\end{verbatim}
The word pointed to by the above descriptor is a header-word, and it has the
same format as an other-immediate:
\begin{verbatim}
----------------------------------------------------------------
| Data (24 bits) | Type (8 bits with low-tag) | 0 1 0 |
----------------------------------------------------------------
\end{verbatim}
This is convenient for scanning the heap when GC'ing, but it does mean that
whenever GC encounters an other-immediate word, it has to do a range check on
the low byte to see if it is a header-word or just a character (for example).
This is easily acceptable performance hit for scanning.
The system interprets the data portion of the header-word for non-vector
data-blocks as the word length excluding the header-word. For example, the
data field of the header for ratio and complex numbers is two, one word each
for the numerator and denominator or for the real and imaginary parts.
For vectors and data-blocks representing Lisp objects stored like vectors, the
system (usually) ignores the data portion of the header-word:
\begin{verbatim}
----------------------------------------------------------------
| Unused Data (24 bits) | Type (8 bits with low-tag) | 0 1 0 |
----------------------------------------------------------------
| Element Length of Vector (30 bits) | 0 0 |
----------------------------------------------------------------
\end{verbatim}
Using a separate word allows for much larger vectors, and it allows {\tt
length} to simply access a single word without masking or shifting. Similarly,
the header for complex arrays and vectors has a second word, following the
header-word, the system uses for the fill pointer, so computing the length of
any array is the same code sequence.
For normal Lisp vectors, the data portion MUST be zero. For hash
tables, a vector is used to store information about the hash key and
value, and the data portion is non-zero to indicate to GC that this is
the key/value vector for the hash table. GENCGC uses this to
determine scavenge the key/value pairs correctly. Cheney GC also uses
this to determine if rehashing (for EQ hash tables) is needed.
\section{Data-Blocks and Other-immediates Typing}
\label{sec:data-blocks-and-o-i}
These are the other-immediate types. We specify them including all low eight
bits, including the other-immediate tag, so we can think of the type bits as
one type -- not an other-immediate major type and a subtype. Also, fetching a
byte and comparing it against a constant is more efficient than wasting even a
small amount of time shifting out the other-immediate tag to compare against a
five bit constant. (The current values can be obtained from the
generated \code{internals.h} file.)
\begin{verbatim}
HEX
Number (< 36)
bignum 10 0A
ratio 14 0E
single-float 18 12
double-float 22 16
double-double-float 26 1A
complex 30 1E
(complex single-float) 34 22
(complex double-float) 38 26
(complex double-double-float) 42 2A
Array (<= 46 code 118)
Simple-Array (<= 46 code 118)
simple-array 46 2E
Vector (<= 50 code 118)
simple-string 50 32
simple-bit-vector 54 36
simple-vector 58 3A
(simple-array (unsigned-byte 2) (*)) 62 3E
(simple-array (unsigned-byte 4) (*)) 66 42
(simple-array (unsigned-byte 8) (*)) 70 46
(simple-array (unsigned-byte 16) (*)) 74 4A
(simple-array (unsigned-byte 32) (*)) 78 4E
(simple-array (signed-byte 8) (*)) 82 52
(simple-array (signed-byte 16) (*)) 86 56
(simple-array (signed-byte 30) (*)) 90 5A
(simple-array (signed-byte 32) (*)) 94 5E
(simple-array single-float (*)) 98 62
(simple-array double-float (*)) 102 66
(simple-array double-double-float (*)) 106 6A
(simple-array (complex single-float) (*) 110 6E
(simple-array (complex double-float) (*) 114 72
(simple-array (complex double-double) (*) 118 76
complex-string 122 7A
complex-bit-vector 126 7E
(array * (*)) -- general complex vector. 130 82
complex-array 134 86
code-header-type 138 8A
function-header-type 142 8E
closure-header-type 146 92
funcallable-instance-header-type 150 96
byte-code-function-header-type 154 9A
byte-code-closure-header-type 158 9E
closure-function-header-type 162 A2
return-pc-header-type (a.k.a LRA) 166 A6
value-cell-header-type 170 AA
symbol-header-type 174 AE
base-character-type 178 B2
system-area-pointer-type (header type) 182 B6
unbound-marker 186 BA
weak-pointer-type 190 BE
instance-header-type 194 C2
fdefn-type 198 C6
scavenger-hook-type 202 CA
\end{verbatim}
\section{Strings}
All strings in the system are C-null terminated. This saves copying the bytes
when calling out to C. The only time this wastes memory is when the string
contains a multiple of eight characters, and then the system allocates two more
words (since Lisp objects are dual-word aligned) to hold the C-null byte.
Since the system will make heavy use of C routines for systems calls and
libraries that save reimplementation of higher level operating system
functionality (such as pathname resolution or current directory computation),
saving on copying strings for C should make C call out more efficient.
The length word in a string header, see ``Data-Blocks and Header-Word
Format'', section~\ref{sec:data-blocks-and-header}, counts only the characters truly in the Common Lisp string.
Allocation and GC will have to know to handle the extra C-null byte, and GC
already has to deal with rounding up various objects to dual-word alignment.
\section{Symbols and NIL}
Symbol data-block has the following format:
\begin{verbatim}
-------------------------------------------------------
| 5 (data-block words) | Symbol Type (8 bits) |
-------------------------------------------------------
| Value Descriptor |
-------------------------------------------------------
| Hash Value (x86/amd64/sparc) Unused (other arch.) |
-------------------------------------------------------
| Property List |
-------------------------------------------------------
| Print Name |
-------------------------------------------------------
| Package |
-------------------------------------------------------
\end{verbatim}
All of these slots are self-explanatory given what symbols must do in Common
Lisp.
The issues with nil are that we want it to act like a symbol, and we need list
operations such as CAR and CDR to be fast on it. CMU Common Lisp solves this
by putting nil as the first object in static space, where other global values
reside, so it has a known address in the system:
\begin{verbatim}
------------------------------------------------------- <-- space
| 6 (data-block words) | 0 | start
-------------------------------------------------------
| 0 (data-block words) | Symbol Type (8 bits) |
------------------------------------------------------- <-- nil
| Value/CAR |
-------------------------------------------------------
| Hash Value/CDR |
-------------------------------------------------------
| Property List |
-------------------------------------------------------
| Print Name |
-------------------------------------------------------
| Package |
-------------------------------------------------------
| ... |
-------------------------------------------------------
\end{verbatim}
In addition, we make the list typed pointer to nil actually point past the
header word of the nil symbol data-block. This has usefulness explained below.
The value and hash-value of nil are nil. Therefore, any reference to nil used
as a list has quick list type checking, and CAR and CDR can go right through
the first and second words as if nil were a cons object.
When there is a reference to nil used as a symbol, the system adds offsets to
the address the same as it does for any symbol. This works due to a
combination of nil pointing past the symbol header-word and the chosen list and
other-pointer type tags. The list type tag is four less than the other-pointer
type tag, but nil points four additional bytes into its symbol data-block.
\section{Array Headers}
The array-header data-block has the following format:
\begin{verbatim}
----------------------------------------------------------------
| Header Len (24 bits) = Array Rank +6 | Array Type (8 bits) |
----------------------------------------------------------------
| Fill Pointer (30 bits) | 0 0 |
----------------------------------------------------------------
| Fill Pointer p (29 bits) -- t or nil | 1 1 1 |
----------------------------------------------------------------
| Available Elements (30 bits) | 0 0 |
----------------------------------------------------------------
| Data Vector (29 bits) | 1 1 1 |
----------------------------------------------------------------
| Displacement (30 bits) | 0 0 |
----------------------------------------------------------------
| Displacedp (29 bits) -- t or nil | 1 1 1 |
----------------------------------------------------------------
| Range of First Index (30 bits) | 0 0 |
----------------------------------------------------------------
.
.
.
\end{verbatim}
The array type in the header-word is one of the eight-bit patterns from
``Data-Blocks and Other-immediates Typing'', section~\ref{sec:data-blocks-and-header}, indicating that this is a complex
string, complex vector, complex bit-vector, or a multi-dimensional array. The
data portion of the other-immediate word is the length of the array header
data-block. Due to its format, its length is always six greater than the
array's number of dimensions. The following words have the following
interpretations and types:
\begin{description}
\item[Fill Pointer:]
This is a fixnum indicating the number of elements in the data vector
actually in use. This is the logical length of the array, and it is
typically the same value as the next slot. This is the second word, so
LENGTH of any array, with or without an array header, is just four bytes
off the pointer to it.
\item[Fill Pointer P:]
This is either T or NIL and indicates whether the array uses the
fill-pointer or not.
\item[Available Elements:]
This is a fixnum indicating the number of elements for which there is
space in the data vector. This is greater than or equal to the logical
length of the array when it is a vector having a fill pointer.
\item[Data Vector:]
This is a pointer descriptor referencing the actual data of the array.
This a data-block whose first word is a header-word with an array type as
described in ``Data-Blocks and Header-Word Format'', section~\ref{sec:data-blocks-and-header} and
``Data-Blocks and Other-immediates Typing'', section~\ref{sec:data-blocks-and-o-i}
\item[Displacement:]
This is a fixnum added to the computed row-major index for any array.
This is typically zero.
\item[Displacedp:]
This is either t or nil. This is separate from the displacement slot, so
most array accesses can simply add in the displacement slot. The rare
need to know if an array is displaced costs one extra word in array
headers which probably aren't very frequent anyway.
\item[Range of First Index:]
This is a fixnum indicating the number of elements in the first dimension
of the array. Legal index values are zero to one less than this number
inclusively. IF the array is zero-dimensional, this slot is
non-existent.
\item[... (remaining slots):]
There is an additional slot in the header for each dimension of the
array. These are the same as the Range of First Index slot.
\end{description}
\section{Bignums}
Bignum data-blocks have the following format:
\begin{verbatim}
-------------------------------------------------------
| Length (24 bits) | Bignum Type (8 bits) |
-------------------------------------------------------
| least significant bits |
-------------------------------------------------------
.
.
.
\end{verbatim}
The elements contain the two's complement representation of the integer with
the least significant bits in the first element or closer to the header. The
sign information is in the high end of the last element.
\section{Code Data-Blocks}
A code data-block is the run-time representation of a ``component''. A component
is a connected portion of a program's flow graph that is compiled as a single
unit, and it contains code for many functions. Some of these functions are
callable from outside of the component, and these are termed ``entry points''.
Each entry point has an associated user-visible function data-block (of type
{\tt function}). The full call convention provides for calling an entry point
specified by a function object.
Although all of the function data-blocks for a component's entry points appear
to the user as distinct objects, the system keeps all of the code in a single
code data-block. The user-visible function object is actually a pointer into
the middle of a code data-block. This allows any control transfer within a
component to be done using a relative branch.
Besides a function object, there are other kinds of references into the middle
of a code data-block. Control transfer into a function also occurs at the
return-PC for a call. The system represents a return-PC somewhat similarly to
a function, so GC can also recognize a return-PC as a reference to a code
data-block. This representation is known as a Lisp Return Address (LRA).
It is incorrect to think of a code data-block as a concatenation of ``function
data-blocks''. Code for a function is not emitted in any particular order with
respect to that function's function-header (if any). The code following a
function-header may only be a branch to some other location where the
function's ``real'' definition is.
The following are the three kinds of pointers to code data-blocks:
\begin{description}
\item[Code pointer (labeled A below):]
A code pointer is a descriptor, with other-pointer low-tag bits, pointing
to the beginning of the code data-block. The code pointer for the
currently running function is always kept in a register (CODE). In
addition to allowing loading of non-immediate constants, this also serves
to represent the currently running function to the debugger.
\item[LRA (labeled B below):]
The LRA is a descriptor, with other-pointer low-tag bits, pointing
to a location for a function call. Note that this location contains no
descriptors other than the one word of immediate data, so GC can treat
LRA locations the same as instructions.
\item[Function (labeled C below):]
A function is a descriptor, with function low-tag bits, that is user
callable. When a function header is referenced from a closure or from
the function header's self-pointer, the pointer has other-pointer low-tag
bits, instead of function low-tag bits. This ensures that the internal
function data-block associated with a closure appears to be uncallable
(although users should never see such an object anyway).
Information about functions that is only useful for entry points is kept
in some descriptors following the function's self-pointer descriptor.
All of these together with the function's header-word are known as the
``function header''. GC must be able to locate the function header. We
provide for this by chaining together the function headers in a NIL
terminated list kept in a known slot in the code data-block.
\end{description}
A code data-block has the following format:
\begin{verbatim}
A -->
****************************************************************
| Header-Word count (24 bits) | Code-Type (8 bits) |
----------------------------------------------------------------
| Number of code words (fixnum tag) |
----------------------------------------------------------------
| Pointer to first function header (other-pointer tag) |
----------------------------------------------------------------
| Debug information (structure tag) |
----------------------------------------------------------------
| First constant (a descriptor) |
----------------------------------------------------------------
| ... |
----------------------------------------------------------------
| Last constant (and last word of code header) |
----------------------------------------------------------------
| Some instructions (non-descriptor) |
----------------------------------------------------------------
| (pad to dual-word boundary if necessary) |
B -->
****************************************************************
| Word offset from code header (24) | Return-PC-Type (8) |
----------------------------------------------------------------
| First instruction after return |
----------------------------------------------------------------
| ... more code and LRA header-words |
----------------------------------------------------------------
| (pad to dual-word boundary if necessary) |
C -->
****************************************************************
| Offset from code header (24) | Function-Header-Type (8) |
----------------------------------------------------------------
| x86/amd64/sparc: Address of start of instructions for |
| function (non-descriptor) |
| other architectures: |
| Self-pointer back to previous word (with other-pointer tag) |
----------------------------------------------------------------
| Pointer to next function (other-pointer low-tag) or NIL |
----------------------------------------------------------------
| Function name (a string or a symbol) |
----------------------------------------------------------------
| Function debug arglist (a string) |
----------------------------------------------------------------
| Function type (a list-style function type specifier) |
----------------------------------------------------------------
| Start of instructions for function (non-descriptor) |
----------------------------------------------------------------
| More function headers and instructions and return PCs, |
| until we reach the total size of header-words + code |
| words. |
----------------------------------------------------------------
\end{verbatim}
The following are detailed slot descriptions:
\begin{description}
\item[Code data-block header-word:]
The immediate data in the code data-block's header-word is the number of
leading descriptors in the code data-block, the fixed overhead words plus
the number of constants. The first non-descriptor word, some code,
appears at this word offset from the header.
\item[Number of code words:]
The total number of non-header-words in the code data-block. The total
word size of the code data-block is the sum of this slot and the
immediate header-word data of the previous slot.
header-word.
\item[Pointer to first function header:]
A NIL-terminated list of the function headers for all entry points to
this component.
\item[Debug information:]
The DEBUG-INFO structure describing this component. All information that
the debugger wants to get from a running function is kept in this
structure. Since there are many functions, the current PC is used to
locate the appropriate debug information. The system keeps the debug
information separate from the function data-block, since the currently
running function may not be an entry point. There is no way to recover
the function object for the currently running function, since this
data-block may not exist.
\item[First constant ... last constant:]
These are the constants referenced by the component, if there are any.
\vspace{1ex}
\item[LRA header word:]
The immediate header-word data is the word offset from the enclosing code
data-block's header-word to this word. This allows GC and the debugger
to easily recover the code data-block from an LRA. The code at the
return point restores the current code pointer using a subtract immediate
of the offset, which is known at compile time.
\vspace{1ex}
\item[Function entry point header-word:]
The immediate header-word data is the word offset from the enclosing code
data-block's header-word to this word. This is the same as for the
return-PC header-word.
\item[Address of start of instructions for function:] This is
implemented on x86, amd64, and sparc only. In a non-closure
function, this address allows the call sequence to always
indirect through the second word in a user callable function.
See section ``Closure Format''. With a closure, indirecting
through the second word also gets you the start of instructions
of a function. This pointer is a raw address, not a descriptor.
\item[Self-pointer back to header-word:]
In a non-closure function, this self-pointer to the previous header-word
allows the call sequence to always indirect through the second word in a
user callable function. See section ``Closure Format''. With a closure,
indirecting through the second word gets you a function header-word. The
system ignores this slot in the function header for a closure, since it
has already indirected once, and this slot could be some random thing
that causes an error if you jump to it. This pointer has an
other-pointer tag instead of a function pointer tag, indicating it is not
a user callable Lisp object.
\item[Pointer to next function:]
This is the next link in the thread of entry point functions found in
this component. This value is NIL when the current header is the last
entry point in the component.
\item[Function name:]
This function's name (for printing). If the user defined this function
with DEFUN, then this is the defined symbol, otherwise it is a
descriptive string.
\item[Function debug arglist:]
A printed string representing the function's argument list, for human
readability. If it is a macroexpansion function, then this is the
original DEFMACRO arglist, not the actual expander function arglist.
\item[Function type:]
A list-style function type specifier representing the argument signature
and return types for this function. For example,
\begin{verbatim}
(function (fixnum fixnum fixnum) fixnum)
\end{verbatim}
or
\begin{verbatim}
(function (string &key (:start unsigned-byte)) string)
\end{verbatim}
This information is intended for machine readablilty, such as by the
compiler.
\end{description}
\section{Closure Format}
A closure data-block has the following format:
\begin{verbatim}
----------------------------------------------------------------
| Word size (24 bits) | Closure-Type (8 bits) |
----------------------------------------------------------------
| Pointer to function header (other-pointer low-tag) |
----------------------------------------------------------------
| . |
| Environment information |
| . |
----------------------------------------------------------------
\end{verbatim}
A closure descriptor has function low-tag bits. This means that a descriptor
with function low-tag bits may point to either a function header or to a
closure. The idea is that any callable Lisp object has function low-tag bits.
Insofar as call is concerned, we make the format of closures and non-closure
functions compatible. This is the reason for the self-pointer in a function
header. Whenever you have a callable object, you just jump through the second
word, offset some bytes, and go.
\section{Function call}
Due to alignment requirements and low-tag codes, it is not possible to use a
hardware call instruction to compute the LRA. Instead the LRA
for a call is computed by doing an add-immediate to the start of the code
data-block.
An advantage of using a single data-block to represent both the descriptor and
non-descriptor parts of a function is that both can be represented by a
single pointer. This reduces the number of memory accesses that have to be
done in a full call. For example, since the constant pool is implicit in an
LRA, a call need only save the LRA, rather than saving both the
return PC and the constant pool.
\section{Memory Layout}
\cmucl{} has four spaces, read-only, static, dynamic-0, and dynamic-1.
Read-only contains objects that the system never modifies, moves, or reclaims.
Static space contains some global objects necessary for the system's runtime or
performance (since they are located at a known offset at a known address), and
the system never moves or reclaims these. However, GC does need to scan static
space for references to moved objects. Dynamic-0 and dynamic-1 are the two
heap areas for stop-and-copy GC algorithms.
What global objects are at the head of static space???
\begin{verbatim}
NIL
eval::*top-of-stack*
lisp::*current-catch-block*
lisp::*current-unwind-protect*
FLAGS (RT only)
BSP (RT only)
HEAP (RT only)
\end{verbatim}
In addition to the above spaces, the system has a control stack, binding stack,
and a number stack. The binding stack contains pairs of descriptors, a symbol
and its previous value. The number stack is the same as the C stack, and the
system uses it for non-Lisp objects such as raw system pointers, saving
non-Lisp registers, parts of bignum computations, etc.
\section{System Pointers}
The system pointers reference raw allocated memory, data returned by foreign
function calls, etc. The system uses these when you need a pointer to a
non-Lisp block of memory, using an other-pointer. This provides the greatest
flexibility by relieving contraints placed by having more direct references
that require descriptor type tags.
A system area pointer data-block has the following format:
\begin{verbatim}
-------------------------------------------------------
| 1 (data-block words) | SAP Type (8 bits) |
-------------------------------------------------------
| system area pointer |
-------------------------------------------------------
\end{verbatim}
``SAP'' means ``system area pointer'', and much of our code contains this naming
scheme. We don't currently restrict system pointers to one area of memory, but
if they do point onto the heap, it is up to the user to prevent being screwed
by GC or whatever.
\section{Weak Pointers}
\label{sec:weak-pointers}
A weak-pointer data-block has the following format:
\begin{verbatim}
-------------------------------------------------------
| 4 (data-block words) | Weak pointer Type (8 bits) |
-------------------------------------------------------
| weak-pointer-value |
-------------------------------------------------------
| weak-pointer-broken |
-------------------------------------------------------
| mark-bit (T or NIL) |
-------------------------------------------------------
| next |
-------------------------------------------------------
\end{verbatim}
The mark-bit is used when gencgc is available. It's used to note if
this weak pointer has been visited before so that scavenging
weak-pointers isn't an $O(n^2)$ process.
The last slot is an internal slot used by the C runtime to chain all
the weak pointers together for GC.

8
doc/internals/cmu/run-time.tex
View file

@ -1,8 +0,0 @@
\part{Run-Time System}
\input{environment}
\input{interpreter}
\input{debugger}
\input{object}
\input{lowlev}
\input{fasl}