Move argument checking before the registers and the stack are
overwritten, this allows for the backtrace to recover all the
arguments exactly as they were passed, including the extra ones.
This is actually implemented for all platforms (albeit not tested),
but not all are either interrupt safe in that place or can produce a
proper backtrace from that interrupt, hence :precise-arg-count-error
feature is only enabled on ARM and x86oids.
The x86-64 patch wasn't fully merged and couldn't be built.
x86 needed a similar simple replacement __FreeBSD__ => LISP_FEATURE_FREEBSD.
Also add :gnu-kfreebsd feature for easier differentiation.
- Add architecture check for NetBSD/ibmnws;
- Clean up resource limit initialization and mmap in small chunks;
- Test for SSE sysctl value when selecting bcopy variant.
This consequently uses the LISP_FEATURE_FREEBSD variable in C code,
modifies the make-config target so `uname` = "GNU/kFreeBSD" is
detected as sbcl_os=freebsd correctly and adds some alternative code
for the FreeBSD target if using the GNU libc.
Running ./make-config.sh --host-location="host:path/to/sbcl" on the
target causes the make-target-N.sh scripts to do all the copying of
files to and from the host using rsync on the target to the specified
host location.
Because this is rsync, managing the reverse ssh to do this on
heavily-firewalled targets is not straightforward; we might have to
extend the syntax, or have an extra --host-port argument. But this
should I think make the port to arm64 more straightforward.
* The overall ARM architecture has several FPU options, such as
soft-float-only, FPA (Floating Point Accelerator), at least three
generations of VFP (Vector Floating Point), plus some system based
on the x86oid MMX instruction sets. And that's off the top of my
head.
* Make an explicit statement that we are supporting VFPv2, and
possibly not even the original VFP, and leave hooks in place to
make it obvious what needs adjusting in order to support other FPU
options.
* Add an arch auto-detect for ARM, so we don't need to use
--arch=arm anymore.
* Add :cheneygc as a default feature for ARM, since it is
required and not otherwise defaulted.
* Both changes from patches supplied by Krystof.
The main idea is that info values are stored in a vector attached to
each symbol when possible. When not possible, the storage reverts
to the volatile [sic] environment, but still using a vector as the
payload instead of the chained hashing/alist approach.
This strives to be very fast at lookup at the expense of some added
complexity during updates. Performance testing suggests that it is
at least 2x to 3x faster at (INFO :class :type name), and FBOUNDP
is almost 4x faster. In a repeatable test, a file that took 1.8 seconds
to compile now takes 1.7 seconds but with more consing (as expected).
sbcl.core itself increases in size by <1% for 64-bit architecture,
and less for 32-bit architecture because there is proportionaly less
wasted space. A compact environment's table is effectively the
concatenation of all info vectors into one, so the added overhead is
in vector headers. However the fallback hash is now smaller,
so there used to be more wasted cells in the compact env.
Eventually the compact and volatile environments will both go away,
but not until the quasi-lockfree hashtable bootstraps properly.
The problem is an inability to use raw slots in early cold init.
It's actually not a problem of using them - the compiled code is ok -
but cold-init drops into 'ldb' due to how defstruct expands.
Among the bugs fixed by this (not straightforwardly testable) is that
the compact environment would hold into symbols that became otherwise
inaccessible. It no longer does, but still holds onto other names.
This patch builds with CCL as host, and for 32-on-64 and vice-versa,
so nothing seems terribly broken in terms of assumptions made.
* Known negative shifts are converted to another function that
only handles machine-friendly right shifts.
* The transforms and VOPs are conditionalised on ash-right-vops,
so other platforms aren't penalised.
* The new transforms trigger a lot of notes; this is suboptimal,
and one test had to be adjusted.
* This turned out to be fairly straightforward. Unlike in a
heap-allocation-only regime, where a VOP is required to :TRANSLATE
ALLOCATE-VECTOR, the :STACK-ALLOCATABLE-VECTORS feature enables an
LTN-ANNOTATE optimizer for ALLOCATE-VECTOR that substitutes
invocations of one of two named VOPs.
* To convert from the old regime to the new, rename the old VOP
to fit the new naming scheme, and write a new VOP to do the stack
allocation.
* As a cleaning-up-a-loose-end matter, lose the :TRANSLATE
option for the old VOP.
* And as a "being somewhat cute about things" matter, make the
support for stack-allocatable-vectors selectable at build time,
which should provide a quick overview of how to make this work on
some other platform, should anyone else be interested later on.
* Alter SYS:SRC;COMPILER;PPC;MACROS.LISP, WITH-FIXED-ALLOCATION
to accept a parameter for requesting stack allocation instead of
heap allocation.
* Alter SYS:SRC;COMPILER;PPC;ALLOC.LISP, VOP FIXED-ALLOC to pass
the new stack-allocation parameter.
* And add :stack-allocatable-fixed-objects to the PPC section in
make-config.sh.
- Mark Lisp signal handlers with a flag `synchronous' to indicate
whether we can (and must) handle them immediately. Conversely,
we understand this flag to imply a guarantee that the signal
does not occur during allocation.
- Any signal with a Lisp handler that is not synchronous is
implemented in the runtime using a trampoline, which (instead of
invoking Lisp code directly) first spawns a new pthread, which
only then calls back into Lisp to invoke the handler function
(with a fake signal context).
- Used in particular for SIGINT.
- For SIGPROF, introduce a second per-thread allocation region,
which gets swapped with the usual region around the call into
SIGPROF-HANDLER. This handler is a special case, because it is
careful not to trigger GC nor non-local unwinds, and we can
safely return to the original region afterwards.
- Add a new subclass SIGNAL-HANDLER-THREAD for this purpose,
making it easy to identify these threads (e.g. in the test
driver).
- Run sprof tests while building the contrib. Add a test stressing
time profiling of allocation sequences.
Enable using :SB-SAFEPOINT-STRICTLY on features.
Quite usable already on x86 and x86-64; PPC still has more prominent
issues, e.g. in threads.impure.lisp.
Simplify arch_os_get_current_thread() such that %fs is not being
loaded directly even on platforms which preserve it reliably (in
particular, Linux/x86 and Solaris/x86).
Aside from a code simplification and reduction of cross-platform
differences, this change is an improvement in the following way:
If arch_os_get_current_thread() gets called in a non-Lisp pthread,
it now returns NULL cleanly even on the platforms mentioned above.
On the other platforms, keep the support for restoring %fs, but
refactor to remove code duplication with arch_os_thread_init().
Coalesce the two differently-spelled Lisp features for this case
into one.
- Formerly static symbols are static no more:
- Always use the linkage table, even in cross compilation. The
runtime retrieves the resulting list of foreign symbols and installs
linkage table entries before calling into Lisp for the first time.
- Simplify/remove various win32 foreign symbol special cases.
- Almost remove scratch().
Conditional on feature SB-DYNAMIC-CORE; required on Windows and optional
on other platforms supporting the linkage table.
With this feature, changes to src/runtime can be built using make,
without requiring a slam.sh step to update core files.
For LINKAGE-TABLE platforms only. Currently supported on x86 and
x86-64. (PowerPC does not yet work.) In the interest of simplicity,
disable this feature permanently on non-SB-THREAD platforms, because
these would require unpleasant changes to allocation macros.
Thanks to Anton Kovalenko.
* Tweak make-config.sh to automatically select GENCGC, and to
update the fallback message for those poor bsd-using folks.
* Copied the gencgc memory map from the SunOS version to use for
Linux (the cheneygc maps were identical, we may as well keep the
parallelism).
* Also tweaked the linker flags to match. Not because it was
required to build, but because I don't know why they were changed
for SunOS, and in the absence of such knowledge I'd rather keep
things more or less parallel.
- A new macro odxprint(flag, "fmt", ...) performs the equivalent of
a printf("fmt", ...), but only if `flag' has been enabled at
run-time.
- Environment variables can be used to set flags, using either
SBCL_DYNDEBUG="flag1 flag2 flag3" syntax, or
SBCL_DYNDEBUG__FLAG1="nonempty string" syntax.
- Lisp feature SB-QSHOW enables support for odxprint-based FSHOW.
(Users who prefer to edit runtime.h to enable QSHOW can still do
so...) SB-QSHOW is enabled by default on Windows, where the
odxprint mechanism was first used.
- Implement FSHOW, FSHOW_SIGNAL on top of odxprint. Corresponding
flags are called fshow, fshow_signal.
- For gencgc_verbose, support a flag of the same name, since it is
conditional on QSHOW (inspite of not being implemented on top of
FSHOW).
- Does not yet support odxprint features specific to Windows debugger
integration; output is currently directed to stderr unconditionally.
This commit backports Anton Kovalenko's Windows-specific odxprint to
POSIX and integrates it with FSHOW.
* When I wrote 14fd5afbac I was
under the impression that there was a :CHENEYGC feature used on
plaforms that used the cheney collector. This was logical, but
incorrect.
* The two ways forward were to back out the use of :CHENEYGC
as a feature when checking compatibility or to move forward and
add :CHENEYGC and use it in preference to (NOT :GENCGC) where
appropriate, opening the door to a third GC option.
* This is the minimal fix, adding :cheneygc to the non-gencgc
targets in make-config.sh.
* Prior to the great de-lutexification, we used futexes on
threaded Linux/PPC. Not having them enabled on PPC was an
oversight, but one which highlighted some problems with the
non-futex threading code. Now that that's fixed, we may as
well restore the status quo.
* Remove all lutex-specific code from the system.
** Use SB-FUTEX for futex-capable platforms, and plain SB-THREAD
otherwise.
** Make non-futex mutexes unfair spinlocks for now, using WAIT-FOR to
provide timeouts and backoff.
** Build non-futex condition variables on top of a queue and WAIT-FOR.
Performance implications: SB-FUTEX builds should perform pretty much the
same, or improve a bit. Threaded non-futex builds are affected as follows:
1. Threads idling on semaphores or condition variables aren't quite as
cheap. Just how costly depends on the OS. On Darwin 1000 idle threads
can chew up a bit over 50% CPU. I will try to address this later.
2. Contested locking around operations that take considerably longer
than a single timeslice suffers mild degradation.
3. Contested locking around operations that don't take long is an order
of magnitude performant.
4. Highly active semaphores perform much better. (Follows from #3.)
* GRAB-MUTEX gets timeout support on all platforms.
* CONDITION-WAIT gets timeout support.
* Disable a bunch of prone-to-hang thread tests on Darwin. (All of them
were already prone to hang prior to this commit.)
* Enable a bunch tests that now /pass/ on Darwin. \o/ This doesn't mean that
the threaded Darwin is fully expected to pass all tests yet, but let's say
it's more likely to do so.
...but still not robust enough to enable threads on Darwin by default.
* GET-MUTEX/GRAB-MUTEX get refactored into two main parts: %TRY-MUTEX and
%WAIT-ON-MUTEX, which are also used directly from CONDITION-WAIT where
appropriate.
* Over time, the process for producing an SBCL build for another
target architecture or OS has become more difficult and less
documented.
* Move all make.sh argument processing to make-config.sh.
* Have make-config.sh produce a file, output/build-config,
containing shell commands to set up important build parameters
such as where to find GNUMAKE, what SBCL_XC_HOST is, and so on.
* Source output/build-config in most of the make*.sh scripts.
* The practical upshot of all this is that a cross-compile
build is now a matter of alternating between target and host,
calling make-config.sh, make-host-1.sh, make-target-1.sh,
make-host-2.sh, make-target-2.sh, and following up with a
make-target-contrib.sh on the target, and you wind up with a
usable product. And, as make-config.sh does all of the arg
parsing, you can pass --xc-host= or --dynamic-space-size= or
whatever and it all works out.
* Does the same thing as only returning the first value of %MULTIPLY,
only better on some platforms.
* Implemented vas VOPs on x86, x86-64 and PPC. The PPC code sequence
is fully untested, and merely looks correct.
* VOPs for fixnum first argument are included, but will only be used
when the result is forcibly marked as fixnum, e.g., with TRULY-THE.
Questionnable, but I'd rather err on the side of straightforwardness
rather than put even more pressure on representation selection.
* Use it in the division-by-multiplication transform for unsigned
TRUNCATE by constant.
for the time being, disable :largefile on SunOS/x86. Unfortunately,
sb-grovel cannot navigate the somewhat sticky way in which struct
dirent gets setup for a 32-bit host with 64-bit ino_t on SunOS, so
sb-posix:readdir breaks
:largefile works fine on SunOS x86_64, and will be re-enabled on x86
after I get some more time to fight with the alien def for struct dirent
A mistake in make-config.sh ("amd64" for "x86-64") was preventing
:sb-lutex from being enabled on Solaris x86-64.
While here, enable :largefile for Solaris (all platforms)
Patch by Jim Wise, lp#667297.
* Patch by Josh Elsasser, lp#615489.
* On OpenBSD/i386, the structure that stores the FPU state during
signals was recently moved.
Add a script to tools-for-build/ which is run from make-config.sh
to detect where the FPU state is located, and create a
src/runtime/openbsd-sigcontext.h header containing a #define which
src/runtime/x86-bsd-os.c can use.
This is necessary to build SBCL on recent -current snapshots of
OpenBSD, and for the upcoming 4.8 release.
* Missing NEWS entry for last commit.
* Based roughly on the x86-64 version and the differences
between x86oid define-full-reffer and the PPC use of VOPs
instead (the VOPs seem smart enough, why do x86oids do the
whole define-full-reffer / -c VOP thing?).
* Compare-and-swap VOPs are full memory barriers.
* Left the VOPs conditionally-compiled in based on the
compare-and-swap-vops feature, more as documentation of what
is involved than anything else.
* Enabled compare-and-swap-vops for all PPC targets.
* Add the VOPs for inlining the various memory-barriers.
* Add the memory-barrier-vops feature to local-target-features on
ppc systems in make-config.sh.
* x86 config -mmacosx-version-min cleanup
* remove dlshim feature in make-config.sh
* fix extern-alien-name and ldso naming conventions
* kludge in load-cold-foreign-symbol-table to strip off the leading
#\_ from the nm parsing
* change dependency on ldso stubs for dlopen and friends to be
#!-dlshim instead of #!-darwin
* remove Config.x86-64-darwin9+ and replace with makefile
conditionals
* add :dlshim feature for darwin in make-config.sh and rework the
darwin9+ stuff
* make grovel-headers.c load genesis/config.h first so that we can
use the LISP_FEATURE_xxx conditionals earlier
* Add new feature UD2-BREAKPOINTS, enabled by default only on x86oid
darwin targets.
* Use said feature instead of DARWIN for breakpoint trap selection.
* Make breakpoints work when using UD2-BREAKPOINTS (tested on x86 and
x86-64 linux).
* This patch brought to you by lp#309067, which remains valid for
three reasons: First, the test case is still disabled. Second, this
only fixes for x86oids, not for PPC. And third, I didn't actually test
this on a darwin system.
* sb-unix:unix-stat/lstat/fstat return the 32bit inode with x86 and
the 64bit inode with x86-64,
* sb-posix:stat/lstat/fstat get stat values using the old stat calls
with x86 and the new ones with x86-64
* re-enable now-working sb-posix stat tests
I'll let FreeBSD folks make the judgement if threads should be
default there as well.
Also: update INSTALL documentation regarding *FEATURES* a bit, and
make the documentation clear about availability of threads on
different builds.
* For consistency with other platforms, build x86-64 by default on OS
X when the OS and hardware support it.
* ... also clean up some stragglers needing .cvsignore entries
* New build-time feature: inline-constants, which specifies that SB!C
and SB!VM implement a protocol described in base-target-features.lisp-expr.
Backends implementing that feature are able to load constants from code
components, in a section that follows the actual executable code.
* Implement the protocol on x86 and x86-64, and use it for float constants,
and, on x86-64 only, mid-sized (> 2^(29-32), but still machine-sized)
integers.
* Use the new feature in integer and float arithmetic VOPs.
* Adjust a few test cases to take newly consing situations into account.
* Clean-up:
- New build-time feature: float-eql-vops, which disable rewriting EQL
of single and double floats in terms of foo-float*-bits.
- Fix a typo (unused variable lookup) in TWO-ARG-+/-
* On all platforms:
- Slightly more stable complex-complex float (double and single)
division;
- New transform for real-complex division;
- complex-real and real-complex float addition and subtraction
behave as though the real was first upgraded to a complex, thus
losing the sign of any imaginary zero.
* On x86-64
- Complexes floats are represented packed in a single SSE register;
- VOPs for all four arithmetic operations, complex-complex, but also
complex-real and real-complex, except for complex-complex and
real-complex division;
- VOPs for =, negate and conjugate of complexes (complex-real and
complex-complex);
- VOPs for EQL of floats (real and complexes).
- Full register moves for float values in SSE registers should also
speed scalar operations up.
* New *FEATURES*: :STACK-ALLOCATABLE-LISTS, :STACK-ALLOCATABLE-VECTORS, and
:STACK-ALLOCATABLE-FIXED-OBJECTS filled in by make-config.sh.
* Use them instead of #!+(or arch1 arch2 ...).
Changed the object filename computation function to use a different
suffix for files in target-compile mode with the :assem flag.
Changed the src/compiler/assembly/ paths in build-order.lisp-expr to
refer to src/assembly/ instead.
Changed the duplicate-stem check in src/cold/shared.lisp to operate
in terms of target-compile object file names.
Changed both the source and object filename computation functions to
call a new function to remap stem path segments matching "/target/" to
the appropriate per-arch directory.
Removed the code for creating the compiler/assembly/ and target/
symlinks from make-config.sh.
With this, source-location information in SBCL is now correct even
when used against an unbuilt source tree.
* Replace %MAKE-INSTANCE-WITH-LAYOUT with %MAKE-STRUCTURE-INSTANCE,
which has an IR2 transform that can handle both initialization and
allocation of the structure. On x86 and x86-64 it can initialize
all slots, whereas on other platforms it only does the layout and
non-raw slots. (See RAW-INSTANCE-INIT/* below.)
* EMIT-INITS needs two new kinds of inits to handle: :SLOT for
instance slots, and :DD for the defstruct-description/layout.
* DEF-ALLOC doesn't anymore use a simple boolean for denoting
variable length allocation, but instead a keyword: either
:VAR-ALLOC, :FIXED-ALLOC, or :STRUCTURE-ALLOC.
* New VOPs: RAW-INSTANCE-INIT/* for all raw slot types, which are
almost identical to RAW-INSTANCE-SET[-C]/* VOPs, except that they
always have a constant index and do not return a result. Structures
with raw slots can be stack allocated only on platforms that
implement these VOPs, denoted in make-config.sh by the
:RAW-INSTANCE-INIT-VOPS feature. ...we really could use a
*VM-FEATURES* or something.
* Print measured times using fixed-width decimal output with the
measured precision, instead of converting to floats for printing.
* Report processor cycle counts on x86 and x86-64.
** Since Intel doesn't seem to consider it necessary to issue a
CPUID both before and after RDTSC, maybe we don't need to do
that either.
** New feature, :CYCLE-COUNTER, for platforms that implement
SB-VM::%READ-CYCLE-COUNTER.
* Instead of reporting %EVAL calls, report "interpreted forms", which means
both %EVAL and SIMPLE-EVAL-IN-LEXENV.
* Report "lambdas converted" for the compiler, not counting TL-XEPs.
* Report CPU percentage (computed from real and run time.)
* Report total run time separately. Condence run time output slightly
by reporting total, user, and system on the same line.
* Report non-GC time as well.
* Condence output by omitting page faults, converted lambdas, and
interpreted forms when they are zero.
* Use GCC's Thread-Local Storage to store current thread.
(Tested on Linux x86 and x86-64, too)
* Restore lisp level TLS segment register at interrupt handler.
* :COMPARE-AND-SWAP-VOPS, not :COMPARE-AND-SWAP-VOP in *FEATURES*.
* Threaded cas tests. (threads.impure.lisp currently broken for
unrelated reasons since .30 or so.)
* New macro SB-EXT:COMPARE-AND-SWAP provides a supported interface to
compare-and-swap functionality.
* New info-type :FUNCTION :STRUCTURE-ACCESSOR allows us to map from
defstruct slot-accessor names to defstruct descriptions.
* Add :CAS-TRANS slot keyword to DEFINE-PRIMITIVE object, and the
compiler machinery needed to support compare and swap on primitive
object slots.
* New VOPs COMPARE-AND-SWAP-SLOT and %COMPARE-AND-SWAP-SYMBOL-VALUE.
* Delete now unnecessary DEFINE-STRUCTURE-SLOT-COMPARE-AND-SWAP.
* Use a consistent %COMPARE-AND-SWAP-FOO naming scheme for CAS
functions.
* Tests.
Tested on x86/Linux & x86/Darwin, x86-64/Darwi, and PPC/Darwin.
* Implement three new VOPs:
** UNWIND-TO-FRAME-AND-CALL constructs a fake catch block for a
given frame pointer, runs all unwinds for that block, sets
the frame pointer to the new value, and calls a given function.
** BIND-SENTINEL (stores a marker on the binding stack, used
to determine how far the binding stack needs to be unwound
during a U-T-F-A-C).
** UNBIND-SENTINEL (pops one of these markers from the stack).
* Modify IR2 to use these VOPs when converting suitable functions.
* Modify the IR1 translation in maybe-insert-debug-catch to only
ensure that tail recursion doesn't happen (needed to match the
BIND-SENTINELs with UNBIND-SENTINELs).
* Use these to implement SB-DEBUG:UNWIND-TO-FRAME-AND-CALL:
** Grovel the binding stack, uwp block chain and the catch block
chain for the values needed to reconstruct the dynamic state.
** Call SB-VM:U-T-F-A-C.
* The new implementation should be substantially the same as the
old one (minor difference in handling of functions with special
variables in the lambda list). Some tests added to verify this.
* New implementation is somewhat faster at runtime (a simple
function call overhead benchmark on (DEBUG 2) improved from 3.4s
to 2.9s), and significantly faster at compiling (generally around
15-30% improvement with (DEBUG 2)).
* Other platforms still use the old implementation that instruments
the code with a CATCH during IR1 translation.
* Based on an earlier hack by Alastair Bridgewater.
* have make-config.sh add mach-exception-handler to the features by
default. This gets around the CrashReporter problems and enables
debugging of SBCL processes with GDB.
* add sb-lutex to features via make-config.sh
* preserve_pointer context registers in gencgc.c
* use lock/cmpxchg on darwin
* GSYM/GNAME asm hacks
* add -arch options when appropriate for building threads-foreign.so
Support files >2GB on Linux/x86.
* Compile the runtime (and the C type grovelers) with various flags
to enable a 64-bit off_t.
* Add C-side wrappers for various POSIX functions, so that we can
reliably get the largefile versions of them from Lisp-side.
Implement SB-THREAD mutexes and waitqueues using only pthread
functionality on platforms that don't support Linux futexes. New
platforms that can be compiled with SB-THREAD:
* Solaris/x86 (seems to be as stable as SBCL threads on Linux)
* OS X/x86 (some known stability problems, but doesn't fail on the
thread regression tests every time)
* FreeBSD/x86 (reportedly "flat-out broken", tends to cause
kernel panics)
While I (Juho) am doing the final merge from lutex-branch to
HEAD, much of the work was done by Cyrus Harmon, based on an
initial implementation by Nathan Froyd. The Solaris work was
funded by Tellme Networks, Inc.
Linkage tables on win32
... heuristically choose a memory range ("does not break on my two
systems, let's ship it")
... added some leftovers to cleanup.sh
... also #!+sb-doc-conditionalize docstrings in run-program.lisp
x86/Darwin support
... Add Config support and ape the darwin/ppc space fixup stuff.
... Align stack to 16-bytes as per ABI in %alien-funcall,
c-call and show.
... Add darwin space params.
... Use an altstack for handling signals in hopes of giving signal
handlers properly aligned stacks.
... Get registers and flags out of the mcontext properly. Needed
to play some games to get sys/_types included in order to get
the full definition of the mcontext struct which Apple tries
to hide from us.
... Moved os_get_runtime_executable_path from ppc-darwin-os.c to
new file darwin-os.c.
... Fix debugging #ifdefs in runtime.h
... #define hacks to x86-assem.S to keep Apple's brain-dead
assembler happy. In particular, it doesn't like things like
$NIL, so I had to play some games with the precompiler to make
it happy, and it wants .globl instead of .global. Also fixed a
couple raw .aligns that should have used the macros
... Align stack in ppc-assem.S:call_into_c
... Added x86-darwin-os.h
... #- out debug.impure test that causes bad things to happen on
darwin/ppc as it seems to cause bad things on x86/darwin as
well.
... adjust fails for float tests
... LDSO stubs for x86/darwin
Merge Cyrus Harmon's 7th gencgc merge candidate
... with the addition of idempotent implementations of
arch_clear_pseudo_atomic_interrupted() for
sparc, mips, alpha and hppa. (the last three completely
untested).
... many, many changes, most of which are documented in
doc/internals-notes/GENCGC-PORTING-NOTES
(This commit may break horribly. Please read, please test)
PPC64 build fixes
... uname returning ppc64 should give us an SBCL_ARCH of ppc;
... move the spaces around for MORE HEAP (512Mb vs 128Mb);
... turn on linkage tables on Linux/PPC.
Merge patch from nyef for load-shared-object on win32.
... there's a scary amount of duplication between
win32-foreign-load and foreign-load; hope some
gardening takes place at some point.
... attempt to make make-config.sh put the relevant target
features in place.
Merge patch from James Bielman fixing self-build under Win32.
... a little bit of an accident ensued; I ran
canonicalize-whitespace on .sh files, which seemed to
work (except on wc.sh) until it scribbled all over
make.sh, confusing the shell interpreter utterly.
... the resulting files have built the system, nevertheless.
Fix saving a core with callbacks on x86 and x86-64, as
discussed on sbcl-devel "CFFI Callbacks on SBCL" on
2005-12-31. Essentially the problem is that the address of
#'ENTER-ALIEN-CALLBACK is hard-coded into the assembly callback
wrappers, and the address of the function can change when
saving a non-purified core.
* Define a static symbol that contains #'ENTER-ALIEN-CALLBACK
in the value slot.
* Change the x86 / x86-64 wrappers to indirect through the
slot.
* Add minimal test case
* Add a make-config.sh-detected :alien-callbacks feature
* move e-a-n from SB-VM to SB-SYS, and from target/vm.lisp
to foreign.lisp.
* move all e-a-n calls to FIND-FOREIGN-SYMBOL-IN-TABLE and
GET-DYNAMIC-FOREIGN-SYMBOL-ADDRESS.
* reader conditionalize e-a-n behaviour on :ELF and :MACH-O; explicitly
add the relevant feature for each OS in make-config.sh.
* delete unused file ppc/print.lisp (duplicates ppc/show.lisp).
* increment fasl-format number.
note: affects all backends; tested on ppc/darwin, x86/freebsd, and
sparc/sunos.
Has it been a month already? Bump +FASL-FILE-VERSION+. Reasons:
* "0.8.20.6: Make FILE-STREAM and STRING-STREAM potential mixins
in CLOS"
* "0.8.20.21: Add immediate single-floats on x86-64."
* Probably others...
Restore a modified version of the "slightly odd-looking stuff"
that was removed in 0.8.20.30. It's there because the arch name
is "x86-64" but the config file suffix is "x86_64-linux".
Slightly nicer behaviour on non-NPTL systems makes it possible
to use thread-enabled binaries in thread-unfriendly environments again
- but a warning is issued at startup and MAKE-THREAD signals an error
Remove some slightly odd-looking stuff in make-config.sh: once
again it is possible to "SBCL_ARCH=x86 sh make.sh" on an x86-64
Merge x86-64-again branch onto HEAD.
Many, many, many 64-bit cleanups in code/, runtime/, compiler/,
compiler/generic/
New SAP-REF-WORD and friends.
Various fixes to the x86-64 backends (and addition of assembly/
and runtime/ files necessary). Implementation of Unicode-related
stuff by CSR. Signed modular arithmetic has not yet been
implemented.
A number of tests fail:
... alien.impure.lisp: enum <-> integer array conversion
... exhaust.impure.lisp: "deferred gubbins"
... float.pure.lisp: float infinities
... foreign.test.sh: "deferred gubbins"
It's possible that this merge will cause alpha32 to break in an
interesting way, probably related to undefined-alien. Needs
debugging. Other architectures have been tested, but of course
it's possible that something has gone wrong.
Though I (CSR) am merging this, the vast majority of the work was
done by Juho Snellman (building on Dan Barlow's initial work to
get it into executing lisp code in cold-init), with guest appearances
by Cheuksan Edward Wang and Vincent Arkesteijn.
* Linkage tables are now implemented on PowerPC.
Right now I've only enabled it on Darwin; to enable it on Linux,
the addresses used for linkage table spaces would need to be tested.
* Lazy foreign functions do not work correctly - the code expects the signal
that occurs when jumping to a write-protected page to be the same as the
signal that occurs when reading from one. On Darwin the former is SIGBUS
and the latter is SIGSEGV; this means that the fault address testing does
not work on Darwin.
* Build with linkage-table by default on x86/NetBSD and
sparc/Linux as well.
* Don't try to be too clever about when to warn user about
alien definitions when saving cores on non-linkage-table
platforms: do it unconditionally.
* Fix parsing of recursive alien record and union types
(reported by Thomas F. Burdick, port of Helmut Eller's
patch for the same problem in CMUCL.)
fixes for build on my old Debian system...
...added missing #!+LINKAGE-TABLE
...converted "cd foo" to "cd ./foo" so that CDPATH-based logic
won't helpfully announce the name of the new dir to
stdout (and thus into local-target-features.lisp-expr);
then "cd ./foo > /dev/null" for overkill
generally grepped for and changed "cd foo" to "cd ./foo" to
reduce the amount of CDPATH ambiguity involved and
the amount of stdout noise generated
* x86/FreeBSD, x86/Linux and Sparc/SunOS now have
linkage-table support, allowing SAVE-LISP-AND-DIE to
function properly in the presence of loaded shared
objects.
* As a related cleanup automate testing for dlopen
support on the plaform, and conditionalize
LOAD-SHARED-OBJECT support on the resulting
:os-provides-dlopen feature.
* Fix dladdr bogosities: test if dladdr is supported on
the platform, and add an ldso_stub for it if so. This
so that SBCL isn't dependant on the dladdr being at the
same location at runtime as it was on the build-host.
Move the dummy definition for FOREIGN-SYMBOL-IN-ADDRESS
to target-load, so that backtraces on target will work
before foreign.lisp is built. Clean up the real F-S-I-A
definition to use the :os-provides-dladdr feature.
* Fix manual bogosities: generate functions signalling
UNSUPPORTED-OPERATOR-ERROR for SB-BSD-SOCKETS
platform-dependant sockopts on platforms where they're
not supported; make these functions have the normal
doctrings. This so that manual building will work on
non-Linux as well. Also clean up the .sbclrc examples
slightly.
More SB-POSIX test fixes
... allow building as root again, since it seems that's a
desired feature
... test for not having EUID of 0 in EACCES posix tests instead
(this will probably annoy antifuchs, but I forgot to commit
version.lisp-expr for the last fix. Since this is logically a
simple continuation, I haven't bothered to increment it again)
Don't allow the system to be built as root
... id(1) check early in the build.
Fix the SB-POSIX test suite
... rewrite mkdir.error.3 not to assume that / is unwriteable;
... continue to assume that the user is not (fake)root, but
nonetheless try to clean up a bit more if we are.
Merge build patch ("building SBCL with Sun toolchain" Nikodemus
Siivola sbcl-devel 2004-02-05)
... add some assertions to ldso-stubs.lisp
... fix the conditional around dlopen()
... delete ~% pasteo from sparc-funcdef.sh
* Revert previous tools-for-build/modify-ldt-struct-name commit
(which never actually included the necessary C file) for the
simpler test of LINUX_VERSION_CODE suggested by Dan Barlow.
* Add tools-for-build program to determine if asm/ldt.h has the
new name for the modify_ldt structure.
* Fixes to properly compile contribs on x86_64 systems
Solaris fixes (from Nikodemus Siivola sbcl-devel 2003-10-22)
... sh(1) apparently doesn't like $(foo). Make it `foo`.
... allow building from cmucl/sparc, by suppressing underflow
traps
... /dev/random doesn't exist. /dev/zero better had, though
Fix bug 213a
... CONS-TYPE-LENGTH-INFO to walk CONS-TYPE lists
... delete the neat but ultimately flawed (CONS NIL T) test
and use a proper test instead
... test suite additions.
Add idea from Michael Hudson (sbcl-devel 2003-08-26) to exit
early from Darwin compilations when the stack size
limit is too small.
PPC build fix (kludge)
... use offsetof() to find where the glibc maintainers have put
the uc_mcontext slot today, and delegate to appropriate
source code
non-x86/linux build fix
... undefine arch_os_get_current_thread() from individual
architecture header files
Everywhere clean.sh fix
... only include src/runtime/Config in tools-for-build/Makefile
if it exists. Now clean.sh works on a clean tree.
MacOS/Darwin merge. Points to note:
... thanks to Patrik Nordebo, Pierre Mai and Brian Mastenbrook;
... mmap()ed simple-streams appear not to work under darwin;
... floating point traps are currently non-functional on both ppc
platforms;
... on darwin, hitting ^C, then choosing CONTINUE results in a
memory fault;
... there's no lchown under darwin;
... x86/bsd building is also slightly non-functional currently;
... There's a OAOOM problem in src/compiler/ppc/c-call.lisp, and in
src/runtime/ppc-assem.S.
Mostly MIPS goodies, though some infrastructural fixes too
... fix hideous DEFINE-PARTIAL-REFFER constant-case bug;
... also adjust the costs so that the constant reffers and
setters are prioritized;
... also build fix for undefined static symbols (reported by
KMR end June 2003)
... include src/runtime/Config in tools-for-build/Makefile
(and adjust order of events in make-config.sh to
ensure that src/runtime/Config exists by the time
we need to build any of the tools-for-build)
Improvements in build technology:
... detect and choose GNUMAKE earlier, once and only once.
... provide recompile and use-anyway restarts for warm load.
As per Johannes Groedem sbcl-devel 2003-01-28, use the simplest
invocation of hostname(1) possible, to avoid e.g.
attempting to set the system hostname.
Fix one "outstanding embarrassment" (as per CSR sbcl-devel
2002-12-10): no more building of compiler files in
warm load.
... play symlink games instead, to build the troublesome
assembly files twice under different names (will need
adjustments to work on non-Unix filesystems (e.g.
NTFS) but no more than the rest of the build).
MIPSel build patch
... write a noddy C program to determine target endianness
(KLUDGE: assumes int is 32 bits)
... use it in make-config.sh if we are building a MIPS target
Merge (alpha-quality, probably) HPPA/Linux port
... added hppa fixups in genesis.lisp
... moved disassem-state definition from target-disassem.lisp to
disassem.lisp, so building it on the host
... lots of new files (thank you, CMUCL)
Minor boilerplate cleanups in src/runtime/
Mostly-tested but still considered "experimental" non-invasive
stack exhaustion checking, using a guard page at the end of the
stack and an extra clause in the sigsegv (on some ports, sigbus)
handler. One day there will be an internals doc with the
gory details: for now, try http://ww.telent.net/diary/2002/7/#23.59392
Ported runtime to OSF/1 a.k.a Digital Unix a.k.a Tru64
current_binding_stack_pointer and related are actual _pointers_
i.e. 8 byte quantities. Asm now uses ldq/stq on them not ldl/stl
Fix bug when gc trigger called while *gc-inhibit* set: see
new comment in interrupt.c for details
Small amount of general comment/dead code cleanup
Merged support for SPARC/SunOS (aka Solaris)
... added relevant runtime and -os.lisp files;
... cleaned up sparc backend runtime, actually _using_ the
abstractions that were written for the SPARC/Linux
port;
... added some #includes for compilation (nothing breaks on
Linux, but BSD has not yet been tested;
... removed some bash-/ksh-isms from build and test scripts;
... abstraced wait3() constants into grovel_headers and
unix.lisp.
more ordinary name (one *FEATURES* symbol, not two)
for the opposed pair of stack-grows-upward and
stack-grows-downward concepts
...replaced old pair of features with new
:CONTROL-STACK-GROWS-DOWNWARD-NOT-UPWARD feature
...#!+STACK-GROWS-DOWNWARD becomes
#!+CONTROL-STACK-GROWS-DOWNWARD-NOT-UPWARD
...#!+STACK-GROWS-UPWARD becomes
#!-CONTROL-STACK-GROWS-DOWNWARD-NOT-UPWARD
Since references to the new STACK-EXHAUSTION stuff can be
automatically written into fasl files, the fasl file
version needs to be incremented.
(Note that the STACK-FOO symbols introduced in this version
probably need to change, since as DB pointed out
on sbcl-devel 2002-03-04, they assume an identity
between C stack and SBCL control stack which doesn't
exist on the non-X86 ports.)
catching stack overflow, part II...
...defined *STACK-EXHAUSTION* and arranged for it to be
initialized in cold init
...added STACK-GROWS-FOOWARD to *FEATURES*, and used it to
reexpress some of the old #!+X86 conditionalization
merged APD "unbound condition slot" patch (sbcl-devel
2002-03-04), so that PARSE-DEFMACRO creates
ARG-COUNT-ERROR conditions correctly in the
(NOT (EQ ERROR-FUN 'ERROR)) case
made parse-defmacro-errors.lisp use pretty-printer
s/defmacro-lambda-list-bind-error/defmacro-ll-bind-error
added SB-DEBUG::*NESTED-DEBUG-CONDITION* to succor those of us
who like to divert ourselves by trashing print methods
bumped *DEBUG-PRINT-LEVEL* and *DEBUG-PRINT-LENGTH* so that
the new highly-structured pretty-printing :REPORT
methods for parse-defmacro-errors.lisp don't truncate
output too ludicrously
merged CSR SPARC port patch (sbcl-devel 2002-02-12, plus
lotso new files through FTP)
tweaking patch...
...s/ARCH_HAS_FOO/ARCH_HAS_FOO_IN_SIGCONTEXT/
...updated sbcl.1
...added 'typedef os_context_register' for OpenBSD
...added ARCH_HAS_STACK_POINTER for x86 (and added
x86-arch.h to hold it)
...renamed fpregister and fp.register to float.register (to
avoid the ambiguity with "frame pointer" that I
experienced when first trying to figure this out,
since even though for a given architecture it's pretty
unambiguous, in architecture-neutral code it's not)
...added x86-bsd-os.h
(This system state is seriously screwed up. It did build,
both on Linux and on OpenBSD, but signalled an
error after writing out sbcl.core on OpenBSD, and
although it runs a little both on Linux and on
OpenBSD, it hangs in the regression tests. It
also has some temporary hacks marked REMOVEME,
including one which suppresses PURIFY when
building the system (!).)
made the system build on OpenBSD again..
..stubbed stuff out as a quick fix to the problem of FFI to
64-bit stat.st_size
tried to make the system run on OpenBSD again..
..initialized current_dynamic_space, since it's now used
instead of DYNAMIC_SPACE_START in PURIFY
..added new assertions about GENCGC alloc_region stuff being
reset when it should be
renamed Lisp-level struct stat stuff to struct wrapped_stat
tried to tidy up Lisp-level stat stuff; removed mysterious
(STRING= NAME "") behavior from UNIX-STAT
added slam.sh to help in low-level compile-and-try cycle
pulled alloc_region-is-reset logic out into separate
functions, and added more assertions on it (hunting
for a bug which broke the old assertions)
renamed gc_alloc_large to gc_alloc_possibly_large, and
gc_quick_alloc_large_unboxed to
gc_quick_alloc_unboxed_possibly_large
enabled various GC checks, e.g. gencgc_zero_check=1
deleted unused (and bizarre..) gencgc_verify_zero_fill()
turned off PURIFY in order to get the system to build, since
PURIFY seems to be misbehaving (leaving INIT-FUNCTION
above the cutoff address..)
Make sure that we "mkdir output/" before we use it in the build
process. (MNA pointed out that we used it before we
made it.)
Use "uname -m" to figure out sbcl_arch default.
merged MNA PCL fixes from sbcl-devel 2001-05-09 (including
port of Pierre Mai's method combination fixes from
cmucl-imp 2001-04-26)
added regression test for method combination fix
Bug 14 was mostly fixed already: the problem with
INVALID-METHOD-ERROR complaining about being outside
a method combination function went away some time ago.
The MNA/Mai patch above improves method combination
error reporting further, so it's definitely time
to retire 14 from BUGS.
made grovel-headers.c run automatically as part of the build
process (so that the automatically-generated file
output/stuff-groveled-from-headers.lisp replaced the
old symlink src/code/target-os-types.lisp)
redid grovel_headers.c DEFCONSTANT macro as function
removed 'ln -sf $sbcl_arch-linux-types.lisp' stuff (from
src/runtime/make-config.sh??)
redid the definition of a stem in stems-and-flags.lisp,
as a relative path from SBCL base directory instead
of from the src/ subdirectory, so that it can now
refer to "output/stuff-groveled-from-headers.lisp
NOTE: This is a slightly screwed-up checkin, which won't
quite build as is, because the checked-in changes from
the Alpha port expect various src/code/*types.lisp
files (e.g. src/code/x86-linux-types.lisp) which
Daniel Barlow created by hand once and for all. Those
are in my working directory but I didn't check them in
because almost immediately, probably 0.6.12.4, I
expect to rearrange things so that the build
process generates a types file on every run of
make.sh. So since CVS add/delete is a little mixed up,
I thought I'd just skip it, even though it means that
this version isn't quite buildable unless you grab
those files from DB's patch.
merged Daniel Barlow's port to the Alpha CPU sbcl-devel
2001-05-04
moved CIRCULAR-LIST-P to early-extensions.lisp (since it
logically belongs there, and DB's new definition,
unlike the old one, doesn't have a physical dependence
on IGNORE-ERRORS which prevents it from being there)
merged boot-extensions.lisp, early-extensions.lisp, and
late-extensions.lisp, since there's no longer any
distinction between the files
deleted CVS files from sbcl-alpha-extra-files.tar, since
they were probably an oversight, I think
also src/compiler/alpha/z050.pdf:-|
The patch change
- (/hexstr arguments)
+ #!+sb-show (/hexstr arguments)
shouldn't be necessary because /SHOW, like all the
/FOO macros, is already conditional on SB-SHOW. So I
undid this change (and added some comments in
src/code/show.lisp explaining the intended behavior
of /FOO macros).
merged stat_wrapper.c into the relatively new, related
wrap.c file; deleted stat_wrapper.h since stat wrappers
seem never intended to be used by C code anyway
renamed build-tools/ to tools-for-build/ in the interest of
unambiguous (even painfully unambiguous)
globally-visible names
tweaked tests/alien.impure.lisp so that the test uses a
non-GENCGC-dependent variable (instead of having to
have the test conditional on #+GENCGC).
