Put "--without-gencgc --with-mark-region-gc" in the make.sh invocation
to enable it (which automatically disables immobile space for now).
There are some known test failures.
Big thanks to Hayley Patton for this.
The C runtime will soon need to support the new mark-region collector which
is generational in nature with the identical heap structure as gencgc.
For clarity's sake, :GENCGC will refer to the specific implementation
contained in the file gencgc.c, whereas :GENERATIONAL shall imply either
gencgc or :MARK-REGION-GC without having to spell the (OR ...) everywhere.
The new feature name is not perfect- there could exist an incompatible
generational GC. Deal with that later, if ever.
That said, a few techniques were used in changing featureization:
- If it was code we would like to work on any GC, then delete the conditional.
Later, conditional code could be re-added if something can't be made to work
for whatever different GC is implemented.
- If it was something such as GENERATION-OF we'd like to remember not to allow
to be called with a non-generational collector, add #+generational.
- If something was #-gencgc but meant "#+cheneygc" then either change to that
or delete the code. Retained dead code is a hint to where subsequent changes
might be required if not obvious, for example the compiler's decision about
L-T-V and value-cells. Absence of a feature marker could leave you wondering.
- No machine-dependent files were touched, as the mark-region collector
will support only x86-64 at present.
(I don't always write long commit messages, but when I do, they concern
multiple work-in-progress GC implementations.)
Client code should henceforth include gc.h and nothing but that,
and in turn, gc.h will include a GC implementation header.
That was the idea from the beginning, by separating gencgc-internal
from cheneygc-internal, except it never worked. Now that cheneygc
is gone we can do a re-design this for the mark-region GC.
Also this change requires that includers of of gc.h understand their own
_direct_ dependencies. So for example it is not correct to assume that gc.h
provides fun_code_header(). It might but only if pinned_p uses that.
Similarly, grovel-headers.c did not directly include any header providing
os_vm_size_t. (clang-tidy is quite upset by the amount of accidental
reliance on indirectly included things in our sources)
* Move a few thread-related prototypes to thread.h
* Removing thread.h exposed the fact that a lot of C files
did not directly include things they needed such as vector_len
* Delete stray copy of 2 duplicated prototypes
and nonexistent instance_scan declaration
The goal is to have good separation between macros that deal with object
representation versus the allocator/collector and other random logic.
Previous attempts have been a total failure, considering that there are
nearly 30 files that include gc-private.h. By starting over with the
manipulators of the object model, it should be possible to arrive at a
healthier state with respect to changing out the GC.
{layout,instance,funcallable-instance} all become part of "instance.h".
Also autgenerate the #defines for known layout IDs.
This will, after subsequent changes, allow putting the layout_of() macro
and other things in that file to improve type-safety and/or clean up the
random mess of logic in gc-*.h headers.
* Don't assume that index 2 in 'adj' is for varyobj space
* Or that PIE ELF cores are ever going to work
* Or that readonly space can be accessed past its free pointer
alloc_pointer as a concept might have made sense for cheneygc, where it is the
address at which to next allocate. But gencgc has only a high water mark which
the "allocation pointer" would certainly be strictly less than.
And dynamic_space_free_pointer itself is redundant, so remove that too.
This logic has always been both suboptimal and wrong, potentially erring
in either direction: not WPing when it could, and WPing when it mustn't.
Only the latter is an actual bug, and nearly impossible to observe.
One bug was already documented as pertains to FDEFNs. This seems never
to happen, so I'm leaving it alone for now. The observable bug was with
lockfree lists where an element is pending pending physical deletion
after being logically deleted, with GC occurring in between those steps.
The suboptimalities are several (but I'm not fixing them all):
- deposit_filler() does not zeroize words that were formerly occupied
by an object that was evacuated from the partially pinned pages.
There were frequently random pointers to lower-generation pages.
- unboxed objects may reside on boxed pages if we were unable to move
them because of pinning, so we'd treat raw bits as pointers. Whether
they look like pointers is about as rare as the situation that a stack
word falsely looks like a pointer, but it's easy enough to skip over
unboxed objects, same as with filler objects.
- mixed raw/boxed instances should use the bitmap. It's unclear whether
doing so would be faster. Treating every word as a pointer is fine
but continues to be slightly suboptimal.
* Far too many calls to fixnum_value() were to extract VECTOR-LENGTH,
and in simple_base_stringize() it wasn't called, which was a minor bug
that resulted in calling malloc() with too large a number.
* Don't constrain C to using unclear terminology: 'scav_vector' is a
a bad name for 'scav_vector_t'. And I dislike simple_vector, which
seems as if it means (AND VECTOR SIMPLE-ARRAY). ANSI could have
picked SIMPLE-T-VECTOR or SIMPLE-VECTOR-T for SIMPLE-VECTOR.
Most places that read the bitmap have an easier time with it now, including
heap relocation and elfination. The 'embiggen' test (which was broken) is
no longer pertinent since there are no external bignums to consider.
There are two ways of iterating over descriptor slots in C, either a test
of "logbitp" at each word index, or the special code in scav_instance() which
avoids a div and mod operations at each step. But the logbitp approach had to
decide at each iteration whether the bitmask was a fixnum or a bignum.
This change removes that decision by canonicalizing the bitmask once up front,
and further anticipates a change in which the bitmask is moved to the end
of the layout so that we never have to chase a pointer outside of the layout
to utilize the layout, which results in generally more understandable C code.
In fact, I don't see how the assertion about a bignum for the bitmask in
coreparse heap relocation was correct, because it used layout->bitmap
without adjustment. But then it used the adjusted value when iterating.
So the code was right, but the assertion probably would fail.
That's one example of why an external bignum is problematic.
* Flag the layout as a raw slot in the bitmap. Tracing an instance will
always examine the layout as a one-off before proceeding to the data slots.
This ensures that we don't accidentally visit a tagged slot twice in
situations where it matters (heap relocation, for one).
* If all words are raw or all fixnums (or characters), nothing needs
examining. The bitmap reduces to 0 which is an early exit.
* It becomes more feasible to unify much of the handling of ordinary
instances and funcallable-instances.
The pseudo-atomic header was more often #included for get_alloc_pointer() than
the p-a accessors,so rename it to 'getallocptr.h', but keep the accessor for
thread->pseudo_atomic_bits in a header that is still named 'pseudo-atomic.h'.
Factor the three major variations into different files. Prior to this change,
too much thinking was needed to figure out which bundle of #defines and #ifs
were applicable to a particular arch x {cheneygc,gencgc} x {+thread,-thread}.
{arm,arm64} + cheney stopped working ever since gencgc was
implemented for the respective architecture.
The repairs were basically the same for either, but 32-bit crashes now
with cheneygc selected. That's better than not compiling at all,
but I'll probably disable that combination of build flags if a fix
isn't readily forthcoming.
Store NAME, ARGLIST, FORM/DOC/XREFS, and TYPE in the boxed portion of
the code header. This is easier on C code that scans lisp code for pointers.
I plan to implement a slightly quicker way of determining where each
simple-fun has its boxed words stored by robbing some bits from the fun->code
backpointer, but that's just an optimization and not critical to this change.
This will be necessary for a couple reasons-
* It might be possible for an ELF text section to contain _strictly_ machine
code, with the code header and simple-funs located in dynamic space.
If done that way, with text (instructions) disembodied from the header,
we can't assume that instructions commence at code + header_size.
* To make heap scanning with concurrent creation of code more robust,
it can't require two memory loads to compute the size of an object.
The first word should convey the full object size, and the dividing line
between boxed/unboxed can be indicated by a different word for GC's use.
It's almost the same issue as atomically inserting into a linked list.
* Move more work from preserve_pointer() into pin_object().
The rationale for doing more in pin_object is that it becomes feasible to
implement a specific use-case of precise pinning with otherwise conservative
GC. preserve_pointer() is the interface given a word that may or may not be
a pointer, and pin_object() is the interface for known good pointers.
This also corrects a newly introduced bug in precise GC where 'descriptor'
wasn't getting recomputed if the object's widetag was RETURN_PC or SIMPLE_FUN.
Conservative GC was fine because scanning the heap for a containing object
will never find a simple-fun.
* update_code_writeprotection() caused invariant violations if compiled
without page protection, because no pages should ever be protected.
* There's no reason to bifurcate the instance_length() function.
* Explain the role of the "transother" functions.
* Mention that object coalescing should probably not accidentally
create hash-table keys that become EQ if the value are different.
Bugs fixed:
- It took care to not do anything on some pointerless objects,
but erred on a character for no good reason.
- Due to a typo, it failed to extract both parts of a complex number.
- It failed to see funcallable-instance-layout if #+compact-instance-header,
and %code-fixups (which could be the bignum or droid you're looking for).
Express the whole thing in terms of a skeletal object traversal macro
into which you can plug small customizations. Share that with
SB-VM::MAP-REFERENCING-OBJECTS, and use only GC-safe accessors, not SAPs.
* It is neither required nor permitted to specify two sets
of dynamic space addresses in the 'parms' files
* Whatever address the core file specifies as the dynamic space
becomes dynamic-0-space. The other semispace is placed anywhere.
* Conditionalization could almost be removed except for win32
