Don't transform it to (setf struct-read-only-struct-slot).
It works when x or read-only-struct-slot are not known at compile
time, no reason to stop it.
* use a loopless final pass after the unrolled pass
* use one fewer temp register
* remove a couple of instructions here and there
Also some stylistic changes:
* give better names to registers depending on their usage
* avoid an #ifdef for SB_SAFEPOINT in x86-64-arch
Co-authored with Gemini
* Add dstate-disp-n to disassem-state
* Add reg/mem arg types for tuple sizes 1,2,4,8,16,32,64.
* Extend avx512-inst-printer-list to accept :disp-n.
* Scale EVEX disp8 by the instruction-specific tuple size in decode-mod-r/m
* Annotate full-vector EVEX moves and emit compressed displacement for them.
* Add disassembler tests for the functionality in this patch
This covers the full-vector move subset. For other EVEX memory
instructions more work is needed.
* Add +evex+ indicator to distinguish EVEX from VEX/legacy
even for EVEX L'L=00 forms
* Decode ModRM.reg registers 16–31 using R'
* Stop incorrectly applying R' to ModRM.r/m operands
* Decode EVEX ModRM.r/m register-direct 16–31 using X as B'
* Decode EVEX vvvv registers 16–31 using V'
* add simd-pack-512-mask as intrinsic type (widetag)
* add associated book-keeping in VM, compiler and interpreter for simd-pack-512-mask
* add VM support for mask registers (mask-reg SC, SB, defregs, ...)
* add VOPs to compiler backend for construction, extraction
* add VOPs for movement: kregs<->kregs, kregs<->gpr and kregs<->mem
* add support for assembler in insts, avx2-insts and avx512-insts
* rewrite most of evex emitter regarding mask registers
* add print support in evex for mask related instructions
* fix some smaller bugs in previous simd-pack-512 support patch
* add tests for creation, extraction, movements, assembly printing and some internal functions
* refactor zmm-registers-used-p into avx512-state-used-p and update call sites
* add support for mask regs to avx512-state-used-p
If RDX ceases to be the first arg-passing register, then this test will fail
because there is no byte-sized register corresponding to byte index 1 of RDI.
The initial problem is that elt derives types on constant lists
without adding a NIL, which can be returned by NTH.
Always check bounds instead.
There's no performance advantage in not signaling an error.
This wasn't working right, and the assertion in LAYOUT-ID about assigning
only structures an ID was written incorrectly.
* Layout IDs are assigned only when applying a :layout-id fixup or when storing
into a descendant type's ID array, whichever occurs first.
* IDs of non-structure instances like STREAM that have nonzero IDs are
stored at the correct place for their depthoid, and not always at index 0
so that LAYOUT-ID doesn't have to test +structure-layout-flag+.
* LAYOUT-ID returns NIL instead of 0 if no ID is assigned.
machine-dependent files will never see the 0.
* The remark that IDs could do type-based dispatch is no longer hypothetical
Which might cause type conflicts in new refs.
This might make some things opaque, but type-annotations are used only
in very specific cases, and preserving control flow with casts or
combinations will be just as opaque.
(defun f ()
(declare (optimize speed))
(labels ((phi (index)
(declare (type (integer 0) index))
(if (> index 9)
nil
(rec 0 index)))
(rec (zoot gindex)
(declare (fixnum zoot))
(if (< zoot 5)
(rec (1+ zoot) gindex)
(phi (1+ gindex)))))
(phi 0)))
now no longer produces any notes.
However, we don't have optimistic type propagation in CP itself, so
neither the lower bound nor the integerness of GINDEX or INDEX can be
derived.
When performing successive tests, if all types have the same depthoid, then
rather than emit an instruction to load instance-layout, and then a series
of instructions to compare the layout-id against various numbers (each
entailing a memory load), instead load the layout-id once up front and
compare that against the various IDs under consideration for equality.
If the depthoids are not all the same, fall back to the slightly suboptimal
code that loads a layout-id every time. (Better would be to try to group
comparisons into sequences of tests that can all use the same loaded ID.)
Test cases by Gemini
This accidental omission of ALLOCATING-FOR-HASH-TABLE had catastrophic
consequences if any system table was involved. I discovered the problem
in particular on SB-DI::*COMPILED-DEBUG-FUNS* which meant that you were
crashing while crashing, because you couldn't extract cached debug funs.
Test case by Gemini
For some reason gen. ai is still awful at writing comprehensible
prose, so re-do the comments manually. (I actually published the last
set of commits prematurely before getting a chance to fix the writing
and other details, though the code was checked already).
A loop written as a local call inferred worse types at its references
than the same loop written with an assignment, and everything that
produced the difference was on the local call side.
Constraint propagation knew nothing about a local function's
parameters. For a LET it adds a TYPEP constraint from the argument the
variable is bound to, but for any other local function it had only
LEAF-DEFINED-TYPE to go on, so the parameters reached the body with
nothing said about them. An imperative loop's variable, meanwhile, is
refined a step per round by exactly this machinery: COMPUTE-BLOCK-IN
joins what each predecessor knows, which around a loop peels one value
of the sequence per iteration until
*DERIVED-NUMERIC-UNION-COMPLEXITY-LIMIT* collapses the rest.
So constrain the parameters at the call. Binding one is an assignment
to it and carries the same two obligations a CSET does: the callee
learns the argument's type, and what was known of the parameter stops
holding -- which matters when the call is recursive, since the
parameter is then one the calling block has constraints about.
COMPUTE-BLOCK-IN already joins these across the call sites.
That alone changes nothing, because PROPAGATE-LOCAL-CALL-ARGS was
narrowing the same variables from a union recomputed out of arguments
derived from the variables themselves. Around a loop that sharpens on
every visit and never settles, and being the narrower of the two it
won. Leave a parameter that is being solved for optimistically to
PUBLISH-OPTIMISTIC-TYPES, the way a variable with sets is left to
PROPAGATE-FROM-SETS. The optimistic side reaches the same answer
convergently and constraint propagation sharpens the references from
there.
With the optimistic side now the only thing narrowing these
parameters, the rate at which it publishes is the rate at which they
sharpen, and IR1-OPTIMIZE-UNTIL-DONE was charging each publication to
*MAX-OPTIMIZE-ITERATIONS*. A pass that found nothing itself and only
published no longer counts against that budget: types only narrow, so
there are finitely many such passes, and the budget still bounds the
passes that do work. Without this, DELETED-CALL-TYPE and
LOCAL-CALLS-TO-&REST run out of rounds with types still stale.
All nine step shapes in (:LOCAL-CALL-ARG-TYPE :SPELLING-PARITY) now
derive identically to their imperative spelling. (ASH X 1), which had
no fixpoint before and drifted with *MAX-OPTIMIZE-ITERATIONS*, settles
on the answer DO reaches and stays there at 3, 6 and 12 iterations.
Implementation and test cases written by Claude Opus 5.
A loop variable carried through a local call still got a worse type
than the same loop written imperatively, in two ways.
An argument that steps the parameter through a known function of
itself contributed whatever IR1 derived for it, and IR1 derived that
from the function's ftype applied to a parameter that was still T:
(do ((x (list 1 2 3) (nreverse x))) (...)) => CONS
(labels ((rec (i x) (... (rec (1- i) (nreverse x)))))
(rec n (list 1 2 3))) => SEQUENCE
And a numeric parameter stepped by an increment got no bounds at all,
where an assigned one gets them from the direction of the step:
(do ((n 0 (1+ n))) (...)) => UNSIGNED-BYTE
(labels ((rec (i n) (... (rec (1- i) (1+ n)))))
(rec k 0)) => NUMBER
PROPAGATE-FROM-SETS has both answers already, in
SET-TYPE-OF-COMBINATION and MAYBE-INFER-ITERATION-VAR-TYPE, and
neither was reachable from the local call side.
So recognize an argument that carries a parameter forward from its own
previous value -- a step -- and keep it apart from the arguments that
deliver an unrelated value. What the steps say about the parameter is
then decided by the same means and in the same order as for a variable
with assignments: a loop counter's bounds if the steps are increments,
otherwise a fixpoint derived through the stepping function if there is
a single one, otherwise the plain union as before. A SETQ and an
argument in the parameter's own position are the same thing written two
ways, so the machinery is shared rather than mirrored:
%ANALYZE-SET-USES, SETS-NUMERIC-CONTAGION and
MAYBE-INFER-ITERATION-VAR-TYPE now take the lvars the stepped values
arrive on instead of a list of CSETs, the ({+,-} VAR STEP) test both of
the first two opened with becomes ITERATION-STEP-VALUE, and
CONVERGED-TYPE-OF-COMBINATION is the fixpoint half of
SET-TYPE-OF-COMBINATION.
A step is the only kind of contribution where an optimistic type
reaches a derivation, which is where iterating a union upward would
need widening, since (REC (1- I) (CONS 1 X)) grows X by a level every
round. Neither route iterates: the counter case reads the direction of
the step and keeps only the bound the variable moves away from, and the
fixpoint case derives twice and accepts its answer only if the second
derivation agrees with the first.
This closes the FIXME on the counter case in (:LOCAL-CALL-ARG-TYPE
:CYCLE), which now derives UNSIGNED-BYTE as the imperative loop does.
A step by more than one is still not at parity and is left as a failing
test. Each round of optimization re-derives the step from the
variable's current type and unions one more value in, and the two
spellings get a different number of rounds before they settle: for
(+ X 2) from 1, DO peels 1,3,5,7,9,11 and then takes [13,inf) while the
local call peels 1,3 and then takes [5,inf). Nothing different is
inferred about the loop -- at the variable itself the local call is the
narrower of the two -- and it is only at a reference, where constraint
propagation has had those extra rounds to work with, that DO comes out
ahead.
Implementation and test cases written by Claude Opus 5.
Instead of trying to compare cleanups modulo harmlessness directly as
in the code disabled in 6cbab77cdb,
check for harmlessness of the cleanup nesting from the call node to
the lambda block directly. This allows us to re-enable the Fluet &
Weeks tests.
We also catch another case of assignment conversion where mutually
tail recursive functions are entered by tail calls from different
functions.
Trying to assignment convert more lambdas whose calls have different
cleanups but not modulo harmfulness only *almost* works. While
conceptually it's fine, it causes problems during cleanup
emission. When we merge lambdas, we anchor to a specific
lexenv/cleanup, not to the messiest common ancestor. Since cleaup
emission itself has no notion of skipping harmless cleanups, it tries
to emit cleanups when it shouldn't, breaking things. I think the
actual correct as in optimal way to do it is to make cleanup emission
also ignore harmless cleanups as well, but that gets a bit hairy with
the existing cleanup sharing code in EMIT-CLEANUPS. So give up and go
back to the slightly dumber cleanup checking code.
No LLM/AI was used to diagnose this issue, so the responsibility for
this rambling is mine.
Unfortunately, a few of the assignment conversion tests (but not all of
the new ones) have to be turned off because the block tag introduced
by labels gets in the way of the optimization. Even for these tests we
still contify more than before though.
Fixes #lp2162990.
PROPAGATE-LOCAL-CALL-ARGS computes a local function's parameter types
by unioning the argument types across its call sites, starting each
parameter at T and narrowing. That descends the lattice, and it cannot
converge once the argument flow has a cycle in it: a call that hands a
parameter back to itself contributes the parameter's own current type,
so the union comes out (UNION <whatever> T) = T on every round and
stays there.
This means that a loop written tail recursively often gets less
precise types than its imperative variant:
(do ((i n (1- i)) (x (list 1) x)) ((zerop i) x)) => CONS
(labels ((rec (i x) (if (zerop i) x (rec (1- i) x))))
(rec n (list 1))) => T
We solve this by solving the local call argument type equations from
the bottom of the type lattice (i.e. from the empty type) as well,
only propagating the final optimistic type once the least fixpoint is
hit.
Fixes lp#486416.
Test cases and implementation sketch written by Claude Opus 5, code
and comments heavily simplified and edited by me.
The technique is to find "equivalent" loads in between which there is no
computation that affects the result of the load. Also it needs a surrounding
LET, which is augmented with a new temp variable as if the user did that.
This is slightly deficient for various reasons:
1. it won't do anything without the LET
2. the more kinds of common subexpressions we allow (such as math),
the slower IR1-OPTIMIZE-COMBINATION is going to run
3. there are possibly other node types that should be allowed to intervene
On the plus side, it's not all that hard to extend the logic to accept
other functions as participants in common subexpressions.
All test cases plus a little bit of assistance from Gemini
concatenated-stream wasn't processing buffered input correctly.
two-way-stream and echo-stream should just return NIL, as
file-position can't apply to both streams.
Improve assignment conversion even more with what I believe to be a
maximal contification algorithm, finally closing the loop on the
comments headed by ###. It can now handle transitive tail calls, even
in mutual recursive scenarios. The cleanup limitation was also relaxed
to allow more local functions to get converted. We now have 356
assignment converted lambdas compared to the baseline of 340 in
self-build. State machine code using local functions heavily should
benefit even more (where higher order control flow gets transformed
into first-order TAGBODY/GO-like code, which is beneficial
particularly for dataflow analyses).
Here is the list of functions in self-build where we now assignment
convert more. The codegen can be easily verified to be better by
e.g. disassembling the code component for
DFO-SCAVENGE-DEPENDENCY-GRAPH.
* PROCESS-REF, PROCESS-LVAR in SB-C::NODE-CONSTANT
* FAST-NTHCDR in CL:NTHCDR
* SET-VALUE in SB-KERNEL::RESTART-UNBOUND
* (LAMBDA (K)) in PRINT-FLOAT, FLONUM-TO-DIGITS, FLONUM-EXPONENT
* EMIT-STRING in SB-FORMAT::COMBINE-DIRECTIVES
* CURRENT-FRAME in SB-DEBUG::BACKTRACE-START-FRAME
* TERMINATE-INITS in ATTEMPT-PSEUDO-ATOMIC-STORE-BUNCHING
* SCAVENGE-POSSIBLY-DELETED-LAMBDA in DFO-SCAVENGE-DEPENDENCY-GRAPH
* FAIL in CL:NCONC, FINISH in SB-SEQUENCE:DELETE-IF-NOT,
RETURN-NO-TIMEOUT in DECODE-TIMEOUT, BOUNDS-ERROR in ir1util.lisp,
CALL-NEXT-METHOD-0 in sb-simple-streams
As a side note for those interested in AI benchmarking: I tried to use
Claude Code Opus 4.6 for this task after writing the Fluet and Weeks
5.1 and 5.2 test cases manually. It completely failed and did nothing
useful after several days of grinding and even tripped over mismatched
close parentheses several times. The task was too difficult.
Opus 5 essentially one-shotted the task. The code from it is used here
almost verbatim with some minor changes, though the comments were
heavily edited to be more concise and understandable. It was checked
thoroughly to be close to something I would write myself and it even
came up with some more test cases here that exercise additional cases
that need to be considered while fixing them immediately after
prompting. So, color me impressed.
Not compiled in yet but potentially part of the adaptive CONSET which
dynamically chooses between simple-bit-vector or sparse set of integers.
(Since constraints are indexed by small integers, CONSETs don't need SSETs
that store the constraints themselves - integers will do just fine.)
Iterating over the second set is preferred as it avoids an intermediate list,
and is worse only if the second set is much larger than the first.
Test cases by Gemini
* Since tombstones do not exist, there is no reason to separately
record the available cell count and in-use element count.
* SSET-ADJOIN need not eagerly grow the SSET except when the vector
length is 0. Only if it discovers that it must insert and the
capacity is reached should it call SSET-GROW.
* SSET-GROW can avoid some conditional branching now that it won't
be called with a vector length of 0.
Test cases and some assistance by Gemini
