It's a little hard to envision a use for the GET/SET/END??END routines
that the DO- macros don't cover. Since the GET/SET/END routines are
not reentrant, calling them during a DO- iteration can only serve to
mess up that iteration, so making those functions available seems only
to expose error possibilities. And if those functions are not external
to SB-POSIX, then users don't need WITH-*-DATABASE macros either.
Have GETPWNAM, GETPWUID implicitly use WITH-PASSWD-DATABASE; GETGRNAM,
GETGRGID use WITH-GROUP-DATABASE. Require that GETPWENT, SETPWENT,
ENDPWENT be called during the extent of a WITH-PASSWD-DATABASE, and
likewise for the group functions.
Add simple tests for these. (These tests add 2 seconds to the
sb-posix test suite runtime, unfortunately.)
These protect against concurrent uses of the respective databases in
distinct threads, as well as prohibit attempts at reentrant use. It's
up to users to wrap their getpw* and getgr* calls in these macros.
(No tests are added for these because it seemed as if doing so would
merely exercise WITH-MUTEX, which is tested for elsewhere.)
Specifically: getpwnam, getpwuid, getgrnam, getgruid, readdir, and
getenv. For all but getenv, POSIX describes the convention that a NULL
return value and "unchanged" errno is a non-error (including
end-of-directory for readdir, no-such-user for getpw*, etc), but NULL
and a changed errno is an error. SB-POSIX hasn't ever signaled errors
for a changed errno before, but probably should. (As for getenv, POSIX
does not define any errors for it, but FreeBSD and OSX document
setting errno to EINVAL sometimes; and FreeBSD actually behaves as
documented; so it seems reasonable to bind to whatever the OS does.)
Unfortunately, it's unclear how to force these errors for testing. For
the ones having to do with the passwd and group databases, SBCL
probably can't and certainly shouldn't try to interfere with them. For
readdir, I was unable to induce an error in a few experiments
concurrently modifying a directory while SBCL was enumerating it; and
the getenv case isn't required to set errno anywhere.
Put "--without-gencgc --with-mark-region-gc" in the make.sh invocation
to enable it (which automatically disables immobile space for now).
There are some known test failures.
Big thanks to Hayley Patton for this.
This is a modified version of Douglas Katzman's commit 869f2acd62
(Remove non-pauseless-threadstart code), which gets the
pauseless-threadstart feature to work on Windows. Since Windows is the
last outstanding platform that builds with pauseless-threadstart
disabled by default, this commit also removes the pauseless-threadstart
feature flag entirely.
The original commit was modified so that it applies to 2.2.7 as well as
to remove the setup semaphore and to set th->no_tls_value_marker to
NO_TLS_VALUE_MARKER in alloc_thread_struct.
Note: The original commit was reverted by 15f2ccc551 (Revert "Remove
non-pauseless-threadstart code") due to failing tests.
This change should resolve both lp#1906571 and lp#1907872.
Delaying finalizer thread creation until just-in-time was too complicated.
Doing it at startup is less confusing because the state of the world is known,
and synchronously stopping/restarting around any call to FORK, as well as
immediately prior to SAVE-LISP-AND-DIE, is easier too.
Use a native condition variable to inform the thread when to run or exit.
Deadlock can't happen because triggering finalizers never involves make-thread
or a mutex (though native condition vars may use one internally), and there's
no hidden "possibly starting a thread" state.
Add a setup semaphore for win32 so the new code resembles the old.
I don't know why it's needed, but tests fail without. The differences
in safepoint logic for unix versus win32 might explain things.
* Remove the create_thread_lock and thread setup semaphore. The start function
is handed off in a GC-safe way and the creating thread is allowed to continue
as soon as pthread_create() returns.
* Retain the memory from up to 2 exited threads for reuse by new threads.
This is not shared with the FOREIGN-THREAD memory pool, but perhaps could be.
While this adds more conditionalization to both C and Lisp, it should be
adaptable to OS_THREAD_STACK so that ultimately we can have only one way
of performing post-mortem freeing instead of three.
Benchmarks show about 6x to 7x faster lisp thread creation.
Incidentally this fixes lp#1595699.
It's not needed to protect the AVL tree.
It was overloaded to not only do that but also prevent creation of new threads
which is better done with the make-thread lock; as well as prevent thread exit,
for which its use was completely horrible.
Since each thread already has a mutex for interruptions, use that for
SYMBOL-VALUE-IN-THREAD (and INTERRUPT-THREAD) to ensure that the memory
exists (in the former case) or the pthread ID exists (in the latter).
There is absolutely no reason to look in the C variable all_threads,
unless you're using one of the platforms/configs that I don't understand.
I assumed that this would help find the cause of crashing with #-sb-thread,
and miraculously the change actually seems to have entirely fixed the problem.
Given that observation, I surmised that the bug lay with misuse of LR as a
temp register which this change removes. Further confirmation was obtained by
reverting the patch and merely adding (LOAD-IMMEDIATE-WORD ,LIP 0) after each
inline allocation. Simply clearing LIP made the random crashing disappear.
I don't know if there was a point in the past when the non-thread build was
perfectly reliable despite the allocation macro being relatively unchanged.
It didn't seem sufficiently interesting a question for me to answer, but in
the interest of code health, I think the LIP argument should be removed.
* #.(1- (ash 1 sb-vm:n-word-bits)) can be spelled MOST-POSITIVE-WORD.
* instead of '(mumble #.(expr)) use `(mumble ,(expr)) when expr is constant.
backquote is constant-foldable in that case.
* but in particular `(signed-byte ,sb-vm:n-word-bits) is just SIGNED-WORD,
respectively `(unsigned-byte ,sb-vm:n-word-bits) is WORD.
* unrelatedly, sb-vm:n-machine-word-bits is external in its package.
Use a single source of truth for stack addresses and set of running threads.
The AVL tree is of the pure functional variety, and the root node is swapped
out for any insert or delete. The STACK-ALLOCATED-P function becomes safer -
at worst it sees a stale tree but never one that violates any invariants.
I do not know whether this fixes the duplicate insertion problem observed
in https://sourceforge.net/p/sbcl/mailman/message/36517264/
but it should allow us to figure out whether the problem pre-dated use of a
tree structure in the sense that tree insertion failure would happen only if
*ALL-THREADS* itself would have contained zombie threads.
I believe that to be the case, as the code for tree insert and delete were
guarded by the same critical section that guarded *ALL-THREADS*.
Therefore if deletion failed to happen in the tree for a previous thread
death, then it ought to have failed to happen in the list representations too.
Of course it's also possible that the treap code was buggy, and/or the
AVL code, but they were/are regression tested by a random tester. Another
scenario would be the code inserting a new thread somehow running before
the code deleting the old, but I don't understand in that case how you would
get a stack address that is the same, because the deletion runs on the stack
of the very thread that is dying. i.e. a physical stack is freeable only
after having removed yourself from *all-threads*. Therefore no new thread can
lay claim to that address space. I really think that if a thread died a
hard (horrible) death, it could fail to remove itself from any lists.
A final note - it seem that the "portable-threads" library directly uses
SB-THREAD::*ALL-THREADS* which is obviously wrong. If we can't find someone
to shoot, then we could rename the global var to something different,
and then (DEFINE-SYMBOL-MACRO *ALL-THREADS* (LIST-ALL-THREADS))
which restores the look-and-feel of the sb-thread internals.
An error from readdir is indicated as a non zero errno and a zero
return value, otherwise errno is unchanged. Set errno to 0 in the
sb_readdir wrapper.
* cfsetispeed(3) and cfsetospeed(3) are defined to mutate an
existing termios structure, but while SB-POSIX:CFSETISPEED and
SB-POSIX:CFSETOSPEED were accepting a TERMIOS structure as input,
they were not setting up the "alien" termios structure based on
the input, thus clobbering most of the termios state.
* Fix, by adding a couple of calls to TERMIOS-TO-ALIEN.
* Note that this does not help the other major part of
lp#1500951, which is confirmed to affect FreeBSD and NetBSD, and
possibly other systems: the termios structure may have fields
beyond the minimum specified set, and these fields may be used to
hold the ispeed and ospeed values.
It's not easy to determine the layout of struct stat, and it fails to
do so on Android, sb-unix:stat is implemented with a wrapper which
parses everything on the C side. Just reuse that wrapper for
sb-posix:stat.
