sbcl.sbcl/tools-for-build
Eric Timmons 4f4269f7e7 run-program: Use close_range syscall or libc wrapper before exec
FreeBSD and Linux have added the close_range syscall (in April 2020 and 5.9,
respectively). This allows for a range of fds to be closed using a single
syscall. The behavior of closefrom can be emulated using ~0U as the end fd.

FreeBSD immediately added it to their libc, but Linux libcs have lagged
behind. To support Linux systems running kernel 5.9+ without a libc wrapper for
close_range, we test for the presence of a wrapper at build time and use it if
it exists. Otherwise, fall back to the syscall directly (if the number of the
syscall can be determined at build time).

If close_range can't be found at build time, or the run time kernel does not
support it, fall back to close() in a for loop.

Linux tests were run on an x86-64 machine with ulimit -n 524288, using

    (time (dotimes (i 100) (sb-ext:run-program "/bin/true" nil)))

Typical "slow" path (no use of close_range) results were:

    Evaluation took:
      11.630 seconds of real time
      0.088556 seconds of total run time (0.013697 user, 0.074859 system)
      0.77% CPU
      1,491 forms interpreted
      26,799,410,672 processor cycles
      425,376 bytes consed

Typical "fast" path results were:

    Evaluation took:
      0.153 seconds of real time
      0.055602 seconds of total run time (0.002995 user, 0.052607 system)
      36.60% CPU
      1,491 forms interpreted
      351,242,436 processor cycles
      424,192 bytes consed

This was tested with a 5.4 kernel and a 5.13 kernel, using Ubuntu 20.04, 21.04,
and 21.10 userspaces. The 5.4 kernel has no close_range support. Ubuntu 20.04
has neither a close_range wrapper (-os-provides-close-range-wrapper) nor a
version of Linux headers with __NR_close_range defined. Ubuntu 21.04 has no
wrapper, but does have __NR_close_range defined. Ubuntu 21.10 has both a
wrapper (+os-provides-close-range-wrapper) and __NR_close_range.

The results for which path was taken are summarized below. The top row of each
table states which kernel was used at build time, the left-most column states
which kernel was used at run time.

| 20.04 | 5.4  | 5.13 |
|-------|------|------|
| 5.4   | slow | slow |
| 5.13  | slow | slow |

| 21.04 | 5.4  | 5.13 |
|-------|------|------|
| 5.4   | slow | slow |
| 5.13  | fast | fast |

| 21.10 | 5.4  | 5.13 |
|-------|------|------|
| 5.4   | slow | slow |
| 5.13  | fast | fast |

This was additionally tested on FreeBSD amd64 13.0-RELEASE. All easily
available versions of FreeBSD have close_range support. So lack of close_range
was emulated by building with --without-os-provides-close-range-wrapper.

Additionally, FreeBSD has closefrom(2), so a slightly more complicated test was
needed to force the use of close_range.

    (time (dotimes (i 100) (sb-ext:run-program "/bin/true" nil :preserve-fds (list 100000))))

As expected, the test took less time to run with
+os-provides-close-range-wrapper as opposed to
-os-provides-close-range-wrapper.
2022-04-18 20:05:14 +03:00
..
allkeys.txt rename a previously-renamed Unicode data file 2019-09-28 18:16:57 +01:00
arm-softfp-test.c Correctly detect hard/soft float ABI on arm32 2019-11-20 21:03:01 -08:00
BidiMirroring.txt update to Unicode 8.0.0 2019-09-19 21:50:24 +01:00
Blocks.txt update to Unicode 8.0.0 2019-09-19 21:50:24 +01:00
canonicalize-whitespace.lisp Support building & testing in a tree whose leaves are symlinks. 2021-05-02 14:28:34 -04:00
CaseFolding.txt build using the original CaseFolding.txt 2019-10-09 07:29:41 +01:00
CompositionExclusions.txt update to Unicode 8.0.0 2019-09-19 21:50:24 +01:00
ConfusablesEdited.txt Change confusables handling. 2019-04-09 17:14:52 +03:00
corefile.lisp Callbacks on darwin-arm64 2021-02-07 20:23:57 +03:00
DerivedAge.txt update to Unicode 8.0.0 2019-09-19 21:50:24 +01:00
DerivedNormalizationProps.txt update to Unicode 8.0.0 2019-09-19 21:50:24 +01:00
determine-endianness.c Add useless noise for the sake of Windows CI machine 2018-06-21 18:20:20 -04:00
ea-refactor.lisp x86-64: Create a BOA constructor named EA for instances of EA 2018-08-01 17:51:36 -04:00
EastAsianWidth.txt update to Unicode 8.0.0 2019-09-19 21:50:24 +01:00
editcore.lisp x86-64: never cons in GENERIC-NEGATE 2022-03-28 19:17:12 -04:00
GraphemeBreakProperty.txt update to Unicode 8.0.0 2019-09-19 21:50:24 +01:00
grovel-features.sh run-program: Use close_range syscall or libc wrapper before exec 2022-04-18 20:05:14 +03:00
grovel-headers-win32.inc win64: handle heap corruption exceptions in VEH 2021-09-15 15:16:39 +03:00
grovel-headers.c win64: handle heap corruption exceptions in VEH 2021-09-15 15:16:39 +03:00
Jamo.txt Improves Unicode support and upgrades SBCL's database to Unicode 7.0 2014-10-01 22:35:00 -05:00
LineBreak.txt update to Unicode 8.0.0 2019-09-19 21:50:24 +01:00
Makefile win32: Thoroughly test for usability of WakeByAddress 2020-09-01 11:48:42 -04:00
mmap-rwx.c Add a test for working mmap() of RWX pages on openbsd 6.0 and newer. 2018-11-08 11:05:27 -08:00
more-ucd-consts.lisp-expr update to Unicode 8.0.0 2019-09-19 21:50:24 +01:00
NormalizationCorrections.txt update to Unicode 8.0.0 2019-09-19 21:50:24 +01:00
openbsd-sigcontext.sh 1.0.42.8: fix build on OpenBSD/i386 -current and upcoming 4.8 release 2010-08-31 13:04:54 +00:00
os-provides-blksize-t-test.c 1.0.4.51: no blksize_t on OS X Panther 2007-04-08 17:34:45 +00:00
os-provides-close-range-wrapper-test.c run-program: Use close_range syscall or libc wrapper before exec 2022-04-18 20:05:14 +03:00
os-provides-dladdr-test.c Remove a check for old NetBSD. 2019-12-09 15:59:17 +03:00
os-provides-dlopen-test.c 0.8.14.5: Join the foreign legion! 2004-09-09 12:10:11 +00:00
os-provides-getprotoby-r-test.c Fix bug in os-getprotoby test 2020-02-17 21:42:45 -05:00
os-provides-poll-test.c 1.0.42.43: FD-STREAMS no longer hook into SERVE-EVENT by default 2010-09-19 20:08:47 +00:00
os-provides-suseconds-t-test.c 1.0.10.3: Grovel suseconds_t 2007-09-26 11:37:34 +00:00
os-provides-wakebyaddr-test.c win32: Thoroughly test for usability of WakeByAddress 2020-09-01 11:48:42 -04:00
pending-deprecations.lisp Use PRINT-SYMBOL-WITH-PREFIX from SB-EXT not SB-IMPL 2017-09-11 21:05:30 -04:00
PropList.txt update to Unicode 8.0.0 2019-09-19 21:50:24 +01:00
rtf.lisp Upgrade to Windows Installer XML 3.5 2012-12-05 17:34:29 +01:00
sbcl.1.4.14.nupkg CI: download windows SBCL from github. 2019-02-02 22:08:14 +03:00
Scripts.txt update to Unicode 8.0.0 2019-09-19 21:50:24 +01:00
SpecialCasing.txt update to Unicode 8.0.0 2019-09-19 21:50:24 +01:00
ucd.lisp build CaseFolding data identically on all hosts 2019-10-11 08:48:41 +01:00
UnicodeData.txt update to Unicode 8.0.0 2019-09-19 21:50:24 +01:00
where-is-mcontext.c 0.9.5.18: 2005-10-03 09:46:39 +00:00
wxs.lisp Fix instances of loop repeat .. for .. pattern 2022-04-10 22:02:11 +01:00