sbcl.sbcl/doc/manual/streams.texinfo

@c Generated by the sb-manual contrib. Do not edit.

@node streams
@chapter Streams

@menu
* Stream External Formats: stream external formats.
* Bivalent Streams: bivalent streams.
* Gray Streams: gray streams.
* Simple Streams: sb simple streams.
@end menu

Streams which read or write Lisp character data from or to the outside
world -- files, sockets or other external entities -- require the
specification of a conversion between the external, binary data and
the Lisp characters. In ANSI Common Lisp, this is done by specifying
the @code{:external-format} argument when the stream is created. The major
information required is an @emph{encoding}, specified by a keyword naming
that encoding; however, it is also possible to specify refinements
to that encoding as additional options to the external format
designator.

In addition, SBCL supports various extensions of ANSI Common Lisp
streams:

@itemize
@item @emph{Bivalent Streams}: A type of stream that can read and write both
  @code{character} and @code{(unsigned-byte 8)} values.

@item @emph{Gray Streams}: User-overloadable CLOS classes whose instances can
  be used as Lisp streams (e.g. passed as the first argument to
  @code{format}).

@item @emph{Simple Streams}: The bundled contrib module @code{sb-simple-streams}
  implements a subset of the Franz Allegro simple-streams proposal.
@end itemize

@node stream external formats
@section Stream External Formats

@cindex external format
@cindex format, external
The function @code{stream-external-format} returns the canonical name of
the external format (see @ref{external formats}) used by the stream for
character-based input and/or output.

When constructing file streams, for example using @code{open} or
@code{with-open-file}, the external format to use is specified via the
@code{:external-format} argument which accepts an external format
designator (see @ref{external format designators}).

@node bivalent streams
@section Bivalent Streams

A @emph{bivalent stream} can be used to read and write both
@code{character} and @code{(unsigned-byte 8)} values. A bivalent stream is
created by calling @code{open} with the argument @code{:element-type}
@code{:default}. On such a stream, both binary and character data can be
read and written with the usual input and output functions.

Streams are @emph{not} created bivalent by default for performance
reasons. Bivalent streams are incompatible with @code{fast-read-char}, an
internal optimization in SBCL's stream machinery that bulk-converts
octets to characters and implements a fast path through @code{read-char}.

@node gray streams
@section Gray Streams

@menu
* Gray Streams classes: gray streams classes.
* Methods common to all streams: methods common to all streams.
* Input stream methods: input stream methods.
* Character input stream methods: character input stream methods.
* Output stream methods: output stream methods.
* Character output stream methods: character output stream methods.
* Binary stream methods: binary stream methods.
* Gray Streams Examples: gray streams examples.
@end menu

The Gray Streams interface is a widely supported extension that
provides for definition of CLOS-extensible stream classes. Gray
stream classes are implemented by adding methods to generic
functions analogous to Common Lisp's standard I/O functions.
Instances of Gray stream classes may be used with any I/O operation
where a non-Gray stream can, provided that all required methods have
been implemented suitably.

@node gray streams classes
@subsection Gray Streams classes

The defined Gray Stream classes are these:

@anchor{Class sb-gray fundamental-stream}
@ttindex @sortas{fundamental-stream sb-gray} fundamental-stream [sb-gray]
@deffn{Class} sb-gray:fundamental-stream
Base class for all Gray streams.
@end deffn
@anchor{Class sb-gray fundamental-input-stream}
@ttindex @sortas{fundamental-input-stream sb-gray} fundamental-input-stream [sb-gray]
@deffn{Class} sb-gray:fundamental-input-stream
Superclass of all Gray input streams.
@end deffn
The function @code{input-stream-p} will return true of any generalized
instance of @code{sb-gray:fundamental-input-stream}.

@anchor{Class sb-gray fundamental-output-stream}
@ttindex @sortas{fundamental-output-stream sb-gray} fundamental-output-stream [sb-gray]
@deffn{Class} sb-gray:fundamental-output-stream
Superclass of all Gray output streams.
@end deffn
The function @code{output-stream-p} will return true of any generalized
instance of @code{sb-gray:fundamental-output-stream}.

@anchor{Class sb-gray fundamental-binary-stream}
@ttindex @sortas{fundamental-binary-stream sb-gray} fundamental-binary-stream [sb-gray]
@deffn{Class} sb-gray:fundamental-binary-stream
Superclass of all Gray streams whose element-type
is a subtype of unsigned-byte or signed-byte.
@end deffn
Note that instantiable subclasses of @code{sb-gray:fundamental-binary-stream}
should provide (or inherit) an applicable method for the generic
function @code{stream-element-type}.

@anchor{Class sb-gray fundamental-character-stream}
@ttindex @sortas{fundamental-character-stream sb-gray} fundamental-character-stream [sb-gray]
@deffn{Class} sb-gray:fundamental-character-stream
Superclass of all Gray streams whose element-type is a subtype of character.
@end deffn
@anchor{Class sb-gray fundamental-binary-input-stream}
@ttindex @sortas{fundamental-binary-input-stream sb-gray} fundamental-binary-input-stream [sb-gray]
@deffn{Class} sb-gray:fundamental-binary-input-stream
Superclass of all Gray input streams whose element-type
is a subtype of unsigned-byte or signed-byte.
@end deffn
@anchor{Class sb-gray fundamental-binary-output-stream}
@ttindex @sortas{fundamental-binary-output-stream sb-gray} fundamental-binary-output-stream [sb-gray]
@deffn{Class} sb-gray:fundamental-binary-output-stream
Superclass of all Gray output streams whose element-type
is a subtype of unsigned-byte or signed-byte.
@end deffn
@anchor{Class sb-gray fundamental-character-input-stream}
@ttindex @sortas{fundamental-character-input-stream sb-gray} fundamental-character-input-stream [sb-gray]
@deffn{Class} sb-gray:fundamental-character-input-stream
Superclass of all Gray input streams whose element-type
is a subtype of character.
@end deffn
@anchor{Class sb-gray fundamental-character-output-stream}
@ttindex @sortas{fundamental-character-output-stream sb-gray} fundamental-character-output-stream [sb-gray]
@deffn{Class} sb-gray:fundamental-character-output-stream
Superclass of all Gray output streams whose element-type
is a subtype of character.
@end deffn
@node methods common to all streams
@subsection Methods common to all streams

These generic functions can be specialized on any generalized instance
of fundamental-stream.

@anchor{Generic function common-lisp stream-element-type}
@ffindex @sortas{stream-element-type common-lisp} stream-element-type [common-lisp]
@deffn{Generic function} stream-element-type stream
Return a type specifier for the kind of object returned by the
@code{stream}. The class @code{sb-gray:fundamental-character-stream} provides a
default method which returns @code{character}.
@end deffn
@anchor{Generic function common-lisp close}
@ffindex @sortas{close common-lisp} close [common-lisp]
@deffn{Generic function} close stream &key abort
Close the given @code{stream}. No more I/O may be performed, but
inquiries may still be made. If @code{:abort} is true, an attempt is made
to clean up the side effects of having created the stream.
@end deffn
@anchor{Generic function sb-gray stream-file-position}
@ffindex @sortas{stream-file-position sb-gray} stream-file-position [sb-gray]
@deffn{Generic function} sb-gray:stream-file-position stream &optional position-spec
Used by @code{file-position}. Returns or changes the current position within @code{stream}.
@end deffn
@node input stream methods
@subsection Input stream methods

These generic functions may be specialized on any generalized instance
of fundamental-input-stream.

@anchor{Generic function sb-gray stream-clear-input}
@ffindex @sortas{stream-clear-input sb-gray} stream-clear-input [sb-gray]
@deffn{Generic function} sb-gray:stream-clear-input stream
This is like @code{cl:clear-input}, but for Gray streams, returning @code{nil}.
The default method does nothing.
@end deffn
@anchor{Generic function sb-gray stream-read-sequence}
@ffindex @sortas{stream-read-sequence sb-gray} stream-read-sequence [sb-gray]
@deffn{Generic function} sb-gray:stream-read-sequence stream seq &optional start end
This is like @code{cl:read-sequence}, but for Gray streams.
@end deffn
@node character input stream methods
@subsection Character input stream methods

These generic functions are used to implement subclasses of
@code{sb-gray:fundamental-input-stream}:

@anchor{Generic function sb-gray stream-peek-char}
@ffindex @sortas{stream-peek-char sb-gray} stream-peek-char [sb-gray]
@deffn{Generic function} sb-gray:stream-peek-char stream
This is used to implement @code{peek-char}; this corresponds to @code{peek-type}
of @code{nil}. It returns either a character or @code{:eof}. The default method
calls @code{stream-read-char} and @code{stream-unread-char}.
@end deffn
@anchor{Generic function sb-gray stream-read-char-no-hang}
@ffindex @sortas{stream-read-char-no-hang sb-gray} stream-read-char-no-hang [sb-gray]
@deffn{Generic function} sb-gray:stream-read-char-no-hang stream
This is used to implement @code{read-char-no-hang}. It returns either a
character, or @code{nil} if no input is currently available, or @code{:eof} if
end-of-file is reached. The default method provided by
@code{fundamental-character-input-stream} simply calls @code{stream-read-char}; this
is sufficient for file streams, but interactive streams should define
their own method.
@end deffn
@anchor{Generic function sb-gray stream-read-char}
@ffindex @sortas{stream-read-char sb-gray} stream-read-char [sb-gray]
@deffn{Generic function} sb-gray:stream-read-char stream
Read one character from the stream. Return either a
character object, or the symbol @code{:eof} if the stream is at end-of-file.
Every subclass of @code{fundamental-character-input-stream} must define a
method for this function.
@end deffn
@anchor{Generic function sb-gray stream-read-line}
@ffindex @sortas{stream-read-line sb-gray} stream-read-line [sb-gray]
@deffn{Generic function} sb-gray:stream-read-line stream
This is used by @code{read-line}. A string is returned as the first value. The
second value is true if the string was terminated by end-of-file
instead of the end of a line. The default method uses repeated
calls to @code{stream-read-char}.
@end deffn
@anchor{Generic function sb-gray stream-listen}
@ffindex @sortas{stream-listen sb-gray} stream-listen [sb-gray]
@deffn{Generic function} sb-gray:stream-listen stream
This is used by @code{listen}. It returns true or false. The default method uses
@code{stream-read-char-no-hang} and @code{stream-unread-char}. Most streams should
define their own method since it will usually be trivial and will
always be more efficient than the default method.
@end deffn
@anchor{Generic function sb-gray stream-unread-char}
@ffindex @sortas{stream-unread-char sb-gray} stream-unread-char [sb-gray]
@deffn{Generic function} sb-gray:stream-unread-char stream character
Undo the last call to @code{stream-read-char}, as in @code{unread-char}.
Return @code{nil}. Every subclass of @code{fundamental-character-input-stream}
must define a method for this function.
@end deffn
@node output stream methods
@subsection Output stream methods

These generic functions are used to implement subclasses of
@code{sb-gray:fundamental-output-stream}:

@anchor{Generic function sb-gray stream-clear-output}
@ffindex @sortas{stream-clear-output sb-gray} stream-clear-output [sb-gray]
@deffn{Generic function} sb-gray:stream-clear-output stream
This is like @code{cl:clear-output}, but for Gray streams: clear the given
output @code{stream}. The default method does nothing.
@end deffn
@anchor{Generic function sb-gray stream-finish-output}
@ffindex @sortas{stream-finish-output sb-gray} stream-finish-output [sb-gray]
@deffn{Generic function} sb-gray:stream-finish-output stream
Attempts to ensure that all output sent to the Stream has reached
its destination, and only then returns false. Implements
@code{finish-output}. The default method does nothing.
@end deffn
@anchor{Generic function sb-gray stream-force-output}
@ffindex @sortas{stream-force-output sb-gray} stream-force-output [sb-gray]
@deffn{Generic function} sb-gray:stream-force-output stream
Attempts to force any buffered output to be sent. Implements
@code{force-output}. The default method does nothing.
@end deffn
@anchor{Generic function sb-gray stream-write-sequence}
@ffindex @sortas{stream-write-sequence sb-gray} stream-write-sequence [sb-gray]
@deffn{Generic function} sb-gray:stream-write-sequence stream seq &optional start end
This is like @code{cl:write-sequence}, but for Gray streams.
@end deffn
@node character output stream methods
@subsection Character output stream methods

These generic functions are used to implement subclasses of
@code{sb-gray:fundamental-character-output-stream}:

@anchor{Generic function sb-gray stream-advance-to-column}
@ffindex @sortas{stream-advance-to-column sb-gray} stream-advance-to-column [sb-gray]
@deffn{Generic function} sb-gray:stream-advance-to-column stream column
Write enough blank space so that the next character will be
written at the specified column. Returns true if the operation is
successful, or @code{nil} if it is not supported for this stream. This is
intended for use by by @code{pprint} and @code{format} @code{~t}. The default method
uses @code{stream-line-column} and repeated calls to @code{stream-write-char}
with a #SPACE character; it returns @code{nil} if @code{stream-line-column}
returns @code{nil}.
@end deffn
@anchor{Generic function sb-gray stream-fresh-line}
@ffindex @sortas{stream-fresh-line sb-gray} stream-fresh-line [sb-gray]
@deffn{Generic function} sb-gray:stream-fresh-line stream
Outputs a new line to the Stream if it is not positioned at the
beginning of a line. Returns @code{t} if it output a new line, nil
otherwise. Used by @code{fresh-line}. The default method uses
@code{stream-start-line-p} and @code{stream-terpri}.
@end deffn
@anchor{Generic function sb-gray stream-line-column}
@ffindex @sortas{stream-line-column sb-gray} stream-line-column [sb-gray]
@deffn{Generic function} sb-gray:stream-line-column stream
Return the column number where the next character
will be written, or @code{nil} if that is not meaningful for this stream.
The first column on a line is numbered 0. This function is used in
the implementation of @code{pprint} and the @code{format} @code{~t} directive. For every
character output stream class that is defined, a method must be
defined for this function, although it is permissible for it to
always return @code{nil}.
@end deffn
@anchor{Generic function sb-gray stream-line-length}
@ffindex @sortas{stream-line-length sb-gray} stream-line-length [sb-gray]
@deffn{Generic function} sb-gray:stream-line-length stream
Return the stream line length or @code{nil}.
@end deffn
@anchor{Generic function sb-gray stream-start-line-p}
@ffindex @sortas{stream-start-line-p sb-gray} stream-start-line-p [sb-gray]
@deffn{Generic function} sb-gray:stream-start-line-p stream
Is @code{stream} known to be positioned at the beginning of a line?
It is permissible for an implementation to always return
@code{nil}. This is used in the implementation of @code{fresh-line}. Note that
while a value of 0 from @code{stream-line-column} also indicates the
beginning of a line, there are cases where @code{stream-start-line-p} can be
meaningfully implemented although @code{stream-line-column} can't be. For
example, for a window using variable-width characters, the column
number isn't very meaningful, but the beginning of the line does have
a clear meaning. The default method for @code{stream-start-line-p} on class
@code{fundamental-character-output-stream} uses @code{stream-line-column}, so if
that is defined to return @code{nil}, then a method should be provided for
either @code{stream-start-line-p} or @code{stream-fresh-line}.
@end deffn
@anchor{Generic function sb-gray stream-terpri}
@ffindex @sortas{stream-terpri sb-gray} stream-terpri [sb-gray]
@deffn{Generic function} sb-gray:stream-terpri stream
Writes an end of line, as for @code{terpri}. Returns @code{nil}. The default
method does (@code{stream-write-char} stream @code{#\Newline}).
@end deffn
@anchor{Generic function sb-gray stream-write-char}
@ffindex @sortas{stream-write-char sb-gray} stream-write-char [sb-gray]
@deffn{Generic function} sb-gray:stream-write-char stream character
Write @code{character} to @code{stream} and return @code{character}. Every
subclass of @code{fundamental-character-output-stream} must have a method
defined for this function.
@end deffn
@anchor{Generic function sb-gray stream-write-string}
@ffindex @sortas{stream-write-string sb-gray} stream-write-string [sb-gray]
@deffn{Generic function} sb-gray:stream-write-string stream string &optional start end
This is used by @code{write-string}. It writes the string to the stream,
optionally delimited by start and end, which default to 0 and @code{nil}.
The string argument is returned. The default method provided by
@code{fundamental-character-output-stream} uses repeated calls to
@code{stream-write-char}.
@end deffn
@node binary stream methods
@subsection Binary stream methods

The following generic functions are available for subclasses of
@code{sb-gray:fundamental-binary-stream}:

@anchor{Generic function sb-gray stream-read-byte}
@ffindex @sortas{stream-read-byte sb-gray} stream-read-byte [sb-gray]
@deffn{Generic function} sb-gray:stream-read-byte stream
Used by @code{read-byte}; returns either an integer, or the symbol @code{:eof}
if the stream is at end-of-file.
@end deffn
@anchor{Generic function sb-gray stream-write-byte}
@ffindex @sortas{stream-write-byte sb-gray} stream-write-byte [sb-gray]
@deffn{Generic function} sb-gray:stream-write-byte stream integer
Implements @code{write-byte}; writes the integer to the stream and
returns the integer as the result.
@end deffn
@node gray streams examples
@subsection Gray Streams Examples

@menu
* Character Counting Input Stream: character counting input stream.
* Output Prefixing Character Stream: output prefixing character stream.
@end menu

Below are two classes of stream that can be conveniently defined as
wrappers for Common Lisp streams. These are meant to serve as
examples of minimal implementations of the protocols that must be
followed when defining Gray streams. Realistic uses of the Gray
Streams API would implement the various methods that can do I/O in
batches, such as @code{sb-gray:stream-read-line},
@code{sb-gray:stream-write-string}, @code{sb-gray:stream-read-sequence}, and
@code{sb-gray:stream-write-sequence}.

@node character counting input stream
@subsubsection Character Counting Input Stream

  It is occasionally handy for programs that process input files to
count the number of characters and lines seen so far, and the number
of characters seen on the current line, so that useful messages may
be reported in case of parsing errors, etc. Here is a character
input stream class that keeps track of these counts. Note that all
character input streams must implement @code{sb-gray:stream-read-char} and
@code{sb-gray:stream-unread-char}.

@example
(defclass wrapped-stream (fundamental-stream)
  ((stream :initarg :stream :reader stream-of)))

(defmethod stream-element-type ((stream wrapped-stream))
  (stream-element-type (stream-of stream)))

(defmethod close ((stream wrapped-stream) &key abort)
  (close (stream-of stream) :abort abort))

(defclass wrapped-character-input-stream
    (wrapped-stream fundamental-character-input-stream)
  ())

(defmethod stream-read-char ((stream wrapped-character-input-stream))
  (read-char (stream-of stream) nil :eof))

(defmethod stream-unread-char ((stream wrapped-character-input-stream)
                               char)
  (unread-char char (stream-of stream)))

(defclass counting-character-input-stream
    (wrapped-character-input-stream)
  ((char-count :initform 1 :accessor char-count-of)
   (line-count :initform 1 :accessor line-count-of)
   (col-count :initform 1 :accessor col-count-of)
   (prev-col-count :initform 1 :accessor prev-col-count-of)))

(defmethod stream-read-char ((stream counting-character-input-stream))
  (with-accessors ((inner-stream stream-of) (chars char-count-of)
                   (lines line-count-of) (cols col-count-of)
                   (prev prev-col-count-of)) stream
      (let ((char (call-next-method)))
        (cond ((eql char :eof)
               :eof)
              ((char= char #Newline)
               (incf lines)
               (incf chars)
               (setf prev cols)
               (setf cols 1)
               char)
              (t
               (incf chars)
               (incf cols)
               char)))))

(defmethod stream-unread-char ((stream counting-character-input-stream)
                               char)
  (with-accessors ((inner-stream stream-of) (chars char-count-of)
                   (lines line-count-of) (cols col-count-of)
                   (prev prev-col-count-of)) stream
      (cond ((char= char #Newline)
             (decf lines)
             (decf chars)
             (setf cols prev))
            (t
             (decf chars)
             (decf cols)
             char))
      (call-next-method)))
@end example

The default methods for @code{sb-gray:stream-read-char-no-hang},
@code{sb-gray:stream-peek-char}, @code{sb-gray:stream-listen},
@code{sb-gray:stream-clear-input}, @code{sb-gray:stream-read-line}, and
@code{sb-gray:stream-read-sequence} should be sufficient (though the last
two will probably be slower than methods that forwarded directly).

Here's a sample use of this class:

@example
(with-input-from-string (input "1 2
 3 :foo  ")
  (let ((counted-stream (make-instance 'counting-character-input-stream
                         :stream input)))
    (loop for thing = (read counted-stream) while thing
       unless (numberp thing) do
         (error "Non-number ~S (line ~D, column ~D)" thing
                (line-count-of counted-stream)
                (- (col-count-of counted-stream)
                   (length (format nil "~S" thing))))
       end
       do (print thing))))
@end example

Output:

@example
1
2
3
Non-number :FOO (line 2, column 5)
  [Condition of type SIMPLE-ERROR]
@end example

@node output prefixing character stream
@subsubsection Output Prefixing Character Stream

One use for a wrapped output stream might be to prefix each line of
text with a timestamp, e.g. for a logging stream. Here's a simple
stream that does this, though without any fancy line-wrapping. Note
that all character output stream classes must implement
@code{sb-gray:stream-write-char} and @code{sb-gray:stream-line-column}.

@example
(defclass wrapped-stream (fundamental-stream)
  ((stream :initarg :stream :reader stream-of)))

(defmethod stream-element-type ((stream wrapped-stream))
  (stream-element-type (stream-of stream)))

(defmethod close ((stream wrapped-stream) &key abort)
  (close (stream-of stream) :abort abort))

(defclass wrapped-character-output-stream
    (wrapped-stream fundamental-character-output-stream)
  ((col-index :initform 0 :accessor col-index-of)))

(defmethod stream-line-column ((stream wrapped-character-output-stream))
  (col-index-of stream))

(defmethod stream-write-char ((stream wrapped-character-output-stream)
                              char)
  (with-accessors ((inner-stream stream-of) (cols col-index-of)) stream
    (write-char char inner-stream)
    (if (char= char #Newline)
        (setf cols 0)
        (incf cols))))

(defclass prefixed-character-output-stream
    (wrapped-character-output-stream)
  ((prefix :initarg :prefix :reader prefix-of)))

(defgeneric write-prefix (prefix stream)
  (:method ((prefix string) stream) (write-string prefix stream))
  (:method ((prefix function) stream) (funcall prefix stream)))

(defmethod stream-write-char ((stream prefixed-character-output-stream)
                              char)
  (with-accessors ((inner-stream stream-of) (cols col-index-of)
                   (prefix prefix-of)) stream
    (when (zerop cols)
      (write-prefix prefix inner-stream))
    (call-next-method)))
@end example

As with the example input stream, this implements only the minimal
protocol. A production implementation should also provide methods
for at least @code{sb-gray:stream-write-string},
@code{sb-gray:stream-write-sequence}.

And here's a sample use of this class:

@example
(flet ((format-timestamp (stream)
         (apply #'format stream "[~2@@*~2,' D:~1@@*~2,'0D:~0@@*~2,'0D] "
                (multiple-value-list (get-decoded-time)))))
  (let ((output (make-instance 'prefixed-character-output-stream
                               :stream *standard-output*
                               :prefix #'format-timestamp)))
    (loop for string in '("abc" "def" ")ghi") do
         (write-line string output)
         (sleep 1))))
@end example

Output:

@example
[ 0:30:05] abc
[ 0:30:06] def
[ 0:30:07] ghi
NIL
@end example

@include ../../contrib/sb-simple-streams/sb-simple-streams.texinfo