Generic FIFO buffer library HOWTO
Hans Petter Jansson, hpj@teletopia.no
Release 1, 19990217

This document describes the FIFO buffering facilities in libflux. It consists
of an introduction to its practical and technical background, and documents
some of the functions.


1.0.0  Introduction

The FIFO buffer is a very simple, but very useful, facility. It can hold data
which is "passing through" as a stream from one place to another - for
example over a network connection, to a storage medium, or to a terminal. An
important point which accounts for most of its usefulness is that the data
stream can be irregular - data can come in and be taken out at an irregular
pace.

This means that whenever the reading unit is ready to receive some data, it
can take it out of the buffer, instead of waiting for the writing unit to
deliver it directly (the units will probably not correspond, and they may
have to negotiate for one character at a time). Using a buffer, the writer
can deliver data whenever it's ready, and the reader can likewise take data
out immediately, when it is.

Depending on how fast data is written/read, the amount of buffered data
grows/shrinks. Therefore, the buffer should not take up much more memory than
required, although it should keep some free space in anticipation of sudden
data influx (growing or shrinking the memory allocated to the buffer takes
time). The FIFO buffers in Fluxlib are adaptive in this manner, and all the
user has to specify are the upper and lower bounds of its memory usage, and
the size of the steps with which the buffer grows/shrinks.

1.1.0  Technicalities

Reading this section is not required for general use.

Data is kept in nodes of similar size, given on initialization of the buffer.
They're linked together to form a ring. A number of spare nodes is usually
kept to accomodate large amounts of data without too much overhead. Here's a
schematic:

.-> . <-> . <-> . <-> . <-> . <-> . <-.
|                                     |
|               node_out    node_in   |
|               |           |         |
`-> . <-> . <-> o <-> O <-> o <-> . <-'
      | |
      | next
      prev

New nodes are always inserted right in front of the "in" node, while node
removal occurs right behind the "out" node.

The number of nodes is tailored to the FIFO's data flux level. Less traffic,
nodes are freed up - more traffic, nodes are allocated.

Within nodes, data is (not surprisingly) arranged like this:

    byte_out                                  byte_in
    |                                         |
[...oooooooooooo] [ooooooooooooooo] [ooooooooo......] [...............]
node_out                            node_in

1.2.0  Important functions

1.2.1  Creating and freeing a buffer structure

When creating a buffer structure, you have to specify the number of buffer
nodes (see section 1.1.0) it will allocate at a minimum (must be 2 or more),
and how many you would like to allocate as a potential maximum. You must also
specify the size, in bytes, of these buffer nodes. These parameters are
passed to fifobuf_new() which returns a pointer to the type FIFOBUF.

Recommended values: Minimum buffers, 2. Maximum buffers, 10. Buffer size,
1024 bytes (this figure should always be a power of two, as this in most
environments leads to a more efficient allocation scheme).

Freeing buffers is trivial. A pointer to the FIFOBUF is passed to
fifobuf_free() and all memory (and remaining data) associated with it is
liberated.

1.2.2  Putting data in and taking it out

Use fifobuf_enqueue() to write data to the buffer and fifobuf_dequeue() to
take it out. The header files actually describe these functions adequately -
you specify the buffer, where the data resides (or where you want it put, in
the case of fifobuf_dequeue()) and its length. Enqueueing will return -1 if
there wasn't room for all the data (and it won't enqueue anything), and
dequeuing returns the amount actually dequeued (may be less than the amount
asked for).

A special case of fifobuf_dequeue() is fifobuf_dequeue_dyn(). This function
returns a pointer to a copy of the dequeued data. Requests for more data than
the queued amount results in a NULL return value, as do requests from an
empty buffer.

1.2.3  Special functions

You can look at data without taking it out of the buffer, and you can take
data out without looking at it. This is accomplished through the functions
fifobuf_peek() and fifobuf_drop() (see header file for details).

There's also a function for operating directly on buffered data. It's called
fifobuf_do() and takes pointers to the buffer, a function for operating on
the data in it, and any extra data to pass to the function. The function is
invoked with a pointer to a data chunk, its length and a pointer to any extra
data supplied on invocation of fifobuf_do(). It may be invoked several times,
once for each chunk of data in the buffer (remember it is split across nodes).

