Introduction
============
-In this chapter we give an [overview](#Overview) of the interactions of
-the two main programs contained in the paraslash package, followed by
+In this chapter we give an [overview](#Overview) of the interactions
+of the programs contained in the paraslash package, followed by
[brief descriptions](#The.paraslash.executables) of all executables.
Overview
--------
The core functionality of the para suite is provided by two main
-executables, para_server and para_audiod. The former maintains a
-database of audio files and streams these files to para_audiod which
-receives and plays the stream.
-
-In a typical setting, both para_server and para_audiod act as
-background daemons whose functionality is controlled by client
-programs: the para_audioc client controls para_audiod over a local
-socket while the para_client program connects to para_server over a
-local or remote networking connection.
-
-Typically, these two daemons run on different hosts but a local setup
-is also possible.
+applications, para_server and para_audiod. para_server maintains
+the audio file database and acts as the streaming source, while
+para_audiod is the streaming client. Usually, both run in the
+background on different hosts but a local setup is also possible.
A simplified picture of a typical setup is as follows
- server_host client_host
- ~~~~~~~~~~~ ~~~~~~~~~~~
- +-----------+ audio stream +-----------+
- |para_server| -----------------------------> |para_audiod|
- +-----------+ +-----------+
- ^ ^
- | |
- | | connect
- | |
- | |
- | +-----------+
- | |para_audioc|
- | +-----------+
- |
- |
- | connect +-----------+
- +-------------------------------------- |para_client|
- +-----------+
+ .____________________.
+ | ______ |
+ .-----------------------. | .d########b. |
+ |.---------------------.| | .d############b |
+ || || | .d######""####//b. |
+ || || | 9######( )######P |
+ || || | 'b######++######d' |
+ || Screen || | "9############P" |
+ || || | "9a########P" |
+ || || | `""""'' |
+ |`---------------------'| | ________________ |
+ `-----------------------' | |________________| |
+ ___) (___ |____________________|
+ `-._______.-' loudspeaker
+ | |
+ | |
+ | |
+ .____/ \___. ._____________. ._____/ \_____.
+ | | | | | |
+ | para_gui |-----| para_audioc |-----| para_audiod |
+ |____ ___| |_____________| |_____ _____|
+ \ / \ /
+ | |
+ | |
+ | |
+ ._____/ \_____. ._____/ \_____.
+ | | | |
+ | para_client |-----------------------| para_server |
+ |_____________| |_____ _____|
+ \ /
+ |
+ |
+ .-'"""`-.
+ ( )
+ |`-.___.-'|
+ | |
+ |. ' " ` .|
+ | |
+ `-.___.-'
+ Database
+
+The two client programs, para_client and para_audioc communicate with
+para_server and para_audiod, respectively.
+
+para_gui controls para_server and para_audiod by executing para_client
+and para_audioc. In particular, it runs a command to obtain the state
+of para_audiod and para_server, and the metadata of the current audio
+file. This information is pretty-printed in a curses window.
+
The paraslash executables
-------------------------
-### para_server ###
+<h3> para_server </h3>
para_server streams binary audio data (MP3, ...) over local and/or
remote networks. It listens on a TCP port and accepts commands such
-as play, stop, pause, next from authenticated clients. There are
-many more commands though, see the man page of para_server for a
-description of all commands.
-
-It supports three built-in network streaming protocols
-(senders/receivers): HTTP, DCCP, or UDP. This is explained in more
-detail in the section on [networking](#Networking).
-
-The built-in audio file selector of paraslash is used to manage your
-audio files. It maintains statistics on the usage of all available
-audio files such as last-played time, and the number of times each
-file was selected.
-
-Additional information may be added to the database to allow
+as play, stop, pause, next from authenticated clients. The components
+of para_server are illustrated in the following diagram:
+
+ ______________________________________________________________________ network
+ | | | | |
+ | .-'""""`-. | | | |
+ | ( ) | | | |
+ .____/ \_____. |`-.____.-'| .____/ \____/ \____/ \____. |
+ | | | | | | |
+ | dispatcher | | database | | senders (http/udp/dccp) | |
+ |____ _____| | | |___________ ___________| |
+ \ / |. ' "" ` .| \ / |
+ | | | | |
+ | `-.____.-' | |
+ | | | |
+ | | | |
+ | | | |
+ | ._____/ \_____. .________/ \________. |
+ | | | | | |
+ | | audio file |________| virtual streaming | |
+ | | selector | | system | |
+ | |_____ _____| |________ ________| |
+ | \ / \ / |
+ | | | |
+ | | | |
+ | | ._________________. | |
+ | | | | | |
+ | `---| command handler |---' |
+ | |____ ___ ____| |
+ | \ / \ / |
+ | | | |
+ | | | |
+ | | | |
+ `-------------------------' `--------------------------'
+
+
+Incoming connections arrive at the dispatcher which creates a process
+dedicated to the connection. Its task is to authenticate the client
+and to run the command handler which forwards the client request to
+either the audio file selector or the virtual streaming system. Results
+(if any) are sent back to the client.
+
+The audio file selector manages audio files using various database
+tables. It maintains statistics on the usage of all audio files such as
+last-played time and the number of times each file was selected. It
+is also responsible for selecting and loading audio files for
+streaming. Additional information may be added to the database to allow
fine-grained selection based on various properties of the audio file,
-including information found in (ID3) tags. However, old-fashioned
-playlists are also supported.
-
-It is also possible to store images (album covers) and lyrics in the
-database and associate these to the corresponding audio files.
-
+including information found in (ID3) tags. Simple playlists are also
+supported. It is possible to store images (album covers) and lyrics
+in the database and associate these to the corresponding audio files.
The section on the [audio file selector](#The.audio.file.selector)
-discusses this topic.
+discusses this topic in more detail.
+
+Another component of para_server is the virtual streaming system,
+which controls the paraslash senders. During streaming it requests
+small chunks of data (e.g., mp3 frames) from the audio file selector
+and feeds them to the senders which forward the chunks to connected
+clients.
+The three senders of para_server correspond to network streaming
+protocols based on HTTP, DCCP, or UDP. This is explained in the
+section on [networking](#Networking).
-### para_client ###
+<h3> para_client </h3>
The client program to connect to para_server. paraslash commands
are sent to para_server and the response is dumped to STDOUT. This
session (shell) is started. Command history and command completion are
supported through libreadline.
-### para_audiod ###
-
-The local daemon that collects information from para_server.
-
-It runs on the client side and connects to para_server. As soon as
-para_server announces the availability of an audio stream, para_audiod
-starts an appropriate receiver, any number of filters and a paraslash
-writer to play the stream.
-
-Moreover, para_audiod listens on a local socket and sends status
-information about para_server and para_audiod to local clients on
-request. Access via this local socket may be restricted by using Unix
-socket credentials, if available.
-
-
-### para_audioc ###
+<h3> para_audiod </h3>
+
+The purpose of para_audiod is to download, decode and play an audio
+stream received from para_server. A typical setup looks as follows.
+
+
+ .----------------------------.
+ | |
+ | |
+ ._____/ \_____. .___/ \____.
+ | | .----------| |
+ | para_server | | .______| receiver |
+ |_____ ____| | | |___ ____|
+ \ / | | \ /
+ | | | |
+ | | | |
+ | | | |
+ ._____/ \_____. | | .___/ \____.
+ | | | | | |
+ | status task |-----+ | | filter 1 |
+ |_____________| | |___ ____|
+ | \ /
+ | | .____________________.
+ | | | ______ |
+ .____________. | .___/ \____. | .d########b. |
+ | | | | | | .d############b |
+ | dispatcher |----------' | filter 2 | | .d######""####//b. |
+ |_____ ____| |___ ____| | 9######( )######P |
+ \ / \ / | 'b######++######d' |
+ | | | "9############P" |
+ | | | "9a########P" |
+ ._____/ \_____. .___/ \____. | `""""'' |
+ | | | | | ________________ |
+ | para_audioc | | writer |------| |________________| |
+ |_____________| |__________| |____________________|
+
+
+The status task of para_audiod connects to para_server and runs the
+"stat" command to retrieve the current server status. If an audio
+stream is available, para_audiod starts a so-called buffer tree to
+play the stream.
+
+The buffer tree consists of a receiver, any number of filters and a
+writer. The receiver downloads the audio stream from para_server and
+the filters decode or modify the received data. The writer plays the
+decoded stream.
+
+The dispatcher of para_audiod listens on a local socket and runs
+audiod commands on behalf of para_audioc. For example, para_gui runs
+para_audioc to obtain status information about para_audiod and the
+current audio file. Access to the local socket may be restricted by
+means of Unix socket credentials.
+
+<h3> para_audioc </h3>
The client program which talks to para_audiod. Used to control
para_audiod, to receive status info, or to grab the stream at any
point of the decoding process. Like para_client, para_audioc supports
interactive sessions on systems with libreadline.
-### para_recv ###
+<h3> para_recv </h3>
A command line HTTP/DCCP/UDP stream grabber. The http mode is
compatible with arbitrary HTTP streaming sources (e.g. icecast).
first decoding it, and it enables third-party software which is unaware
of the particular audio format to send complete frames in real time.
-### para_filter ###
+<h3> para_filter </h3>
A filter program that reads from STDIN and writes to STDOUT.
Like para_recv, this is an atomic building block which can be used to
formats and a number of processing filters, among these a normalizer
for audio volume.
-### para_afh ###
+<h3> para_afh </h3>
A small stand-alone program that prints tech info about the given
audio file to STDOUT. It can be instructed to print a "chunk table",
an array of offsets within the audio file.
-### para_write ###
+<h3> para_write </h3>
A modular audio stream writer. It supports a simple file writer
-output plug-in and optional WAV/raw players for ALSA (Linux) and for
-coreaudio (Mac OS). para_write can also be used as a stand-alone WAV
-or raw audio player.
+output plug-in and optional WAV/raw players for ALSA (Linux) and OSS.
+para_write can also be used as a stand-alone WAV or raw audio player.
-### para_play ###
+<h3> para_play </h3>
-A command line audio player.
+A command line audio player which supports the same audio formats as
+para_server. It differs from other players in that it has an insert
+and a command mode, like the vi editor. Line editing is based on
+libreadline, and tab completion and command history are supported.
-### para_gui ###
+<h3> para_gui </h3>
Curses-based gui that presents status information obtained in a curses
window. Appearance can be customized via themes. para_gui provides
key-bindings for the most common server commands and new key-bindings
can be added easily.
-### para_fade ###
+<h3> para_mixer </h3>
An alarm clock and volume-fader for OSS and ALSA.
Requirements
------------
-### For the impatient ###
+<h3> For the impatient </h3>
+ git clone git://git.tuebingen.mpg.de/lopsub
+ cd lopsub && make && sudo make install
git clone git://git.tuebingen.mpg.de/osl
cd osl && make && sudo make install && sudo ldconfig
- sudo apt-get install autoconf libssl-dev help2man gengetopt m4 \
+ sudo apt-get install autoconf libssl-dev m4 \
libmad0-dev libid3tag0-dev libasound2-dev libvorbis-dev \
libfaad-dev libspeex-dev libFLAC-dev libsamplerate-dev realpath \
libasound2-dev libao-dev libreadline-dev libncurses-dev \
libopus-dev
-### Detailed description ###
+<h3> Detailed description </h3>
In any case you will need
-- [libosl](http://people.tuebingen.mpg.de/maan/osl/). The _object
-storage layer_ library is used by para_server. To clone the source
-code repository, execute
+- [lopsub](http://people.tuebingen.mpg.de/maan/lopsub/). The long
+option parser for subcommands generates the command line and config
+file parsers for all paraslash executables. Clone the source code
+repository with
- git clone git://git.tuebingen.mpg.de/osl
+ git clone git://git.tuebingen.mpg.de/lopsub
- [gcc](ftp://ftp.gnu.org/pub/gnu/gcc) or
[clang](http://clang.llvm.org). All gcc versions >= 4.2 are currently
during compilation require the _Bourne again shell_. It is most
likely already installed.
-- [gengetopt](ftp://ftp.gnu.org/pub/gnu/gengetopt/) is needed to
-generate the C code for the command line parsers of all paraslash
-executables.
-
-- [help2man](ftp://ftp.gnu.org/pub/gnu/help2man) is used to create
-the man pages.
-
- [m4](ftp://ftp.gnu.org/pub/gnu/m4/). Some source files are generated
from templates by the m4 macro processor.
Optional:
-- [openssl](http://www.openssl.org/) or
+- [libosl](http://people.tuebingen.mpg.de/maan/osl/). The _object
+storage layer_ library is used by para_server. To clone the source
+code repository, execute
+
+ git clone git://git.tuebingen.mpg.de/osl
+
+- [openssl](https://www.openssl.org/) or
[libgcrypt](ftp://ftp.gnupg.org/gcrypt/libgcrypt/). At least one
of these two libraries is needed as the backend for cryptographic
routines on both the server and the client side. Both openssl and
to install the development package (`libssl-dev` or `libgcrypt-dev`
on debian systems) as well.
+- [flex](https://github.com/westes/flex) and
+[bison](https://www.gnu.org/software/bison) are needed to build the
+mood parser of para_server. The build system will skip para_server
+if these tools are not installed.
+
- [libmad](http://www.underbit.com/products/mad/). To compile in MP3
support for paraslash, the development package must be installed. It
is called `libmad0-dev` on debian-based systems. Note that libmad is
not necessary on the server side, i.e., for sending MP3 files.
- [libid3tag](http://www.underbit.com/products/mad/). For version-2
-ID3 tag support, you willl need the libid3tag development package
+ID3 tag support, you will need the libid3tag development package
`libid3tag0-dev`. Without libid3tag, only version-1 tags are
recognized. The mp3 tagger also needs this library for modifying
(id3v1 and id3v2) tags.
-- [ogg vorbis](http://www.xiph.org/downloads/). For ogg vorbis streams
+- [ogg vorbis](https://www.xiph.org/downloads/). For ogg vorbis streams
you need libogg, libvorbis, libvorbisfile. The corresponding Debian
packages are called `libogg-dev` and `libvorbis-dev`.
-- [libfaad](http://www.audiocoding.com/). For aac files (m4a) you
-need libfaad (`libfaad-dev`).
+- [libfaad and mp4ff](http://www.audiocoding.com/). For aac files
+(m4a) you need libfaad and libmp4ff (package: `libfaad-dev`). Note
+that for some distributions, e.g. Ubuntu, mp4ff is not part of the
+libfaad package. Install the faad library from sources (available
+through the above link) to get the mp4ff library and header files.
-- [speex](http://www.speex.org/). In order to stream or decode speex
+- [speex](https://www.speex.org/). In order to stream or decode speex
files, libspeex (`libspeex-dev`) is required.
-- [flac](http://flac.sourceforge.net/). To stream or decode files
+- [flac](https://xiph.org/flac/). To stream or decode files
encoded with the _Free Lossless Audio Codec_, libFLAC (`libFLAC-dev`)
must be installed.
- [alsa-lib](ftp://ftp.alsa-project.org/pub/lib/). On Linux, you will
need to have the ALSA development package `libasound2-dev` installed.
-- [libao](http://downloads.xiph.org/releases/ao/). Needed to build
+- [libao](https://ftp.osuosl.org/pub/xiph/releases/ao/). Needed to build
the ao writer (ESD, PulseAudio,...). Debian package: `libao-dev`.
- [curses](ftp://ftp.gnu.org/pub/gnu/ncurses). Needed for
Configuration
-------------
-### Create a paraslash user ###
+<h3> Create a paraslash user </h3>
In order to control para_server at runtime you must create a paraslash
user. As authentication is based on the RSA crypto system you'll have
echo 'hostname server_host' > $conf
-### Start para_server ###
+<h3> Start para_server </h3>
For this first try, we'll use the info loglevel to make the output
of para_server more verbose.
to retrieve the list of available commands and some server info.
Don't proceed if this doesn't work.
-### Create and populate the database ###
+<h3> Create and populate the database </h3>
An empty database is created with
para_client ls
-### Configure para_audiod ###
+<h3> Configure para_audiod </h3>
We will have to tell para_audiod that it should receive the audio
stream from server_host via http:
Troubleshooting
---------------
-If you receive a socket related error on server or audiod startup,
-make sure you have write permissions to the /var/paraslash directory:
-
- sudo chown $LOGNAME /var/paraslash
-
-Alternatively, use the --afs-socket (para_server) or --socket
-(para_audiod) option to specify a different socket pathname.
-
To identify streaming problems try to receive, decode and play the
stream manually using para_recv, para_filter and para_write as follows.
For simplicity we assume that you're running Linux/ALSA and that only
para_server uses a challenge-response mechanism to authenticate
requests from incoming connections, similar to ssh's public key
authentication method. Authenticated connections are encrypted using
-a stream cipher, either RC4 or AES in integer counter mode.
+the AES stream cipher in integer counter mode.
-In this chapter we briefly describe RSA, RC4 and AES, and sketch the
+In this chapter we briefly describe RSA and AES, and sketch the
[authentication handshake](#Client-server.authentication)
between para_client and para_server. User management is discussed
in the section on [the user_list file](#The.user_list.file).
server. Connecting para_audiod is a different matter and is described
in a [separate section](#Connecting.para_audiod).
-RSA, RC4, AES
--------------
+RSA and AES
+-----------
-RSA is an asymmetric block cipher which is used in many applications,
-including ssh and gpg. An RSA key consists in fact of two keys,
+A block cipher is a transformation which operates on fixed-length
+blocks. For symmetric block ciphers the transformation is determined
+by a single key for both encryption and decryption. For asymmetric
+block ciphers, on the other hand, the key consists of two parts,
called the public key and the private key. A message can be encrypted
-with either key and only the counterpart of that key can decrypt
-the message. While RSA can be used for both signing and encrypting
-a message, paraslash uses RSA only for the latter purpose. The
-RSA public key encryption and signatures algorithms are defined in
-detail in RFC 2437.
-
-RC4 is a stream cipher, i.e. the input is XORed with a pseudo-random
-key stream to produce the output. Decryption uses the same function
-calls as encryption. While RC4 supports variable key lengths,
-paraslash uses a fixed length of 256 bits, which is considered a
-strong encryption by today's standards. Since the same key must never
-be used twice, a different, randomly-generated key is used for every
-new connection.
+with either key and only the counterpart of that key can decrypt the
+message. Asymmetric block ciphers can be used for both signing and
+encrypting a message.
+
+RSA is an asymmetric block cipher which is used in many applications,
+including ssh and gpg. The RSA public key encryption and signatures
+algorithms are defined in detail in RFC 2437. Paraslash relies on
+RSA for authentication.
+
+Stream ciphers XOR the input with a pseudo-random key stream to produce
+the output. Decryption uses the same function calls as encryption.
+Any block cipher can be turned into a stream cipher by generating the
+pseudo-random key stream by encrypting successive values of a counter
+(counter mode).
AES, the advanced encryption standard, is a well-known symmetric block
-cipher, i.e. a transformation operating on fixed-length blocks which
-is determined by a single key for both encryption and decryption. Any
-block cipher can be turned into a stream cipher by generating
-a pseudo-random key stream by encrypting successive values of a
-counter. The AES_CTR128 stream cipher used in paraslash is obtained
-in this way from the AES block cipher with a 128 bit block size.
+cipher. Paraslash employs AES in counter mode as described above to
+encrypt communications. Since a stream cipher key must not be used
+twice, a random key is generated for every new connection.
Client-server authentication
----------------------------
the session key known to both peers.
paraslash relies on the quality of the pseudo-random bytes provided
-by the crypto library (openssl or libgcrypt), on the security of the
-implementation of the RSA, RC4 and AES crypto routines and on the
+by the crypto library (openssl or libgcrypt), on the security of
+the implementation of the RSA and AES crypto routines and on the
infeasibility to invert the SHA1 function.
Neither para_server or para_client create RSA keys on their
para_client help
-contains in the third column the permissions needed to execute the
-command.
+contains the permissions needed to execute the command.
It is possible to make para_server reread the user_list file by
executing the paraslash "hup" command or by sending SIGHUP to the
paraslash comes with a sophisticated audio file selector (AFS),
whose main task is to determine which file to stream next, based on
information on the audio files stored in a database. It communicates
-also with para_client whenever an AFS command is executed, for example
-to answer a database query.
+also with para_client via the command handler whenever an AFS command
+is executed, for example to answer a database query.
-Besides the traditional playlists, AFS supports audio file selection
+Besides the simple playlists, AFS supports audio file selection
based on _moods_ which act as a filter that limits the set of all
-known audio files to those which satisfy certain criteria. It also
+known audio files to those which satisfy certain criteria. It also
maintains tables containing images (e.g. album cover art) and lyrics
that can be associated with one or more audio files.
-AFS employs [libosl](http://people.tuebingen.mpg.de/maan/osl/), the
-object storage layer library, as the backend library for storing
-information on audio files, playlists, etc. This library offers
-functionality similar to a relational database, but is much more
-lightweight than a full database backend.
-
In this chapter we sketch the setup of the [AFS
process](#The.AFS.process) during server startup and proceed with the
description of the [layout](#Database.layout) of the various database
tables. The section on [playlists and moods](#Playlists.and.moods)
explains these two audio file selection mechanisms in detail
-and contains pratical examples. The way [file renames and content
+and contains practical examples. The way [file renames and content
changes](#File.renames.and.content.changes) are detected is discussed
briefly before the [Troubleshooting](#Troubleshooting) section
concludes the chapter.
---------------
On startup, para_server forks to create the AFS process which opens
-the OSL database tables. The server process communicates with the
-AFS process via pipes and shared memory. Usually, the AFS process
-awakes only briefly whenever the current audio file changes. The AFS
-process determines the next audio file, opens it, verifies it has
-not been changed since it was added to the database and passes the
-open file descriptor to the server process, along with audio file
-meta-data such as file name, duration, audio format and so on. The
-server process then starts to stream the audio file.
-
-The AFS process also accepts connections from local clients via
-a well-known socket. However, only child processes of para_server
-may connect through this socket. All server commands that have the
-AFS_READ or AFS_WRITE permission bits use this mechanism to query or
-change the database.
+the database tables. The AFS process accepts incoming connections
+which arrive either on a pipe which is shared with para_server,
+or on the local socket it is listening on. The setup is as follows.
+
+ .___________________. .______________.
+ | | | |
+ | virtual streaming | | audio format |
+ | system | | handler |
+ |_________ _______| |_____ ______|
+ \ / \ /
+ | |
+ .-'""""`-. | | .-'""""`-.
+ ( ) | | ( )
+ |`-.____.-'| .__/ \________________/ \___. |`-.____.-'|
+ | | | | | |
+ | file |----| AFS (audio file selector) |----| OSL |
+ | system | | process | | database |
+ | | |___________________________| | |
+ |. ' "" ` .| | |. ' "" ` .|
+ | | | | |
+ `-.____.-' | `-.____.-'
+ ._______/ \_______.
+ | |
+ | command handler |
+ |_______ _______|
+ \ /
+ |
+ |
+ |
+ ._____/ \_____.
+ | |
+ | para_client |
+ |_____________|
+
+The virtual streaming system, which is part of the server process,
+communicates with the AFS process via pipes and shared memory. When
+the current audio file changes, it sends a notification through the
+shared pipe. The AFS process queries the database to determine the
+next audio file, opens it, verifies that it has not been changed since
+it was added to the database and passes the open file descriptor back
+to the virtual streaming system, along with audio file meta-data such
+as file name, duration, audio format and so on. The virtual streaming
+system then starts to stream the file.
+
+The command handlers of all AFS server commands use the local socket
+to query or update the database. For example, the command handler of
+the add command sends the path of an audio file to the local socket.
+The AFS process opens the file and tries to find an audio format
+handler which recognizes the file. If all goes well, a new database
+entry with metadata obtained from the audio format handler is added
+to the database.
+
+Note that AFS employs
+[libosl](http://people.tuebingen.mpg.de/maan/osl/), the object
+storage layer library, as the database backend. This library offers
+functionality similar to a relational database, but is much more
+lightweight than a full featured database management system.
Database layout
---------------
-### The audio file table ###
+Metadata about the known audio files is stored in an OSL database. This
+database consists of the following tables:
+
+- The audio file table contains path, hash and metadata of each
+known file.
+
+- The "attributes" table maps each of the 64 possible attributes to a
+string.
+
+- The "blob" tables store images, lyrics, moods, playlists. All of
+these are optional.
+
+- The "score" table describes the subset of admissible files for the
+current playlist or mood.
+
+All tables are described in more detail below.
+
+<h3> The audio file table </h3>
This is the most important and usually also the largest table of the
AFS database. It contains the information needed to stream each audio
and all files which are recognized as valid audio files are added to
the database.
-### The attribute table ###
+<h3> The attribute table </h3>
The attribute table contains two columns, _name_ and _bitnum_. An
attribute is simply a name for a certain bit number in the attribute
for more information and a complete list of command line options to
these commands.
-### Blob tables ###
+<h3> Blob tables </h3>
The image, lyrics, moods and playlists tables are all blob tables.
Blob tables consist of three columns each: The identifier which is
-a positive non-negative number that is auto-incremented, the name
-(an arbitrary string) and the content (the blob).
+a positive number that is auto-incremented, the name (an arbitrary
+string) and the content (the blob).
All blob tables support the same set of actions: cat, ls, mv, rm
and add. Of course, _add_ is used for adding new blobs to the table
the playlist and the mood blobs are only investigated when the mood
or playlist is activated with the select command.
-### The score table ###
+<h3> The score table </h3>
The score table describes those audio files which are admissible for
the current mood or playlist (see below). The table has two columns:
audio file table. As an example, a mood can define a filename pattern,
which is then matched against the names of audio files in the table.
-### Playlists ###
+<h3> Playlists </h3>
Playlists are accommodated in the playlist table of the afs database,
using the aforementioned blob format for tables. A new playlist is
file could not be opened for streaming, its entry is removed from
the score table (but not from the playlist).
-### Moods ###
-
-A mood consists of a unique name and its *mood definition*, which is
-a set of *mood lines* containing expressions in terms of attributes
-and other data contained in the database.
-
-At any time at most one mood can be *active* which means that
-para_server is going to select only files from that subset of
-admissible files.
-
-So in order to create a mood definition one has to write a set of
-mood lines. Mood lines come in three flavours: Accept lines, deny
-lines and score lines.
-
-The general syntax of the three types of mood lines is
+<h3> Moods </h3>
+
+A mood consists of a unique name and a definition. The definition
+is an expression which describes which audio files are considered
+admissible. At any time at most one mood can be active, meaning
+that para_server will only stream files which are admissible for the
+active mood.
+
+The expression may refer to attributes and other metadata stored in
+the database. Expressions may be combined by means of logical and
+arithmetical operators in a natural way. Moreover, string matching
+based on regular expression or wildcard patterns is supported.
+
+The set of admissible files is determined by applying the expression
+to each audio file in turn. For a mood definition to be valid, its
+expression must evaluate to a number, a string or a boolean value
+("true" or "false"). For numbers, any value other than zero means the
+file is admissible. For strings, any non-empty string indicates an
+admissible file. For boolean values, true means admissible and false
+means not admissible. As a special case, the empty expression treats
+all files as admissible.
+
+<h3> Mood grammar </h3>
+
+Expressions are based on a context-free grammar which distinguishes
+between several types for syntactic units or groupings. The grammar
+defines a set of keywords which have a type and a corresponding
+semantic value, as shown in the following table.
+
+Keyword | Type | Semantic value
+:--------------------|--------:|:----------------------------------
+`path` | string | Full path of the current audio file
+`artist` | string | Content of the artist meta tag
+`title` | string | Content of the title meta tag
+`album` | string | Content of the album meta tag
+`comment` | string | Content of the somment meta tag
+`num_attributes_set` | integer | Number of attributes which are set
+`year` | integer | Content of the year meta tag [\*]
+`num_played` | integer | How many times the file has been streamed
+`image_id` | integer | The identifier of the (cover art) image
+`lyrics_id` | integer | The identifier of the lyrics blob
+`bitrate` | integer | The average bitrate
+`frequency` | integer | The output sample rate
+`channels` | integer | The number of channels
+`is_set("foo")` | boolean | True if attribute "foo" is set.
+
+[\*] For most audio formats, the year tag is stored as a string. It
+is converted to an integer by the mood parser. If the audio file
+has no year tag or the content of the year tag is not a number, the
+semantic value is zero. A special convention applies if the year tag
+is a one-digit or a two-digit number. In this case 1900 is added to
+the tag value.
+
+Expressions may be grouped using parentheses, logical and
+arithmetical operators or string matching operators. The following
+table lists the available operators.
+
+Token | Meaning
+:------|:-------
+`\|\|` | Logical Or
+`&&` | Logical And
+`!` | Logical Not
+`==` | Equal (can be applied to all types)
+`!=` | Not equal. Likewise
+`<` | Less than
+`<=` | Less or equal
+`>=` | Greater or equal
+`+` | Arithmetical minus
+`-` | Binary/unary minus
+`*` | Multiplication
+`/` | Division
+`=~` | Regular expression match
+`=\|` | Filename match
+
+Besides integers, strings and booleans there is an additional type
+which describes regular expression or wildcard patterns. Patterns
+are not just strings because they also include a list of flags which
+modify matching behaviour.
+
+Regular expression patterns are of the form `/pattern/[flags]`. That
+is, the pattern is delimited by slashes, and is followed by zero or
+more characters, each specifying a flag according to the following
+table
+
+Flag | POSIX name | Meaning
+:----|--------------:|--------
+`i` | `REG_ICASE` | Ignore case in match
+`n` | `REG_NEWLINE` | Treat newline as an ordinary character
+
+Note that only extended regular expression patterns are supported. See
+regex(3) for details.
+
+Wildcard patterns are similar, but the pattern must be delimited by
+`'|'` characters rather than slashes. For wildcard patterns different
+flags exist, as shown below.
+
+Flag | POSIX name | Meaning
+:----|-----------------------:|--------
+`n` | `FNM_NOESCAPE` | Treat backslash as an ordinary character
+`p` | `FNM_PATHNAME` | Match a slash only with a slash in pattern
+`P` | `FNM_PERIOD` | Leading period has to be matched exactly
+`l` | `FNM_LEADING_DIR` [\*] | Ignore "/\*" rest after successful matching
+`i` | `FNM_CASEFOLD` [\*] | Ignore case in match
+`e` | `FNM_EXTMATCH` [\*\*] | Enable extended pattern matching
+
+[\*] Not in POSIX, but both FreeBSD and NetBSD have it.
+
+[\*\*] GNU extension, silently ignored on non GNU systems.
+
+See fnmatch(3) for details.
+
+Mood definitions may contain arbitrary whitespace and comments.
+A comment is a word beginning with #. This word and all remaining
+characters of the line are ignored.
+
+<h3> Example moods </h3>
+
+* Files with no/invalid year tag: `year == 0`
+
+* Only oldies: `year != 0 && year < 1980`
+
+* Only 80's Rock or Metal: `(year >= 1980 && year < 1990) &&
+ (is_set("rock") || is_set("metal"))`
+
+* Files with incomplete tags: `artist == "" || title == "" || album =
+"" || comment == "" || year == 0`
+
+* Files with no attributes defined so far: `num_attributes_set == 0`
+
+* Only newly added files: `num_played == 0`
+
+* Only poor quality files: `bitrate < 96`
+
+* Cope with different spellings of Motörhead: `artist =~ /mot(ö|oe{0,1})rhead/i`
+
+* The same with extended wildcard patterns: `artist =| |mot+(o\|oe\|ö)rhead|ie`
-
- accept [with score <score>] [if] [not] <mood_method> [options]
- deny [with score <score>] [if] [not] <mood_method> [options]
- score <score> [if] [not] <mood_method> [options]
-
-
-Here <score> is either an integer or the string "random" which assigns
-a random score to all matching files. The score value changes the
-order in which admissible files are going to be selected, but is of
-minor importance for this introduction.
-
-So we concentrate on the first two forms, i.e. accept and deny
-lines. As usual, everything in square brackets is optional, i.e.
-accept/deny lines take the following form when ignoring scores:
-
- accept [if] [not] <mood_method> [options]
-
-and analogously for the deny case. The "if" keyword is only syntactic
-sugar and has no function. The "not" keyword just inverts the result,
-so the essence of a mood line is the mood method part and the options
-following thereafter.
-
-A *mood method* is realized as a function which takes an audio file
-and computes a number from the data contained in the database.
-If this number is non-negative, we say the file *matches* the mood
-method. The file matches the full mood line if it either
-
- - matches the mood method and the "not" keyword is not given,
-or
- - does not match the mood method, but the "not" keyword is given.
-
-The set of admissible files for the whole mood is now defined as those
-files which match at least one accept mood line, but no deny mood line.
-More formally, an audio file F is admissible if and only if
-
- (F ~ AL1 or F ~ AL2...) and not (F ~ DL1 or F ~ DN2 ...)
-
-where AL1, AL2... are the accept lines, DL1, DL2... are the deny
-lines and "~" means "matches".
-
-The cases where no mood lines of accept/deny type are defined need
-special treatment:
-
- - Neither accept nor deny lines: This treats all files as
- admissible (in fact, that is the definition of the dummy mood
- which is activated automatically if no moods are available).
-
- - Only accept lines: A file is admissible iff it matches at
- least one accept line:
-
- F ~ AL1 or F ~ AL2 or ...
-
- - Only deny lines: A file is admissible iff it matches no
- deny line:
-
- not (F ~ DL1 or F ~ DN2 ...)
-
-
-
-### List of mood_methods ###
-
- no_attributes_set
-
-Takes no arguments and matches an audio file if and only if no
-attributes are set.
-
- is_set <attribute_name>
-
-Takes the name of an attribute and matches iff that attribute is set.
-
- path_matches <pattern>
-
-Takes a filename pattern and matches iff the path of the audio file
-matches the pattern.
-
- artist_matches <pattern>
- album_matches <pattern>
- title_matches <pattern>
- comment_matches <pattern>
-
-Takes an extended regular expression and matches iff the text of the
-corresponding tag of the audio file matches the pattern. If the tag
-is not set, the empty string is matched against the pattern.
-
- year ~ <num>
- bitrate ~ <num>
- frequency ~ <num>
- channels ~ <num>
- num_played ~ <num>
- image_id ~ <num>
- lyrics_id ~ <num>
-
-Takes a comparator ~ of the set {<, =, <=, >, >=, !=} and a number
-<num>. Matches an audio file iff the condition <val> ~ <num> is
-satisfied where val is the corresponding value of the audio file
-(value of the year tag, bitrate in kbit/s, etc.).
-
-The year tag is special as its value is undefined if the audio file
-has no year tag or the content of the year tag is not a number. Such
-audio files never match. Another difference is the special treatment
-if the year tag is a two-digit number. In this case either 1900 or
-2000 is added to the tag value, depending on whether the number is
-greater than 2000 plus the current year.
-
-
-### Mood usage ###
+<h3> Mood usage </h3>
To create a new mood called "my_mood", write its definition into
some temporary file, say "tmpfile", and add it to the mood table
para ls -a
-
-### Example mood definition ###
-
-Suppose you have defined attributes "punk" and "rock" and want to define
-a mood containing only Punk-Rock songs. That is, an audio file should be
-admissible if and only if both attributes are set. Since
-
- punk and rock
-
-is obviously the same as
-
- not (not punk or not rock)
-
-(de Morgan's rule), a mood definition that selects only Punk-Rock
-songs is
-
- deny if not is_set punk
- deny if not is_set rock
-
-
-
File renames and content changes
--------------------------------
The following audio formats are supported by paraslash:
-### MP3 ###
+<h3> MP3 </h3>
Mp3, MPEG-1 Audio Layer 3, is a common audio format for audio storage,
designed as part of its MPEG-1 standard. An MP3 file is made up of
stereo), encoded with a bit rate of 128 kbit, an MP3 frame is about
400 bytes large.
-### OGG/Vorbis ###
+<h3> OGG/Vorbis </h3>
OGG is a standardized audio container format, while Vorbis is an
open source codec for lossy audio compression. Since Vorbis is most
Vorbis codec creates variable-bitrate (VBR) data, where the bitrate
may vary considerably.
-### OGG/Speex ###
+<h3> OGG/Speex </h3>
Speex is an open-source speech codec that is based on CELP (Code
Excited Linear Prediction) coding. It is designed for voice
in OGG files. As of 2012 this codec is considered obsolete since the
Oppus codec, described below, surpasses its performance in all areas.
-### OGG/Opus ###
+<h3> OGG/Opus </h3>
Opus is a lossy audio compression format standardized through RFC
6716 in 2012. It combines the speech-oriented SILK codec and the
containers. All known software patents which cover Opus are licensed
under royalty-free terms.
-### AAC ###
+<h3> AAC </h3>
Advanced Audio Coding (AAC) is a standardized, lossy compression
and encoding scheme for digital audio which is the default audio
the container format and audio files encoded with AAC have the .m4a
extension. A typical AAC frame is about 700 bytes large.
-### WMA ###
+<h3> WMA </h3>
Windows Media Audio (WMA) is an audio data compression technology
developed by Microsoft. A WMA file is usually encapsulated in the
is composed of superframes, each containing one or more frames of
2048 samples. For 16 bit stereo a WMA superframe is about 8K large.
-### FLAC ###
+<h3> FLAC </h3>
The Free Lossless Audio Codec (FLAC) compresses audio without quality
loss. It gives better compression ratios than a general purpose
- UDP. Recommended for multicast LAN streaming.
-See the Appendix on [network protocols](/#Network.protocols)
+See the Appendix on [network protocols](#Network.protocols)
for brief descriptions of the various protocols relevant for network
audio streaming with paraslash.
sample is multiplied with a scaling factor (>= 1) which makes amp
and compress quite expensive in terms of computing power.
-### amp ###
+<h3> amp </h3>
The amp filter amplifies the audio stream by a fixed scaling factor
that must be known in advance. For para_audiod this factor is derived
is used. The reader is encouraged to write a script that performs
these computations :)
-### compress ###
+<h3> compress </h3>
Unlike the amplification filter, the compress filter adjusts the volume
of the audio stream dynamically without prior knowledge about the peak
also limited. For example only one application can open the device
at any time. The OSS writer is activated by default on BSD Systems.
-- *OSX*. Mac OS X has yet another API called CoreAudio. The OSX writer
-for this API is only compiled in on such systems and is of course
-the default there.
-
- *FILE*. The file writer allows to capture the audio stream and
write the PCM data to a file on the file system rather than playing
it through a sound device. It is supported on all platforms and is
Development
===========
+Contributing
+------------
+
+Paraslash is an open source project and contributions are
+welcome. Here's a list of things you can do to help the project:
+
+- Report problems with building, installing or running the software.
+ In particular, test the experimental git branches ("next" and "pu").
+ This helps to identify and fix problems before the code gets merged
+ and thus keeps the master branch as stable as possible.
+- Proofread the documentation (manual, web pages, man pages, source
+ code documentation) and point out unclear or poorly written parts. If
+ you are a native English speaker you will easily find a lot of text
+ that could be improved.
+- Run analysis tools (coverity, afl, sparse, etc.) and report issues
+ found by those tools.
+- Suggest new features you would like to see implemented.
+- Compile and test on your favorite architecture or operating
+ system. The code is tested only on a limited set of systems, so you
+ will probably encounter problems when building on different systems.
+- Post about paraslash on your blog or on social networks.
+- Build and maintain Debian/RPM packages for your favorite distribution.
+
+Note that there is no mailing list, no bug tracker and no discussion
+forum for paraslash. If you'd like to contribute, or have questions
+about contributing, send email to Andre Noll <maan@tuebingen.mpg.de>.
+New releases are announced by email. If you would like to receive
+these announcements, contact the author through the above address.
+
Tools
-----
the configure file which is shipped in the tarballs but has to be
generated when compiling from git.
-- [discount](http://www.pell.portland.or.us/~orc/Code/discount). The
+- [discount](http://www.pell.portland.or.us/~orc/Code/discount/). The
HTML version of this manual and some of the paraslash web pages are
written in the Markdown markup language and are translated into html
with the converter of the *Discount* package.
The preferred coding style for paraslash coincides more or less
with the style of the Linux kernel. So rather than repeating what is
-written [there](http://www.kernel.org/doc/Documentation/CodingStyle),
+written [there](https://www.kernel.org/doc/Documentation/process/coding-style.rst),
here are the most important points.
- Burn the GNU coding standards.
Network protocols
-----------------
-### IP ###
+<h3> IP </h3>
The _Internet Protocol_ is the primary networking protocol used for
the Internet. All protocols described below use IP as the underlying
layer. Both the prevalent IPv4 and the next-generation IPv6 variant
are being deployed actively worldwide.
-### Connection-oriented and connectionless protocols ###
+<h3> Connection-oriented and connectionless protocols </h3>
Connectionless protocols differ from connection-oriented ones in
that state associated with the sending/receiving endpoints is treated
that the sending endpoints perform congestion control, adapting to
qualitative changes of the connection medium.
-### Reliability ###
+<h3> Reliability </h3>
In IP networking, packets can be lost, duplicated, or delivered
out of order, and different network protocols handle these
delivery. Unreliable protocols, in contrast, do not guarantee ordering
or data integrity.
-### Classification ###
+<h3> Classification </h3>
With these definitions the protocols which are used by paraslash for
steaming audio data may be classified as follows.
Below we give a short descriptions of these protocols.
-### TCP ###
+<h3> TCP </h3>
The _Transmission Control Protocol_ provides reliable, ordered delivery
of a stream and a classic window-based congestion control. In contrast
Transfer Protocol), also FTP (the File Transfer protocol), SMTP (Simple
Mail Transfer Protocol), SSH (Secure Shell) all sit on top of TCP.
-### UDP ###
+<h3> UDP </h3>
The _User Datagram Protocol_ is the simplest transport-layer protocol,
built as a thin layer directly on top of IP. For this reason, it offers
congestion. Error checking and correction (if at all) are performed
in the application.
-### DCCP ###
+<h3> DCCP </h3>
The _Datagram Congestion Control Protocol_ combines the
connection-oriented state maintenance known from TCP with the
control known from TCP is available as CCID-2, rate-based, "smooth"
congestion control is offered as CCID-3.
-### HTTP ###
+<h3> HTTP </h3>
The _Hypertext Transfer Protocol_ is an application layer protocol
on top of TCP. It is spoken by web servers and is most often used
protocol, the semantics of the protocol also allow the delivery of
multimedia content, such as audio over http.
-### Multicast ###
+<h3> Multicast </h3>
IP multicast is not really a protocol but a technique for one-to-many
communication over an IP network. The challenge is to deliver
---------------------
- [paraslash](http://people.tuebingen.mpg.de/maan/paraslash/)
-- [xmms](http://xmms2.org/wiki/Main_Page)
+- [xmms](https://xmms2.org/wiki/Main_Page)
- [mpg123](http://www.mpg123.de/)
-- [gstreamer](http://gstreamer.freedesktop.org/)
+- [gstreamer](https://gstreamer.freedesktop.org/)
- [icecast](http://www.icecast.org/)
-- [Audio Compress](http://beesbuzz.biz/code/audiocompress.php)
+- [Audio Compress](https://beesbuzz.biz/code/audiocompress.php)
External documentation
----------------------
- [The mathematics of
-Raid6](http://kernel.org/pub/linux/kernel/people/hpa/raid6.pdf)
+Raid6](https://www.kernel.org/pub/linux/kernel/people/hpa/raid6.pdf)
by H. Peter Anvin
- [Effective Erasure Codes for reliable Computer Communication
projects / paraslash.git / blobdiff