X-Git-Url: http://git.tuebingen.mpg.de/?p=paraslash.git;a=blobdiff_plain;f=web%2Fmanual.md;h=904ac18077691aacc3444b9bdd6e477609b4a840;hp=c8f1a9d87d139ae309a2e5327354c4c3a8c7bf84;hb=3d3a2f50a05501cf27f1155629799953f952bd4b;hpb=ff13641489b00540d8b8ca0eb27adf0ed49b70f7 diff --git a/web/manual.md b/web/manual.md index c8f1a9d8..904ac180 100644 --- a/web/manual.md +++ b/web/manual.md @@ -80,7 +80,7 @@ file. This information is pretty-printed in a curses window. The paraslash executables ------------------------- -### para_server ### +

para_server

para_server streams binary audio data (MP3, ...) over local and/or remote networks. It listens on a TCP port and accepts commands such @@ -148,7 +148,7 @@ The three senders of para_server correspond to network streaming protocols based on HTTP, DCCP, or UDP. This is explained in the section on [networking](#Networking). -### para_client ### +

para_client

The client program to connect to para_server. paraslash commands are sent to para_server and the response is dumped to STDOUT. This @@ -163,7 +163,7 @@ If para_client is started without non-option arguments, an interactive session (shell) is started. Command history and command completion are supported through libreadline. -### para_audiod ### +

para_audiod

The purpose of para_audiod is to download, decode and play an audio stream received from para_server. A typical setup looks as follows. @@ -216,14 +216,14 @@ para_audioc to obtain status information about para_audiod and the current audio file. Access to the local socket may be restricted by means of Unix socket credentials. -### para_audioc ### +

para_audioc

The client program which talks to para_audiod. Used to control para_audiod, to receive status info, or to grab the stream at any point of the decoding process. Like para_client, para_audioc supports interactive sessions on systems with libreadline. -### para_recv ### +

para_recv

A command line HTTP/DCCP/UDP stream grabber. The http mode is compatible with arbitrary HTTP streaming sources (e.g. icecast). @@ -234,7 +234,7 @@ optionally 'just in time'. This allows to cut an audio file without first decoding it, and it enables third-party software which is unaware of the particular audio format to send complete frames in real time. -### para_filter ### +

para_filter

A filter program that reads from STDIN and writes to STDOUT. Like para_recv, this is an atomic building block which can be used to @@ -243,30 +243,33 @@ different functionalities in one tool: decoders for multiple audio formats and a number of processing filters, among these a normalizer for audio volume. -### para_afh ### +

para_afh

A small stand-alone program that prints tech info about the given audio file to STDOUT. It can be instructed to print a "chunk table", an array of offsets within the audio file. -### para_write ### +

para_write

A modular audio stream writer. It supports a simple file writer output plug-in and optional WAV/raw players for ALSA (Linux) and OSS. para_write can also be used as a stand-alone WAV or raw audio player. -### para_play ### +

para_play

-A command line audio player. +A command line audio player which supports the same audio formats as +para_server. It differs from other players in that it has an insert +and a command mode, like the vi editor. Line editing is based on +libreadline, and tab completion and command history are supported. -### para_gui ### +

para_gui

Curses-based gui that presents status information obtained in a curses window. Appearance can be customized via themes. para_gui provides key-bindings for the most common server commands and new key-bindings can be added easily. -### para_mixer ### +

para_mixer

An alarm clock and volume-fader for OSS and ALSA. @@ -282,7 +285,7 @@ source code and the steps that have to be performed in order to Requirements ------------ -### For the impatient ### +

For the impatient

git clone git://git.tuebingen.mpg.de/lopsub cd lopsub && make && sudo make install @@ -294,16 +297,10 @@ Requirements libasound2-dev libao-dev libreadline-dev libncurses-dev \ libopus-dev -### Detailed description ### +

Detailed description

In any case you will need -- [libosl](http://people.tuebingen.mpg.de/maan/osl/). The _object -storage layer_ library is used by para_server. To clone the source -code repository, execute - - git clone git://git.tuebingen.mpg.de/osl - - [lopsub](http://people.tuebingen.mpg.de/maan/lopsub/). The long option parser for subcommands generates the command line and config file parsers for all paraslash executables. Clone the source code @@ -327,6 +324,12 @@ from templates by the m4 macro processor. Optional: +- [libosl](http://people.tuebingen.mpg.de/maan/osl/). The _object +storage layer_ library is used by para_server. To clone the source +code repository, execute + + git clone git://git.tuebingen.mpg.de/osl + - [openssl](http://www.openssl.org/) or [libgcrypt](ftp://ftp.gnupg.org/gcrypt/libgcrypt/). At least one of these two libraries is needed as the backend for cryptographic @@ -335,6 +338,11 @@ libgcrypt are usually shipped with the distro, but you might have to install the development package (`libssl-dev` or `libgcrypt-dev` on debian systems) as well. +- [flex](https://github.com/westes/flex) and +[bison](https://www.gnu.org/software/bison) are needed to build the +mood parser of para_server. The build system will skip para_server +if these tools are not installed. + - [libmad](http://www.underbit.com/products/mad/). To compile in MP3 support for paraslash, the development package must be installed. It is called `libmad0-dev` on debian-based systems. Note that libmad is @@ -413,7 +421,7 @@ to install executables under /usr/local/bin and the man pages under Configuration ------------- -### Create a paraslash user ### +

Create a paraslash user

In order to control para_server at runtime you must create a paraslash user. As authentication is based on the RSA crypto system you'll have @@ -457,7 +465,7 @@ Finally, tell para_client to connect to server_host: echo 'hostname server_host' > $conf -### Start para_server ### +

Start para_server

For this first try, we'll use the info loglevel to make the output of para_server more verbose. @@ -473,7 +481,7 @@ commands. Open a new shell as bar@client_host and try to retrieve the list of available commands and some server info. Don't proceed if this doesn't work. -### Create and populate the database ### +

Create and populate the database

An empty database is created with @@ -501,7 +509,7 @@ You may print the list of all known audio files with para_client ls -### Configure para_audiod ### +

Configure para_audiod

We will have to tell para_audiod that it should receive the audio stream from server_host via http: @@ -790,7 +798,7 @@ current playlist or mood. All tables are described in more detail below. -### The audio file table ### +

The audio file table

This is the most important and usually also the largest table of the AFS database. It contains the information needed to stream each audio @@ -839,7 +847,7 @@ directory. In the latter case, the directory is traversed recursively and all files which are recognized as valid audio files are added to the database. -### The attribute table ### +

The attribute table

The attribute table contains two columns, _name_ and _bitnum_. An attribute is simply a name for a certain bit number in the attribute @@ -899,7 +907,7 @@ Read the output of for more information and a complete list of command line options to these commands. -### Blob tables ### +

Blob tables

The image, lyrics, moods and playlists tables are all blob tables. Blob tables consist of three columns each: The identifier which is @@ -926,7 +934,7 @@ Note that the images and lyrics are not interpreted at all, and also the playlist and the mood blobs are only investigated when the mood or playlist is activated with the select command. -### The score table ### +

The score table

The score table describes those audio files which are admissible for the current mood or playlist (see below). The table has two columns: @@ -952,7 +960,7 @@ terms of attributes and other type of information available in the audio file table. As an example, a mood can define a filename pattern, which is then matched against the names of audio files in the table. -### Playlists ### +

Playlists

Playlists are accommodated in the playlist table of the afs database, using the aforementioned blob format for tables. A new playlist is @@ -971,128 +979,144 @@ in descending order so that files will be selected in order. If a file could not be opened for streaming, its entry is removed from the score table (but not from the playlist). -### Moods ### - -A mood consists of a unique name and its *mood definition*, which is -a set of *mood lines* containing expressions in terms of attributes -and other data contained in the database. - -At any time at most one mood can be *active* which means that -para_server is going to select only files from that subset of -admissible files. - -So in order to create a mood definition one has to write a set of -mood lines. Mood lines come in three flavours: Accept lines, deny -lines and score lines. - -The general syntax of the three types of mood lines is - - - accept [with score ] [if] [not] [options] - deny [with score ] [if] [not] [options] - score [if] [not] [options] - - -Here is either an integer or the string "random" which assigns -a random score to all matching files. The score value changes the -order in which admissible files are going to be selected, but is of -minor importance for this introduction. - -So we concentrate on the first two forms, i.e. accept and deny -lines. As usual, everything in square brackets is optional, i.e. -accept/deny lines take the following form when ignoring scores: - - accept [if] [not] [options] - -and analogously for the deny case. The "if" keyword is only syntactic -sugar and has no function. The "not" keyword just inverts the result, -so the essence of a mood line is the mood method part and the options -following thereafter. - -A *mood method* is realized as a function which takes an audio file -and computes a number from the data contained in the database. -If this number is non-negative, we say the file *matches* the mood -method. The file matches the full mood line if it either - - - matches the mood method and the "not" keyword is not given, -or - - does not match the mood method, but the "not" keyword is given. - -The set of admissible files for the whole mood is now defined as those -files which match at least one accept mood line, but no deny mood line. -More formally, an audio file F is admissible if and only if - - (F ~ AL1 or F ~ AL2...) and not (F ~ DL1 or F ~ DN2 ...) - -where AL1, AL2... are the accept lines, DL1, DL2... are the deny -lines and "~" means "matches". - -The cases where no mood lines of accept/deny type are defined need -special treatment: - - - Neither accept nor deny lines: This treats all files as - admissible (in fact, that is the definition of the dummy mood - which is activated automatically if no moods are available). +

Moods

+ +A mood consists of a unique name and a definition. The definition +is an expression which describes which audio files are considered +admissible. At any time at most one mood can be active, meaning +that para_server will only stream files which are admissible for the +active mood. + +The expression may refer to attributes and other metadata stored in +the database. Expressions may be combined by means of logical and +arithmetical operators in a natural way. Moreover, string matching +based on regular expression or wildcard patterns is supported. + +The set of admissible files is determined by applying the expression +to each audio file in turn. For a mood definition to be valid, its +expression must evaluate to a number, a string or a boolean value +("true" or "false"). For numbers, any value other than zero means the +file is admissible. For strings, any non-empty string indicates an +admissible file. For boolean values, true means admissible and false +means not admissible. As a special case, the empty expression treats +all files as admissible. + +

Mood grammar

+ +Expressions are based on a context-free grammar which distinguishes +between several types for syntactic units or groupings. The grammar +defines a set of keywords which have a type and a corresponding +semantic value, as shown in the following table. + +Keyword | Type | Semantic value +:--------------------|--------:|:---------------------------------- +`path` | string | Full path of the current audio file +`artist` | string | Content of the artist meta tag +`title` | string | Content of the title meta tag +`album` | string | Content of the album meta tag +`comment` | string | Content of the somment meta tag +`num_attributes_set` | integer | Number of attributes which are set +`year` | integer | Content of the year meta tag [\*] +`num_played` | integer | How many times the file has been streamed +`image_id` | integer | The identifier of the (cover art) image +`lyrics_id` | integer | The identifier of the lyrics blob +`bitrate` | integer | The average bitrate +`frequency` | integer | The output sample rate +`channels` | integer | The number of channels +`is_set("foo")` | boolean | True if attribute "foo" is set. + +[\*] For most audio formats, the year tag is stored as a string. It +is converted to an integer by the mood parser. If the audio file +has no year tag or the content of the year tag is not a number, the +semantic value is zero. A special convention applies if the year tag +is a one-digit or a two-digit number. In this case 1900 is added to +the tag value. + +Expressions may be grouped using parentheses, logical and +arithmetical operators or string matching operators. The following +table lists the available operators. + +Token | Meaning +:------|:------- +`\|\|` | Logical Or +`&&` | Logical And +`!` | Logical Not +`==` | Equal (can be applied to all types) +`!=` | Not equal. Likewise +`<` | Less than +`<=` | Less or equal +`>=` | Greater or equal +`+` | Arithmetical minus +`-` | Binary/unary minus +`*` | Multiplication +`/` | Division +`=~` | Regular expression match +`=\|` | Filename match + +Besides integers, strings and booleans there is an additional type +which describes regular expression or wildcard patterns. Patterns +are not just strings because they also include a list of flags which +modify matching behaviour. + +Regular expression patterns are of the form `/pattern/[flags]`. That +is, the pattern is delimited by slashes, and is followed by zero or +more characters, each specifying a flag according to the following +table + +Flag | POSIX name | Meaning +:----|--------------:|-------- +`i` | `REG_ICASE` | Ignore case in match +`n` | `REG_NEWLINE` | Treat newline as an ordinary character + +Note that only extended regular expression patterns are supported. See +regex(3) for details. + +Wildcard patterns are similar, but the pattern must be delimited by +`'|'` characters rather than slashes. For wildcard patterns different +flags exist, as shown below. + +Flag | POSIX name | Meaning +:----|-----------------------:|-------- +`n` | `FNM_NOESCAPE` | Treat backslash as an ordinary character +`p` | `FNM_PATHNAME` | Match a slash only with a slash in pattern +`P` | `FNM_PERIOD` | Leading period has to be matched exactly +`l` | `FNM_LEADING_DIR` [\*] | Ignore "/\*" rest after successful matching +`i` | `FNM_CASEFOLD` [\*] | Ignore case in match +`e` | `FNM_EXTMATCH` [\*\*] | Enable extended pattern matching + +[\*] Not in POSIX, but both FreeBSD and NetBSD have it. + +[\*\*] GNU extension, silently ignored on non GNU systems. + +See fnmatch(3) for details. + +Mood definitions may contain arbitrary whitespace and comments. +A comment is a word beginning with #. This word and all remaining +characters of the line are ignored. + +

Example moods

+ +* Files with no/invalid year tag: `year == 0` + +* Only oldies: `year != 0 && year < 1980` + +* Only 80's Rock or Metal: `(year >= 1980 && year < 1990) && + (is_set("rock") || is_set("metal"))` + +* Files with incomplete tags: `artist == "" || title == "" || album = +"" || comment == "" || year == 0` + +* Files with no attributes defined so far: `num_attributes_set == 0` + +* Only newly added files: `num_played == 0` + +* Only poor quality files: `bitrate < 96` + +* Cope with different spellings of Motörhead: `artist =~ /mot(ö|oe{0,1})rhead/i` + +* The same with extended wildcard patterns: `artist =| |mot+(o\|oe\|ö)rhead|ie` - - Only accept lines: A file is admissible iff it matches at - least one accept line: - - F ~ AL1 or F ~ AL2 or ... - - - Only deny lines: A file is admissible iff it matches no - deny line: - - not (F ~ DL1 or F ~ DN2 ...) - - - -### List of mood_methods ### - - no_attributes_set - -Takes no arguments and matches an audio file if and only if no -attributes are set. - - is_set - -Takes the name of an attribute and matches iff that attribute is set. - - path_matches - -Takes a filename pattern and matches iff the path of the audio file -matches the pattern. - - artist_matches - album_matches - title_matches - comment_matches - -Takes an extended regular expression and matches iff the text of the -corresponding tag of the audio file matches the pattern. If the tag -is not set, the empty string is matched against the pattern. - - year ~ - bitrate ~ - frequency ~ - channels ~ - num_played ~ - image_id ~ - lyrics_id ~ - -Takes a comparator ~ of the set {<, =, <=, >, >=, !=} and a number -. Matches an audio file iff the condition ~ is -satisfied where val is the corresponding value of the audio file -(value of the year tag, bitrate in kbit/s, etc.). - -The year tag is special as its value is undefined if the audio file -has no year tag or the content of the year tag is not a number. Such -audio files never match. Another difference is the special treatment -if the year tag is a two-digit number. In this case either 1900 or -2000 is added to the tag value, depending on whether the number is -greater than 2000 plus the current year. - - -### Mood usage ### +

Mood usage

To create a new mood called "my_mood", write its definition into some temporary file, say "tmpfile", and add it to the mood table @@ -1119,27 +1143,6 @@ if the "-a" switch is given: para ls -a - -### Example mood definition ### - -Suppose you have defined attributes "punk" and "rock" and want to define -a mood containing only Punk-Rock songs. That is, an audio file should be -admissible if and only if both attributes are set. Since - - punk and rock - -is obviously the same as - - not (not punk or not rock) - -(de Morgan's rule), a mood definition that selects only Punk-Rock -songs is - - deny if not is_set punk - deny if not is_set rock - - - File renames and content changes -------------------------------- @@ -1209,7 +1212,7 @@ Audio formats The following audio formats are supported by paraslash: -### MP3 ### +

MP3

Mp3, MPEG-1 Audio Layer 3, is a common audio format for audio storage, designed as part of its MPEG-1 standard. An MP3 file is made up of @@ -1219,7 +1222,7 @@ of channels. For a typical CD-audio file (sample rate of 44.1 kHz stereo), encoded with a bit rate of 128 kbit, an MP3 frame is about 400 bytes large. -### OGG/Vorbis ### +

OGG/Vorbis

OGG is a standardized audio container format, while Vorbis is an open source codec for lossy audio compression. Since Vorbis is most @@ -1229,7 +1232,7 @@ chunks called OGG pages. A typical OGG page is about 4KB large. The Vorbis codec creates variable-bitrate (VBR) data, where the bitrate may vary considerably. -### OGG/Speex ### +

OGG/Speex

Speex is an open-source speech codec that is based on CELP (Code Excited Linear Prediction) coding. It is designed for voice @@ -1239,7 +1242,7 @@ supported. As for Vorbis audio, Speex bit-streams are often stored in OGG files. As of 2012 this codec is considered obsolete since the Oppus codec, described below, surpasses its performance in all areas. -### OGG/Opus ### +

OGG/Opus

Opus is a lossy audio compression format standardized through RFC 6716 in 2012. It combines the speech-oriented SILK codec and the @@ -1248,7 +1251,7 @@ OGG/Vorbis and OGG/Speex, Opus data is usually encapsulated in OGG containers. All known software patents which cover Opus are licensed under royalty-free terms. -### AAC ### +

AAC

Advanced Audio Coding (AAC) is a standardized, lossy compression and encoding scheme for digital audio which is the default audio @@ -1256,7 +1259,7 @@ format for Apple's iPhone, iPod, iTunes. Usually MPEG-4 is used as the container format and audio files encoded with AAC have the .m4a extension. A typical AAC frame is about 700 bytes large. -### WMA ### +

WMA

Windows Media Audio (WMA) is an audio data compression technology developed by Microsoft. A WMA file is usually encapsulated in the @@ -1265,7 +1268,7 @@ how meta data about the file is to be encoded. The bit stream of WMA is composed of superframes, each containing one or more frames of 2048 samples. For 16 bit stereo a WMA superframe is about 8K large. -### FLAC ### +

FLAC

The Free Lossless Audio Codec (FLAC) compresses audio without quality loss. It gives better compression ratios than a general purpose @@ -1730,7 +1733,7 @@ they are usually placed directly after the decoding filter. Each sample is multiplied with a scaling factor (>= 1) which makes amp and compress quite expensive in terms of computing power. -### amp ### +

amp

The amp filter amplifies the audio stream by a fixed scaling factor that must be known in advance. For para_audiod this factor is derived @@ -1757,7 +1760,7 @@ To store V in the audio file table, the command is used. The reader is encouraged to write a script that performs these computations :) -### compress ### +

compress

Unlike the amplification filter, the compress filter adjusts the volume of the audio stream dynamically without prior knowledge about the peak @@ -1978,12 +1981,14 @@ welcome. Here's a list of things you can do to help the project: - Compile and test on your favorite architecture or operating system. The code is tested only on a limited set of systems, so you will probably encounter problems when building on different systems. -- Post about about paraslash on your blog or on social networks. +- Post about paraslash on your blog or on social networks. - Build and maintain Debian/RPM packages for your favorite distribution. Note that there is no mailing list, no bug tracker and no discussion forum for paraslash. If you'd like to contribute, or have questions about contributing, send email to Andre Noll . +New releases are announced by email. If you would like to receive +these announcements, contact the author through the above address. Tools ----- @@ -2172,14 +2177,14 @@ Appendix Network protocols ----------------- -### IP ### +

IP

The _Internet Protocol_ is the primary networking protocol used for the Internet. All protocols described below use IP as the underlying layer. Both the prevalent IPv4 and the next-generation IPv6 variant are being deployed actively worldwide. -### Connection-oriented and connectionless protocols ### +

Connection-oriented and connectionless protocols

Connectionless protocols differ from connection-oriented ones in that state associated with the sending/receiving endpoints is treated @@ -2195,7 +2200,7 @@ up-to-date internal state of the connection also in general means that the sending endpoints perform congestion control, adapting to qualitative changes of the connection medium. -### Reliability ### +

Reliability

In IP networking, packets can be lost, duplicated, or delivered out of order, and different network protocols handle these @@ -2207,7 +2212,7 @@ out-of-order. Retransmission is used to guarantee loss-free delivery. Unreliable protocols, in contrast, do not guarantee ordering or data integrity. -### Classification ### +

Classification

With these definitions the protocols which are used by paraslash for steaming audio data may be classified as follows. @@ -2218,7 +2223,7 @@ steaming audio data may be classified as follows. Below we give a short descriptions of these protocols. -### TCP ### +

TCP

The _Transmission Control Protocol_ provides reliable, ordered delivery of a stream and a classic window-based congestion control. In contrast @@ -2229,7 +2234,7 @@ extensively by many application layers. Besides HTTP (the Hypertext Transfer Protocol), also FTP (the File Transfer protocol), SMTP (Simple Mail Transfer Protocol), SSH (Secure Shell) all sit on top of TCP. -### UDP ### +

UDP

The _User Datagram Protocol_ is the simplest transport-layer protocol, built as a thin layer directly on top of IP. For this reason, it offers @@ -2240,7 +2245,7 @@ means that there is no protection against packet loss or network congestion. Error checking and correction (if at all) are performed in the application. -### DCCP ### +

DCCP

The _Datagram Congestion Control Protocol_ combines the connection-oriented state maintenance known from TCP with the @@ -2257,7 +2262,7 @@ the choice of congestion control: classic, window-based congestion control known from TCP is available as CCID-2, rate-based, "smooth" congestion control is offered as CCID-3. -### HTTP ### +

HTTP

The _Hypertext Transfer Protocol_ is an application layer protocol on top of TCP. It is spoken by web servers and is most often used @@ -2267,7 +2272,7 @@ delivery of web pages only. Being a simple request/response based protocol, the semantics of the protocol also allow the delivery of multimedia content, such as audio over http. -### Multicast ### +

Multicast

IP multicast is not really a protocol but a technique for one-to-many communication over an IP network. The challenge is to deliver