Merge branch 't/filter_parse_config_cleanup'
[paraslash.git] / web / manual.m4
1 dnl To generate the html version, execute
2 dnl m4 web/manual.m4 | grutatxt --toc
3
4 define(`LOCAL_LINK_NAME', `translit(`$1', `A-Z
5 ', `a-z__')')
6 define(`REMOVE_NEWLINE', `translit(`$1',`
7 ', ` ')')
8
9 define(`REFERENCE', ./``#''`LOCAL_LINK_NAME($1)' (`REMOVE_NEWLINE($2)'))
10 define(`XREFERENCE', `$1' (`REMOVE_NEWLINE($2)'))
11 define(`EMPH', ``_''`REMOVE_NEWLINE($1)'``_'')
12
13 Paraslash user manual
14 =====================
15
16 This document describes how to install, configure and use the paraslash
17 network audio streaming system. Most chapters start with a chapter
18 overview and conclude with an example section. We try to focus on
19 general concepts and on the interaction of the various pieces of the
20 paraslash package. Hence this user manual is not meant as a replacement
21 for the manual pages that describe all command line options of each
22 paraslash executable.
23
24 ------------
25 Introduction
26 ------------
27
28 In this chapter we give an REFERENCE(Overview, overview) of the
29 interactions of the two main programs contained in the paraslash
30 package, followed by REFERENCE(The paraslash executables, brief
31 descriptions) of all executables.
32
33 Overview
34 ~~~~~~~~
35
36 The core functionality of the para suite is provided by two main
37 executables, para_server and para_audiod. The former maintains a
38 database of audio files and streams these files to para_audiod which
39 receives and plays the stream.
40
41 In a typical setting, both para_server and para_audiod act as
42 background daemons whose functionality is controlled by client
43 programs: the para_audioc client controls para_audiod over a local
44 socket while the para_client program connects to para_server over a
45 local or remote networking connection.
46
47 Typically, these two daemons run on different hosts but a local setup
48 is also possible.
49
50 A simplified picture of a typical setup is as follows
51 <<
52 <pre>
53 server_host client_host
54 ~~~~~~~~~~~ ~~~~~~~~~~~
55
56 +-----------+ audio stream +-----------+
57 |para_server| -----------------------------> |para_audiod|
58 +-----------+ +-----------+
59 ^ ^
60 | |
61 | | connect
62 | |
63 | |
64 | +-----------+
65 | |para_audioc|
66 | +-----------+
67 |
68 |
69 | connect +-----------+
70 +-------------------------------------- |para_client|
71 +-----------+
72 </pre>
73 >>
74
75 The paraslash executables
76 ~~~~~~~~~~~~~~~~~~~~~~~~~
77
78 *para_server*
79
80 para_server streams binary audio data (MP3, OGG/Vorbis, OGG/Speex,
81 M4A, WMA files) over local and/or remote networks. It listens on a
82 TCP port and accepts commands such as play, stop, pause, next from
83 authenticated clients. There are many more commands though, see the
84 man page of para_server for a description of all commands.
85
86 It supports three built-in network streaming protocols
87 (senders/receivers): HTTP, DCCP, or UDP. This is explained in more
88 detail in the section on REFERENCE(Networking, networking).
89
90 The built-in audio file selector of paraslash is used to manage your
91 audio files. It maintains statistics on the usage of all available
92 audio files such as last-played time, and the number of times each
93 file was selected.
94
95 Additional information may be added to the database to allow
96 fine-grained selection based on various properties of the audio file,
97 including information found in (ID3) tags. However, old-fashioned
98 playlists are also supported.
99
100 It is also possible to store images (album covers) and lyrics in the
101 database and associate these to the corresponding audio files.
102
103 The section on the REFERENCE(The audio file selector, audio file
104 selector) discusses this topic.
105
106
107 *para_client*
108
109 The client program to connect to para_server. paraslash commands
110 are sent to para_server and the response is dumped to STDOUT. This
111 can be used by any scripting language to produce user interfaces with
112 little programming effort.
113
114 All connections between para_server and para_client are encrypted
115 with a symmetric RC4 session key. For each user of paraslash you must
116 create a public/secret RSA key pair for authentication.
117
118 If para_client is started without non-option arguments, an interactive
119 session (shell) is started. Command history and command completion are
120 supported through libreadline.
121
122 *para_audiod*
123
124 The local daemon that collects information from para_server.
125
126 It runs on the client side and connects to para_server. As soon as
127 para_server announces the availability of an audio stream, para_audiod
128 starts an appropriate receiver, any number of filters and a paraslash
129 writer to play the stream.
130
131 Moreover, para_audiod listens on a local socket and sends status
132 information about para_server and para_audiod to local clients on
133 request. Access via this local socket may be restricted by using Unix
134 socket credentials, if available.
135
136
137 *para_audioc*
138
139 The client program which talks to para_audiod. Used to control
140 para_audiod, to receive status info, or to grab the stream at any
141 point of the decoding process. Like para_client, para_audioc supports
142 interactive sessions on systems with libreadline.
143
144 *para_recv*
145
146 A command line HTTP/DCCP/UDP stream grabber. The http mode is
147 compatible with arbitrary HTTP streaming sources (e.g. icecast).
148
149 *para_filter*
150
151 A filter program that reads from STDIN and writes to STDOUT.
152 Like para_recv, this is an atomic building block which can be used to
153 assemble higher-level audio receiving facilities. It combines several
154 different functionalities in one tool: decoders for multiple audio
155 formats and a number of processing filters, among these a normalizer
156 for audio volume.
157
158 *para_afh*
159
160 A small stand-alone program that prints tech info about the given
161 audio file to STDOUT. It can be instructed to print a "chunk table",
162 an array of offsets within the audio file or to write the content of
163 the audio file in complete chunks 'just in time'.
164
165 This allows third-party streaming software that is unaware of the
166 particular audio format to send complete frames in real time.
167
168 *para_write*
169
170 A modular audio stream writer. It supports a simple file writer
171 output plug-in and optional WAV/raw players for ALSA (Linux) and for
172 coreaudio (Mac OS). para_write can also be used as a stand-alone WAV
173 or raw audio player.
174
175
176 *para_gui*
177
178 Curses-based gui that presents status information obtained in a curses
179 window. Appearance can be customized via themes. para_gui provides
180 key-bindings for the most common server commands and new key-bindings
181 can be added easily.
182
183
184 *para_fade*
185
186 An alarm clock and volume-fader for OSS and ALSA.
187
188 -----------
189 Quick start
190 -----------
191
192 This chapter lists the REFERENCE(Requirements, necessary software)
193 that must be installed to compile the paraslash package, describes
194 how to REFERENCE(Installation, compile and install) the paraslash
195 source code and the steps that have to be performed in order to
196 REFERENCE(Quick start, set up) a typical server and client.
197
198 Requirements
199 ~~~~~~~~~~~~
200
201 In any case you'll need
202
203 - XREFERENCE(http://systemlinux.org/~maan/osl/, libosl).
204 The _object storage layer_ library is used by para_server. To
205 clone the source code repository, execute
206
207 git clone git://git.tuebingen.mpg.de/osl
208
209 - XREFERENCE(ftp://ftp.gnu.org/pub/gnu/gcc, gcc) or
210 XREFERENCE(http://clang.llvm.org, clang). All gcc versions
211 >= 3.3 are currently supported. Clang version 1.1 or newer
212 should work as well.
213
214 - XREFERENCE(ftp://ftp.gnu.org/pub/gnu/make, gnu make) is
215 also shipped with the disto. On BSD systems the gnu make
216 executable is often called gmake.
217
218 - XREFERENCE(ftp://ftp.gnu.org/pub/gnu/bash, bash). Some
219 scripts which run during compilation require the EMPH(Bourne
220 again shell). It is most likely already installed.
221
222 - XREFERENCE(http://www.openssl.org/, openssl) or
223 XREFERENCE(ftp://ftp.gnupg.org/gcrypt/libgcrypt/, libgcrypt).
224 At least one of these two libraries is needed as the backend
225 for cryptographic routines on both the server and the client
226 side. Both openssl and libgcrypt are usually shipped with the
227 distro, but you might have to install the development package
228 (libssl-dev or libgcrypt-dev on debian systems) as well.
229
230 - XREFERENCE(ftp://ftp.gnu.org/pub/gnu/gengetopt/, gengetopt)
231 is needed to generate the C code for the command line parsers
232 of all paraslash executables.
233
234 - XREFERENCE(ftp://ftp.gnu.org/pub/gnu/help2man, help2man)
235 is used to create the man pages.
236
237 Optional:
238
239 - XREFERENCE(http://www.underbit.com/products/mad/, libmad).
240 To compile in MP3 support for paraslash, the development
241 package must be installed. It is called libmad0-dev on
242 debian-based systems. Note that libmad is not necessary on
243 the server side, i.e. for sending MP3 files.
244
245 - XREFERENCE(http://www.underbit.com/products/mad/,
246 libid3tag). For version-2 ID3 tag support, you'll need
247 the libid3tag development package libid3tag0-dev. Without
248 libid3tag, only version one tags are recognized.
249
250 - XREFERENCE(http://www.xiph.org/downloads/, ogg vorbis).
251 For ogg vorbis streams you'll need libogg, libvorbis,
252 libvorbisfile. The corresponding Debian packages are called
253 libogg-dev and libvorbis-dev.
254
255 - XREFERENCE(http://www.audiocoding.com/, libfaad). For aac
256 files (m4a) you'll need libfaad (libfaad-dev).
257
258 - XREFERENCE(http://www.speex.org/, speex). In order to stream
259 or decode speex files, libspeex (libspeex-dev) is required.
260
261 - XREFERENCE(http://flac.sourceforge.net/, flac). To stream
262 or decode files encoded with the _Free Lossless Audio Codec_,
263 libFLAC (libFLAC-dev) must be installed.
264
265 - XREFERENCE(ftp://ftp.alsa-project.org/pub/lib/, alsa-lib). On
266 Linux, you'll need to have ALSA's development package
267 libasound2-dev installed.
268
269 - XREFERENCE(http://downloads.xiph.org/releases/ao/,
270 libao). Needed to build the ao writer (ESD, PulseAudio,...).
271 Debian package: libao-dev.
272
273 - XREFERENCE(http://cnswww.cns.cwru.edu/php/chet/readline/rltop.html,
274 GNU Readline). If this library (libreadline-dev) is installed,
275 para_client and para_audioc support interactive sessions.
276
277 Installation
278 ~~~~~~~~~~~~
279
280 First make sure all non-optional packages listed in the section on
281 REFERENCE(Requirements, required software) are installed on your
282 system.
283
284 You don't need everything listed there. In particular, MP3, OGG/Vorbis,
285 OGG/Speex and AAC support are all optional. The configure script will
286 detect what is installed on your system and will only try to build
287 those executables that can be built with your setup.
288
289 Note that no special decoder library (not even the MP3 decoding library
290 libmad) is needed for para_server if you only want to stream MP3 or WMA
291 files. Also, it's fine to use para_server on a box without sound card.
292
293 Next, install the paraslash package on all machines, you'd like this
294 software to run on. If you compile from a released tarball, execute
295
296 (./configure && make) > /dev/null
297
298 When compiling from git or from snapshots downloaded via gitweb,
299 the above command will not work because the configure script is not
300 included in the git repository. In this case the following command
301 should be used instead:
302
303 ./autogen.sh
304
305 This runs autoconf to generate the configure script, then runs it as
306 above. Therefore you'll need autoconf for this to work.
307
308 There should be no errors but probably some warnings about missing
309 packages which usually implies that not all audio formats will be
310 supported. If headers or libs are installed at unusual locations you
311 might need to tell the configure script where to find them. Try
312
313 ./configure --help
314
315 to see a list of options. If the paraslash package was compiled
316 successfully, execute (optionally)
317
318 make test
319
320 to run the paraslash test suite. If all tests pass, execute as root
321
322 make install
323
324 to install executables under /usr/local/bin and the man pages under
325 /usr/local/man.
326
327 Configuration
328 ~~~~~~~~~~~~~
329
330 *Step 1*: Create a paraslash user
331
332 In order to control para_server at runtime you must create a paraslash
333 user. As authentication is based on the RSA crypto system you'll have
334 to create an RSA key pair. If you already have a user and an RSA key
335 pair, you may skip this step.
336
337 In this section we'll assume a typical setup: You would like to run
338 para_server on some host called server_host as user foo, and you want
339 to connect to para_server from another machine called client_host as
340 user bar.
341
342 As foo@server_host, create ~/.paraslash/server.users by typing the
343 following commands:
344
345 user=bar
346 target=~/.paraslash/server.users
347 key=~/.paraslash/id_rsa.pub.$user
348 perms=AFS_READ,AFS_WRITE,VSS_READ,VSS_WRITE
349 mkdir -p ~/.paraslash
350 echo "user $user $key $perms" >> $target
351
352 Next, change to the "bar" account on client_host and generate the
353 key pair with the commands
354
355 ssh-keygen -t rsa -b 2048
356 # hit enter twice to create a key with no passphrase
357
358 This generates the two files id_rsa and id_rsa.pub in ~/.ssh. Note
359 that paraslash can also read keys generated by the "openssl genrsa"
360 command. However, since keys created with ssh-keygen can also be used
361 for ssh, this method is recommended.
362
363 Note that para_server refuses to use a key if it is shorter than 2048
364 bits. In particular, the RSA keys of paraslash 0.3.x will not work
365 with version 0.4.x. Moreover, para_client refuses to use a (private)
366 key which is world-readable.
367
368 para_server only needs to know the public key of the key pair just
369 created. Copy this public key to server_host:
370
371 src=~/.ssh/id_rsa.pub
372 dest=.paraslash/id_rsa.pub.$LOGNAME
373 scp $src foo@server_host:$dest
374
375 Finally, tell para_client to connect to server_host:
376
377 conf=~/.paraslash/client.conf
378 echo 'hostname server_host' > $conf
379
380
381 *Step 2*: Start para_server
382
383 Before starting the server make sure you have write permissions to
384 the directory /var/paraslash that has been created during installation:
385
386 sudo chown $LOGNAME /var/paraslash
387
388 Alternatively, use the --afs_socket Option to specify a different
389 location for the AFS command socket.
390
391 For this first try, we'll use the info loglevel to make the output
392 of para_server more verbose.
393
394 para_server -l info
395
396 Now you can use para_client to connect to the server and issue
397 commands. Open a new shell as bar@client_host and try
398
399 para_client help
400 para_client si
401
402 to retrieve the list of available commands and some server info.
403 Don't proceed if this doesn't work.
404
405 *Step 3*: Create and populate the database
406
407 An empty database is created with
408
409 para_client init
410
411 This initializes a couple of empty tables under
412 ~/.paraslash/afs_database-0.4. You normally don't need to look at these
413 tables, but it's good to know that you can start from scratch with
414
415 rm -rf ~/.paraslash/afs_database-0.4
416
417 in case something went wrong.
418
419 Next, you need to add some audio files to that database so that
420 para_server knows about them. Choose an absolute path to a directory
421 containing some audio files and add them to the audio file table:
422
423 para_client add /my/mp3/dir
424
425 This might take a while, so it is a good idea to start with a directory
426 containing not too many files. Note that the table only contains data
427 about the audio files found, not the files themselves.
428
429 You may print the list of all known audio files with
430
431 para_client ls
432
433 *Step 4*: Configure para_audiod
434
435 para_audiod needs to create a "well-known" socket for the clients to
436 connect to. The default path for this socket is
437
438 /var/paraslash/audiod_socket.$HOSTNAME
439
440 In order to make this directory writable for para_audiod, execute
441 as bar@client_host
442
443 sudo chown $LOGNAME /var/paraslash
444
445
446 We will also have to tell para_audiod that it should receive the
447 audio stream from server_host via http:
448
449 para_audiod -l info -r '.:http -i server_host'
450
451 You should now be able to listen to the audio stream once para_server
452 starts streaming. To activate streaming, execute
453
454 para_client play
455
456 Since no playlist has been specified yet, the "dummy" mode which
457 selects all known audio files is activated automatically. See the
458 section on the REFERENCE(The audio file selector, audio file selector)
459 for how to use playlists and moods to specify which files should be
460 streamed in which order.
461
462 *Troubleshooting*
463
464 It did not work? To find out why, try to receive, decode and play the
465 stream manually using para_recv, para_filter and para_write as follows.
466
467 For simplicity we assume that you're running Linux/ALSA and that only
468 MP3 files have been added to the database.
469
470 para_recv -r 'http -i server_host' > file.mp3
471 # (interrupt with CTRL+C after a few seconds)
472 ls -l file.mp3 # should not be empty
473 para_filter -f mp3dec -f wav < file.mp3 > file.wav
474 ls -l file.wav # should be much bigger than file.mp3
475 para_write -w alsa < file.wav
476
477 Double check what is logged by para_server and use the --loglevel
478 option of para_recv, para_filter and para_write to increase verbosity.
479
480 ---------------
481 User management
482 ---------------
483
484 para_server uses a challenge-response mechanism to authenticate
485 requests from incoming connections, similar to ssh's public key
486 authentication method. Authenticated connections are encrypted using
487 the RC4 stream cipher.
488
489 In this chapter we briefly describe RSA and RC4 and sketch the
490 REFERENCE(Client-server authentication, authentication handshake)
491 between para_client and para_server. User management is discussed
492 in the section on REFERENCE(The user_list file, the user_list file).
493 These sections are all about communication between the client and the
494 server. Connecting para_audiod is a different matter and is described
495 in a REFERENCE(Connecting para_audiod, separate section).
496
497
498
499 RSA and RC4
500 ~~~~~~~~~~~
501
502 RSA is an asymmetric block cipher which is used in many applications,
503 including ssh and gpg. An RSA key consists in fact of two keys,
504 called the public key and the private key. A message can be encrypted
505 with either key and only the counterpart of that key can decrypt
506 the message. While RSA can be used for both signing and encrypting
507 a message, paraslash uses RSA only for the latter purpose. The
508 RSA public key encryption and signatures algorithms are defined in
509 detail in RFC 2437.
510
511 RC4 is a stream cipher, i.e. the input is XORed with a pseudo-random
512 key stream to produce the output. Decryption uses the same function
513 calls as encryption. While RC4 supports variable key lengths,
514 paraslash uses a fixed length of 256 bits, which is considered a
515 strong encryption by today's standards. Since the same key must never
516 be used twice, a different, randomly-generated key is used for every
517 new connection.
518
519 Client-server authentication
520 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
521
522 The authentication handshake between para_client and para_server goes
523 as follows:
524
525 - para_client connects to para_server and sends an
526 authentication request for a user. It does so by connecting
527 to TCP port 2990 of the server host. This port is called the
528 para_server _control port_.
529
530 - para_server accepts the connection and forks a child process
531 which handles the incoming request. The parent process keeps
532 listening on the control port while the child process (also
533 called para_server below) continues as follows.
534
535 - para_server loads the RSA public key of that user, fills a
536 fixed-length buffer with random bytes, encrypts that buffer
537 using the public key and sends the encrypted buffer to the
538 client. The first part of the buffer is the challenge which
539 is used for authentication while the second part is the RC4
540 session key.
541
542 - para_client receives the encrypted buffer and decrypts it
543 with the user's private key, thereby obtaining the challenge
544 buffer and the session key. It sends the SHA1 hash value of
545 the challenge back to para_server and stores the session key
546 for further use.
547
548 - para_server also computes the SHA1 hash of the challenge
549 and compares it against what was sent back by the client.
550
551 - If the two hashes do not match, the authentication has
552 failed and para_server closes the connection.
553
554 - Otherwise the user is considered authenticated and the client
555 is allowed to proceed by sending a command to be executed. From
556 this point on the communication is encrypted using the RC4
557 stream cipher with the session key known to both peers.
558
559 paraslash relies on the quality of the pseudo-random bytes provided
560 by the crypto library (openssl or libgcrypt), on the security of
561 the implementation of the RSA and RC4 crypto routines and on the
562 infeasibility to invert the SHA1 function.
563
564 Neither para_server or para_client create RSA keys on their own. This
565 has to be done once for each user as sketched in REFERENCE(Quick start,
566 Quick start) and discussed in more detail REFERENCE(The user_list
567 file, below).
568
569 The user_list file
570 ~~~~~~~~~~~~~~~~~~
571
572 At startup para_server reads the user list file which contains one
573 line per user. The default location of the user list file may be
574 changed with the --user_list option.
575
576 There should be at least one user in this file. Each user must have
577 an RSA key pair. The public part of the key is needed by para_server
578 while the private key is needed by para_client. Each line of the
579 user list file must be of the form
580
581 user <username> <key> <perms>
582
583 where _username_ is an arbitrary string (usually the user's login
584 name), _key_ is the full path to that user's public RSA key, and
585 _perms_ is a comma-separated list of zero or more of the following
586 permission bits:
587
588 +---------------------------------------------------------+
589 | AFS_READ | read the contents of the databases |
590 +-----------+---------------------------------------------+
591 | AFS_WRITE | change database contents |
592 +-----------+---------------------------------------------+
593 | VSS_READ | obtain information about the current stream |
594 +-----------+---------------------------------------------+
595 | VSS_WRITE | change the current stream |
596 +---------------------------------------------------------+
597
598 The permission bits specify which commands the user is allowed to
599 execute. The output of
600
601 para_client help
602
603 contains in the third column the permissions needed to execute the
604 command.
605
606 It is possible to make para_server reread the user_list file by
607 executing the paraslash "hup" command or by sending SIGHUP to the
608 PID of para_server.
609
610
611 Connecting para_audiod
612 ~~~~~~~~~~~~~~~~~~~~~~
613
614 para_audiod listens on a Unix domain socket. Those sockets are
615 for local communication only, so only local users can connect to
616 para_audiod. The default is to let any user connect but this can be
617 restricted on platforms that support UNIX socket credentials which
618 allow para_audiod to obtain the Unix credentials of the connecting
619 process.
620
621 Use para_audiod's --user_allow option to allow connections only for
622 a limited set of users.
623
624 -----------------------
625 The audio file selector
626 -----------------------
627
628 paraslash comes with a sophisticated audio file selector (AFS),
629 whose main task is to determine which file to stream next, based on
630 information on the audio files stored in a database. It communicates
631 also with para_client whenever an AFS command is executed, for example
632 to answer a database query.
633
634 Besides the traditional playlists, AFS supports audio file selection
635 based on _moods_ which act as a filter that limits the set of all
636 known audio files to those which satisfy certain criteria. It also
637 maintains tables containing images (e.g. album cover art) and lyrics
638 that can be associated with one or more audio files.
639
640 AFS uses XREFERENCE(http://systemlinux.org/~maan/osl/, libosl), the
641 object storage layer library, as the backend library for storing
642 information on audio files, playlists, etc. This library offers
643 functionality similar to a relational database, but is much more
644 lightweight than a full database backend.
645
646 In this chapter we sketch the setup of the REFERENCE(The AFS process,
647 AFS process) during server startup and proceed with the description
648 of the REFERENCE(Database layout, layout) of the various database
649 tables. The section on REFERENCE(Playlists and moods, playlists
650 and moods) explains these two audio file selection mechanisms
651 in detail and contains pratical examples. The way REFERENCE(File
652 renames and content changes, file renames and content changes) are
653 detected is discussed briefly before the REFERENCE(Troubleshooting,
654 Troubleshooting) section concludes the chapter.
655
656 The AFS process
657 ~~~~~~~~~~~~~~~
658
659 On startup, para_server forks to create the AFS process which opens
660 the OSL database tables. The server process communicates with the
661 AFS process via pipes and shared memory. Usually, the AFS process
662 awakes only briefly whenever the current audio file changes. The AFS
663 process determines the next audio file, opens it, verifies it has
664 not been changed since it was added to the database and passes the
665 open file descriptor to the server process, along with audio file
666 meta-data such as file name, duration, audio format and so on. The
667 server process then starts to stream the audio file.
668
669 The AFS process also accepts connections from local clients via
670 a well-known socket. However, only child processes of para_server
671 may connect through this socket. All server commands that have the
672 AFS_READ or AFS_WRITE permission bits use this mechanism to query or
673 change the database.
674
675 Database layout
676 ~~~~~~~~~~~~~~~
677
678 *The audio file table*
679
680 This is the most important and usually also the largest table of the
681 AFS database. It contains the information needed to stream each audio
682 file. In particular the following data is stored for each audio file.
683
684 - SHA1 hash value of the audio file contents. This is computed
685 once when the file is added to the database. Whenever AFS
686 selects this audio file for streaming the hash value is
687 recomputed and checked against the value stored in the
688 database to detect content changes.
689
690 - The time when this audio file was last played.
691
692 - The number of times the file has been played so far.
693
694 - The attribute bitmask.
695
696 - The image id which describes the image associated with this
697 audio file.
698
699 - The lyrics id which describes the lyrics associated with
700 this audio file.
701
702 - The audio format id (MP3, OGG, ...).
703
704 - An amplification value that can be used by the amplification
705 filter to pre-amplify the decoded audio stream.
706
707 - The chunk table. It describes the location and the timing
708 of the building blocks of the audio file. This is used by
709 para_server to send chunks of the file at appropriate times.
710
711 - The duration of the audio file.
712
713 - Tag information contained in the audio file (ID3 tags,
714 Vorbis comments, ...).
715
716 - The number of channels
717
718 - The encoding bitrate.
719
720 - The sampling frequency.
721
722 To add or refresh the data contained in the audio file table, the _add_
723 command is used. It takes the full path of either an audio file or a
724 directory. In the latter case, the directory is traversed recursively
725 and all files which are recognized as valid audio files are added to
726 the database.
727
728 *The attribute table*
729
730 The attribute table contains two columns, _name_ and _bitnum_. An
731 attribute is simply a name for a certain bit number in the attribute
732 bitmask of the audio file table.
733
734 Each of the 64 bits of the attribute bitmask can be set for each
735 audio file individually. Hence up to 64 different attributes may be
736 defined. For example, "pop", "rock", "blues", "jazz", "instrumental",
737 "german_lyrics", "speech", whatever. You are free to choose as
738 many attributes as you like and there are no naming restrictions
739 for attributes.
740
741 A new attribute "test" is created by
742
743 para_client addatt test
744 and
745 para_client lsatt
746
747 lists all available attributes. You can set the "test" attribute for
748 an audio file by executing
749
750 para_client setatt test+ /path/to/the/audio/file
751
752 Similarly, the "test" bit can be removed from an audio file with
753
754 para_client setatt test- /path/to/the/audio/file
755
756 Instead of a path you may use a shell wildcard pattern. The attribute
757 is applied to all audio files matching this pattern:
758
759 para_client setatt test+ '/test/directory/*'
760
761 The command
762
763 para_client -- ls -lv
764
765 gives you a verbose listing of your audio files also showing which
766 attributes are set.
767
768 In case you wonder why the double-dash in the above command is needed:
769 It tells para_client to not interpret the options after the dashes. If
770 you find this annoying, just say
771
772 alias para='para_client --'
773
774 and be happy. In what follows we shall use this alias.
775
776 The "test" attribute can be dropped from the database with
777
778 para rmatt test
779
780 Read the output of
781
782 para help ls
783 para help setatt
784
785 for more information and a complete list of command line options to
786 these commands.
787
788 *Blob tables*
789
790 The image, lyrics, moods and playlists tables are all blob tables.
791 Blob tables consist of three columns each: The identifier which is
792 a positive non-negative number that is auto-incremented, the name
793 (an arbitrary string) and the content (the blob).
794
795 All blob tables support the same set of actions: cat, ls, mv, rm
796 and add. Of course, _add_ is used for adding new blobs to the table
797 while the other actions have the same meaning as the corresponding
798 Unix commands. The paraslash commands to perform these actions are
799 constructed as the concatenation of the table name and the action. For
800 example addimg, catimg, lsimg, mvimg, rmimg are the commands that
801 manipulate or query the image table.
802
803 The add variant of these commands is special as these commands read
804 the blob contents from stdin. To add an image to the image table the
805 command
806
807 para addimg image_name < file.jpg
808
809 can be used.
810
811 Note that the images and lyrics are not interpreted at all, and also
812 the playlist and the mood blobs are only investigated when the mood
813 or playlist is activated with the select command.
814
815 *The score table*
816
817 Unlike all other tables the contents of the score table remain in
818 memory and are never stored on disk. The score table contains two
819 columns: The SHA1 hash value (of an audio file) and its current
820 score.
821
822 However, only those files which are admissible for the current mood
823 or playlist are contained in the score table. The audio file selector
824 always chooses the row with the highest score as the file to stream
825 next. While doing so, it computes the new score and updates the
826 last_played and the num_played fields in the audio file table.
827
828 The score table is recomputed by the select command which loads a
829 mood or playlist. Audio files are chosen for streaming from the rows
830 of the score table on a highest-score-first basis.
831
832
833 Playlists and moods
834 ~~~~~~~~~~~~~~~~~~~
835
836 Playlists and moods offer two different ways of specifying the set of
837 admissible files. A playlist in itself describes a set of admissible
838 files. A mood, in contrast, describes the set of admissible files in
839 terms of attributes and other type of information available in the
840 audio file table. As an example, a mood can define a filename pattern,
841 which is then matched against the names of audio files in the table.
842
843 *Playlists*
844
845 Playlists are accommodated in the playlist table of the afs database,
846 using the aforementioned blob format for tables. A new playlist is
847 created with the addpl command by specifying the full (absolute)
848 paths of all desired audio files, separated by newlines. Example:
849
850 find /my/mp3/dir -name "*.mp3" | para addpl my_playlist
851
852 If _my_playlist_ already exists it is overwritten. To activate the
853 new playlist, execute
854
855 para select p/my_playlist
856
857 The audio file selector will assign scores to each entry of the list,
858 in descending order so that files will be selected in order. If a
859 file could not be opened for streaming, its entry is removed from
860 the score table (but not from the playlist).
861
862 *Moods*
863
864 A mood consists of a unique name and its *mood definition*, which is
865 a set of *mood lines* containing expressions in terms of attributes
866 and other data contained in the database.
867
868 At any time at most one mood can be *active* which means that
869 para_server is going to select only files from that subset of
870 admissible files.
871
872 So in order to create a mood definition one has to write a set of
873 mood lines. Mood lines come in three flavours: Accept lines, deny
874 lines and score lines.
875
876 The general syntax of the three types of mood lines is
877
878
879 accept [with score <score>] [if] [not] <mood_method> [options]
880 deny [with score <score>] [if] [not] <mood_method> [options]
881 score <score> [if] [not] <mood_method> [options]
882
883
884 Here <score> is either an integer or the string "random" which assigns
885 a random score to all matching files. The score value changes the
886 order in which admissible files are going to be selected, but is of
887 minor importance for this introduction.
888
889 So we concentrate on the first two forms, i.e. accept and deny
890 lines. As usual, everything in square brackets is optional, i.e.
891 accept/deny lines take the following form when ignoring scores:
892
893 accept [if] [not] <mood_method> [options]
894
895 and analogously for the deny case. The "if" keyword is only syntactic
896 sugar and has no function. The "not" keyword just inverts the result,
897 so the essence of a mood line is the mood method part and the options
898 following thereafter.
899
900 A *mood method* is realized as a function which takes an audio file
901 and computes a number from the data contained in the database.
902 If this number is non-negative, we say the file *matches* the mood
903 method. The file matches the full mood line if it either
904
905 - matches the mood method and the "not" keyword is not given,
906 or
907 - does not match the mood method, but the "not" keyword is given.
908
909 The set of admissible files for the whole mood is now defined as those
910 files which match at least one accept mood line, but no deny mood line.
911 More formally, an audio file F is admissible if and only if
912
913 (F ~ AL1 or F ~ AL2...) and not (F ~ DL1 or F ~ DN2 ...)
914
915 where AL1, AL2... are the accept lines, DL1, DL2... are the deny
916 lines and "~" means "matches".
917
918 The cases where no mood lines of accept/deny type are defined need
919 special treatment:
920
921 - Neither accept nor deny lines: This treats all files as
922 admissible (in fact, that is the definition of the dummy mood
923 which is activated automatically if no moods are available).
924
925 - Only accept lines: A file is admissible iff it matches at
926 least one accept line:
927
928 F ~ AL1 or F ~ AL2 or ...
929
930 - Only deny lines: A file is admissible iff it matches no
931 deny line:
932
933 not (F ~ DL1 or F ~ DN2 ...)
934
935
936
937 *List of mood_methods*
938
939 no_attributes_set
940
941 Takes no arguments and matches an audio file if and only if no
942 attributes are set.
943
944 is_set <attribute_name>
945
946 Takes the name of an attribute and matches iff that attribute is set.
947
948 path_matches <pattern>
949
950 Takes a filename pattern and matches iff the path of the audio file
951 matches the pattern.
952
953 artist_matches <pattern>
954 album_matches <pattern>
955 title_matches <pattern>
956 comment_matches <pattern>
957
958 Takes an extended regular expression and matches iff the text of the
959 corresponding tag of the audio file matches the pattern. If the tag
960 is not set, the empty string is matched against the pattern.
961
962 year ~ <num>
963 bitrate ~ <num>
964 frequency ~ <num>
965 channels ~ <num>
966 num_played ~ <num>
967
968 Takes a comparator ~ of the set {<, =, <=, >, >=, !=} and a number
969 <num>. Matches an audio file iff the condition <val> ~ <num> is
970 satisfied where val is the corresponding value of the audio file
971 (value of the year tag, bitrate in kbit/s, frequency in Hz, channel
972 count, play count).
973
974 The year tag is special as its value is undefined if the audio file
975 has no year tag or the content of the year tag is not a number. Such
976 audio files never match. Another difference is the special treatment
977 if the year tag is a two-digit number. In this case either 1900 or
978 2000 is added to the tag value, depending on whether the number is
979 greater than 2000 plus the current year.
980
981
982 *Mood usage*
983
984 To create a new mood called "my_mood", write its definition into
985 some temporary file, say "tmpfile", and add it to the mood table
986 by executing
987
988 para addmood my_mood < tmpfile
989
990 If the mood definition is really short, you may just pipe it to the
991 client instead of using temporary files. Like this:
992
993 echo "$MOOD_DEFINITION" | para addmood my_mood
994
995 There is no need to keep the temporary file since you can always use
996 the catmood command to get it back:
997
998 para catmood my_mood
999
1000 A mood can be activated by executing
1001
1002 para select m/my_mood
1003
1004 Once active, the list of admissible files is shown by the ls command
1005 if the "-a" switch is given:
1006
1007 para ls -a
1008
1009
1010 *Example mood definition*
1011
1012 Suppose you have defined attributes "punk" and "rock" and want to define
1013 a mood containing only Punk-Rock songs. That is, an audio file should be
1014 admissible if and only if both attributes are set. Since
1015
1016 punk and rock
1017
1018 is obviously the same as
1019
1020 not (not punk or not rock)
1021
1022 (de Morgan's rule), a mood definition that selects only Punk-Rock
1023 songs is
1024
1025 deny if not is_set punk
1026 deny if not is_set rock
1027
1028
1029
1030 File renames and content changes
1031 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1032
1033 Since the audio file selector knows the SHA1 of each audio file that
1034 has been added to the afs database, it recognizes if the content of
1035 a file has changed, e.g. because an ID3 tag was added or modified.
1036 Also, if a file has been renamed or moved to a different location,
1037 afs will detect that an entry with the same hash value already exists
1038 in the audio file table.
1039
1040 In both cases it is enough to just re-add the new file. In the
1041 first case (file content changed), the audio table is updated, while
1042 metadata such as the num_played and last_played fields, as well as
1043 the attributes, remain unchanged. In the other case, when the file
1044 is moved or renamed, only the path information is updated, all other
1045 data remains as before.
1046
1047 It is possible to change the behaviour of the add command by using the
1048 "-l" (lazy add) or the "-f" (force add) option.
1049
1050 Troubleshooting
1051 ~~~~~~~~~~~~~~~
1052
1053 Use the debug loglevel (option -l debug for most commands) to show
1054 debugging info. Almost all paraslash executables have a brief online
1055 help which is displayed by using the -h switch. The --detailed-help
1056 option prints the full help text.
1057
1058 If para_server crashed or was killed by SIGKILL (signal 9), it
1059 may refuse to start again because of "dirty osl tables". In this
1060 case you'll have to run the oslfsck program of libosl to fix your
1061 database. It might be necessary to use --force (even if your name
1062 isn't Luke). However, make sure para_server isn't running before
1063 executing oslfsck --force.
1064
1065 If you don't mind to recreate your database you can start
1066 from scratch by removing the entire database directory, i.e.
1067
1068 rm -rf ~/.paraslash/afs_database-0.4
1069
1070 Be aware that this removes all attribute definitions, all playlists
1071 and all mood definitions and requires to re-initialize the tables.
1072
1073 Although oslfsck fixes inconsistencies in database tables it doesn't
1074 care about the table contents. To check for invalid table contents, use
1075
1076 para_client check
1077
1078 This prints out references to missing audio files as well as invalid
1079 playlists and mood definitions.
1080
1081 ---------------------------------------
1082 Audio formats and audio format handlers
1083 ---------------------------------------
1084
1085 Audio formats
1086 ~~~~~~~~~~~~~
1087
1088 The following audio formats are supported by paraslash:
1089
1090 *MP3*
1091
1092 Mp3, MPEG-1 Audio Layer 3, is a common audio format for audio storage,
1093 designed as part of its MPEG-1 standard. An MP3 file is made up of
1094 multiple MP3 frames, which consist of a header and a data block. The
1095 size of an MP3 frame depends on the bit rate and on the number
1096 of channels. For a typical CD-audio file (sample rate of 44.1 kHz
1097 stereo), encoded with a bit rate of 128 kbit, an MP3 frame is about
1098 400 bytes large.
1099
1100 *OGG/Vorbis*
1101
1102 OGG is a standardized audio container format, while Vorbis is an
1103 open source codec for lossy audio compression. Since Vorbis is most
1104 commonly made available via the OGG container format, it is often
1105 referred to as OGG/Vorbis. The OGG container format divides data into
1106 chunks called OGG pages. A typical OGG page is about 4KB large. The
1107 Vorbis codec creates variable-bitrate (VBR) data, where the bitrate
1108 may vary considerably.
1109
1110 *OGG/Speex*
1111
1112 Speex is an open-source speech codec that is based on CELP (Code
1113 Excited Linear Prediction) coding. It is designed for voice
1114 over IP applications, has modest complexity and a small memory
1115 footprint. Wideband and narrowband (telephone quality) speech are
1116 supported. As for Vorbis audio, Speex bit-streams are often stored
1117 in OGG files.
1118
1119 *AAC*
1120
1121 Advanced Audio Coding (AAC) is a standardized, lossy compression
1122 and encoding scheme for digital audio which is the default audio
1123 format for Apple's iPhone, iPod, iTunes. Usually MPEG-4 is used as
1124 the container format and audio files encoded with AAC have the .m4a
1125 extension. A typical AAC frame is about 700 bytes large.
1126
1127 *WMA*
1128
1129 Windows Media Audio (WMA) is an audio data compression technology
1130 developed by Microsoft. A WMA file is usually encapsulated in the
1131 Advanced Systems Format (ASF) container format, which also specifies
1132 how meta data about the file is to be encoded. The bit stream of WMA
1133 is composed of superframes, each containing one or more frames of
1134 2048 samples. For 16 bit stereo a WMA superframe is about 8K large.
1135
1136 *FLAC*
1137
1138 The Free Lossless Audio Codec (FLAC) compresses audio without quality
1139 loss. It gives better compression ratios than a general purpose
1140 compressor like zip or bzip2 because FLAC is designed specifically
1141 for audio. A FLAC-encoded file consits of frames of varying size, up
1142 to 16K. Each frame starts with a header that contains all information
1143 necessary to decode the frame.
1144
1145 Meta data
1146 ~~~~~~~~~
1147
1148 Unfortunately, each audio format has its own conventions how meta
1149 data is added as tags to the audio file.
1150
1151 For MP3 files, ID3, version 1 and 2 are widely used. ID3 version 1
1152 is rather simple but also very limited as it supports only artist,
1153 title, album, year and comment tags. Each of these can only be at most
1154 32 characters long. ID3, version 2 is much more flexible but requires
1155 a separate library being installed for paraslash to support it.
1156
1157 Ogg vorbis, ogg speex and flac files contain meta data as Vorbis
1158 comments, which are typically implemented as strings of the form
1159 "[TAG]=[VALUE]". Unlike ID3 version 1 tags, one may use whichever
1160 tags are appropriate for the content.
1161
1162 AAC files usually use the MPEG-4 container format for storing meta
1163 data while WMA files wrap meta data as special objects within the
1164 ASF container format.
1165
1166 paraslash only tracks the most common tags that are supported by
1167 all tag variants: artist, title, year, album, comment. When a file
1168 is added to the AFS database, the meta data of the file is extracted
1169 and stored in the audio file table.
1170
1171 Chunks and chunk tables
1172 ~~~~~~~~~~~~~~~~~~~~~~~
1173
1174 paraslash uses the word "chunk" as common term for the building blocks
1175 of an audio file. For MP3 files, a chunk is the same as an MP3 frame,
1176 while for OGG files a chunk is an OGG page, etc. Therefore the chunk
1177 size varies considerably between audio formats, from a few hundred
1178 bytes (MP3) up to 16K (FLAC).
1179
1180 The chunk table contains the offsets within the audio file that
1181 correspond to the chunk boundaries of the file. Like the meta data,
1182 the chunk table is computed and stored in the database whenever an
1183 audio file is added.
1184
1185 The paraslash senders (see below) always send complete chunks. The
1186 granularity for seeking is therefore determined by the chunk size.
1187
1188 Audio format handlers
1189 ~~~~~~~~~~~~~~~~~~~~~
1190
1191 For each audio format paraslash contains an audio format handler whose
1192 first task is to tell whether a given file is a valid audio file of
1193 this type. If so, the audio file handler extracts some technical data
1194 (duration, sampling rate, number of channels etc.), computes the
1195 chunk table and reads the meta data.
1196
1197 The audio format handler code is linked into para_server and executed
1198 via the _add_ command. The same code is also available as a stand-alone
1199 tool, para_afh, which can be used to print the technical data, the
1200 chunk table and the meta data of a file. Furthermore, one can use
1201 para_afh to cut an audio file, i.e. to select some of its chunks to
1202 produce a new file containing only these chunks.
1203
1204 ----------
1205 Networking
1206 ----------
1207
1208 Paraslash uses different network connections for control and data.
1209 para_client communicates with para_server over a dedicated TCP control
1210 connection. To transport audio data, separate data connections are
1211 used. For these data connections, a variety of transports (UDP, DCCP,
1212 HTTP) can be chosen.
1213
1214 The chapter starts with the REFERENCE(The paraslash control
1215 service, control service), followed by a section on the various
1216 REFERENCE(Streaming protocols, streaming protocols) in which the data
1217 connections are described. The way audio file headers are embedded into
1218 the stream is discussed REFERENCE(Streams with headers and headerless
1219 streams, briefly) before the REFERENCE(Networking examples, example
1220 section) which illustrates typical commands for real-life scenarios.
1221
1222 Both IPv4 and IPv6 are supported.
1223
1224 The paraslash control service
1225 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1226
1227 para_server is controlled at runtime via the paraslash control
1228 connection. This connection is used for server commands (play, stop,
1229 ...) as well as for afs commands (ls, select, ...).
1230
1231 The server listens on a TCP port and accepts connections from clients
1232 that connect the open port. Each connection causes the server to fork
1233 off a client process which inherits the connection and deals with that
1234 client only. In this classical accept/fork approach the server process
1235 is unaffected if the child dies or goes crazy for whatever reason. In
1236 fact, the child process can not change address space of server process.
1237
1238 The section on REFERENCE(Client-server authentication, client-server
1239 authentication) above described the early connection establishment
1240 from the crypto point of view. Here it is described what happens
1241 after the connection (including crypto setup) has been established.
1242 There are four processes involved during command dispatch as sketched
1243 in the following diagram.
1244
1245 <<
1246 <pre>
1247 server_host client_host
1248 ~~~~~~~~~~~ ~~~~~~~~~~~
1249
1250 +-----------+ connect +-----------+
1251 |para_server|<------------------------------ |para_client|
1252 +-----------+ +-----------+
1253 | ^
1254 | fork +---+ |
1255 +----------> |AFS| |
1256 | +---+ |
1257 | ^ |
1258 | | |
1259 | | connect (cookie) |
1260 | | |
1261 | | |
1262 | fork +-----+ inherited connection |
1263 +---------->|child|<--------------------------+
1264 +-----+
1265 </pre>
1266 >>
1267
1268 Note that the child process is not a child of the afs process,
1269 so communication of these two processes has to happen via local
1270 sockets. In order to avoid abuse of the local socket by unrelated
1271 processes, a magic cookie is created once at server startup time just
1272 before the server process forks off the AFS process. This cookie is
1273 known to the server, AFS and the child, but not to unrelated processes.
1274
1275 There are two different kinds of commands: First there are commands
1276 that cause the server to respond with some answer such as the list
1277 of all audio files. All but the addblob commands (addimg, addlyr,
1278 addpl, addmood) are of this kind. The addblob commands add contents
1279 to the database, so they need to transfer data the other way round,
1280 from the client to the server.
1281
1282 There is no knowledge about the server commands built into para_client,
1283 so it does not know about addblob commands. Instead, it inspects the
1284 first data package sent by the server for a magic string. If this
1285 string was found, it sends STDIN to the server, otherwise it dumps
1286 data from the server to STDOUT.
1287
1288 Streaming protocols
1289 ~~~~~~~~~~~~~~~~~~~
1290
1291 A network (audio) stream usually consists of one streaming source,
1292 the _sender_, and one or more _receivers_ which read data over the
1293 network from the streaming source.
1294
1295 Senders are thus part of para_server while receivers are part of
1296 para_audiod. Moreover, there is the stand-alone tool para_recv which
1297 can be used to manually download a stream, either from para_server
1298 or from a web-based audio streaming service.
1299
1300 The following three streaming protocols are supported by paraslash:
1301
1302 - HTTP. Recommended for public streams that can be played by
1303 any player like mpg123, xmms, itunes, winamp, etc. The HTTP
1304 sender is supported on all operating systems and all platforms.
1305
1306 - DCCP. Recommended for LAN streaming. DCCP is currently
1307 available only for Linux.
1308
1309 - UDP. Recommended for multicast LAN streaming.
1310
1311 See the Appendix on REFERENCE(Network protocols, network protocols)
1312 for brief descriptions of the various protocols relevant for network
1313 audio streaming with paraslash.
1314
1315 It is possible to activate more than one sender simultaneously.
1316 Senders can be controlled at run time and via config file and command
1317 line options.
1318
1319 Note that audio connections are _not_ encrypted. Transport or Internet
1320 layer encryption should be used if encrypted data connections are
1321 needed.
1322
1323 Since DCCP and TCP are both connection-oriented protocols, connection
1324 establishment/teardown and access control are very similar between
1325 these two streaming protocols. UDP is the most lightweight option,
1326 since in contrast to TCP/DCCP it is connectionless. It is also the
1327 only protocol supporting IP multicast.
1328
1329 The HTTP and the DCCP sender listen on a (TCP/DCCP) port waiting for
1330 clients to connect and establish a connection via some protocol-defined
1331 handshake mechanism. Both senders maintain two linked lists each:
1332 The list of all clients which are currently connected, and the list
1333 of access control entries which determines who is allowed to connect.
1334 IP-based access control may be configured through config file and
1335 command line options and via the "allow" and "deny" sender subcommands.
1336
1337 Upon receiving a GET request from the client, the HTTP sender sends
1338 back a status line and a message. The body of this message is the
1339 audio stream. This is common practice and is supported by many popular
1340 clients which can thus be used to play a stream offered by para_server.
1341 For DCCP things are a bit simpler: No messages are exchanged between
1342 the receiver and sender. The client simply connects and the sender
1343 starts to stream.
1344
1345 DCCP is an experimental protocol which offers a number of new features
1346 not available for TCP. Both ends can negotiate these features using
1347 a built-in negotiation mechanism. In contrast to TCP/HTTP, DCCP is
1348 datagram-based (no retransmissions) and thus should not be used over
1349 lossy media (e.g. WiFi networks). One useful feature offered by DCCP
1350 is access to a variety of different congestion-control mechanisms
1351 called CCIDs. Two different CCIDs are available per default on Linux:
1352
1353
1354 - _CCID 2_. A Congestion Control mechanism similar to that
1355 of TCP. The sender maintains a congestion window and halves
1356 this window in response to congestion.
1357
1358
1359 - _CCID-3_. Designed to be fair when competing for bandwidth.
1360 It has lower variation of throughput over time compared with
1361 TCP, which makes it suitable for streaming media.
1362
1363 Unlike the HTTP and DCCP senders, the UDP sender maintains only a
1364 single list, the _target list_. This list describes the set of clients
1365 to which the stream is sent. There is no list for access control and
1366 no "allow" and "deny" commands for the UDP sender. Instead, the "add"
1367 and "delete" commands can be used to modify the target list.
1368
1369 Since both UDP and DCCP offer an unreliable datagram-based transport,
1370 additional measures are necessary to guard against disruptions over
1371 networks that are lossy or which may be subject to interference (as
1372 is for instance the case with WiFi). Paraslash uses FEC (Forward
1373 Error Correction) to guard against packet losses and reordering. The
1374 stream is FEC-encoded before it is sent through the UDP socket and
1375 must be decoded accordingly on the receiver side.
1376
1377 The packet size and the amount of redundancy introduced by FEC can
1378 be configured via the FEC parameters which are dictated by server
1379 and may also be configured through the "sender" command. The FEC
1380 parameters are encoded in the header of each network packet, so no
1381 configuration is necessary on the receiver side. See the section on
1382 REFERENCE(Forward error correction, FEC) below.
1383
1384 Streams with headers and headerless streams
1385 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1386
1387 For OGG/Vorbis, OGG/Speex and wma streams, some of the information
1388 needed to decode the stream is only contained in the audio file
1389 header of the container format but not in each data chunk. Clients
1390 must be able to obtain this information in case streaming starts in
1391 the middle of the file or if para_audiod is started while para_server
1392 is already sending a stream.
1393
1394 This is accomplished in different ways, depending on the streaming
1395 protocol. For connection-oriented streams (HTTP, DCCP) the audio file
1396 header is sent prior to audio file data. This technique however does
1397 not work for the connectionless UDP transport. Hence the audio file
1398 header is periodically being embedded into the UDP audio data stream.
1399 By default, the header is resent after five seconds. The receiver has
1400 to wait until the next header arrives before it can start decoding
1401 the stream.
1402
1403 Examples
1404 ~~~~~~~~
1405
1406 The sender command of para_server allows to (de-)activate senders
1407 and to change the access permissions senders at runtime. The "si"
1408 (server info) command is used to list the streaming options of the
1409 currently running server as well as the various sender access lists.
1410
1411 -> Show client/target/access lists:
1412
1413 para_client si
1414
1415 -> Obtain general help for the sender command:
1416
1417 para_client help sender
1418
1419 -> Get help for a specific sender (contains further examples):
1420
1421 s=http # or dccp or udp
1422 para_client sender $s help
1423
1424 By default para_server activates both the HTTP and th DCCP sender on
1425 startup. This can be changed via command line options or para_server's
1426 config file.
1427
1428 -> List config file options for senders:
1429
1430 para_server -h
1431
1432 All senders share the "on" and "off" commands, so senders may be
1433 activated and deactivated independently of each other.
1434
1435 -> Switch off the http sender:
1436
1437 para_client sender http off
1438
1439 -> Receive a DCCP stream using CCID2 and write the output into a file:
1440
1441 host=foo.org; ccid=2; filename=bar
1442 para_recv --receiver "dccp --host $host --ccid $ccid" > $filename
1443
1444 Note the quotes around the arguments for the dccp receiver. Each
1445 receiver has its own set of command line options and its own command
1446 line parser, so arguments for the dccp receiver must be protected
1447 from being interpreted by para_recv.
1448
1449 -> Start UDP multicast, using the default multicast address:
1450
1451 para_client sender udp add 224.0.1.38
1452
1453 -> Receive FEC-encoded multicast stream and write the output into a file:
1454
1455 filename=foo
1456 para_recv -r udp > $filename
1457
1458 -> Add an UDP unicast for a client to the target list of the UDP sender:
1459
1460 t=client.foo.org
1461 para_client sender udp add $t
1462
1463 -> Receive this (FEC-encoded) unicast stream:
1464
1465 filename=foo
1466 para_recv -r 'udp -i 0.0.0.0' > $filename
1467
1468 -> Create a minimal config for para_audiod for HTTP streams:
1469
1470 c=$HOME/.paraslash/audiod.conf.min; s=server.foo.com
1471 echo receiver \".:http -i $s\" > $c
1472 para_audiod --config $c
1473
1474 -------
1475 Filters
1476 -------
1477
1478 A paraslash filter is a module which transforms an input stream into
1479 an output stream. Filters are included in the para_audiod executable
1480 and in the stand-alone tool para_filter which usually contains the
1481 same modules.
1482
1483 While para_filter reads its input stream from STDIN and writes
1484 the output to STDOUT, the filter modules of para_audiod are always
1485 connected to a receiver which produces the input stream and a writer
1486 which absorbs the output stream.
1487
1488 Some filters depend on a specific library being installed and are
1489 not compiled in if this library was not found at compile time. To
1490 see the list of supported filters, run para_filter and para_audiod
1491 with the --help option. The output looks similar to the following:
1492
1493 Available filters:
1494 compress wav amp fecdec wmadec prebuffer oggdec aacdec mp3dec
1495
1496 Out of these filter modules, a chain of filters can be constructed,
1497 much in the way Unix pipes can be chained, and analogous to the use
1498 of modules in gstreamer: The output of the first filter becomes the
1499 input of the second filter. There is no limitation on the number of
1500 filters and the same filter may occur more than once.
1501
1502 Like receivers, each filter has its own command line options which
1503 must be quoted to protect them from the command line options of
1504 the driving application (para_audiod or para_filter). Example:
1505
1506 para_filter -f 'mp3dec --ignore-crc' -f 'compress --damp 1'
1507
1508 For para_audiod, each audio format has its own set of filters. The
1509 name of the audio format for which the filter should be applied can
1510 be used as the prefix for the filter option. Example:
1511
1512 para_audiod -f 'mp3:prebuffer --duration 300'
1513
1514 The "mp3" prefix above is actually interpreted as a POSIX extended
1515 regular expression. Therefore
1516
1517 para_audiod -f '.:prebuffer --duration 300'
1518
1519 activates the prebuffer filter for all supported audio formats (because
1520 "." matches all audio formats) while
1521
1522 para_audiod -f 'wma|ogg:prebuffer --duration 300'
1523
1524 activates it only for wma and ogg streams.
1525
1526 Decoders
1527 ~~~~~~~~
1528
1529 For each supported audio format there is a corresponding filter
1530 which decodes audio data in this format to 16 bit PCM data which
1531 can be directly sent to the sound device or any other software that
1532 operates on undecoded PCM data (visualizers, equalizers etc.). Such
1533 filters are called _decoders_ in general, and xxxdec is the name of
1534 the paraslash decoder for the audio format xxx. For example, the mp3
1535 decoder filter is called mp3dec.
1536
1537 Note that the output of the decoder is about 10 times larger than
1538 its input. This means that filters that operate on the decoded audio
1539 stream have to deal with much more data than filters that transform
1540 the audio stream before it is fed to the decoder.
1541
1542 Paraslash relies on external libraries for most decoders, so these
1543 libraries must be installed for the decoder to be included in the
1544 para_filter and para_audiod executables. The oggdec filter depends
1545 on the libogg and libvorbis libraries for example.
1546
1547 Forward error correction
1548 ~~~~~~~~~~~~~~~~~~~~~~~~
1549
1550 As already mentioned REFERENCE(Streaming protocols, earlier),
1551 paraslash uses forward error correction (FEC) for the unreliable UDP
1552 and DCCP transports. FEC is a technique which was invented already
1553 in 1960 by Reed and Solomon and which is widely used for the parity
1554 calculations of storage devices (RAID arrays). It is based on the
1555 algebraic concept of finite fields, today called Galois fields, in
1556 honour of the mathematician Galois (1811-1832). The FEC implementation
1557 of paraslash is based on code by Luigi Rizzo.
1558
1559 Although the details require a sound knowledge of the underlying
1560 mathematics, the basic idea is not hard to understand: For positive
1561 integers k and n with k < n it is possible to compute for any k given
1562 data bytes d_1, ..., d_k the corresponding r := n -k parity bytes p_1,
1563 ..., p_r such that all data bytes can be reconstructed from *any*
1564 k bytes of the set
1565
1566 {d_1, ..., d_k, p_1, ..., p_r}.
1567
1568 FEC-encoding for unreliable network transports boils down to slicing
1569 the audio stream into groups of k suitably sized pieces called _slices_
1570 and computing the r corresponding parity slices. This step is performed
1571 in para_server which then sends both the data and the parity slices
1572 over the unreliable network connection. If the client was able
1573 to receive at least k of the n = k + r slices, it can reconstruct
1574 (FEC-decode) the original audio stream.
1575
1576 From these observations it is clear that there are three different
1577 FEC parameters: The slice size, the number of data slices k, and the
1578 total number of slices n. It is crucial to choose the slice size
1579 such that no fragmentation of network packets takes place because
1580 FEC only guards against losses and reordering but fails if slices are
1581 received partially.
1582
1583 FEC decoding in paralash is performed through the fecdec filter which
1584 usually is the first filter (there can be other filters before fecdec
1585 if these do not alter the audio stream).
1586
1587
1588 Volume adjustment (amp and compress)
1589 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1590
1591 The amp and the compress filter both adjust the volume of the audio
1592 stream. These filters operate on uncompressed audio samples. Hence
1593 they are usually placed directly after the decoding filter. Each
1594 sample is multiplied with a scaling factor (>= 1) which makes amp
1595 and compress quite expensive in terms of computing power.
1596
1597 *amp*
1598
1599 The amp filter amplifies the audio stream by a fixed scaling factor
1600 that must be known in advance. For para_audiod this factor is derived
1601 from the amplification field of the audio file's entry in the audio
1602 file table while para_filter uses the value given at the command line.
1603
1604 The optimal scaling factor F for an audio file is the largest real
1605 number F >= 1 such that after multiplication with F all samples still
1606 fit into the sample interval [-32768, 32767]. One can use para_filter
1607 in combination with the sox utility to compute F:
1608
1609 para_filter -f mp3dec -f wav < file.mp3 | sox -t wav - -e stat -v
1610
1611 The amplification value V which is stored in the audio file table,
1612 however, is an integer between 0 and 255 which is connected to F
1613 through the formula
1614
1615 V = (F - 1) * 64.
1616
1617 To store V in the audio file table, the command
1618
1619 para_client -- touch -a=V file.mp3
1620
1621 is used. The reader is encouraged to write a script that performs
1622 these computations :)
1623
1624 *compress*
1625
1626 Unlike the amplification filter, the compress filter adjusts the volume
1627 of the audio stream dynamically without prior knowledge about the peak
1628 value. It maintains the maximal volume of the last n samples of the
1629 audio stream and computes a suitable amplification factor based on that
1630 value and the various configuration options. It tries to chose this
1631 factor such that the adjusted volume meets the desired target level.
1632
1633 Note that it makes sense to combine amp and compress.
1634
1635 Misc filters (wav and prebuffer)
1636 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1637
1638 These filters are rather simple and do not modify the audio stream at
1639 all. The wav filter is only useful with para_filter and in connection
1640 with a decoder. It asks the decoder for the number of channels and the
1641 sample rate of the stream and adds a Microsoft wave header containing
1642 this information at the beginning. This allows to write wav files
1643 rather than raw PCM files (which do not contain any information about
1644 the number of channels and the sample rate).
1645
1646 The prebuffer filter simply delays the output until the given time has
1647 passed (starting from the time the first byte was available in its
1648 input queue) or until the given amount of data has accumulated. It
1649 is mainly useful for para_audiod if the standard parameters result
1650 in buffer underruns.
1651
1652 Both filters require almost no additional computing time, even when
1653 operating on uncompressed audio streams, since data buffers are simply
1654 "pushed down" rather than copied.
1655
1656 Examples
1657 ~~~~~~~~
1658
1659 -> Decode an mp3 file to wav format:
1660
1661 para_filter -f mp3dec -f wav < file.mp3 > file.wav
1662
1663 -> Amplify a raw audio file by a factor of 1.5:
1664
1665 para_filter -f amp --amp 32 < foo.raw > bar.raw
1666
1667 ------
1668 Output
1669 ------
1670
1671 Once an audio stream has been received and decoded to PCM format,
1672 it can be sent to a sound device for playback. This part is performed
1673 by paraslash _writers_ which are described in this chapter.
1674
1675 Writers
1676 ~~~~~~~
1677
1678 A paraslash writer acts as a data sink that consumes but does not
1679 produce audio data. Paraslash writers operate on the client side and
1680 are contained in para_audiod and in the stand-alone tool para_write.
1681
1682 The para_write program reads uncompressed audio data from STDIN. If
1683 this data starts with a wav header, sample rate, sample format and
1684 channel count are read from the header. Otherwise CD audio (44.1KHz
1685 16 bit little endian, stereo) is assumed but this can be overridden
1686 by command line options. para_audiod, on the other hand, obtains
1687 the sample rate and the number of channels from the decoder.
1688
1689 Like receivers and filters, each writer has an individual set of
1690 command line options, and for para_audiod writers can be configured
1691 per audio format separately. It is possible to activate more than
1692 one writer for the same stream simultaneously.
1693
1694 OS-dependent APIs
1695 ~~~~~~~~~~~~~~~~~
1696
1697 Unfortunately, the various flavours of Unix on which paraslash
1698 runs on have different APIs for opening a sound device and starting
1699 playback. Hence for each such API there is a paraslash writer that
1700 can play the audio stream via this API.
1701
1702 *ALSA*. The _Advanced Linux Sound Architecture_ is only available on
1703 Linux systems. Although there are several mid-layer APIs in use by
1704 the various Linux distributions (ESD, Jack, PulseAudio), paraslash
1705 currently supports only the low-level ALSA API which is not supposed
1706 to be change. ALSA is very feature-rich, in particular it supports
1707 software mixing via its DMIX plugin. ALSA is the default writer on
1708 Linux systems.
1709
1710 *OSS*. The _Open Sound System_ is the only API on *BSD Unixes and
1711 is also available on Linux systems, usually provided by ALSA as an
1712 emulation for backwards compatibility. This API is rather simple but
1713 also limited. For example only one application can open the device
1714 at any time. The OSS writer is activated by default on BSD Systems.
1715
1716 *OSX*. Mac OS X has yet another API called CoreAudio. The OSX writer
1717 for this API is only compiled in on such systems and is of course
1718 the default there.
1719
1720 *FILE*. The file writer allows to capture the audio stream and
1721 write the PCM data to a file on the file system rather than playing
1722 it through a sound device. It is supported on all platforms and is
1723 always compiled in.
1724
1725 *AO*. _Libao_ is a cross-platform audio library which supports a wide
1726 variety of platforms including PulseAudio (gnome), ESD (Enlightened
1727 Sound Daemon), AIX, Solaris and IRIX. The ao writer plays audio
1728 through an output plugin of libao.
1729
1730 Examples
1731 ~~~~~~~~
1732
1733 -> Use the OSS writer to play a wav file:
1734
1735 para_write --writer oss < file.wav
1736
1737 -> Enable ALSA software mixing for mp3 streams
1738
1739 para_audiod --writer 'mp3:alsa -d plug:swmix'
1740
1741
1742 ---
1743 Gui
1744 ---
1745
1746 para_gui executes an arbitrary command which is supposed to print
1747 status information to STDOUT. It then displays this information in
1748 a curses window. By default the command
1749
1750 para_audioc -- stat -p
1751
1752 is executed, but this can be customized via the --stat_cmd option. In
1753 particular it possible to use
1754
1755 para_client -- stat -p
1756
1757 to make para_gui work on systems on which para_audiod is not running.
1758
1759 Key bindings
1760 ~~~~~~~~~~~~
1761
1762 It is possible to bind keys to arbitrary commands via custom
1763 key-bindings. Besides the internal keys which can not be changed (help,
1764 quit, loglevel, version...), the following flavours of key-bindings
1765 are supported:
1766
1767 - external: Shutdown curses before launching the given command.
1768 Useful for starting other ncurses programs from within
1769 para_gui, e.g. aumix or dialog scripts. Or, use the mbox
1770 output format to write a mailbox containing one mail for each
1771 (admissible) file the audio file selector knows about. Then
1772 start mutt from within para_gui to browse your collection!
1773
1774 - display: Launch the command and display its stdout in
1775 para_gui's bottom window.
1776
1777 - para: Like display, but start "para_client <specified
1778 command>" instead of "<specified command>".
1779
1780 The general form of a key binding is
1781
1782 key_map k:m:c
1783
1784 which maps key k to command c using mode m. Mode may be x, d or p
1785 for external, display and paraslash commands, respectively.
1786
1787 Themes
1788 ~~~~~~
1789
1790 Currently there are only two themes for para_gui. It is easy, however,
1791 to add more themes. To create a new theme one has to define the
1792 position, color and geometry for for each status item that should be
1793 shown by this theme. See gui_theme.c for examples.
1794
1795 The "." and "," keys are used to switch between themes.
1796
1797 Examples
1798 ~~~~~~~~
1799
1800 -> Show server info:
1801
1802 key_map "i:p:si"
1803
1804 -> Jump to the middle of the current audio file by pressing F5:
1805
1806 key_map "<F5>:p:jmp 50"
1807
1808 -> vi-like bindings for jumping around:
1809
1810 key_map "l:p:ff 10"
1811 key_map "h:p:ff 10-"
1812 key_map "w:p:ff 60"
1813 key_map "b:p:ff 60-"
1814
1815 -> Print the current date and time:
1816
1817 key_map "D:d:date"
1818
1819 -> Call other curses programs:
1820
1821 key_map "U:x:aumix"
1822 key_map "!:x:/bin/bash"
1823 key_map "^E:x:/bin/sh -c 'vi ~/.paraslash/gui.conf'"
1824
1825 -----------
1826 Development
1827 -----------
1828
1829 Tools
1830 ~~~~~
1831
1832 In order to compile the sources from the git repository (rather than
1833 from tar balls) and for contributing non-trivial changes to the
1834 paraslash project, some additional tools should be installed on a
1835 developer machine.
1836
1837 http://git.or.cz/ (git). As described in more detail REFERENCE(Git
1838 branches, below), the git source code management tool is used for
1839 paraslash development. It is necessary for cloning the git repository
1840 and for getting updates.
1841
1842 ftp://ftp.gnu.org/pub/gnu/m4/ (m4). Some input files for gengetopt
1843 are generated from templates by the m4 macro processor.
1844
1845 ftp://ftp.gnu.org/pub/gnu/autoconf/ (autoconf) GNU autoconf creates
1846 the configure file which is shipped in the tarballs but has to be
1847 generated when compiling from git.
1848
1849 http://www.triptico.com/software/grutatxt.html (grutatxt). The
1850 HTML version of this manual and some of the paraslash web pages are
1851 generated by the grutatxt plain text to HTML converter. If changes
1852 are made to these text files the grutatxt package must be installed
1853 to regenerate the HTML files.
1854
1855 http://www.stack.nl/~dimitri/doxygen/ (doxygen). The documentation
1856 of paraslash's C sources uses the doxygen documentation system. The
1857 conventions for documenting the source code is described in the
1858 REFERENCE(Doxygen, Doxygen section).
1859
1860 ftp://ftp.gnu.org/pub/gnu/global (global). This is used to generate
1861 browsable HTML from the C sources. It is needed by doxygen.
1862
1863 Git branches
1864 ~~~~~~~~~~~~
1865
1866 Paraslash has been developed using the git source code management
1867 tool since 2006. Development is organized roughly in the same spirit
1868 as the git development itself, as described below.
1869
1870 The following text passage is based on "A note from the maintainer",
1871 written by Junio C Hamano, the maintainer of git.
1872
1873 There are four branches in the paraslash repository that track the
1874 source tree: "master", "maint", "next", and "pu".
1875
1876 The "master" branch is meant to contain what is well tested and
1877 ready to be used in a production setting. There could occasionally be
1878 minor breakages or brown paper bag bugs but they are not expected to
1879 be anything major, and more importantly quickly and easily fixable.
1880 Every now and then, a "feature release" is cut from the tip of this
1881 branch, named with three dotted decimal digits, like 0.4.2.
1882
1883 Whenever changes are about to be included that will eventually lead to
1884 a new major release (e.g. 0.5.0), a "maint" branch is forked off from
1885 "master" at that point. Obvious, safe and urgent fixes after the major
1886 release are applied to this branch and maintenance releases are cut
1887 from it. New features never go to this branch. This branch is also
1888 merged into "master" to propagate the fixes forward.
1889
1890 A trivial and safe enhancement goes directly on top of "master".
1891 New development does not usually happen on "master", however.
1892 Instead, a separate topic branch is forked from the tip of "master",
1893 and it first is tested in isolation; Usually there are a handful such
1894 topic branches that are running ahead of "master". The tip of these
1895 branches is not published in the public repository to keep the number
1896 of branches that downstream developers need to worry about low.
1897
1898 The quality of topic branches varies widely. Some of them start out as
1899 "good idea but obviously is broken in some areas" and then with some
1900 more work become "more or less done and can now be tested by wider
1901 audience". Luckily, most of them start out in the latter, better shape.
1902
1903 The "next" branch is to merge and test topic branches in the latter
1904 category. In general, this branch always contains the tip of "master".
1905 It might not be quite rock-solid production ready, but is expected to
1906 work more or less without major breakage. The maintainer usually uses
1907 the "next" version of paraslash for his own pleasure, so it cannot
1908 be _that_ broken. The "next" branch is where new and exciting things
1909 take place.
1910
1911 The two branches "master" and "maint" are never rewound, and "next"
1912 usually will not be either (this automatically means the topics that
1913 have been merged into "next" are usually not rebased, and you can find
1914 the tip of topic branches you are interested in from the output of
1915 "git log next"). You should be able to safely build on top of them.
1916
1917 However, at times "next" will be rebuilt from the tip of "master" to
1918 get rid of merge commits that will never be in "master". The commit
1919 that replaces "next" will usually have the identical tree, but it
1920 will have different ancestry from the tip of "master".
1921
1922 The "pu" (proposed updates) branch bundles the remainder of the
1923 topic branches. The "pu" branch, and topic branches that are only in
1924 "pu", are subject to rebasing in general. By the above definition
1925 of how "next" works, you can tell that this branch will contain quite
1926 experimental and obviously broken stuff.
1927
1928 When a topic that was in "pu" proves to be in testable shape, it
1929 graduates to "next". This is done with
1930
1931 git checkout next
1932 git merge that-topic-branch
1933
1934 Sometimes, an idea that looked promising turns out to be not so good
1935 and the topic can be dropped from "pu" in such a case.
1936
1937 A topic that is in "next" is expected to be polished to perfection
1938 before it is merged to "master". Similar to the above, this is
1939 done with
1940
1941 git checkout master
1942 git merge that-topic-branch
1943 git branch -d that-topic-branch
1944
1945 Note that being in "next" is not a guarantee to appear in the next
1946 release (being in "master" is such a guarantee, unless it is later
1947 found seriously broken and reverted), nor even in any future release.
1948
1949 Coding Style
1950 ~~~~~~~~~~~~
1951
1952 The preferred coding style for paraslash coincides more or less
1953 with the style of the Linux kernel. So rather than repeating what is
1954 written XREFERENCE(http://www.kernel.org/doc/Documentation/CodingStyle,
1955 there), here are the most important points.
1956
1957 - Burn the GNU coding standards.
1958 - Never use spaces for indentation.
1959 - Tabs are 8 characters, and thus indentations are also 8 characters.
1960 - Don't put multiple assignments on a single line.
1961 - Avoid tricky expressions.
1962 - Don't leave whitespace at the end of lines.
1963 - The limit on the length of lines is 80 columns.
1964 - Use K&R style for placing braces and spaces:
1965
1966 if (x is true) {
1967 we do y
1968 }
1969
1970 - Use a space after (most) keywords.
1971 - Do not add spaces around (inside) parenthesized expressions.
1972 - Use one space around (on each side of) most binary and ternary operators.
1973 - Do not use cute names like ThisVariableIsATemporaryCounter, call it tmp.
1974 - Mixed-case names are frowned upon.
1975 - Descriptive names for global variables are a must.
1976 - Avoid typedefs.
1977 - Functions should be short and sweet, and do just one thing.
1978 - The number of local variables shouldn't exceed 10.
1979 - Gotos are fine if they improve readability and reduce nesting.
1980 - Don't use C99-style "// ..." comments.
1981 - Names of macros defining constants and labels in enums are capitalized.
1982 - Enums are preferred when defining several related constants.
1983 - Always use the paraslash wrappers for allocating memory.
1984 - If the name of a function is an action or an imperative.
1985 command, the function should return an error-code integer
1986 (<0 means error, >=0 means success). If the name is a
1987 predicate, the function should return a "succeeded" boolean.
1988
1989
1990 Doxygen
1991 ~~~~~~~
1992
1993 Doxygen is a documentation system for various programming
1994 languages. The paraslash project uses Doxygen for generating the API
1995 reference on the web pages, but good source code documentation is
1996 also beneficial to people trying to understand the code structure
1997 and the interactions between the various source files.
1998
1999 It is more illustrative to look at the source code for examples than
2000 to describe the conventions for documenting the source in this manual,
2001 so we only describe which parts of the code need doxygen comments,
2002 but leave out details on documentation conventions.
2003
2004 As a rule, only the public part of the C source is documented with
2005 Doxygen. This includes structures, defines and enumerations in header
2006 files as well as public (non-static) C functions. These should be
2007 documented completely. For example each parameter and the return
2008 value of a public function should get a descriptive comment.
2009
2010 No doxygen comments are necessary for static functions and for
2011 structures and enumerations in C files (which are used only within
2012 this file). This does not mean, however, that those entities need
2013 no documentation at all. Instead, common sense should be applied to
2014 document what is not obvious from reading the code.
2015
2016 --------
2017 Appendix
2018 --------
2019
2020 Network protocols
2021 ~~~~~~~~~~~~~~~~~
2022
2023 *IP*. The _Internet Protocol_ is the primary networking protocol
2024 used for the Internet. All protocols described below use IP as the
2025 underlying layer. Both the prevalent IPv4 and the next-generation
2026 IPv6 variant are being deployed actively worldwide.
2027
2028 *Connection-oriented and connectionless protocols*. Connectionless
2029 protocols differ from connection-oriented ones in that state
2030 associated with the sending/receiving endpoints is treated
2031 implicitly. Connectionless protocols maintain no internal knowledge
2032 about the state of the connection. Hence they are not capable of
2033 reacting to state changes, such as sudden loss or congestion on the
2034 connection medium. Connection-oriented protocols, in contrast, make
2035 this knowledge explicit. The connection is established only after
2036 a bidirectional handshake which requires both endpoints to agree
2037 on the state of the connection, and may also involve negotiating
2038 specific parameters for the particular connection. Maintaining an
2039 up-to-date internal state of the connection also in general means
2040 that the sending endpoints perform congestion control, adapting to
2041 qualitative changes of the connection medium.
2042
2043 *Reliability*. In IP networking, packets can be lost, duplicated,
2044 or delivered out of order, and different network protocols handle
2045 these problems in different ways. We call a transport-layer protocol
2046 _reliable_, if it turns the unreliable IP delivery into an ordered,
2047 duplicate- and loss-free delivery of packets. Sequence numbers
2048 are used to discard duplicates and re-arrange packets delivered
2049 out-of-order. Retransmission is used to guarantee loss-free
2050 delivery. Unreliable protocols, in contrast, do not guarantee ordering
2051 or data integrity.
2052
2053 *Classification*. With these definitions the protocols which are used
2054 by paraslash for steaming audio data may be classified as follows.
2055
2056 - HTTP/TCP: connection-oriented, reliable,
2057 - UDP: connectionless, unreliable,
2058 - DCCP: connection-oriented, unreliable.
2059
2060 Below we give a short descriptions of these protocols.
2061
2062 *TCP*. The _Transmission Control Protocol_ provides reliable,
2063 ordered delivery of a stream and a classic window-based congestion
2064 control. In contrast to UDP and DCCP (see below), TCP does not have
2065 record-oriented or datagram-based syntax, i.e. it provides a stream
2066 which is unaware and independent of any record (packet) boundaries.
2067 TCP is used extensively by many application layers. Besides HTTP (the
2068 Hypertext Transfer Protocol), also FTP (the File Transfer protocol),
2069 SMTP (Simple Mail Transfer Protocol), SSH (Secure Shell) all sit on
2070 top of TCP.
2071
2072 *UDP*. The _User Datagram Protocol_ is the simplest transport-layer
2073 protocol, built as a thin layer directly on top of IP. For this reason,
2074 it offers the same best-effort service as IP itself, i.e. there is no
2075 detection of duplicate or reordered packets. Being a connectionless
2076 protocol, only minimal internal state about the connection is
2077 maintained, which means that there is no protection against packet
2078 loss or network congestion. Error checking and correction (if at all)
2079 are performed in the application.
2080
2081 *DCCP*. The _Datagram Congestion Control Protocol_ combines the
2082 connection-oriented state maintenance known from TCP with the
2083 unreliable, datagram-based transport of UDP. This means that it
2084 is capable of reacting to changes in the connection by performing
2085 congestion control, offering multiple alternative approaches. But it
2086 is bound to datagram boundaries (the maximum packet size supported
2087 by a medium), and like UDP it lacks retransmission to protect
2088 against loss. Due to the use of sequence numbers, it is however
2089 able to react to loss (interpreted as a congestion indication) and
2090 to ignore out-of-order and duplicate packets. Unlike TCP it allows
2091 to negotiate specific, binding features for a connection, such as
2092 the choice of congestion control: classic, window-based congestion
2093 control known from TCP is available as CCID-2, rate-based, "smooth"
2094 congestion control is offered as CCID-3.
2095
2096 *HTTP*. _The Hypertext Transfer Protocol_ is an application layer
2097 protocol on top of TCP. It is spoken by web servers and is most often
2098 used for web services. However, as can be seen by the many Internet
2099 radio stations and YouTube/Flash videos, http is by far not limited to
2100 the delivery of web pages only. Being a simple request/response based
2101 protocol, the semantics of the protocol also allow the delivery of
2102 multimedia content, such as audio over http.
2103
2104 *Multicast*. IP multicast is not really a protocol but a technique
2105 for one-to-many communication over an IP network. The challenge is to
2106 deliver information to a group of destinations simultaneously using
2107 the most efficient strategy to send the messages over each link of
2108 the network only once. This has benefits for streaming multimedia:
2109 the standard one-to-one unicast offered by TCP/DCCP means that
2110 n clients listening to the same stream also consume n-times the
2111 resources, whereas multicast requires to send the stream just once,
2112 irrespective of the number of receivers. Since it would be costly to
2113 maintain state for each listening receiver, multicast often implies
2114 connectionless transport, which is the reason that it is currently
2115 only available via UDP.
2116
2117 License
2118 ~~~~~~~
2119
2120 Paraslash is licensed under the GPL, version 2. Most of the code
2121 base has been written from scratch, and those parts are GPL V2
2122 throughout. Notable exceptions are FEC and the WMA decoder. See the
2123 corresponding source files for licencing details for these parts. Some
2124 code sniplets of several other third party software packages have
2125 been incorporated into the paraslash sources, for example log message
2126 coloring was taken from the git sources. These third party software
2127 packages are all published under the GPL or some other license
2128 compatible to the GPL.
2129
2130 Acknowledgements
2131 ~~~~~~~~~~~~~~~~
2132
2133 Many thanks to Gerrit Renker who read an early draft of this manual
2134 and contributed significant improvements.
2135
2136 ----------
2137 References
2138 ----------
2139
2140 Articles
2141 ~~~~~~~~
2142 - Reed, Irving S.; Solomon, Gustave (1960),
2143 XREFERENCE(http://kom.aau.dk/~heb/kurser/NOTER/KOFA01.PDF,
2144 Polynomial Codes over Certain Finite Fields), Journal of the
2145 Society for Industrial and Applied Mathematics (SIAM) 8 (2):
2146 300-304, doi:10.1137/0108018)
2147
2148 RFCs
2149 ~~~~
2150
2151 - XREFERENCE(http://www.ietf.org/rfc/rfc768.txt, RFC 768) (1980):
2152 User Datagram Protocol
2153 - XREFERENCE(http://www.ietf.org/rfc/rfc791.txt, RFC 791) (1981):
2154 Internet Protocol
2155 - XREFERENCE(http://www.ietf.org/rfc/rfc2437.txt, RFC 2437) (1998):
2156 RSA Cryptography Specifications
2157 - XREFERENCE(http://www.ietf.org/rfc/rfc4340.txt, RFC 4340)
2158 (2006): Datagram Congestion Control Protocol (DCCP)
2159 - XREFERENCE(http://www.ietf.org/rfc/rfc4341.txt, RFC 4341) (2006):
2160 Congestion Control ID 2: TCP-like Congestion Control
2161 - XREFERENCE(http://www.ietf.org/rfc/rfc4342.txt, RFC 4342) (2006):
2162 Congestion Control ID 3: TCP-Friendly Rate Control (TFRC)
2163
2164 Application web pages
2165 ~~~~~~~~~~~~~~~~~~~~~
2166
2167 - XREFERENCE(http://paraslash.systemlinux.org/, paraslash)
2168 - XREFERENCE(http://xmms2.org/wiki/Main_Page, xmms)
2169 - XREFERENCE(http://www.mpg123.de/, mpg123)
2170 - XREFERENCE(http://gstreamer.freedesktop.org/, gstreamer)
2171 - XREFERENCE(http://www.icecast.org/, icecast)
2172 - XREFERENCE(http://beesbuzz.biz/code/audiocompress.php, Audio Compress)
2173
2174 External documentation
2175 ~~~~~~~~~~~~~~~~~~~~~~
2176
2177 - XREFERENCE(http://kernel.org/pub/linux/kernel/people/hpa/raid6.pdf,
2178 H. Peter Anvin: The mathematics of Raid6)
2179 - XREFERENCE(http://info.iet.unipi.it/~luigi/fec_ccr.ps.gz,
2180 Luigi Rizzo: Effective Erasure Codes for reliable Computer
2181 Communication Protocols)
2182
2183 Code
2184 ~~~~
2185 - XREFERENCE(http://info.iet.unipi.it/~luigi/vdm.tar.gz,
2186 Original FEC implementation by Luigi Rizzo)
2187