Add GSU logo.
[gsu.git] / README
1 <<
2 <img src='gsu.png' alt="gsu logo"><h2>The global subcommand utility</h2>
3 >>
4
5 gsu is a small library of bash functions intended to ease the task of
6 writing and documenting large shell scripts with multiple subcommands,
7 each providing different functionality. gsu is known to work on Linux,
8 FreeBSD, NetBSD and MacOS.
9
10 This document describes how to install and use the gsu library.
11
12 Setting up gsu
13 --------------
14 gsu is very easy to install:
15
16 Requirements
17 ~~~~~~~~~~~~
18 gsu is implemented in bash, and thus gsu depends on bash. Bash version
19 3 is required, version 4 is recommended. Besides bash, gsu depends
20 only on programs which are usually installed on any Unix system (awk,
21 grep, sort, ...). Care has been taken to not rely on GNU specific
22 behavior of these programs, so it should work on non GNU systems
23 (MacOS, *BSD) as well. The gui module depends on the dialog utility.
24
25 Download
26 ~~~~~~~~
27 All gsu modules are contained in a git repository. Get a copy with
28
29 git clone git://git.tuebingen.mpg.de/gsu.git
30
31 There is also a http://ilm.eb.local/gitweb/?p=gsu;a=summary (gitweb) page.
32
33 Installation
34 ~~~~~~~~~~~~
35 gsu consists of several independent modules which are all located
36 at the top level directory of the git repository. gsu requires no
37 installation beyond downloading. In particular it is not necessary
38 to make the downloaded files executable. The library modules can
39 be sourced directly, simply tell your application where to find
40 it. The examples of this document assume that gsu is installed in
41 `/usr/local/lib/gsu' but this is not mandatory.`~/.gsu' is another
42 reasonable choice.
43
44 Conventions
45 -----------
46 Public and private functions and variables
47 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
48 Although there is no way in bash to annotate symbols (functions
49 and variables) as private or public, gsu distinguishes between the
50 two. The `gsu_*' name space is reserved for public symbols while all
51 private symbols start with `_gsu'.
52
53 Private symbols are meant for internal use only. Applications should
54 never use them directly because name and semantics might change
55 between gsu versions.
56
57 The public symbols, on the other hand, define the gsu API. This API
58 must not change in incompatible ways that would break existing
59 applications.
60
61 $ret and $result
62 ~~~~~~~~~~~~~~~~
63 All public gsu functions set the $ret variable to an integer value to
64 indicate success or failure. As a convention, $ret < 0 means failure
65 while a non-negative value indicates success.
66
67 The $result variable contains either the result of a function (if any)
68 or further information in the error case. A negative value of $ret is
69 in fact an error code similar to the errno variable used in C programs.
70 It can be turned into a string that describes the error. The public
71 gsu_err_msg() function can be used to pretty-print a suitable error
72 message provided $ret and $result are set appropriately.
73
74 The subcommand module
75 ---------------------
76 This gsu module provides helper functions to ease the repetitious task
77 of writing applications which operate in several related modes, where
78 each mode of operation corresponds to a subcommand of the application.
79
80 With gsu, for each subcommand one must only write a _command handler_
81 which is simply a function that implements the subcommand. All
82 processing is done by the gsu library. Functions starting with the
83 string `com_' are automatically recognized as subcommand handlers.
84
85 The startup part of the script has to source the subcommand file of
86 gsu and must then call
87
88 gsu "$@"
89
90 Minimal example:
91
92 #!/bin/bash
93 com_world()
94 {
95 echo 'hello world'
96 }
97 . /usr/local/lib/gsu/subcommand || exit 1
98 gsu "$@"
99
100 Save this code in a file called `hello' (adjusting the installation
101 directory if necessary), make it executable (`chmod +x hello') and try
102
103 ./hello
104 ./hello world
105 ./hello invalid
106
107 Here, we have created a bash script ("hello") that has a single "mode"
108 of operation, specified by the subcommand "world".
109
110 gsu automatically generates several reserved subcommands, which should
111 not be specified: `help, man, prefs, complete'.
112
113 Command handler structure
114 ~~~~~~~~~~~~~~~~~~~~~~~~~
115 For the automatically generated help and man subcommands to work
116 properly, all subcommand handlers must be documented. In order to be
117 recognized as subcommand help text, comments must be prefixed with
118 two `#' characters and the subcommand documentation must be located
119 between the function "declaration", com_world() in the example above,
120 and the opening brace that starts the function body.
121
122 Example:
123
124 com_world()
125 ##
126 ##
127 ##
128 {
129 echo 'hello world'
130 }
131
132 The subcommand documentation consists of three parts:
133
134 - The summary. One line of text,
135 - the usage/synopsis string,
136 - free text section.
137
138 The three parts should be separated by lines consisting of two # characters
139 only. Example:
140
141 com_world()
142 ##
143 ## Print the string "hello world" to stdout.
144 ##
145 ## Usage: world
146 ##
147 ## Any arguments to this function are ignored.
148 ##
149 ## Warning: This subcommand may cause the top most line of your terminal to
150 ## disappear and may cause DATA LOSS in your scrollback buffer. Use with
151 ## caution.
152 {
153 echo 'hello world'
154 }
155
156 Replace 'hello' with the above and try:
157
158 ./hello help
159 ./hello help world
160 ./hello help invalid
161 ./hello man
162
163 to check the automatically generated help and man subcommands.
164
165 Error codes
166 ~~~~~~~~~~~
167 As mentioned above, all public functions of gsu return an error code
168 in the $ret variable. A negative value indicates failure, and in this
169 case $result contains more information about the error. The same
170 convention applies for subcommand handlers: gsu will automatically
171 print an error message to stderr if a subcommand handler returns with
172 $ret set to a negative value.
173
174 To allow for error codes defined by the application, the $gsu_errors
175 variable must be set before calling gsu(). Each non-empty line in this
176 variable should contain an identifier and error string. Identifiers
177 are written in upper case and start with `E_'. For convenience the
178 $GSU_SUCCESS variable is defined to non-negative value. Subcommand
179 handlers should set $ret to $GSU_SUCCESS on successful return.
180
181 To illustrate the $gsu_errors variable, assume the task is to
182 print all mount points which correspond to an ext3 file system in
183 `/etc/fstab'. We'd like to catch two possible errors: (a) the file
184 does not exist or is not readable, and (b) it contains no ext3 entry.
185 A possible implementation of the ext3 subcommand could look like this
186 (documentation omitted):
187
188 #!/bin/bash
189
190 gsu_errors='
191 E_NOENT No such file or directory
192 E_NOEXT3 No ext3 file system detected
193 '
194
195 com_ext3()
196 {
197 local f='/etc/fstab'
198 local ext3_lines
199
200 if [[ ! -r "$f" ]]; then
201 ret=-$E_NOENT
202 result="$f"
203 return
204 fi
205 ext3_lines=$(awk '{if ($3 == "ext3") print $2}' "$f")
206 if [[ -z "$ext3_lines" ]]; then
207 ret=-$E_NOEXT3
208 result="$f"
209 return
210 fi
211 printf 'ext3 mount points:\n%s\n' "$ext3_lines"
212 ret=$GSU_SUCCESS
213 }
214
215 Printing diagnostic output
216 ~~~~~~~~~~~~~~~~~~~~~~~~~~
217 gsu provides a couple of convenience functions for output. All
218 functions write to stderr.
219
220 - *gsu_msg()*. Writes the name of the application and the given text.
221
222 - *gsu_short_msg()*. Like gsu_msg(), but does not print the application name.
223
224 - *gsu_date_msg()*. Writes application name, date, and the given text.
225
226 - *gsu_err_msg()*. Prints an error message according to $ret and $result.
227
228 Subcommands with options
229 ~~~~~~~~~~~~~~~~~~~~~~~~
230 Bash's getopts builtin provides a way to define and parse command line
231 options, but it is cumbersome to use because one must loop over all
232 given arguments and check the OPTIND and OPTARG variables during each
233 iteration. The gsu_getopts() function makes this repetitive task easier.
234
235 gsu_getopts() takes a single argument: the optstring which contains
236 the option characters to be recognized. As usual, if a character is
237 followed by a colon, the option is expected to have an argument. On
238 return $result contains bash code that should be eval'ed to parse the
239 position parameters $1, $2, ... of the subcommand according to the
240 optstring.
241
242 The shell code returned by gsu_getopts() creates a local variable $o_x
243 for each defined option `x'. It contains `true/false' for options
244 without argument and either the empty string or the given argument for
245 options that take an argument.
246
247 To illustrate gsu_getopts(), assume the above com_ext3() subcommand
248 handler is to be extended to allow for arbitrary file systems, and
249 that it should print either only the mount point as before or the
250 full line of `/etc/fstab', depending on whether the verbose switch
251 `-v' was given at the command line.
252
253 Hence our new subcommand handler must recognize two options: `-t' for
254 the file system type and `-v'. Note that `-t' takes an argument but `-v'
255 does not. Hence we shall use the optstring `t:v' as the argument for
256 gsu_getopts() as follows:
257
258 com_fs()
259 {
260 local f='/etc/fstab'
261 local fstype fstab_lines
262 local -i awk_field=2
263
264 gsu_getopts 't:v'
265 eval "$result"
266 (($ret < 0)) && return
267
268 [[ -z "$o_t" ]] && o_t='ext3' # default to ext3 if -t is not given
269 [[ "$o_v" == 'true' ]] && awk_field=0 # $0 is the whole line
270 fstab_lines=$(awk -v fstype="$o_t" -v n="$awk_field" \
271 '{if ($3 == fstype) print $n}' "$f")
272 printf '%s entries:\n%s\n' "$o_t" "$fstab_lines"
273 ret=$GSU_SUCCESS
274 }
275
276 Another repetitive task is to check the number of non-option arguments
277 and to report an error if this number turns out to be invalid for
278 the subcommand in question. The gsu_check_arg_count() function performs
279 this check and sets $ret and $result as appropriate. This function
280 takes three arguments: the actual argument count and the minimal and
281 maximal number of non-option arguments allowed. The last argument may
282 be omitted in which case any number of arguments is considered valid.
283
284 Our com_world() subcommand handler above ignored any given
285 arguments. Let's assume we'd like to handle this case and
286 print an error message if one or more arguments are given. With
287 gsu_check_arg_count() this can be achieved as follows:
288
289 com_world()
290 {
291 gsu_check_arg_count $# 0 0 # no arguments allowed
292 (($ret < 0)) && return
293 echo 'hello world'
294 }
295
296 Global documentation
297 ~~~~~~~~~~~~~~~~~~~~
298 Besides the documentation for subcommands, one might also want to
299 include an overall description of the application which provides
300 general information that is not related to any particular subcommand.
301
302 If such a description is included at the top of the script, the
303 automatically generated man subcommand will print it. gsu recognizes
304 the description only if it is enclosed by two lines consisting of at
305 least 70 # characters.
306
307 Example:
308
309 #/bin/bash
310
311 #######################################################################
312 # gsu-based hello - a cumbersome way to write a hello world program
313 # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
314 # It not only requires one to download and install some totally weird
315 # git repo, it also takes about 50 lines of specially written code
316 # to perform what a simple echo 'hello world' would do equally well.
317 #######################################################################
318
319 HTML output
320 ~~~~~~~~~~~
321 The output of the auto-generated man subcommand is a suitable input for the
322 grutatxt plain text to html converter. Hence
323
324 ./hello man | grutatxt > index.html
325
326 is all it takes to produce an html page for your application.
327
328 Interactive completion
329 ~~~~~~~~~~~~~~~~~~~~~~
330 The auto-generated `complete' subcommand provides interactive bash
331 completion. To activate completion for the hello program, it is
332 enough to put the following into your `~/.bashrc':
333
334 _hello()
335 {
336 eval $(hello complete 2>/dev/null)
337 }
338 complete -F _hello hello
339
340 This will give you completion for the first argument of the hello
341 program: the subcommand.
342
343 In order to get subcommand-sensitive completion you must provide a
344 _completer_ in your application for each subcommand that is to support
345 completion. Like subcommand handlers, completers are recognized by name:
346 If a function xxx_complete() is defined, gsu will call it on the
347 attempt to complete the `xxx' subcommand at the subcommand line. gsu
348 has a few functions to aid you in writing a completer.
349
350 Let's have a look at the completer for the above `fs' subcommand.
351
352 complete_fs()
353 {
354 local f='/etc/fstab'
355 local optstring='t:v'
356
357 gsu_complete_options $optstring "$@"
358 (($ret > 0)) && return
359
360 gsu_cword_is_option_parameter $optstring "$@"
361 [[ "$result" == 't' ]] && awk '{print $3}' "$f"
362 }
363
364 Completers are always called with $1 set to the index into the array
365 of words in the current command line when tab completion was attempted
366 (see `COMP_CWORD' in the bash manual). These words are passed to the
367 completer as $2, $3,...
368
369 gsu_complete_options() receives the option string as $1, the word
370 index as $2 and the individual words as $3, $4,... Hence we may simply
371 pass the $optstring and `"$@"'. gsu_complete_options() checks if the
372 current word begins with `-', i.e., whether an attempt to complete
373 an option was performed. If yes gsu_complete_options() prints all
374 possible command line options and sets $ret to a positive value.
375
376 The last two lines of complete_fs() check whether the word preceding
377 the current word is an option that takes an argument. If it is,
378 that option is returned in $result, otherwise $result is the empty
379 string. Hence, if we are completing the argument to `-t', the awk
380 command is executed to print all file system types of /etc/fstab as
381 the possible completions.
382
383 See the comments to gsu_complete_options(),
384 gsu_cword_is_option_parameter() and gsu_get_unnamed_arg_num()
385 (which was not covered here) in the `subcommand' file for a more
386 detailed description.
387
388 The gui module
389 --------------
390 This module can be employed to create interactive dialog boxes from a
391 bash script. It depends on the dialog(1) utility which is available on
392 all Unix systems. On Debian and Ubuntu Linux it can be installed with
393
394 apt-get install dialog
395
396 The core of the gui module is the gsu_gui() function which receives
397 a _menu tree_ as its single argument. The menu tree defines a tree
398 of menus for the user to navigate with the cursor keys. As for a
399 file system tree, internal tree nodes represent folders. Leaf nodes,
400 on the other hand, correspond to _actions_. Pressing enter activates a
401 node. On activation, for internal nodes a new menu with the contents of
402 the subfolder is shown. For leaf nodes the associated _action handler_
403 is executed.
404
405 Hence the application has to provide a menu tree and an action handler
406 for each leaf node defined in the tree. The action handler is simply a
407 function which is named according to the node. In most cases the action
408 handler will run dialog(1) to show some dialog box on its own. Wrappers
409 for some widgets of dialog are provided by the gui module, see below.
410
411 Menu trees
412 ~~~~~~~~~~
413 The concept of a menu tree is best illustrated by an example. Assume
414 we'd like to write a system utility for the not-so-commandline-affine
415 Linux sysadmin next door. For the implementation we confine ourselves
416 with giving some insight in the system by running lean system commands
417 like `df' to show the list of file system, or `dmesg' to print the
418 contents of the kernel log buffer. Bash code which defines the menu
419 tree could look like this:
420
421 menu_tree='
422 load_average
423 processes
424 hardware/
425 cpu
426 scsi
427 storage/
428 df
429 mdstat
430 log/
431 syslog
432 dmesg
433 '
434
435 In this tree, `hardware/', `block_devices/' and `log/' are the only
436 internal nodes. Note that these are written with a trailing slash
437 character while the leaf nodes have no slash at the end. All entries
438 of the menu tree must be indented by tab characters.
439
440 Action handlers
441 ~~~~~~~~~~~~~~~
442 Action handlers are best explained via example:
443
444 Our application, let's call it `lsi' for _lean system information_,
445 must provide action handlers for all leaf nodes. Here is the action
446 handler for the `df' node:
447
448 lsi_df()
449 {
450 gsu_msgbox "$(df -h)"
451 }
452
453 The function name `lsi_df' is derived from the name of the script
454 (`lsi') and the name of the leaf node (`df'). The function simply
455 passes the output of the `df(1)' command as the first argument to
456 the public gsu function gsu_msgbox() which runs dialog(1) to display
457 a message box that shows the given text.
458
459 gsu_msgbox() is suitable for small amounts of output. For essentially
460 unbounded output like log files that can be arbitrary large, it is
461 better to use gsu_textbox() instead which takes a path to the file
462 that contains the text to show.
463
464 To illustrate gsu_input_box() function, assume the action handler
465 for the `processes' leaf node should ask for a username, and display
466 all processes owned by the given user. This could be implemented
467 as follows.
468
469 lsi_processes()
470 {
471 local username
472
473 gsu_inputbox 'Enter username' "$LOGNAME"
474 (($ret != 0)) && return
475 username="$result"
476 gsu_msgbox "$(pgrep -lu "$username")"
477 }
478
479 Once all other action handlers have been defined, the only thing left
480 to do is to source the gsu gui module and to call gsu_gui():
481
482 . /usr/local/lib/gsu/gui || exit 1
483 gsu_gui "$menu_tree"
484
485 Example
486 ~~~~~~~
487 The complete lsi script below can be used as a starting point
488 for your own gsu gui application. If you cut and paste it, be
489 sure to not turn tab characters into space characters.
490
491 #!/bin/bash
492
493 menu_tree='
494 load_average
495 processes
496 hardware/
497 cpu
498 scsi
499 storage/
500 df
501 mdstat
502 log/
503 syslog
504 dmesg
505 '
506
507 lsi_load_average()
508 {
509 gsu_msgbox "$(cat /proc/loadavg)"
510 }
511
512 lsi_processes()
513 {
514 local username
515
516 gsu_inputbox 'Enter username' "$LOGNAME"
517 (($ret < 0)) && return
518 username="$result"
519 gsu_msgbox "$(pgrep -lu "$username")"
520 }
521
522 lsi_cpu()
523 {
524 gsu_msgbox "$(lscpu)"
525 }
526
527 lsi_scsi()
528 {
529 gsu_msgbox "$(lsscsi)"
530 }
531
532 lsi_df()
533 {
534 gsu_msgbox "$(df -h)"
535 }
536
537 lsi_mdstat()
538 {
539 gsu_msgbox "$(cat /proc/mdstat)"
540 }
541
542 lsi_dmesg()
543 {
544 local tmp="$(mktemp)" || exit 1
545
546 trap "rm -f $tmp" EXIT
547 dmesg > $tmp
548 gsu_textbox "$tmp"
549 }
550
551 lsi_syslog()
552 {
553 gsu_textbox '/var/log/syslog'
554 }
555
556 . /usr/local/lib/gsu/gui || exit 1
557 gsu_gui "$menu_tree"
558
559 The config module
560 -----------------
561 Some applications need config options which are not related to
562 any particular subcommand, like the URL of a web service, the path
563 to some data directory, or a default value which is to be used by
564 several subcommands. Such options do not change frequently and are
565 hence better stored in a configuration file rather than passed to
566 every subcommand that needs the information.
567
568 The config module of gsu makes it easy to maintain such options and
569 performs routine tasks like reading and checking the values given in
570 the config file, or printing out the current configuration. It can
571 be used stand-alone, or in combination with either the subcommand or
572 the gui module.
573
574 Defining config options
575 ~~~~~~~~~~~~~~~~~~~~~~~
576 To use the config module, you must define the $gsu_options bash array.
577 Each config option is represented by one slot in this array. Here is
578 an example which defines two options:
579
580 gsu_options=(
581 "
582 name=fs_type
583 option_type=string
584 default_value=ext3
585 required=false
586 description='file system type to consider'
587 help_text='
588 This option is used in various contexts. All
589 subcommands which need a file system type
590 use the value specified here as the default.
591 '
592 "
593 "
594 name=limit
595 option_type=num
596 default_value=3
597 required=no
598 description='print at most this many lines of output'
599 "
600 )
601
602 Each config option consists of the following fields:
603
604 - *name*. This must be a valid bash variable name. Hence no special
605 characters are allowed.
606
607 - *option_type*. Only `string' and `num' are supported but additional
608 types might be supported in future versions. While string variables
609 may have arbitrary content, only integers are accepted for variables
610 of type `num'.
611
612 - *default_value*. The value to use if the option was not specified.
613
614 - *required*. Whether gsu considers it an error if the option was
615 not specified. It does not make sense to set this to `true' and set
616 *default_value* at the same time.
617
618 - *description*. Short description of the variable. It is printed by
619 the `prefs' subcommand.
620
621 - *help_text*. Optional long description, also printed by `prefs'.
622
623 To enable the config module you must source the config module of gsu
624 after $gsu_options has been defined:
625
626 . /usr/local/lib/gsu/config || exit 1
627
628 Passing config options to the application
629 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
630 There are two ways to pass the value of an option to a gsu application:
631 environment variable and config file. The default config file is
632 ~/.$gsu_name.rc where $gsu_name is the basename of the application,
633 but this can be changed by setting $gsu_config_file. Thus, the
634 following two statements are equivalent
635
636 fs_type=xfs hello fs
637 echo 'fs_type=xfs' > ~/.hello.rc && hello fs
638
639 If an option is set both in the environment and in the config file,
640 the environment takes precedence.
641
642 Checking config options
643 ~~~~~~~~~~~~~~~~~~~~~~~
644 The gsu config module defines two public functions for this purpose:
645 gsu_check_options() and gsu_check_options_or_die(). The latter function
646 exits on errors while the former function only sets $ret and $result
647 as appropriate and lets the application deal with the error. The best
648 place to call one of these functions is after sourcing the config
649 module but before calling gsu() or gsu_gui().
650
651 Using config values
652 ~~~~~~~~~~~~~~~~~~~
653 The name of an option as specified in $gsu_options (`fs_type' in
654 the example above) is what users of your application may specify at
655 the command line or in the config file. This leads to a mistake that
656 is easy to make and difficult to debug: The application might use a
657 variable name which is also a config option.
658
659 To reduce the chance for this to happen, gsu_check_options() creates
660 a different set of variables for the application where each variable
661 is prefixed with ${gsu_name}. For example, if $gsu_options as above
662 is part of the hello script, $hello_fs_type and $hello_limit are
663 defined after gsu_check_options() returned successfully. Only the
664 prefixed variants are guaranteed to contain the proper value, so this
665 variable should be used exclusively in the application. The
666 prefix may be changed by setting $gsu_config_var_prefix before calling
667 gsu_check_options().
668
669 com_prefs()
670 ~~~~~~~~~~~
671 For scripts which source both the subcommand and the config module, the
672 auto-generated 'prefs' subcommand prints out the current configuration
673 and exits. The description and help text of the option as specified
674 in the `description' and `help_text' fields of $gsu_options are shown
675 as comments in the output. Hence this output can be used as a template
676 for the config file.
677
678 List of public variables
679 ------------------------
680 - *$gsu_dir*. Where gsu is installed. If unset, gsu guesses
681 its installation directory by examining the $BASH_SOURCE array.
682
683 - *$gsu_name*. The name of the application. Defaults to $0 with
684 all leading directories removed.
685
686 - *$gsu_banner_txt*. Used by both the subcommand and the gui
687 module. It is printed by the man subcommand, and as the title for
688 dialog windows.
689
690 - *$gsu_errors*. Identifier/text pairs for custom error reporting.
691
692 - *$gsu_config_file*. The name of the config file of the application.
693 Defaults to `~/.${gsu_name}.rc'.
694
695 - *$gsu_options*.
696
697 - *$gsu_config_var_prefix*. Used by the config module to set up
698 the variables defined in $gsu_options.
699
700 License
701 -------
702 gsu is licensed under the GNU LESSER GENERAL PUBLIC LICENSE (LGPL), version 3.
703 See COPYING and COPYING.LESSER.
704
705 Contact
706 -------
707 Send beer, pizza, patches, improvements, bug reports, flames,
708 (in this order), to Andre Noll `<maan@tuebingen.mpg.de>'.
709
710 References
711 ----------
712 - http://www.gnu.org/software/bash/bash.html (bash)
713 - http://www.invisible-island.net/dialog/dialog.html (dialog)
714 - http://triptico.com/software/grutatxt.html (grutatxt)