Return from filter_setup() so callers can reset the terminal.

Return from filter_setup() so callers can reset the terminal.

This function of filter_common.c calls exit(3) on errors. This is OK
for para_filter and para_audiod, but not for para_play because there
the function is called after readline has initialized the terminal for
its own use. If the function calls exit(3), the terminal settings are
not reset as they should have been. This was observed on the attempt
to open an mp3 file with an executable that lacked mp3 support because
the libmad package was missing.

This commit changes the function to return an error code instead and
deals with the fallout in the three callers. Although play.c already
had error checking for calls to filter_setup(), it still needs a minor
tweak because we now have to deal with the fact that the filter and
writer node don't exist in eof_cleanup().