fd: Simplify and move for_each_file_in_dir(). With only one user it can be static in aft.c. Modify the function so that it no longer changes the current working directory, remove para_opendir() because it is unused now, dedox the documentation and streamline it a bit.
fd.c: Improve error checking of para_mkdir(). The old code returned success in case the pathname existed but was no directory, so try to improve on this a bit. However, don't be over-zealous as any pathname based approach won't be bullet-proof because the file identified by the pathname may change at any time.
fd: Revamp para_mkdir(). It has two callers which both pass the mode value 0777 and contain extra code to regard the EEXIST error case as a success. Move the common bits into the wrapper and improve the documentation.
fd: Improve read_pattern(), rename it to read_and_compare(). The old name was a poor choice because the pattern argument actually is neither a regular expression nor a filename pattern. More importantly, the function receives a buffer size and tries to read this many bytes but then compares only the first part of the received buffer to the expected string. This is a rather weird calling convention. The only two callers are the http sender and receiver which both call the function during the initial handshake where no other data is available. Thus we can change the function to read only the minimal amount of data (length of the expected string), and drop the bufsize parameter. Remove the unnecessary log message in the error case and streamline the documentation while at it.
fd: Remove log message from para_munmap(). Low-level functions like this should leave it to the caller to log the error. Extend the documentation a bit while at it to document the fact that passing NULL is OK.
fd: Open-code para_chdir(). Another public trivial wrapper that can go away because it has only a single caller. POSIX says Upon successful completion, 0 shall be returned. Otherwise, −1 shall be returned, the current working directory shall remain unchanged, and errno shall be set to indicate the error. So the new check against zero is equivalent to the old code which checked whether the return value is non-negative.
fd: Remove file_exists(). Open-coding this function actually improves code readability. The function name was a misnomer anyway because any error from the stat() call (such as EACCES) was reported as "file does not exist".
fd: Improve documentation of xwritev(). This is another essential helper which was poorly documented. In particular the EINTR and short write cases were not covered, and it was left open what a return value of zero means. Omit the DCCP-specific part and the sentence which explains what EAGAIN means.
fd: Improve documentation of write_all(). The name of this public function is a bit of a misnomer, so be at least clear in the one-line description what the function does in the EAGAIN case or if a short write occurs.
fd: Improve documentation of write_va_buffer(). This public function was a bit underdocumented, so explain in more detail what the function does, and how it differs from fprintf(3).
Merge topic branch t/overflow into master This series implements a new memory allocation API which checks for overflows. The first part of the series just renames the main allocation functions. Later patches in the series implement allocators which take two size_t arguments (like calloc(3)) and check whether the multiplication overflows by employing the __builtin_mul_overflow() primitive supported by gcc and clang. This requires us to bump the lowest supported gcc and clang version. * refs/heads/t/overflow: build: Compile with -ftrapv. string: Introduce arr_zalloc(). string: Introduce arr_alloc(). string: Introduce arr_realloc() and check for integer overflow. string: Rename para_calloc() -> zalloc(). string: Rename para_malloc() -> alloc(). string: Overhaul para_strdup().
Switch from select(2) to poll(2). The select(2) API is kind of obsolete because it does not work for file descriptors greater or equal than 1024, The general advice is to switch to poll(2), which offers equivalent functionality and does not suffer from this restriction. This patch implements this switch. The fd sets of select(2) have one nice feature: One can determine in O(1) time whether the bit for a given fd is turned on in an fd set. For poll(2), the monitored file descriptors are organized in an array of struct pollfd. Without information about the given fd's index in the pollfd array, one can only perform a linear search which requires O(n) time, with n being the number of fds being watched. Since this would have to be done for each fd, the running time becomes quadratic in the number of monitored fds, which is bad. Keeping the pollfd array sorted would reduce that to n * log(n) at the cost of additional work at insert time. This patch implements a different approach. The scheduler now maintains an additional array of unsigned integers which map fds to indices into the pollfd array. This new index array is transparent to the individual tasks, which still simply pass one or more fds from their ->pre_monitor() method to the scheduler. The length of the index array equals the highest fd given. This might become prohibitive in theory, but should not be an issue for the time being. Care needs to be taken in order to deal with callers which ask for the readiness of an fd without having called sched_monitor_readfd() or sched_monitor_writefd() in the ->pre_monitor() step. Before the patch, thanks to the FD_ZERO() call at the beginning of each iteration of the scheduler's main loop, both sched_read_ok() and sched_write_ok() returned false for fds which were not asked to be watched. We need to keep it this way for a seamless transition. We achieve this by replacing the FD_ZERO() call by a memset(3) call which fills the index array with 0xff bytes. Both sched_read_ok() and sched_write_ok() call the new get_revents() helper, where we check the fd argument against the allocation sizes of the two arrays. If either function is called with an fd that was not asked to be monitored in the ->pre_monitor() step, the checks notice that the index of this fd, 0xffffffff, is larger than the highest open fd and we return "not ready for I/O". Another issue is the case where the same file descriptor is submitted twice in ->pre_monitor() to check for readiness with respect to both reading and writing. The code in client_comon.c currently does that. To keep it working, the scheduler needs to detect this case and re-use the existing slot in both arrays.
Hide implementation of para_fd_set(). This preparatory patch for replacing select() renames para_fd_set() to sched_fd_set(), moves it to sched.c and makes it static. All users are modified to call either of the two new public functions sched_monitor_{read,write}fd() which take a pointer to struct sched rather than an fd set pointer.
fd: Drop fd_set parameter from read_nonblock() and friends. This parameter is not necessary because its only purpose is to avoid the readv(2) system call in case it would likely return EAGAIN because we just called select(2) which reported that there is no data to read. Since the parameter is an obstacle for the conversion of the code base from select(2) to poll(2), get rid of it for the time being. If needed we can add back an equivalent optimization which checks for POLLIN after the conversion.
interactive: Avoid select(2) in input_available(). In analogy to write_ok(), introduce read_ok() which uses poll(2) rather than select(2). To avoid duplications, abstract out the common code to the new xpoll() helper. We could avoid the timeout parameter of xpoll() at this point because both callers call it with a zero timeout (causing poll() to return immediately), but later patches introduce other callers which specify non-zero timeouts.
fd.c: Prefer poll(2) over select(2) for write_ok(). This is easy to do and avoids the old and well-known shortcomings of select(2). See http://0pointer.net/blog/file-descriptor-limits.html for a short discussion, or the references in the log message of commit e4a403876d2c of the man-pages repository. The linux poll manpage says: On some other UNIX systems, poll() can fail with the error EAGAIN if the system fails to allocate kernel-internal resources, rather than ENOMEM as Linux does. POSIX permits this behavior. Portable programs may wish to check for EAGAIN and loop, just as with EINTR. We do not follow this approach since failing the call in the out of memory case seems to be the right thing to do while busy looping without trying to free memory between the calls is not likely to help. Also, looping on EAGAIN would be inconsistent since in the OOM case the code would fail on Linux but loop on those other UNIX systems. To be consistent, one must check for both EAGAIN and ENOMEM.
sched: Use integer value for select timeout. This modifies the public struct sched so that users pass in the default timeout as an integer value in milliseconds rather than a struct timeval. This simplifies the code a little and eases the transition from select(2) to poll(2) because poll(2) also takes a plain integer for the timeout. Since para_select() of fd.c now calls ms2tv() to convert the timeout back to a struct timeval, all executables which link with fd.o must also link with time.o. This was not the case for para_mixer and para_audioc, so configure.ac needs to be adjusted accordingly.
string: Rename para_malloc() -> alloc(). Just because it's shorter and matches the naming of the new allocators we are about to introduce. The bulk of this patch was created with sed -i 's/para_malloc/alloc/g' *.c *.h yy/mp.y
Merge branch 'refs/heads/t/clean_server_exit' This series removes many memory leaks of para_server by refactoring the shutdown and signal handling code. Most of the leaks happen only at shutdown and are hence harmless. But it is still good to plug the leaks because this puts more focus on real memory leaks in the valgrind output. The merge conflicted rather badly due to the changes introduced with the crypt branch that was merged last week. The resolution has been thoroughly tested, though. * refs/heads/t/clean_server_exit: (32 commits) command.c: Document return value of handle_connect(). user_list: Make list head static. afs: Allow database switching on sighup. afs: Free current mood or playlist on exit. afs: Free status items on exit. afs: Shutdown signals on exit. server: Free parse result also in afs. afs: Deplete user list at startup. server: Free audio file header on exit. sender: Deplete ACLs on exit. Remove some unused includes from {dccp,http}_send.c. server: Make argument of user_list_init() constant. server: Deplete user list on exit. server: Combine user_list_init() and populate(). server: Move para_fgets() to user_list.c. server: Initialize user list at compile time. server: Rename functions related to user lists. server: Constify return value of lookup_user(). server: Let stat command handler perform cleanup on signals. server: Have afs process close the current mood on exit(). ...
fd: Let readv_nonblock() recover from EINTR. No need to fail the operation if the read was interrupted by a signal. The patch also fixes some minor issues in the documentation: a typo ("The" was incorrectly capitalized) and a missing \ref for xwrite(). We use the opportunity to get rid of the \a and \p font annotations. This improves the readability of the source code, which is more imporatant than nice looking web pages. However, we only touch those parts of the documentation which are modified anyway.