server: Do not fail if client does not request sideband.