In handle_connect() we really need a blocking fd for the client.