This addresses CVE-2014-3636.

Based on a patch by Alban Crequy. Now that it's the same on all
platforms, there's little point in it being set by configure/cmake.

This change fixes two distinct denials of service:

fd.o#82820, part A
------------------

Before this patch, the system bus had the following default configuration:
- max_connections_per_user: 256
- DBUS_DEFAULT_MESSAGE_UNIX_FDS: usually 1024 (or 256 on QNX, see fd.o#61176)
  as defined by configure.ac
- max_incoming_unix_fds: DBUS_DEFAULT_MESSAGE_UNIX_FDS*4 = usually 4096
- max_outgoing_unix_fds: DBUS_DEFAULT_MESSAGE_UNIX_FDS*4 = usually 4096
- max_message_unix_fds: DBUS_DEFAULT_MESSAGE_UNIX_FDS = usually 1024

This means that a single user could create 256 connections and transmit
256*4096 = 1048576 file descriptors.

The file descriptors stay attached to the dbus-daemon process while they are
in the message loader, in the outgoing queue or waiting to be dispatched before
D-Bus activation.

dbus-daemon is usually limited to 65536 file descriptors (ulimit -n). If the
limit is reached and dbus-daemon needs to receive a message with a file
descriptor attached, this is signalled by recvfrom with the flag MSG_CTRUNC.
Dbus-daemon cannot recover from that error because the kernel does not have any
API to retrieve a file descriptor which has been discarded with MSG_CTRUNC.
Therefore, it closes the connection of the sender. This is not necessarily the
connection which generated the most file descriptors so it can lead to
denial-of-service attacks.

In order to prevent DoS issues, this patch reduces DEFAULT_MESSAGE_UNIX_FDS to
16:

max_connections_per_user * max_incoming_unix_fds = 256 * 64 = 16384

This is less than the usual "ulimit -n" (65536) with a good margin to
accomodate the other sources of file descriptors (stdin/stdout/stderr,
listening sockets, message loader, etc.).

Distributors on non-Linux may need to configure a smaller limit in
system.conf, if their limit on the number of fds is smaller than
Linux's.

fd.o#82820, part B
------------------

On Linux, it's not possible to send more than 253 fds in a single sendmsg()
call: sendmsg() would return -EINVAL.
  #define SCM_MAX_FD      253

SCM_MAX_FD changed value during Linux history:
- it used to be (OPEN_MAX-1)
- commit c09edd6eb (Jul 2007) changed it to 255
- commit bba14de98 (Nov 2010) changed it to 253

Libdbus always sends all of a message's fds, and the beginning
of the message itself, in a single sendmsg() call. Combining these
two, a malicious sender could split a message across two or more
sendmsg() calls to construct a composite message with 254 or more
fds. When dbus-daemon attempted to relay that message to its
recipient in a single sendmsg() call, it would receive EINVAL,
interpret that as a fatal socket error and disconnect the recipient,
resulting in denial of service.

This is fixed by keeping max_message_unix_fds <= SCM_MAX_FD.

Bug: https://bugs.freedesktop.org/show_bug.cgi?id=82820


Reviewed-by: Alban Crequy <alban.crequy@collabora.co.uk>