path: root/Networking.m4

TITLE(«

Network down, IP packets delivered via UPS. -- BOFH excuse #427

», __file__)

OVERVIEW(«

Networking is a complex and diverse area of computer science. This page
can only scratch the surface of some essential networking concepts,
aiming to convey enough background knowledge to understand more
specific and thorough articles on the subject matter and to encourage
the reader to explore the vast freely available literature. We cover
the four layers of the TCP/IP interconnection model in some detail
and look at a small subset of networking tools, including SSH. The
chapter concludes with a short overview of the Linux-specific Netlink
subsystem.

»)

SECTION(«Network Layers»)

<div>
define(«nl_width», «260»)
define(«nl_height», «200»)
define(«nl_box_width», «100»)
define(«nl_text_offset», «110»)
define(«nl_box_height», «eval((nl_height() - 10) / 5)»)
define(«nl_layer_width», «eval(nl_box_width() / 4)»)
define(«nl_font_size», «15»)
dnl $1: layer (link/internet/transport/application)
dnl $2: box number (0-4), $3: row-span, $4: column-span, $5: color
define(«nl_box», «
	<rect
		stroke="black"
		stroke-width="1"
		ifelse(«$1», «link», «x="1"»)
		ifelse(«$1», «internet», «x="eval(1 + nl_layer_width())"»)
		ifelse(«$1», «transport», «x="eval(1 + nl_layer_width() * 2)"»)
		ifelse(«$1», «application», «x="eval(1 + nl_layer_width() * 3)"»)
		y="eval($2 * nl_box_height())"
		height="eval($3 * nl_box_height())"
		width="nl_layer_width()"
		fill="$4"
	/>
»)
dnl $1: box number (see nl_box()), $2: text
define(«nl_text», «
	<text
		x="nl_text_offset()"
		y="eval($1 * nl_box_height() + nl_box_height() / 2)"
		font-size="nl_font_size()"
		dy="0.3em"
	>
		$2
	</text>
»)
<svg
	width="nl_width()" height="nl_height()"
	viewBox="0 0 260 200"
	xmlns="http://www.w3.org/2000/svg"
	xmlns:xlink="http://www.w3.org/1999/xlink"
>
	nl_box(«link», «0», «1», «#a22»)
	nl_box(«link», «1», «3», «#7e5»)
	nl_box(«link», «4», «1», «blue»)
	nl_box(«internet», «1», «1», «yellow»)
	nl_box(«internet», «2», «2», «#7e5»)
	nl_box(«transport», «2», «1», «orange»)
	nl_box(«transport», «3», «1», «#7e5»)
	nl_box(«application», «3», «1», «#7e5»)
	nl_text(«0», «Frame Header»)
	nl_text(«1», «IP Header»)
	nl_text(«2», «TCP/UDP Header»)
	nl_text(«3», «Data»)
	nl_text(«4», «Frame Footer»)
</svg>
</div>

<p> The <em> Open Systems Interconnection </em> (OSI) model describes
network communication by subdividing the data flow into <em>
abstraction layers</em>. This model was published as an ISO standard
in 1984 and comprises seven independent layers. A similar model with
only four layers, known as the <em> TCP/IP interconnection model</em>,
was proposed in RFC 1122 (1989). The TCP/IP model does not consider
physical specifications, so it has no counterpart to the <em> physical
layer </em> of the OSI model. Moreover, the three top layers in the
OSI model are not distinguished in the TCP/IP model. </p>

<p> The four layers of the TCP/IP model (<em>link, internet,
transport</em>, and <em> application</em>) are illustrated in the
diagram on the left. The link layer receives the full ethernet frame
(left column). It reads and interprets the frame header (red) and
footer (blue), and regards the remaining part as data (green), to be
passed uninterpreted to the next layer. The internet layer (second
column) expects an IP packet and interprets the first part of the data
as the IP header (yellow). It hands off the rest as a TCP/UDP packet to
the transport layer (third column) which in turn reads and strips off
its header (orange). The application layer only sees the green part in
the fourth column. Each layer is discussed in a dedicated section. </p>

EXERCISES()

<ul>
	<li> RFC is short for <em> Request for Comments</em>. Make yourself
	familiar with this concept. </li>

	<li> Search the web for "RFC 822" to get an idea how an RFC looks
	like. </li>

	<li> Discuss the pros and cons of many abtraction layers. </li>
</ul>

SECTION(«Link Layer»)

<p> The local network connection of a host is called its <em> link</em>.
The <em> link layer </em> is responsible for transmitting packets
between two hosts on the same link, that is, between directly connected
nodes. The link layer includes the protocols which maintain link states
such as the <em> Address Resolution Protocol </em> (ARP). Several
<em> link types </em> exist, the ubiquitous <em> ethernet </em> being
the only one to be discussed here. For ethernet links, the protocol
is specified in terms of the <em> media access control </em> (MAC)
addresses of ethernet frames. <p>

SUBSECTION(«Ethernet Bridging»)

<p> An <em> ethernet bridge </em> connects two or more networks by relaying
ethernet frames between the participating devices. This is described
in an official standard, the first revision of which was published in
1990. This standard can be implemented within a dedicated hardware
device, for example a <em> network switch</em>, or in software as
part of the operating system. Many soft- and hardware implementations
exist, which are compatible to each other as they all implement the
same protocol. Since ethernet bridges operate on the link layer, they
are transparent to higher level protocols like IP. </p>

<p> At the core of each bridge implementation there is the <em> forwarding
database </em> whose entries are indexed by the MAC addresses that have
recently been seen. Each time the bridge receives an ethernet frame,
the destination MAC address is looked up in the database to determine
the device to which the frame should be relayed. If no entry exists,
the frame is sent to <em> all </em> devices except the one it came
from, with the expectation that all devices but one will ignore the
frame. This is called <em> flooding</em>. From the source address
of the (single) reply a new database entry is created. This prevents
further flooding. Entries are removed from the database by aging: If
no frames have been received from a MAC address for the duration of a
time interval called <em> aging time</em>, the entry is removed from
the database. </p>

<p> The Linux ethernet bridge implementation dates back to 1999. Two
different tools are available to create and configure bridges: <code>
brctl(8) </code> and <code> bridge(8)</code>. The exercises of this
section aim to get the reader started with both tools. </p>

SUBSECTION(«Virtual Ethernet Interfaces»)

<p> A bridge can accommodate physical devices as well as virtual
devices. On Linux systems the common approach to equip virtual machines
with network interfaces employs the <em>virtual ethernet</em> device
driver, veth. This driver provides virtual pairs of devices where
each pair represents an ethernet tunnel. Ethernet frames received
by one end appear on its peer. To set up the network interface for
a virtual machine, one end of the pair is added to a bridge on the
host system while the other end represents the ethernet device of
the virtual machine. </p>

EXERCISES()
<ul>
	<li> Determine the MAC address of the first ethernet device of your
	computer. </li>

	<li> How many times faster has ethernet become since its first protocol
	version that supported a transfer rate of 2.94 Mbit/s in 1973? </li>

	<li> Explain why bridges can not be detected by tools like <code>
	traceroute(1) </code> or <code> tracepath(8)</code>. </li>

	<li> Hardware ethernet bridges are called switches because they
	relay ethernet frames by using a technique called <em> packet
	switching</em>. Define this term. </li>

	<li> The <code> arp(8) </code> command prints the kernel's network
	neighbour cache. Explain the purpose of this cache and the difference
	to the forwarding database of an ethernet bridge. </li>

	<li> Network devices can be set into <em> promiscuous
	mode</em>. Explain what this means, why interfaces which belong
	to an ethernet bridge need to be set into promiscuous mode,
	and the consequences of this fact. </li>

	<li> On your local computer, shut down the <code> eth0 </code>
	interface, create a bridge and add <code> eth0 </code> to the bridge.
	Then configure the bridge device in the same way the physical interface
	was configured before. Run <code>brctl showmacs </code> to see MAC
	addresses and the aging timer. </li>

	<li> Create a virtual ethernet pair by running <code> ip link add v1
	type veth peer name v2</code> and bring up the two interfaces with
	<code> ip link set up v1 </code> and similar for <code> v2</code>. Add
	the <code> v1 </code> end to the bridge. Configure an IP address on
	the <code> v2 </code> end of the pair (<code>ip addr add 192.168.42.42
	dev v2</code>).  Add an entry for the IP routing table with <code>
	ip route add 192.168.42.42/32 dev v2</code>. Start the <code> nc(1)
	</code> tool in listening mode and send IP traffic through the bridge
	by starting <code> nc(1) </code> a second time to connect to <code>
	192.168.42.42</code>. </li>

</ul>

HOMEWORK(«
<ul>

	<li> Recall how the <em> spanning tree algorithm</em> works. Explain
	how the spanning tree algorithm is employed in the <em> spanning
	tree protocol </em> (STP). Name two reasons for activating STP in a
	large network. </li>

	<li> The <em> rapid spanning tree protocol </em> (RSTP) is the
	successor of the traditional STP. Explain the difference between the
	two protocols. </li>

	<li> In each bridged network, there is one bridge which
	plays a special role: the so-called <em> root bridge</em>.
	Explain the purpose of the root bridge and how it is determined
	among the bridges of the network. </li>

	<li> Linux offers two different tools to configure ethernet bridges:
	<code> brctl(8) </code> and <code> bridge(8)</code>. Compare the
	feature sets of these tools. </li>

</ul>
»)


SECTION(«Internet Layer»)

<p> These days the term "internet" has acquired a rather broad meaning
in that it refers to all kind of network services. However, in
the context of the TCP/IP interconnection model, the <em> internet
layer </em> is named aptly because its purpose is to send packets
across different networks, thereby enabling inter-networking. More
precisely, packets are <em> routed </em> from the source network to
the destination network, where both networks are identified by <em>
IP interface addresses</em>.  Although both the prevalent IPv4 and the
next-generation IPv6 variant are being deployed actively worldwide,
we shall only discuss IPv4 here. </p>

<p> The first part of each IP packet is the <em> IP header</em>, which is
usually 20 byte long. Besides the source and destination addresses,
it contains an 8 bit protocol number which refers to the data portion
of the packet. </p>

<p> IP only provides an <em> unreliable </em> datagram transmission
facility, which means that packets may be lost, arrive multiple times,
or out of order. Moreover, packets can be fragmented or defragmented. </p>

EXERCISES()

<ul>
	<li> How many different IPv4 addresses exist? </li>

	<li> What is the difference between the <em> maximum transmission
	unit </em> (MTU) and the <em> path MTU</em>? </li>

	<li> Describe the purpose of the <em> Internet Control Message Protocol
	</em> (ICMP) and its relationship to IP. </li>

	<li> Byte 9 of the header of an IP packet is the so-called <em>
	time to live </em> (TTL) field, which is initially set to 64 by
	the sender. Explain the purpose of this field. </li>

	<li> Explain the connection between the TTL and the <em> Internet
	Control Message Protocol </em> (ICMP). </li>

	<li> What is a netmask in an IPv4 network? What is the purpose of the
	netmask? Why is the network part of an IP address also called <em>
	routing prefix</em>? </li>

	<li> On any host, run <code> ifconfig </code> and <code> ip addr
	show</code>. Both commands print the netmask of each network, but
	in different ways.  Explain which part of the output of the <code>
	ip </code> command contains the netmask. </li>

</ul>

HOMEWORK(«
Discuss the security implications of network services which are based
on MAC addresses or IP addresses alone.
», «
Both the IP address and the MAC address are trivial to fake. So they
should never be used to authenticate a user or a device on a network
to which potential attackers have physical access, i.e., untrusted
devices can be connected.
»)

HOMEWORK(«

Illustrate how <em> network address translation </em> (NAT) works
on the basis of a web search initiated from a desktop computer in a
local network and discuss the implications that NAT has on privacy.

», «

<p> The desktop is configured to route packets which are not destined
for the local network through a dedicated machine, called the <em>
router</em>. In particular, all internet traffic is sent to the router.
The router has two IP addresses: one address in the local network
and a public NAT address. As traffic passes from the desktop through
the router to the web server in the internet, the source address of
each IP packet (the local address of the desktop) is changed on the
fly to the public NAT address of the router. The router tracks each
active connection. When a reply arrives at the router, it uses the
connection tracking data stored during the outbound phase to determine
the address in the local network to which to forward the reply. This
time it overwrites the destination address of the IP packet with the
local address of the desktop. </p>

<p> NAT can be seen as providing a kind of privacy mechanism because
machines on the internet cannot monitor which hosts are sending and
receiving traffic. They only see the NAT address.    NAT has also
downsides though: Pinpointing the source of a problem becomes harder,
and encryption becomes more difficult. For example you can not encrypt
the IP address because the router must be able to change it. </p>

»)

HOMEWORK(«
Run <code> tracepath wikipedia.org</code>. Explain how this command
works and how it can be used to identify networking problems.
»)

SECTION(«Transport Layer»)

<p> The protocols of the transport layer provide message transfer
services which are on one hand independent of the underlying network
type, and on the other hand independent of the application. Different
network services running on the same host are distinguished by
<em>port numbers</em>, which are 16 bit identifiers. Several well
known port numbers are are associated with specific applications.
The two dominant transport layer protocols on top of IP, TCP and UDP,
are discussed in the following subsections. </p>

SUBSECTION(«The User Datagram Protocol»)

<p> The <em> User Datagram Protocol </em> (UDP) is the simplest
transport-layer protocol, built as a thin layer on top of IP. For this
reason, it offers only the same best-effort service as IP itself. For
example, there is no detection of duplicate or reordered packets,
no protection against packet loss or network congestion. However,
UDP generates checksums to catch transmission errors.  Being a
connectionless protocol, only minimal internal state about the
connection is maintained. This makes UDP suitable for applications
which need to avoid the overhead of setting up a TCP connection, or
in situations where on-time arrival is more important than reliability. </p>

SUBSECTION(«The Transmission Control Protocol»)

<p> The <em> Transmission Control Protocol </em> (TCP) provides reliable,
ordered delivery of a stream and a classic window-based congestion
control. In contrast to UDP, TCP provides a stream which is independent
of any packet boundaries. TCP is used extensively by many applications.
Besides HTTP (the Hypertext Transfer Protocol), also FTP (the File
Transfer protocol), SMTP (Simple Mail Transfer Protocol), SSH (Secure
Shell) all sit on top of TCP. </p>

EXERCISES()

<ul>
	<li> Check <code> /etc/services </code> and find the TCP port
	numbers for http (web), ssh and smtp (email).

	<li> Run <code> ls /proc/sys/net/ipv4/udp* </code> and <code> ls
	/proc/sys/net/ipv4/tcp* </code> to see the available UDP and TCP
	parameter settings, and to compare the complexity of UDP and TCP. </li>

	<li> Run <code> netstat -pant </code> to see active TCP sockets. </li>

	<li> Run <code> nmap localhost </code> to determine the listening
	TCP sockets. </li>

	<li> Run netcat in TCP listening mode: <code> nc -l $((19853 +
	UID)))</code>. Invite your neighbour to chat with you by connecting
	to your netcat process. </li>

	<li> Read section 3.1 of RFC 793 (Transmission Control Protocol,
	1981) to get an idea of the format of a TCP header, particularly the
	control bits called ACK, RST, SYN and FIN. </li>

	<li> The name "SYN" for one of the control bits of the TCP header
	stands for <em> synchronize</em>. What is being synchronized when
	this bit is set? And why does it need to be synchronous in the first
	place? </li>

	<li> Make yourself familiar with the 3-way TCP handshake also described
	in RFC 793. Why is it called a 3-way handshake? </li>

	<li> Run <code> ip tcpmetrics </code> to see recent TCP peers and
	discuss the output. </li>
</ul>

HOMEWORK(«
<ul>

	<li> UDP is said to be <em> datagram-oriented</em> while TCP is <em>
	stream-oriented</em>. Define both terms and explain the implications
	for application writers. </li>

	<li> Explain how TCP achieves its goal of controlling the transmission
	speed. </li>

	<li> Explain how the "SYN flooding" denial-of-service attack works and
	how SYN cookies mitigate this attack. </li>

	<li> Explain the idea behind TFO (TCP fast open). </li>

	<li> In contrast to TCP, UDP is a <em> connectionless </em> protocol. In
	particular, there is no handshake necessary to establish a
	connection. Describe the pros and cons of this fact. </li>

	<li> Explain, in no more than two sentences, the idea behind a port
	scan. </li>

	<li> What's a half-open TCP connection? What's a half-open TCP
	port scan? Explain why half-open TCP port scans are reasonably
	stealthy. </li>

	<li> Express your opinion on whether performing an unsolicited TCP
	port scan should be considered a criminal act. </li>
</ul>
»)

SECTION(«Application Layer»)

<p> Application layer protocols define how the server side of a network
service communicates with clients that connect to the server by
connecting a specific TCP or UDP port. Services are often associcated
with port numbers which can be registred at the <em> Internet Assigned
Numbers Authority </em> (IANA). </p>

<p> Examples for application layer protocols which are employed on top of
TCP are the <em> Hypertext Transfer Protocol </em> (HTTP, port 80)
and the <em> Secure Shell Protocol </em> (SSH, port 22). On top of
UDP sit the <em> Domain Name System </em> (DNS, port 53), the <em>
Dynamic Host Configuration Protocol </em> (DHCP, ports 67 and 68)
and the <em> Network Time Protocol </em> (NTP, port 123). </p>

<p> We won't discuss any specific application layer protocols here. Instead,
we look at some client programs. </p>

SUBSECTION(«The Name Service Switch»)

<p> Every Unix system needs a couple of (usually small) system databases
for proper operation. Besides the user database, there are other
databases for Unix group membership, the known hosts, network
protocols, and more. Traditionally, there was only a single source for
this information in the form of a configuration file per database, for
example <code> /etc/hosts</code> for the hosts database. The format of
each database file is described in the POSIX standard and in section
5 of the user manuals. This approach works well if the databases
and the number of hosts which need to share the same databases are
small. Larger organizations, however, have a need to maintain this
information centrally by means of some network service. The <em>
Lightweight Directory Access Protocol </em> (LDAP) and the <em>
Domain Name System </em> (DNS) are popular choices for the user
and the host/domain databases. Often the entries of the centralized
network database have to be merged with the entries of the local file
in <code> /etc</code>. This calls for a flexible method which lets the
administrator specify the sources of information and the search order.
Sun Microsystems came up with a clean solution to this problem named
<em> Name Service Switch </em> (NSS) for the Solaris operating system.
This solution was ported to most other Unix operating systems. The
implementation used on GNU/Linux systems is part of the <em> GNU
C Library </em> (glibc). The central configuration file for NSS is
<code> /etc/nsswitch.conf</code>. </p>

SUBSECTION(«Advanced SSH Features»)

<p> SSH, the <em> secure shell</em>, is a popular client/server software
package for logging into a remote machine. The name is a little
misleading, though. For one, SSH is not a shell; it merely provides
a method to <em> run </em> a shell. Second, it can do much more than
just log in and start the shell. It features a secure encrypted
communication channel between two hosts, and this channel can be
utilized in interesting ways on both ends.  In the exercises we look
at TCP port forwarding, some useful configuration options, and public
key authorization. </p>

EXERCISES()

<ul>

	<li> Inspect <code> /etc/resolv.conf </code> to get the IP address
	of your nameserver(s). Then run <code> dig @$IP $DOMAIN MX </code>
	where <code> $IP </code> is the nameserver IP address, and <code>
	$DOMAIN </code> is the domain of your email adress, e.g. <code>
	tuebingen,mpg.de</code>. Determine the hostname of the mail server
	from the output and run <code> nc $MAILHOST 25 </code> to send a mail
	to yourself. Hint: <code> HELO $MAILHOST</code>, <code> mail from:
	&lt;$LOGNAME@$DOMAIN&gt;</code>, <code> rcpt to: &lt;$LOGNAME@DOMAIN&gt;</code>,
	<code> data</code>. </li>

	<li> Edit <code> /etc/passswd</code>, <code> /etc/shadow</code>,
	and <code> /etc/group </code> to manually create a user account.
	Use the <a href="«#»cryptout.c">cryptout</a> program below to
	generate the second field of <code> /etc/shadow </code> containing
	the encrypted password. </li>

	<li> Understand the <code> hosts </code> line in <code>
	/etc/nsswitch.conf</code>. </li>

	<li> Does <code> host $HOSTNAME </code> always print the same IPv4
	address as <code> ping $HOSTNAME</code>? </li>

	<li> Run <code> nc localhost 22 </code> to determine the SSH server
	version. </li>

	<li> Forward the TCP port 12345 of your local machine to an
	internal server using ssh's <code> -L </code> option for local port
	forwarding.  Check that you can log in with <code> ssh -p 12345
	localhost</code>. </li>

	<li> Search the <code>ssh_config(5)</code> man page for <code>
	NoHostAuthenticationForLocalhost </code> and ponder if it is a good
	idea to set this to <code> yes</code>. </li>

	<li> Add <code> Host </code> and <code> Hostname </code> entries
	to your ssh config file so that you can log in with <code> ssh
	hostname</code>, even though <code> hostname </code> does not resolve
	on your local network. </li>

	<li> Create an ssh key pair and add the public part so that you can
	log in without specifying a password. Discuss the security implications
	of this setup. </li>

	<li> In an ssh session, type <code> ~C </code> to open the ssh command
	prompt. Use a suitable <code> -L </code> command to add a local port
	forward to the existing connection. Type <code> ~? </code> to see
	the available escape sequences. </li>

	<li> Add the lines <code> Host *.eb.local </code> and <code> ProxyJump
	cgw.tuebingen.mpg.de </code> to your ssh config file. Then type <code>
	ssh olt.eb.local</code>. Check <code> ssh(1) </code> to learn how
	this works. </li>

</ul>

HOMEWORK(«
Explain the difference between local and remote port forwarding. Give
a typical example for either type of forwarding.
»)

SECTION(«The Netlink Messaging System»)

<p> The various layers and protocols discussed earlier in this chapter
dealt with the communication between hosts which are connected by
a network. The Linux-specific <em>Netlink Interface</em>, however,
does not fit into this picture because it is a messaging system
for passing network-related information between the kernel and a
user space program, and vice-versa. Among other uses, tools like
<code>ip(8)</code> and <code>ifconfig(8)</code> employ Netlink
to configure network devices. Netlink is implemented on top of the
socket infrastructure, so the communication link between a user space
program and the kernel is estabished by means of the usual system calls
<code>socket(2)</code>, <code>bind(2)</code>, <code>connect(2)</code>,
and messages are transferred by calling <code>sendmsg(2)</code>
and <code>recvmsg(2)</code>. </p>

<p> There are several <em>netlink families</em> which select the
kernel subsystem to communicate with. We shall only be concerned
with the <code>NETLINK_ROUTE</code> family, which is used to
modify network routes, IP addresses, and more. The details of
<code>NETLINK_ROUTE</code> are described in <code>rtnetlink(7)</code>
while <code>netlink(7)</code> covers the general interface and the
currently assigned families. </p>

<p> A Netlink message starts with a 16 byte header as defined by
<code>struct nlmsghdr</code>. To report errors to userspace, Netlink
provides a message type that encapsulates an error header defined
by <code>struct nlmsgerr</code>. Both structures are declared in
in <code>include/linux/netlink.h</code>. Full Netlink messsages,
including the Netlink header are transferred. Therefore the user space
program has to implement a parser for both regular Netlink messages
and Netlink error messages, as well as a primitive for setting up
properly formatted Netlink messages to be sent to the kernel. Several
user space libraries aim to help the programmer with this repetetive
and error-prone task, the <em>minimalistic Netlink library</em>
(libmnl) being the most popular one. </p>

SUPPLEMENTS()

SUBSECTION(«cryptout.c»)

<pre>
	<code>
		#include &lt;stdlib.h&gt;
		#include &lt;crypt.h&gt;
		#include &lt;stdio.h&gt;
		#include &lt;sys/random.h&gt;

		static const char set[] =
			"abcdefghijklmnopqrstuvwxyz"
			"ABCDEFGHIJKLMNOPQRSTUVWXYZ"
			"0123456789./";

		int main(int argc, char **argv)
		{
			unsigned char rnd[2], salt[2], *result;

			if (argc &lt; 2)
				exit(EXIT_FAILURE);
			if (getrandom(rnd, 2, 0) &lt; 0)
				exit(EXIT_FAILURE);
			salt[0] = set[rnd[0] &amp; 63];
			salt[1] = set[rnd[1] &amp; 63];
			result = crypt(argv[1], salt);
			if (!result)
				exit(EXIT_FAILURE);
			printf("%s\n", result);
			exit(EXIT_SUCCESS);
		}
	</code>
</pre>