file and communicates this information to the client. </p>
<p> The file handle which an nfs client received through some earlier
-rpc can become invalid at any time due to operations on a different
+rpc can become invalid at any time due to operations on different
hosts. This happens, for example, if the file was deleted on the server
or on a different nfs client, or when the directory that contains
the file is no longer exported by the server due to a configuration
<li> In an nfs-mounted directory (nfs version 4.0 or earlier), run
<code>cat > foo &</code>. Note that the cat process automatically
receives the STOP signal. Run <code>rm foo; ls -ltra</code>. Read
- section D2 of the <a href="http://nfs.sourceforge.net/">nfs HOWTO</a>
+ section D2 of the <a href="https://nfs.sourceforge.net/">nfs HOWTO</a>
for the explanation. </li>
<li> In an nfs-mounted directory, run <code>{ while :; do echo; sleep
<li> Discuss the pros and cons of hard vs. soft mounts. </li>
- <li> Read section A10 of the <a href="http://nfs.sourceforge.net/">nfs
+ <li> Read section A10 of the <a href="https://nfs.sourceforge.net/">nfs
HOWTO</a> to learn about common reasons for stale nfs handles. </li>
<li> Can every local filesystem be exported via nfs? </li>
<li> Dominic Giampaolo: Practical File System Design </li>
<li> Cormen </li>
<li> Darrick Wong: XFS Filesystem Disk Structures </li>
- <li> The <a href="https://xfs.org/index.php/Main_Page">xfs FAQ</a> </li>
<li> Documentation/filesystems/path-lookup.rst </li>
<li> rfc 5531: Remote Procedure Call Protocol, Version 2 (2009) </li>
<li> Birell, A.D. and Nelson, B.J.: Implementing Remote Procedure Calls
(1984) </li>
+ <li> <a href="https://lwn.net/Articles/897917/">NFS: the early
+ years</a> and <a href="https://lwn.net/Articles/898262/">NFS: the new
+ millennium</a>, two articles on the design and history of NFS by Neil
+ Brown. </li>
</ul>
Tübingen, Germany. Over time they morphed into a generic document
that was made public in December 2019. </p>
-<p> The title is of course a pun on the famous book "Advanced
-Programming in the Unix Environment" by W. Richard Stevens. While
+<p> The title is of course a pun on the famous book <em>Advanced
+Programming in the Unix Environment</em> by W. Richard Stevens. While
Stevens' book centers around C programming, we try to convey
fundamental ideas without assuming substantial programming skills.
An elementary knowledge of shell and C programming is certainly
processed bios then form an I/O request which is handed to an <em>
I/O scheduler </em> (also known as <em> elevator</em>). </p>
-<p> At this time of writing (2018-11) there exist two different sets
-of schedulers: the traditional single-queue schedulers and the
-modern multi-queue schedulers, which are expected to replace the
-single-queue schedulers soon. The three single-queue schedulers,
-noop, deadline and cfq (complete fair queueing), were designed for
-rotating disks. They reorder requests with the aim to minimize seek
-time. The newer multi-queue schedulers, mq-deadline, kyber, and bfq
-(budget fair queueing), aim to max out even the fastest devices. As
-implied by the name "multi-queue", they implement several request
-queues, the number of which depends on the hardware in use. This
-has become necessary because modern storage hardware allows multiple
-requests to be submitted in parallel from different CPUs. Moreover,
-with many CPUs the locking overhead required to put a request into
-a queue increases. Per-CPU queues allow for per-CPU locks, which
-decreases queue lock contention. </p>
+<p> Traditionally, the schedulers were designed for rotating disks.
+They implemented a single request queue and reordered the queued
+I/O requests with the aim to minimize disk seek times. The newer
+multi-queue schedulers mq-deadline, kyber, and bfq (budget fair
+queueing) aim to max out even the fastest devices. As implied by
+the name "multi-queue", they implement several request queues,
+the number of which depends on the hardware in use. This has become
+necessary because modern storage hardware allows multiple requests
+to be submitted in parallel from different CPUs. Moreover, with many
+CPUs the locking overhead required to put a request into a queue
+increases. Per-CPU queues allow for per-CPU locks, which decreases
+queue lock contention. </p>
<p> We will take a look at some aspects of the Linux block layer and on
the various I/O schedulers. An exercise on loop devices enables the
target_args</code>. Determine the correct values for the first three
arguments to encrypt <code> /dev/loop0</code>. </li>
- <li> The <code> target_args </code> for the dm-crypt target are
- of the form <code> cipher key iv_offset device offset</code>. To
- encrypt <code> /dev/loop0 </code> with AES-256, <code> cipher </code>
- is <code> aes</code>, device is <code> /dev/loop0 </code> and both
- offsets are zero. Come up with an idea to create a 256 bit key from
- a passphrase. </li>
+ <li> The <code>target_args</code> for the dm-crypt target are
+ of the form <code>cipher key iv_offset device offset</code>. To
+ encrypt <code>/dev/loop0</code> with AES-256, <code>cipher</code>
+ is <code>aes</code>, <code>device</code> is <code>/dev/loop0</code>
+ and both offsets are zero. Come up with an idea to create a 256 bit
+ key from a passphrase. </li>
<li> The <code> create </code> subcommand of <code> dmsetup(8)
</code> creates a device from the given table. Run a command of
SUBSECTION(«Virtual Ethernet Interfaces»)
-<p> A bridge can accommodate physical devices like <code> eth0 </code>
-as well as virtual devices. On Linux systems the common approach
-to equip virtual machines with network interfaces employs the <em>
-virtual ethernet </em> (veth) device driver. This driver provides
-virtual pairs of devices where each pair represents an ethernet
-tunnel. Ethernet frames received by one end appear on its pair. To
-set up the network interface for a virtual machine, one end of the
-pair is added to a bridge on the host system while the other end
-represents the ethernet device of the virtual machine. </p>
+<p> A bridge can accommodate physical devices as well as virtual
+devices. On Linux systems the common approach to equip virtual machines
+with network interfaces employs the <em>virtual ethernet</em> device
+driver, veth. This driver provides virtual pairs of devices where
+each pair represents an ethernet tunnel. Ethernet frames received
+by one end appear on its peer. To set up the network interface for
+a virtual machine, one end of the pair is added to a bridge on the
+host system while the other end represents the ethernet device of
+the virtual machine. </p>
EXERCISES()
<ul>
SECTION(«Transport Layer»)
-<p> The protocols of the transport layer provide message transfer services
-which are on one hand independent of the underlying network type,
-and on the other hand independent of the application. Different
-network services on running on the same host are distinguished by
-<em> port numbers</em>, which are 16 bit identifiers. Several well
+<p> The protocols of the transport layer provide message transfer
+services which are on one hand independent of the underlying network
+type, and on the other hand independent of the application. Different
+network services running on the same host are distinguished by
+<em>port numbers</em>, which are 16 bit identifiers. Several well
known port numbers are are associated with specific applications.
The two dominant transport layer protocols on top of IP, TCP and UDP,
are discussed in the following subsections. </p>
<em> Extended Page Tables</em>. </li>
<li> Suppose a hacker gained root access to a VM and wishes to proceed
- from there to get also full control over the host OS. Discuss the thread
+ from there to get also full control over the host OS. Discuss the threat
model in the context of the three virtualization frameworks covered
in this section. </li>
which ran a complete open source software stack. </p>
<p> The success of Linux, or <em>GNU/Linux</em> as some prefer to
-call it for reasons that should now be clear, has only increased
-over time, to the point where commercial Unix systems are mostly
-irrelevant. Today Linux runs on a wide variety of machines ranging
-from supercomputers to workstations, smart phones and IOT (internet
-of things) devices with very limited resources.
-
-<p> The same companies which almost killed Unix by commercializing it
-in order to maximize their profit make money with Linux today. However,
-they had to adjust their business model in order to comply with the
-GPL. Rather than selling proprietary software, they bundle open source
-software and sell support to paying customers. Some companies also
-sell hardware with Linux pre-installed. </p>
+call it for reasons that should now be clear, steadily increased over
+time. In 2003 the SCO group, a company which sold a proprietary Unix
+system, was unhappy about this progress and sued IBM, which offered
+various Linux products. SCO claimed to be the owner of Unix, and that
+Linux contained "millions of lines" of code copied from Unix. SCO's
+lawyers argued that the success of Linux originated from this theft
+of intellectual property and asked for $5 billion as compensation
+for the resulting losses. The company also tried to collect taxes
+from other Linux users. Microsoft funded SCO in these efforts. </p>
+
+<p> In the end SCO lost the lawsuit since it was evident that all that
+copied code never existed. In fact, the court ruled that SCO did not
+even own the Unix copyrights to begin with. Another fun fact is that
+the large number of bugs in the early Linux code actually helped to
+prove that Linux was original work. The long term effects of this
+lawsuit, an improved position of Linux and its ecosystem, last until
+the presence. Commercial Unix systems have become irrelevant as Linux
+runs on a wide variety of machines ranging from supercomputers to
+workstations, smart phones and IOT (internet of things) devices with
+very limited resources. </p>
+
+<p> While SCO went bankrupt eventually, some of the companies which
+almost killed Unix by maximizing their own profit still exist, and
+make money with Linux today. However, they had to adjust their
+business model in order to comply with the GPL. Rather than selling
+proprietary software, they bundle open source software and sell
+support to paying customers. Some companies also sell hardware with
+Linux pre-installed. </p>
SUBSECTION(«Linux Distributions»)
HOMEWORK(«
Think about printers, sound cards, or displays as a file. Specifically,
-describe what <code>open, read</code>, and <code>write</code> should
-mean for these devices.
+describe what <code>open</code>, <code>read</code>, and <code>write</code>
+should mean for these devices.
», «
left etc. Writing to the file descriptor would cause output on the
device. This would mean to print the text that is written, play the
audio samples, or show the given text on the display. The point to
-take away is that the <code>open, read, write</code> interface is a
-generic concept that works for different kinds of devices, not only
-for storing data in a file on a hard disk.
+take away is that the <code>open</code>, <code>read</code>,
+<code>write</code> interface is a generic concept that works for
+different kinds of devices, not only for storing data in a file on a
+hard disk.
»)
<p> The <code>pipe(2)</code> system call takes no arguments and
creates two file descriptors for the calling process which are tied
-together as a unidirectional first in, first out data channel that
-works just like a fifo, but without any files being involved. One
+together as a unidirectional first in, first out data channel. One
file descriptor is the <em>read end</em> of the pipe, the other is
the <em>write end</em>. Data written to the write end is buffered by
the kernel and can be obtained by reading from the read end. </p>
communicate with the child by writing a message to the write end of
the pipe for the child to read. </p>
+<p> This approach depends on file descriptor inheritance across
+<code>fork(2)</code>, so it does not work in the situation
+where neither process is an ancestor of the other. Files of
+type <em>fifo</em> (named pipes) overcome this restriction. To
+establish a connection between two <em>unrelated</em> processes,
+both processes call <code>open(2)</code> to obtain a file
+descriptor which is associated with the fifo. One process passes
+the <code>O_WRONLY</code> flag to open the file for writing while
+the other passes <code>O_RDONLY</code> to open it for reading. The
+two processes may then communicate in the same way as with the
+<code>pipe(2)/fork(2)</code> approach. </p>
+
<p> The POSIX <code>dup(2)</code> and <code>dup2(2)</code> system
calls allow a process to manipulate the entries of its file descriptor
array. In particular the standard file descriptors 0, 1, and 2 can be
and <code>wc(1)</code> reads from stdin, <code>wc(1)</code> processes
the output of <code>ls(1)</code>. </p>
-<p> Note that this trick does not work to establish a connection
-between two <em>existing</em> processes because it depends on file
-descriptor inheritance across <code>fork(2)</code>. In the general
-case one has to fall back to sockets or fifos to create the data
-channel. </p>
-
SUBSECTION(«Stdio»)
<p> The POSIX standard requires a compliant Unix system to provide
href="https://www.newyorker.com/business/currency/the-gnu-manifesto-turns-thirty">
The GNU Manifesto Turns Thirty</a>, by Maria Bustillos. </li>
+ <li> <a href="https://lwn.net/Articles/924577/">The SCO lawsuit,
+ 20 years later</a> </li>
+
<li> <a
href="http://www.catb.org/~esr/writings/unix-koans/end-user.html">
The Koan of Master Foo and the End User</a>. </li>
body {
- background-color: #aacccc;
+ color: #000;
+ background-color: #acc;
text-align: justify;
padding: 0px 30px 0px 25px;
-
+ font-size: 110%;
}
h1 {
- font-size: 150%;
+ font-size: 160%;
+}
+
+h2 {
+ font-size: 130%;
+}
+
+h3 {
+ font-size: 110%;
+}
+
+code {
+ font-size: 130%;
}
+p {
+ padding: 0px 0px 0px 0px;
+}
+
+
.logo {
border: 0px;
- padding: 0em 20px 0px 0px;
- margin-left: 0px;
- margin-right: 0px;
- border: 0px;
}
table {
- margin-left: auto;
+ margin-left: 0px;
margin-right: auto;
border: none;
+ padding: 10px 0px 10px 0px;
+}
+
+table.menu {
+ padding: none;
+ border: 0px;
+ border-spacing: 0px;
+}
+
+table.logo {
+ margin-left: 0px;
}
td {
padding: 5px;
}
+td.logo {
+ border: none;
+}
+
+td.menu {
+ border: none;
+ vertical-align: top;
+}
+
svg {
float: left;
}
border: 0px;
}
-#overview_heading {
- text-align: center;
- font-weight: bold;
-}
-
-#overview_text {
+.overview {
margin: 1% 5% 1% 5%;
- font-size: 95%;
+ font-size: 90%;
}
div.solution {
text-decoration: underline;
}
-#chapter_list {
- font-size: 120%;
-}
-
.dropdown {
position: fixed;
top: 0; /* Position the navbar at the top of the page */
.dropdown-content {
display: none;
background-color: #8aa;
- min-width: 300px;
- padding: 12px 12px;
- font-size: 75%;
+ padding: 6px 25px 0px 30px;
}
.dropdown:hover .dropdown-content {
display: block;
+ font-size: 80%;
}
define(«XREFERENCE», «[$2]»«($1)»)
define(«EMPH», ««_»REMOVE_NEWLINE(«$1»)«_»»)
define(«CMD», «`REMOVE_NEWLINE(«$1»)`»)
-define(«SECTION», «
-divert(«0»)
-<li> <a href='«#»LOCAL_LINK_NAME(«$1»)'>$1</a> </li>
-divert(«2»)
-<h2 id="LOCAL_LINK_NAME(«$1»)">$1</h2>
-»)
-define(«SUBSECTION», «<h3 id="LOCAL_LINK_NAME(«$1»)">$1</h3>
-»)
+define(«SECTION», « divert(«0») <a
+href='«#»LOCAL_LINK_NAME(«$1»)'>$1</a><br> divert(«2») <h2
+id="LOCAL_LINK_NAME(«$1»)">$1</h2> ») define(«SUBSECTION»,
+«<h3 id="LOCAL_LINK_NAME(«$1»)">$1</h3> »)
define(«OVERVIEW», «
-<div id="overview_heading">
-ifelse(«$2», «», «Overview», «$2»)
-</div>
-<div id="overview_text">
-$1
+<div class="overview">
+ <center> <strong>
+ ifelse(«$2», «», «Overview», «$2»)
+ </strong> </center>
+ $1
</div>
»)
define(«EXERCISES», «<h3> Exercises </h3>»)
<body>
<table class="logo"> <tr>
<td class="logo">LOGO()</td>
- <td id="title"> <h1>
+ <td class="logo"> <h1>
ifelse(«$3», «», «
substr(translit(«$2», «_», « »), 0, index(«$2», .m4))
», «
»)
define(«TABLE_OF_CONTENTS», «
- <ul id="$1">
- <li> TOC_ENTRY(«Introduction», «$2») </li>
- <li> TOC_ENTRY(«Unix_Concepts», «$2») </li>
- ifelse(PUBLIC(), «false», «
- <li> TOC_ENTRY(Command_Line_Utilities, «$2») </li>
- »)
- <li> TOC_ENTRY(Networking, «$2») </li>
- <li> TOC_ENTRY(LVM, «$2») </li>
- <li> TOC_ENTRY(«Filesystems», «$2») </li>
- <li> TOC_ENTRY(OS-Level_Virtualization, «$2») </li>
- ifelse(PUBLIC(), «false», «
- <li> TOC_ENTRY(Gridengine, «$2») </li>
- <li> TOC_ENTRY(Git, «$2») </li>
- <li> TOC_ENTRY(Bash, «$2») </li>
- <li> TOC_ENTRY(Debugging, «$2») </li>
- »)
- </ul>
+ TOC_ENTRY(«Introduction», «$2»)
+ TOC_ENTRY(«Unix_Concepts», «$2»)
+ ifelse(PUBLIC(), «false», «
+ TOC_ENTRY(Command_Line_Utilities, «$2»)
+ »)
+ TOC_ENTRY(Networking, «$2»)
+ TOC_ENTRY(LVM, «$2»)
+ TOC_ENTRY(«Filesystems», «$2»)
+ TOC_ENTRY(OS-Level_Virtualization, «$2»)
+ ifelse(PUBLIC(), «false», «
+ TOC_ENTRY(Gridengine, «$2»)
+ TOC_ENTRY(Git, «$2»)
+ TOC_ENTRY(Bash, «$2»)
+ TOC_ENTRY(Debugging, «$2»)
+ »)
»)
define(«TITLE», «
HEADER(«$1», «$2», «$3»)
-<hr>
+
<div class="dropdown">
<svg
height="35" width="30"
<rect fill="black" height="3" width="20" x="5" y="11" rx="2"/>
<rect fill="black" height="3" width="20" x="5" y="17" rx="2"/>
</svg>
- <div class="dropdown-content">
- <h3> Chapter </h3>
- TABLE_OF_CONTENTS(«chapter_list»)
- <h4> Section </h4>
- <ul>
+<div class="dropdown-content">
+ <table class="menu">
+ <tr>
+ <th> Chapter </th>
+ <th> Section </th>
+ </tr> <tr>
+ <td class="menu">
+ TABLE_OF_CONTENTS(«chapter_list»)
+ </td> <td class="menu">
divert(«1»)
-</ul>
+ </td>
+ </tr>
+ </table>
</div>
</div>
divert(«2»)
projects / aple.git / commitdiff