Split into files for discussion/feet stepping purposes

second-edition/src/SUMMARY.md

@@ -111,7 +111,12 @@
     - [Advanced Types](ch19-04-advanced-types.md)
     - [Advanced Functions & Closures](ch19-05-advanced-functions-and-closures.md)
 
-- [Final project: a web server](ch20-00-final-project-a-web-server.md)
+- [Final Project: Building a Multithreaded Web Server](ch20-00-final-project-a-web-server.md)
+    - [Accepting a TCP Connection](ch20-01-accepting-a-tcp-connection.md)
+    - [Reading the Request](ch20-02-reading-the-request.md)
+    - [Writing a Response](ch20-03-writing-a-response.md)
+    - [Validating the Request](ch20-04-validating-the-request.md)
+    - [Adding a Thread Pool](ch20-05-adding-a-thread-pool.md)
 
 - [Appendix](appendix-00.md)
     - [A - Keywords](appendix-01-keywords.md)

second-edition/src/ch20-01-accepting-a-tcp-connection.md

@@ -0,0 +1,152 @@
+## Accepting a TCP Connection
+
+The *Hypertext Transfer Protocol* (*HTTP*) that powers the web is built on top
+of the *Transmission Control Protocol* (*TCP*). We won't get into the details
+too much, but here's a short overview: TCP is a low-level protocol, and HTTP
+builds a higher-level protocol on top of TCP. Both protocols are what's called a
+*request-response protocol*, that is, there is a *client* that initiates
+requests, and a *server* that listens to requests and provides a response to
+the client. The contents of those requests and responses are defined by the
+protocols themselves.
+
+TCP describes the low-level details of how information gets from one server to
+another, but doesn't specify what that information is; it's just a bunch of
+ones and zeroes. HTTP builds on top of TCP by defining what the content of the
+requests and responses should be. As such, it's technically possible to use
+HTTP with other protocols, but in the vast majority of cases, HTTP sends its
+data over TCP.
+
+So the first thing we need to build for our web server is to be able to listen
+to a TCP connection. The standard library has a `std::net` module that lets us
+do this. Let's make a new project:
+
+```text
+$ cargo new hello --bin
+     Created binary (application) `hello` project
+$ cd hello
+```
+
+And put the code in Listing 20-1 in `src/main.rs` to start. This code will
+listen at the address `127.0.0.1:8080` for incoming TCP streams. When it gets
+an incoming stream, it will print `Connection established!`:
+
+<span class="filename">Filename: src/main.rs</span>
+
+```rust,no_run
+use std::net::TcpListener;
+
+fn main() {
+    let listener = TcpListener::bind("127.0.0.1:8080").unwrap();
+
+    for stream in listener.incoming() {
+        let stream = stream.unwrap();
+
+        println!("Connection established!");
+    }
+}
+```
+
+<span class="caption">Listing 20-1: Listening for incoming streams and printing
+a message when we receive a stream</span>
+
+A `TcpListener` allows us to listen for TCP connections. We've chosen to listen
+to the address `127.0.0.1:8080`. The part before the colon is an IP address
+representing our own computer, and `8080` is the port. We've chosen this port
+because HTTP is normally accepted on port 80, but connecting to port 80 requires
+administrator privileges. Regular users can listen on ports higher than 1024;
+8080 is easy to remember since it's the HTTP port 80 repeated.
+
+The `bind` function is sort of like `new` in that it returns a new
+`TcpListener` instance, but `bind` is a more descriptive name that fits with
+the domain terminology. In networking, people will often talk about "binding to
+a port", so the function that the standard library defined to create a new
+`TcpListener` is called `bind`.
+
+The `bind` function returns a `Result<T, E>`. Binding may fail, for example, if
+we had tried to connect to port 80 without being an administrator. Another
+example of a case when binding would fail is if we tried to have two programs
+listening to the same port, which would happen if we ran two instances of our
+program. Since we're writing a basic server here, we're not going to worry
+about handling these kinds of errors, and `unwrap` lets us just stop the
+program if they happen.
+
+The `incoming` method on `TcpListener` returns an iterator that gives us a
+sequence of streams (more specifically, streams of type `TcpStream`). A
+*stream* represents an open connection between the client and the server. A
+*connection* is the name for the full request/response process when a client
+connects to the server, the server generates a response, and the server closes
+the connection. As such, the `TcpStream` will let us read from itself to see
+what the client sent, and we can write our response to it. So this `for` loop
+will process each connection in turn and produce a series of streams for us to
+handle.
+
+For now, handling a stream means calling `unwrap` to terminate our program if
+the stream has any errors, then printing a message. Errors can happen because
+we're not actually iterating over connections, we're iterating over *connection
+attempts*. The connection might not work for a number of reasons, many of them
+operating-system specific. For example, many operating systems have a limit to
+the number of simultaneous open connections; new connection attempts will then
+produce an error until some of the open connections are closed.
+
+Let's try this code out! First invoke `cargo run` in the terminal, then load up
+`127.0.0.1:8080` in a web browser. The browser will show an error message that
+will say something similar to "Connection reset", since we're not currently
+sending any data back. If we look at our terminal, though, we'll see a bunch of
+messages that were printed when the browser connected to the server!
+
+```text
+     Running `target/debug/hello`
+Connection established!
+Connection established!
+Connection established!
+```
+
+We got multiple messages printed out for one browser request; these connections
+might be the browser making a request for the page and a request for a
+`favicon.ico` icon that appears in the browser tab, or the browser might be
+retrying the connection. Our browser is expecting to speak HTTP, but we aren't
+replying with anything, just closing the connection by moving on to the next
+loop iteration. When `stream` goes out of scope and dropped at the end of the
+loop, its connection gets closed as part of the `drop` implementation for
+`TcpStream`. Browsers sometimes deal with closed connections by retrying, since
+the problem might be temporary. The important thing is that we've successfully
+gotten a handle on a TCP connection!
+
+Remember to stop the program with `CTRL-C` when you're done running a
+particular version of the code, and restart `cargo run` after you've made each
+set of code changes in order to be running the newest code.
+
+In order to keep our code clean, let's move the code processing the connection
+out to a function. We're about to add more code to actually process a
+connection rather than only printing out a message, so we'll make a function to
+contain the code for this purpose. Modify your code to look like Listing 20-2,
+which starts a `handle_connection` function:
+
+<span class="filename">Filename: src/main.rs</span>
+
+```rust,no_run
+use std::net::TcpListener;
+use std::net::TcpStream;
+
+fn main() {
+    let listener = TcpListener::bind("127.0.0.1:8080").unwrap();
+
+    for stream in listener.incoming() {
+        let stream = stream.unwrap();
+
+        handle_connection(stream);
+    }
+}
+
+fn handle_connection(stream: TcpStream) {
+    println!("Connection established!");
+}
+```
+
+<span class="caption">Listing 20-2: Extracting a `handle_connection`
+function</span>
+
+This should have no effect on the behavior of the program; this was just a
+small refactoring to set up a nice separation of concerns. Now we can
+concentrate on handling the `TcpStream` in `handle_connection` and not worry
+about all of the setup code that we'll leave in `main`.

second-edition/src/ch20-02-reading-the-request.md

@@ -0,0 +1,98 @@
+## Reading the Request
+
+Let's read in the request from our browser! Modify `handle_connection` to read
+data from the `stream` and print it out as shown in Listing 20-3. No changes to
+`main` are needed, but we will need to add the `std::io::prelude` in order to
+bring traits into scope that let us read from and write to the stream:
+
+<span class="filename">Filename: src/main.rs</span>
+
+```rust
+use std::io::prelude::*;
+use std::net::TcpListener;
+use std::net::TcpStream;
+
+// ...snip...
+
+fn handle_connection(mut stream: TcpStream) {
+    let mut buffer = [0; 512];
+
+    stream.read(&mut buffer).unwrap();
+
+    println!("Request: {}", String::from_utf8_lossy(&buffer[..]));
+}
+```
+
+<span class="caption">Listing 20-3: Reading from the `TcpStream`</span>
+
+In `handle_connection`, we had to make `stream` mutable with the `mut` keyword.
+We're going to be reading data from the stream, so it's going to get modified.
+
+Next, we need to actually read from the stream; we do this in two steps. First,
+we declare a `buffer` on the stack; we've made it 512 bytes. Why 512? It's big
+enough to get a basic request, but not super huge. If we wanted to handle
+requests of an arbitrary size, this would need to be more complicated, but
+we're keeping it simple for now! We then pass that buffer to `stream.read`.
+This will read bytes from the `TcpStream` and put them in the buffer.
+
+Next, we print that stream out. The `String::from_utf8_lossy` function takes a
+`&[u8]` and produce a `String`. The 'lossy' part of its name comes from its
+behavior when it sees invalid UTF-8 sequences; it replaces them with <20>, `U+FFFD
+REPLACEMENT CHARACTER`.
+
+Let's give this a try!
+
+```text
+$ cargo run
+   Compiling hello v0.1.0 (file:///projects/hello/src/hello)
+    Finished dev [unoptimized + debuginfo] target(s) in 0.42 secs
+     Running `target/debug/hello`
+Request: GET / HTTP/1.1
+Host: 127.0.0.1:8080
+User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101
+Firefox/52.0
+Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
+Accept-Language: en-US,en;q=0.5
+Accept-Encoding: gzip, deflate
+Connection: keep-alive
+Upgrade-Insecure-Requests: 1
+<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>
+```
+
+You'll probably get slightly different output depending on your browser! You
+also might see this request repeated; if so, we can definitively tell that the
+reason we have multiple connections is because the browser is trying to fetch
+`/` repeatedly.
+
+Let's break this request data down. HTTP is a text-based protocol, and a
+request looks like this:
+
+```text
+Request-Line headers CRLF message-body
+```
+
+First there's a 'request line'. Then, any headers. Next, a CRLF sequence, and
+then, the body of the message. A request line looks like this:
+
+```text
+Request-Line = Method Request-URI HTTP-Version CRLF
+```
+
+First, we have a method, like `GET` or `POST`. Then, the request's URI, which
+is a term the HTTP spec uses. You have probably heard of a 'URL'. All URLs are
+URIs, but not all URIs are URLs. Since this isn't a book about the HTTP
+specification, given this fact, we can just think "URL" when we see "URI" and
+move on. Next, we have the HTTP version, and then a CRLF sequence. That's
+`\r\n` is the CRLF sequence; `\r` is a "carriage return" and `\n` is a "line
+feed"; these terms come from the typewriter days!
+
+If we apply this to our request:
+
+```text
+GET / HTTP/1.1
+Host: 127.0.0.1:8080
+<more headers>
+```
+
+`GET` is our method, `/` is our Request URI, and `HTTP/1.1` is our version. All
+the stuff from `Host` and after are headers. `GET` requests have no body. Neat!

second-edition/src/ch20-03-writing-a-response.md

@@ -0,0 +1,132 @@
+## Writing a Response
+
+Let's respond to our browser with a response. Responses look like this:
+
+```text
+Status-Line headers CRLF message-body
+```
+
+First, we need a status line. Then, any headers. Next, a CRLF sequence, and
+then, the body of the message. What's a status line? Here's an example of one:
+
+```text
+HTTP/1.1 200 OK\r\n\r\n
+```
+
+Status lines look like this:
+
+```text
+Status-Line = HTTP-Version Status-Code Reason-Phrase CRLF
+```
+
+We're using version 1.1 of the protocol, and `200` is the status code. `OK` is
+the "reason phrase", it's like a text description of the status code. Finally,
+`\r\n` is the CRLF sequence; `\r` is a "carriage return" and `\n` is a "line
+feed"; these terms come from the typewriter days!
+
+We don't have any headers, so there's nothing to put there. Next, another CRLF
+to separate the headers from the body, which is empty. Whew! With this text,
+we've got a successful, tiny, HTTP response. We have to write it to the stream
+though! Let's modify our code to write out a response. Remove the `println!`
+line, and add these below:
+
+```rust,ignore
+let response = "HTTP/1.1 200 OK\r\n\r\n";
+
+stream.write(response.as_bytes()).unwrap();
+stream.flush().unwrap();
+```
+
+The first line defines our response. Then, we call `as_bytes` on our
+`response`, as the `write` method on `stream` takes a `&[u8]`, and writes those
+bytes directly down the connection. This could fail, so `write` returns a
+`Result<T, E>`; we continue to use `unwrap` to make progress here. Finally,
+`flush()` will wait until all of the underlying bytes are written to the
+connection; `TcpStream` contains an internal buffer to minimize calls into the
+underlying operating system.
+
+With these changes, let's run our code!
+
+```text
+> cargo run
+   Compiling hello v0.1.0 (file:///projects/hello/src/hello)
+    Finished dev [unoptimized + debuginfo] target(s) in 0.39 secs
+     Running `target\debug\hello.exe`
+```
+
+Once we've loaded `127.0.0.1:8080` in our web browser... we get a blank page!
+How exciting! You've just hand-coded an HTTP request and response. From here on
+out, it's all just details.
+
+### Returning Real HTML
+
+Let's return more than a blank page. Create a new file, `hello.html`, in the
+root of the project; that is, not in the `src` directory. You can put any HTML
+you want in it, here's what the authors used for theirs:
+
+```html
+<!DOCTYPE html>
+<html lang="en">
+  <head>
+    <meta charset="utf-8">
+    <title>Hello!</title>
+  </head>
+  <body>
+  <h1>Hello!</h1>
+  <p>Hi from Rust</p>
+  </body>
+</html>
+```
+
+This is a minimal HTML 5 document, with a heading and a little paragraph. Let's
+modify `handle_connection` to read that file, append it to our header, and send
+it as the response:
+
+```rust,ignore
+// add this import at the top
+use std::fs::File;
+
+// our new handle_connection
+fn handle_connection(mut stream: TcpStream) {
+    let mut buffer = [0; 512];
+    stream.read(&mut buffer).unwrap();
+
+    let mut file = File::open("hello.html").unwrap();
+    let mut contents = String::new();
+
+    file.read_to_string(&mut contents).unwrap();
+
+    let header = "HTTP/1.1 200 OK\r\n\r\n";
+    let response = format!("{}{}", header, contents);
+
+    stream.write(response.as_bytes()).unwrap();
+    stream.flush().unwrap();
+}
+```
+
+Here's opening and reading the file:
+
+```rust,ignore
+let mut file = File::open("hello.html").unwrap();
+let mut contents = String::new();
+
+file.read_to_string(&mut contents).unwrap();
+```
+
+We talked about this in the I/O project chapter, so this should look fairly
+familiar. We open the file with `File::open`, and the read it into a `String`
+with `file.read_to_string`.
+
+Next, we write our response out:
+
+```rust,ignore
+let header = "HTTP/1.1 200 OK\r\n\r\n";
+let response = format!("{}{}", header, contents);
+
+stream.write(response.as_bytes()).unwrap();
+stream.flush().unwrap();
+```
+
+We use `format!` to concatenate our header onto the body, and then change
+`write` to write `response`. Easy! Run it with `cargo run`, load up
+`127.0.0.1:8080` in your browser, and you should see your HTML rendered!

second-edition/src/ch20-04-validating-the-request.md

@@ -0,0 +1,159 @@
+## Validating the Request
+
+Right now, our web server will return this HTML no matter what the request.
+Let's check that the browser is requesting `/`, and then return an error if
+it's not. First, modify `handle_connection` to look like this:
+
+```rust,ignore
+fn handle_connection(mut stream: TcpStream) {
+    let mut buffer = [0; 512];
+    stream.read(&mut buffer).unwrap();
+
+    let get = b"GET / HTTP/1.1\r\n";
+
+    let start = &buffer[..get.len()];
+
+    if start == get {
+        // success!
+    } else {
+        // error :(
+    };
+```
+
+Here, we defined the HTTP request we're looking for with `get`. Because we are
+reading raw bytes into the buffer, we use a byte string, with `b"`, to make
+this a byte string too. Then, we take a slice of the `buffer` that's the same
+length as `get`, and compare them. If they're identical, we've gotten a good
+request. If not, we've gotten a bad request.
+
+Let's add in the code to handle each side:
+
+```rust,ignore
+if start == get {
+    let header = "HTTP/1.1 200 OK\r\n\r\n";
+    let mut file = File::open("hello.html").unwrap();
+    let mut contents = String::new();
+
+    file.read_to_string(&mut contents).unwrap();
+
+    let response = format!("{}{}", header, contents);
+
+    stream.write(response.as_bytes()).unwrap();
+    stream.flush().unwrap();
+} else {
+    let header = "HTTP/1.1 404 NOT FOUND\r\n\r\n";
+    let mut file = File::open("404.html").unwrap();
+    let mut contents = String::new();
+
+    file.read_to_string(&mut contents).unwrap();
+
+    let response = format!("{}{}", header, contents);
+
+    stream.write(response.as_bytes()).unwrap();
+    stream.flush().unwrap();
+};
+```
+
+The interesting bit is in the else case:
+
+```rust,ignore
+let header = "HTTP/1.1 404 NOT FOUND\r\n\r\n";
+let mut file = File::open("404.html").unwrap();
+```
+
+`404 NOT FOUND` is the proper error code here. And we need to make a new file,
+`404.html`, to go along with `hello.html`. Here's its contents:
+
+```html
+<!DOCTYPE html>
+<html lang="en">
+  <head>
+    <meta charset="utf-8">
+    <title>Hello!</title>
+  </head>
+  <body>
+  <h1>Oops!</h1>
+  <p>Sorry, I don't know what you're asking for.</p>
+  </body>
+</html>
+```
+
+With these changes, try running your server again. Requesting `127.0.0.1:8080`
+should return our `hello.html`, and any other request, like
+`127.0.0.1:8080/foo`, should return our error!
+
+There's a lot of repetition in this function; let's pull it out:
+
+```rust,ignore
+   let (header, filename) = if start == get {
+        ("HTTP/1.1 200 OK\r\n\r\n", "hello.html")
+    } else {
+        ("HTTP/1.1 404 NOT FOUND\r\n\r\n", "404.html")
+    };
+
+    let mut file = File::open(filename).unwrap();
+    let mut contents = String::new();
+
+    file.read_to_string(&mut contents).unwrap();
+
+    let response = format!("{}{}", header, contents);
+
+    stream.write(response.as_bytes()).unwrap();
+    stream.flush().unwrap();
+}
+```
+
+Here, the only thing in our `if` is the header and the filename; we then use
+destructuring to assign these two bits to `filename` and `header`. We have to
+change the call to `File::open` to use this new variable.
+
+Here's our final code. Don't forget those two HTML files as well!
+
+```rust,ignore
+use std::fs::File;
+use std::io::prelude::*;
+use std::net::TcpListener;
+use std::net::TcpStream;
+
+fn main() {
+    let listener = TcpListener::bind("127.0.0.1:8080").unwrap();
+
+    for stream in listener.incoming() {
+        let stream = stream.unwrap();
+
+        handle_connection(stream);
+    }
+}
+
+// our new handle_connection
+fn handle_connection(mut stream: TcpStream) {
+    let mut buffer = [0; 512];
+    stream.read(&mut buffer).unwrap();
+
+    let get = b"GET / HTTP/1.1\r\n";
+
+    let start = &buffer[..get.len()];
+
+
+    let (header, filename) = if start == get {
+        ("HTTP/1.1 200 OK\r\n\r\n", "hello.html")
+    } else {
+        ("HTTP/1.1 404 NOT FOUND\r\n\r\n", "404.html")
+    };
+
+    let mut file = File::open(filename).unwrap();
+    let mut contents = String::new();
+
+    file.read_to_string(&mut contents).unwrap();
+
+    let response = format!("{}{}", header, contents);
+
+    stream.write(response.as_bytes()).unwrap();
+    stream.flush().unwrap();
+}
+```
+
+Awesome! We have a simple little web server in ~40 lines of Rust code. So far,
+this project has been relatively straightforward as far as Rust code goes; we
+haven't done much of the more advanced things yet. Let's kick it up a notch and
+add a feature to our web server: a thread pool.