From 6e631237a18af62cfdb2f0563e2d82d4099a6ff3 Mon Sep 17 00:00:00 2001 From: "Carol (Nichols || Goulding)" Date: Tue, 18 Apr 2017 09:39:34 -0400 Subject: [PATCH] Rearrange motivation/introduction of thread pool a bit --- second-edition/src/ch20-01-single-threaded.md | 6 +- second-edition/src/ch20-02-slow-requests.md | 76 ++++++++++--------- 2 files changed, 45 insertions(+), 37 deletions(-) diff --git a/second-edition/src/ch20-01-single-threaded.md b/second-edition/src/ch20-01-single-threaded.md index 8e712a529..627c4cbdb 100644 --- a/second-edition/src/ch20-01-single-threaded.md +++ b/second-edition/src/ch20-01-single-threaded.md @@ -588,6 +588,6 @@ Awesome! We have a simple little web server in about 40 lines of Rust code that responds to one request with a page of content and responds to all other requests with a `404` response. -So far, this project has been relatively straightforward as far as Rust code -goes; we haven't done much of the more advanced things yet. Let's kick it up a -notch and add a feature to our web server: a thread pool. +Since this server runs in a single thread, though, it can only serve one +request at a time. Let's see how that can be a problem by simulating some +slow requests. diff --git a/second-edition/src/ch20-02-slow-requests.md b/second-edition/src/ch20-02-slow-requests.md index 71a88bdcd..a1667386d 100644 --- a/second-edition/src/ch20-02-slow-requests.md +++ b/second-edition/src/ch20-02-slow-requests.md @@ -3,39 +3,12 @@ Right now, the server will process each request in turn. That works for services like ours that aren't expected to get very many requests, but as applications get more complex, this sort of serial execution isn't optimal. -Let's make our web server better by adding a *thread pool*. A thread pool is a -group of spawned threads that are ready to handle some task. When the program -receives a new task, one of the threads in the pool will be assigned the task -and will go off and process it. The remaining threads in the pool are available -to handle any other tasks that come in while the first thread is processing. -When the first thread is done processing its task, it gets returned to the pool -of idle threads ready to handle a new task. Because our current program processes connections sequentially, it won't process a second connection until it's completed processing the first. If we get one request that takes a long time to process, requests coming in during that time will have to wait until the long request is finished, even if the new -requests can be processed quickly. A thread pool allows us to process -connections concurrently: we can start processing a new connection before an -older connection is finished. This increases the throughput of our server. - -Here's what we're going to implement: instead of waiting for each request to -process before starting on the next one, we'll send the processing of each -connection to a different thread. The threads will come from a pool of four -threads that we'll spawn when we start our program. The reason we're limiting -the number of threads to a small number is that if we created a new thread for -each request as the requests come in, someone making ten million requests to -our server could create havoc by using up all of our server's resources and -grinding the processing of all requests to a halt. - -Instead, we'll create a pool of threads with a fixed size of our choosing. As -requests come in, we'll send them to the pool for processing. The pool will -maintain a queue of incoming requests. Each of the threads in the pool will pop -a request off of this queue, handle the request, and then ask the queue for -another request. With this design, we can process `N` requests concurrently, -where `N` is the number of threads. This still means that `N` long-running -requests can cause requests to back up in the queue, but we've increased the -number of long-running requests we can handle before that point from one to `N`. +requests can be processed quickly. Let's see this in action. ### Simulating a Slow Request in the Current Server Implementation @@ -97,12 +70,47 @@ Start the server with `cargo run`, and then open up two browser windows: one for `http://localhost:8080/` and one for `http://localhost:8080/sleep`. If you hit `/` a few times, as before, you'll see it respond quickly. But if you hit `/sleep`, and then load up `/`, you'll see that `/` waits until `sleep` -has slept for its full five seconds before going on. This is the issue we can -improve with our thread pool. +has slept for its full five seconds before going on. + +There are multiple ways we could change how our web server works in order to +avoid having all requests back up behind a slow request; the one we're going to +implement is a thread pool. + +### Improving Throughput with a Thread Pool + +A *thread pool* is a group of spawned threads that are ready to handle some +task. When the program receives a new task, one of the threads in the pool will +be assigned the task and will go off and process it. The remaining threads in +the pool are available to handle any other tasks that come in while the first +thread is processing. When the first thread is done processing its task, it +gets returned to the pool of idle threads ready to handle a new task. + +A thread pool will allow us to process connections concurrently: we can start +processing a new connection before an older connection is finished. This +increases the throughput of our server. + +Here's what we're going to implement: instead of waiting for each request to +process before starting on the next one, we'll send the processing of each +connection to a different thread. The threads will come from a pool of four +threads that we'll spawn when we start our program. The reason we're limiting +the number of threads to a small number is that if we created a new thread for +each request as the requests come in, someone making ten million requests to +our server could create havoc by using up all of our server's resources and +grinding the processing of all requests to a halt. + +Rather than spawning unlimited threads, we'll have a fixed number of threads +waiting in the pool. As requests come in, we'll send the requests to the pool +for processing. The pool will maintain a queue of incoming requests. Each of +the threads in the pool will pop a request off of this queue, handle the +request, and then ask the queue for another request. With this design, we can +process `N` requests concurrently, where `N` is the number of threads. This +still means that `N` long-running requests can cause requests to back up in the +queue, but we've increased the number of long-running requests we can handle +before that point from one to `N`. This design is one of many ways to improve the throughput of our web server. This isn't a book about web servers, though, so it's the one we're going to -cover. Other options are the "fork/join" model, and the "single threaded async -I/O" model. If you're interested in this topic, you may want to read more about -them and try to implement them in Rust; with a low-level language like Rust, -all of these options are possible. +cover. Other options are the fork/join model and the single threaded async I/O +model. If you're interested in this topic, you may want to read more about +other solutions and try to implement them in Rust; with a low-level language +like Rust, all of these options are possible.