Rearrange motivation/introduction of thread pool a bit

This commit is contained in:
Carol (Nichols || Goulding)
2017-04-18 09:39:34 -04:00
parent 0ad425f372
commit 6e631237a1
2 changed files with 45 additions and 37 deletions

View File

@@ -588,6 +588,6 @@ Awesome! We have a simple little web server in about 40 lines of Rust code that
responds to one request with a page of content and responds to all other
requests with a `404` response.
So far, this project has been relatively straightforward as far as Rust code
goes; we haven't done much of the more advanced things yet. Let's kick it up a
notch and add a feature to our web server: a thread pool.
Since this server runs in a single thread, though, it can only serve one
request at a time. Let's see how that can be a problem by simulating some
slow requests.

View File

@@ -3,39 +3,12 @@
Right now, the server will process each request in turn. That works for
services like ours that aren't expected to get very many requests, but as
applications get more complex, this sort of serial execution isn't optimal.
Let's make our web server better by adding a *thread pool*. A thread pool is a
group of spawned threads that are ready to handle some task. When the program
receives a new task, one of the threads in the pool will be assigned the task
and will go off and process it. The remaining threads in the pool are available
to handle any other tasks that come in while the first thread is processing.
When the first thread is done processing its task, it gets returned to the pool
of idle threads ready to handle a new task.
Because our current program processes connections sequentially, it won't
process a second connection until it's completed processing the first. If we
get one request that takes a long time to process, requests coming in during
that time will have to wait until the long request is finished, even if the new
requests can be processed quickly. A thread pool allows us to process
connections concurrently: we can start processing a new connection before an
older connection is finished. This increases the throughput of our server.
Here's what we're going to implement: instead of waiting for each request to
process before starting on the next one, we'll send the processing of each
connection to a different thread. The threads will come from a pool of four
threads that we'll spawn when we start our program. The reason we're limiting
the number of threads to a small number is that if we created a new thread for
each request as the requests come in, someone making ten million requests to
our server could create havoc by using up all of our server's resources and
grinding the processing of all requests to a halt.
Instead, we'll create a pool of threads with a fixed size of our choosing. As
requests come in, we'll send them to the pool for processing. The pool will
maintain a queue of incoming requests. Each of the threads in the pool will pop
a request off of this queue, handle the request, and then ask the queue for
another request. With this design, we can process `N` requests concurrently,
where `N` is the number of threads. This still means that `N` long-running
requests can cause requests to back up in the queue, but we've increased the
number of long-running requests we can handle before that point from one to `N`.
requests can be processed quickly. Let's see this in action.
### Simulating a Slow Request in the Current Server Implementation
@@ -97,12 +70,47 @@ Start the server with `cargo run`, and then open up two browser windows: one
for `http://localhost:8080/` and one for `http://localhost:8080/sleep`. If
you hit `/` a few times, as before, you'll see it respond quickly. But if you
hit `/sleep`, and then load up `/`, you'll see that `/` waits until `sleep`
has slept for its full five seconds before going on. This is the issue we can
improve with our thread pool.
has slept for its full five seconds before going on.
There are multiple ways we could change how our web server works in order to
avoid having all requests back up behind a slow request; the one we're going to
implement is a thread pool.
### Improving Throughput with a Thread Pool
A *thread pool* is a group of spawned threads that are ready to handle some
task. When the program receives a new task, one of the threads in the pool will
be assigned the task and will go off and process it. The remaining threads in
the pool are available to handle any other tasks that come in while the first
thread is processing. When the first thread is done processing its task, it
gets returned to the pool of idle threads ready to handle a new task.
A thread pool will allow us to process connections concurrently: we can start
processing a new connection before an older connection is finished. This
increases the throughput of our server.
Here's what we're going to implement: instead of waiting for each request to
process before starting on the next one, we'll send the processing of each
connection to a different thread. The threads will come from a pool of four
threads that we'll spawn when we start our program. The reason we're limiting
the number of threads to a small number is that if we created a new thread for
each request as the requests come in, someone making ten million requests to
our server could create havoc by using up all of our server's resources and
grinding the processing of all requests to a halt.
Rather than spawning unlimited threads, we'll have a fixed number of threads
waiting in the pool. As requests come in, we'll send the requests to the pool
for processing. The pool will maintain a queue of incoming requests. Each of
the threads in the pool will pop a request off of this queue, handle the
request, and then ask the queue for another request. With this design, we can
process `N` requests concurrently, where `N` is the number of threads. This
still means that `N` long-running requests can cause requests to back up in the
queue, but we've increased the number of long-running requests we can handle
before that point from one to `N`.
This design is one of many ways to improve the throughput of our web server.
This isn't a book about web servers, though, so it's the one we're going to
cover. Other options are the "fork/join" model, and the "single threaded async
I/O" model. If you're interested in this topic, you may want to read more about
them and try to implement them in Rust; with a low-level language like Rust,
all of these options are possible.
cover. Other options are the fork/join model and the single threaded async I/O
model. If you're interested in this topic, you may want to read more about
other solutions and try to implement them in Rust; with a low-level language
like Rust, all of these options are possible.