23.1What async is, and when you want it
Chapter 22 ran code on many threads to do more computing at once. This chapter is about a different problem that looks similar but isn't: handling a great many things that spend most of their time waiting. A web server with ten thousand open connections isn't doing ten thousand computations, it's mostly waiting for ten thousand slow networks to send the next byte. Async ("asynchronous") is the model built for that, and it's one of the most useful tools in modern Rust. It's also, we'll be honest from the start, the corner of the language where the seams show. This lesson is about understanding when you actually want async, because reaching for it reflexively is the most common async mistake.
Two kinds of slow
Work is slow for one of two fundamentally different reasons, and the distinction decides everything.
CPU-bound work is slow because there's a lot of computing to do: resizing a million images, simulating physics, hashing a huge file. The processor is busy the whole time. To go faster, you need more processors working at once, which is threads (chapter 22). Async does nothing for CPU-bound work; the CPU is already the bottleneck.
IO-bound work is slow because it's waiting for something outside the program: a network response, a disk read, a database query, a timer. The processor is mostly idle, twiddling its thumbs until the data arrives. Here, "more processors" doesn't help, the CPU isn't the limit, the waiting is. What helps is not blocking a whole thread while you wait. That's async.
The one-line test: is this slow because the CPU is busy, or because it's waiting? Busy means threads; waiting means async. Get this distinction right and the rest of the chapter is detail; get it wrong and you'll reach for the wrong tool.
Threads versus tasks
You could handle many waiting connections with threads: one thread per connection, each blocking on its slow network. And for a few hundred connections, that's fine, do that, it's simpler. The problem is scale. A thread is a relatively heavy operating-system object: it reserves a chunk of memory for its stack (often megabytes) and the OS spends real effort scheduling it. Ten thousand threads means gigabytes of stacks and a scheduler straining under the load, most of it wasted on threads that are just waiting.
Async replaces the thread-per-connection model with tasks. A task is a lightweight unit of waiting-work, far cheaper than a thread, thousands of bytes, not megabytes. A small number of real OS threads (often one per CPU core) runs a large number of tasks by a simple trick: whenever a task hits a point where it must wait, it steps aside and lets another ready task run on that thread, instead of blocking the thread. When the awaited thing is ready, the task resumes. So a handful of threads can juggle tens of thousands of waiting tasks, because at any instant most tasks are parked waiting and only the few that are ready need the CPU.
Key insight
Async is concurrency without a thread for each concurrent thing. The whole point is to handle many simultaneous waits cheaply: tasks that are waiting cost almost nothing and don't tie up a thread, so one thread can make progress on whichever tasks are actually ready. This is why async dominates network servers (thousands of mostly-idle connections) and does nothing for a tight computational loop (one task, never waiting, CPU pinned). Match the tool to the bottleneck: waiting → tasks (async); computing → threads.
When NOT to use async
This is the part most tutorials skip, and it's the most important. Async has real costs, which the rest of this chapter will make concrete: it's more complex, it requires pulling in a runtime (an external dependency), it "colors" your functions in a way that spreads through your code, and its error messages can be rough. Those costs are worth paying when you have the problem async solves, and pure overhead when you don't.
Most command-line tools should not be async. A program that reads a file, processes it, and writes a result waits for one or two things, briefly, sequentially. Async buys it nothing and costs it complexity. The same goes for CPU-bound work (use threads or rayon) and for anything handling just a handful of concurrent operations (plain threads are simpler). Reach for async when you're handling many concurrent IO-bound operations at once, the classic case being a network service, and prefer the simpler tools everywhere else.
Async is not a performance upgrade you sprinkle on
A common misconception is that async makes code faster, so you should make everything async. It doesn't, and you shouldn't. Async makes waiting cheaper to do at scale; it makes a single sequential computation slightly slower and more complex. Adding async/await to a CPU-bound or low-concurrency program is a pure loss: more dependencies, harder errors, no benefit. Choose async deliberately, for many-concurrent-waits workloads, not as a default. The honest framing throughout this chapter is that async is a specialized tool, not an upgrade.
What's coming
With the when settled, the rest of the chapter covers the how, and stays honest about the rough edges. Lesson 23.2 introduces the async/await syntax and the Future type that underlies it, including why you can't just call an async function like a normal one. Lesson 23.3 brings in Tokio, the runtime that actually runs async code, because, unlike threads, async needs an engine you supply. And lesson 23.4 is a field guide to the specific ways async bites: blocking the executor, Send troubles, cancellation, the seams, named plainly so you can step around them.
Quiz time
Question #1
What's the difference between CPU-bound and IO-bound work, and which one is async for?
Show solution
CPU-bound work is slow because the processor has a lot of computing to do (the CPU is busy); it's sped up by more processors working at once, i.e. threads. IO-bound work is slow because it's waiting for something external (network, disk, timer) while the CPU is mostly idle; here more processors don't help, not blocking a thread during the wait does. Async is for IO-bound work, the waiting kind.
Question #2
Why can a handful of OS threads handle tens of thousands of async tasks, when tens of thousands of threads would be impractical?
Show solution
Threads are heavyweight, each reserves a large stack (often megabytes) and burdens the OS scheduler, so tens of thousands waste enormous memory and scheduling effort, mostly on threads that are just waiting. Tasks are lightweight (kilobytes). When a task must wait, it steps aside and lets another ready task run on the same thread instead of blocking it. Since most tasks are waiting at any moment, a few threads can drive many tasks by always running whichever ones are ready.
Question #3
Give two kinds of program that should not be written with async, and why.
Show solution
(1) CPU-bound programs (image processing, simulation): the bottleneck is computation, not waiting, so async helps nothing, use threads or rayon. (2) Simple command-line tools and low-concurrency programs (read a file, process, write): they wait briefly and sequentially, so async adds complexity, a runtime dependency, and harder errors for no benefit. Async is worth its costs only for many concurrent IO-bound operations, typically network services.
You know when to reach for async. The next lesson (23.2) shows the syntax, async/await, and the lazy Future it produces, including the first surprise: an async function doesn't run when you call it.