No history yet

Virtual Threads Mastery

The Scaling Problem with Threads

For years, the 'thread-per-request' model has been an anti-pattern in high-concurrency Java applications. The reason is simple: platform threads are expensive. Each one is a wrapper around an operating system thread, consuming significant memory and incurring a heavy context-switching penalty. A server could only handle a few thousand at most before performance would grind to a halt.

This forced developers into complex, asynchronous programming models with callbacks and reactive frameworks to manage high throughput, often at the cost of readability and debugging simplicity.

Virtual threads, introduced as part of and refined in modern JDKs, flip this paradigm on its head. They are lightweight threads managed by the JVM, not the OS. Instead of a one-to-one mapping, millions of virtual threads can run on just a handful of OS threads. This makes the simple, readable 'thread-per-request' model not just viable, but highly efficient for I/O-bound applications.

Carrier Threads at Work

The magic behind virtual threads lies in their relationship with a small pool of platform threads known as carrier threads. Think of carrier threads as the actual workers and virtual threads as the individual tasks they perform. The JVM has a scheduler that intelligently assigns (or "mounts") a virtual thread onto a carrier thread to execute its code.

The game-changer occurs when a virtual thread encounters a blocking I/O operation, like waiting for a database query or a network call. Instead of letting the carrier thread sit idle, the JVM automatically "unmounts" the blocked virtual thread and mounts a different, ready-to-run virtual thread in its place. The original virtual thread is parked until its I/O operation completes, at which point it becomes eligible to be mounted again on any available carrier.

This mounting/unmounting happens transparently, allowing a small number of carrier threads (by default, one per CPU core) to service a huge number of concurrent tasks. This is what gives virtual threads their immense scalability for workloads dominated by waiting.

Virtual threads don’t make the CPU work faster. They shine for I/O-bound workloads.

The easiest way to start using virtual threads is with the Executors.newVirtualThreadPerTaskExecutor(). This factory method creates an ExecutorService that starts a new virtual thread for each submitted task. Unlike traditional fixed-size thread pools, this executor doesn't pool threads because creating virtual threads is cheap. It simply creates them on demand.

// Recommended way to use virtual threads
// The try-with-resources ensures the executor is shut down
try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
    // Submit 100,000 tasks. Each runs in a new virtual thread.
    for (int i = 0; i < 100_000; i++) {
        int taskNumber = i;
        executor.submit(() -> {
            // Simulate a blocking I/O operation
            Thread.sleep(Duration.ofSeconds(1));
            System.out.println("Task " + taskNumber + " complete.");
            return taskNumber;
        });
    }
} // executor.close() is called automatically

Pinning and Other Pitfalls

While powerful, virtual threads are not a silver bullet. Their effectiveness depends on the ability to unmount from the carrier during blocking operations. An issue known as thread pinning can prevent this. Pinning occurs when a virtual thread executes code that cannot be safely moved off its carrier. This effectively monopolizes the carrier, defeating the purpose of the lightweight model.

The two most common causes of pinning are:

  1. synchronized blocks: When a virtual thread enters a synchronized block or method, it becomes pinned to its carrier thread. It will not be unmounted, even if it blocks. The modern replacement, java.util.concurrent.locks.ReentrantLock, is virtual-thread-friendly and should be preferred.

  2. Native Methods (JNI): Executing native code or certain foreign functions can also pin the thread. The JVM cannot safely unmount a thread in the middle of a native call.

The JDK is continuously working to reduce pinning scenarios, but it's crucial for developers to be aware of these pitfalls. The key is to avoid extensive or long-running operations inside synchronized blocks when using virtual threads.

Quiz Questions 1/5

What is the primary reason the traditional 'thread-per-request' model was considered an anti-pattern for high-concurrency Java applications?

Quiz Questions 2/5

In the context of virtual threads, what is a 'carrier thread'?

By understanding the interplay between virtual and carrier threads, and by avoiding patterns that lead to pinning, you can leverage virtual threads to build highly scalable and maintainable applications.