What exactly makes Java Virtual Threads better?
One big advantage of coroutines (so virtual threads) is that they can generate high levels of concurrency without the drawback of callbacks.
Let me first introduce Little's Law:
concurrency = arrival_rate * latency
And we can rewrite this to:
arrival_rate = concurrency/latency
In a stable system, the arrival rate equals throughput.
throughput = concurrency/latency
To increase throughput, you have 2 options:
- decrease latency; which typically is very hard since you have little influence on how much time a remote call or a request to disk takes.
- increase concurrency
With regular threads, it is difficult to reach high levels of concurrency with blocking calls due to context switch overhead. Requests can be issued asynchronously in some cases (e.g. NIO + Epoll or Netty io_uring binding), but then you need to deal with callbacks and callback hell.
With a virtual thread, the request can be issued asynchronously and park the virtual thread and schedule another virtual thread. Once the response is received, the virtual thread is rescheduled and this is done completely transparently. The programming model is much more intuitive than using classic threads with callbacks.
Virtual Threads Advantages
- exhibits exact the same behavior as platform threads.
- disposable and can be scaled to millions.
- much more lightweight than platform threads.
- fast creation time, as fast as creating string object.
- the JVM does delimited continuation on IO operations, no IO for virtual threads.
- yet can have the sequential code as previous but way more effective.
- the JVM gives an illusion of virtual threads, underneath whole story goes on platform threads.
- Just with usage of virtual thread CPU core become much more concurrent, the combination of virtual threads and multi core CPU with ComputableFutures to parallelized code is very powerful
Virtual Threads Usage Cautions
Don not use monitor i.e the synchronized block, however this will fix in new release of JDK's, an alternative to do so is to use 'ReentrantLock' with try-final statement.
Blocking with native frames on stack, JNI's. its very rare
Control memory per stack (reduce thread locales and no deep recursion)
Monitoring tools not updated yet like debuggers, JConsole, VisualVM etc
Platform Threads versus Virtual threads. Platform threads take OS threads hostage in IO based tasks and operations limited to number of applicable threads with in thread pool and OS threads, by default they are non Daemon threads
Virtual threads are implemented with JVM, in CPU bound operations the associated to platform threads and retuning them to thread pool, after IO bound operation finished a new thread will be called from thread pool, so no hostage in this case.
Sometimes people have to build systems able to handle an enormous number of simultaneous clients. Native threads are inadequate means for doing that due to RAM consumption and context switching costs.
Virtual threads give us an ability to run millions of I/O bound tasks simultaneously without changing our mental model.
That's why Golang made its way into the industry (besides Google support). Goroutines are a concept very similar to Java's virtual threads and they solve the same problem.
There are other ways to achieve what virtual thread do (such as NIO and the related Reactor pattern). This, however, entails using message loops and callbacks which warp your mind (that's why so many people hate JavaScript). There are layers of abstractions on top of them making things a bit easier but they also have a cost.