Sharing data between processes

When working with threads or coroutines, data is shared across them by virtue of the fact that they share the same memory space. So, you can access any object from any thread, as long as attention is paid to avoiding race conditions and providing proper locking.

With processes, instead, things get far more complicated and no data is shared across them. So when using ProcessPool or ProcessPoolExecutor, we need to find a way to pass data across the processes and make them able to share a common state.

The Python standard library provides many tools to create a communication channel between processes: multiprocessing.Queues, multiprocessing.Pipe, multiprocessing.Value, and multiprocessing.Array can be used to create queues that one process can feed and the other consume, or simply values shared between multiple processes in a shared memory.

While all these are viable solutions, they have some limits: you must create all shared values before creating any process,...