How To Best Implement Multiprocessing In Python?

2019 has been a very big year for technological development all around the globe. Starting from computer manufacturers adding more cores to their CPUs and processors to the launch of smarter chips in mobile phones, multiprocessing is no longer a dream. Today, one of the most prominent programming languages that supports Multiprocessing is Python. Thanks to its introduction of parallel processing power in its release, coders around the world can now seamlessly create codes to be executed simultaneously and thus their run time by a significant amount. 

Following pointers will be covered in this article,

• What is Multiprocessing?
• Code For Multiprocessing In Python
• Example 1
• Example 2

Let us begin,

Multiprocessing In Python

What is Multiprocessing?

Multiprocessing can simply be defined as the ability of a system to support more than one operation at any given instance. What this means is that applications in multiprocessing systems are broken down into small pieces and then run independently of each other in order to increase efficiency and reduce the overall run time. The processor in the system allocates each small piece an independent thread of its own, thus allowing it to function as a standalone entity. 

The Need for Multiprocessing

Imagine a computer system which only has one core in its processor. If multiple tasks are assigned to this single core, then it has break each task in between and then switch over to the next. This will not increase the amount of time that is needed to complete each task, but will also reduce the overall efficiency of the system. 

On the other hand a multiprocessing computer, can have one processor which in turn has multiple functioning units inside it, called individual cores which have the ability to run several different tasks independently at the same time. This not only increases the efficiency of the system but also in the long run reduces the runtime of the system by a significant amount. 

A Multiprocessing system in Python can be of two types. 

Multi Processor System

This system basically has multiple processors present under its hood, each of which can perform one task at a time and function as an independent component. 

Multi-core Processor System

This system has multiple cores present in the same processor, within which core functions as a separate unit and performs tasks independently that are assigned to it. 

Code For Multiprocessing In Python

Now that you are accustomed to the basic concept of multiprocessing, let us explore how multiprocessing can be achieved in Python. 

In Python, the interpreter contains a very simple and intuitive API which takes a single task, breaks it down into multiple components and gets them processed independently. 

Take a look at the program module below to understand this concept of Multiprocessing in python better.

Example 1

# importing the multiprocessing module
import multiprocessing
def print_cube(num):
"""
function to print cube of given num
"""
print("Cube: {}".format(num * num * num))
def print_square(num):
"""
function to print square of given num
"""
print("Square: {}".format(num * num))
if __name__ == "__main__":
# creating processes
p1 = multiprocessing.Process(target=print_square, args=(10, ))
p2 = multiprocessing.Process(target=print_cube, args=(10, ))
# starting process 1
p1.start()
# starting process 2
p2.start()
# wait until process 1 is finished
p1.join()
# wait until process 2 is finished
p2.join()
# both processes finished
print("Done!") 

Output

Square: 100

Cube: 1000

Done!

Now let us analyze this program to understand it better.

1. The first step is to import the multiprocessing module. In order to do this, make use of the following syntax: import multiprocessing.

2. Now that the multiprocessing module has been imported, let us move ahead and create a process. In order to do this, we create an object of Process class and assign it the following arguments. Target: the function that needs to be executed by this process and args: the arguments that needs to be passed to the target function. 

Note: A process constructor has the ability to take on multiple targets as well as arguments; but in the above example we have assigned our Process only two targets and arguments as shown below. 

p1 = multiprocessing.Process(target=print_square, args=(10, ))

p2 = multiprocessing.Process(target=print_cube, args=(10, ))

1. Now that the process has been created, let us write the syntax for starting the same. 

p1.start()

p2.start()

Once the process is started, the current program and the already executing one are being run simultaneously. If in a certain situation, you need to stop the execution of the current program and only focus on the execution of the pre existing one, we make use of the join function as shown below. 

p1.join()

p2.join()

Once you input this syntax, the interpreter will wait for the program p1 to finish execution and then move on to program p2.

To understand this concept further, take a look at another example for Multiprocessing In Python below. 

Example 2

# importing the multiprocessing module
import multiprocessing
import os
def worker1():
# printing process id
print("ID of process running worker1: {}".format(os.getpid()))
def worker2():
# printing process id
print("ID of process running worker2: {}".format(os.getpid()))
if __name__ == "__main__":
# printing main program process id
print("ID of main process: {}".format(os.getpid()))
# creating processes
p1 = multiprocessing.Process(target=worker1)
p2 = multiprocessing.Process(target=worker2)
# starting processes
p1.start()
p2.start()
# process IDs
print("ID of process p1: {}".format(p1.pid))
print("ID of process p2: {}".format(p2.pid))
# wait until processes are finished
p1.join()
p2.join()
# both processes finished
print("Both processes finished execution!")
# check if processes are alive
print("Process p1 is alive: {}".format(p1.is_alive()))
print("Process p2 is alive: {}".format(p2.is_alive())) 

Output

ID of main process: 18938

ID of process running worker1: 18939

ID of process running worker2: 18940

ID of process p1: 18939

ID of process p2: 18940

Both processes finished execution!

Process p1 is alive: False

Process p2 is alive: False

Notice in the above program that both process p1 and p2 run independently of their individual memories. As soon as both have finished execution, the program is terminated.

This brings us to the end of this article on Multiprocessing in Python

To get in-depth knowledge on Python along with its various applications, you can enroll here for live online training with 24/7 support and lifetime access.

Got a question for us? Mention them in the comments section of  article and we will get back to you.