How to train a Keras model on multiple GPUs

Question

Hi Guys,

I am new to Keras. I want to run the Mask-RCNN model using the Keras module. But it is taking lots of time to execute the code. Is there any way so that I can train the Keras model on Multiple GPUs?

Answer 1

Hi@akhtar,

There are two ways to run a single model on multiple GPUs, data parallelism and device parallelism. In most cases, what you need is most likely data parallelism.

1) Data parallelism

Data parallelism consists of replicating the target model once on each device and using each replica to process a different fraction of the input data. The best way to do data parallelism with Keras models is to use the tf.distribute API.

2) Model parallelism

Model parallelism consists of running different parts of the same model on different devices. It works best for models that have a parallel architecture, e.g. a model with two branches. This can be achieved by using TensorFlow device scopes.

Ready to dive into the world of machine learning? Supercharge your career and join our comprehensive Machine Learning Course today!

Answer 2

Hello,

One of the biggest hangups with Keras is that it can be a pain to perform multi-GPU training. Between the boilerplate code and configuring TensorFlow it can be a bit of a process…

…but not anymore.

With the latest commit and release of Keras (v2.0.9) it’s now extremely easy to train deep neural networks using multiple GPUs.

In fact, it’s as easy as a single function call!

Thanks.

Answer 3

Hello there,

With the latest commit and release of Keras (v2.0.9) it’s now extremely easy to train deep neural networks using multiple GPUs.