How to handle exceptions in Spark and Scala

Question

Could you please help me to understand exceptions in Scala and Spark. And what are the common exceptions that we need to handle while writing spark code?
 

Answer 1

There is no particular format to handle exception caused in spark. There are a couple of exceptions that you will face on everyday basis, such as StringOutOfBoundException/FileNotFoundException which actually explains itself like if the number of columns mentioned in the dataset is more than number of columns mentioned in dataframe schema then you will find a StringOutOfBoundException or if the dataset path is incorrect while creating an rdd/dataframe then you will face FileNotFoundException.

So, in short, it completely depends on the type of code you are executing or mistakes you are going to commit while coding them. That is why we have interpreter such as spark shell that helps you execute the code line by line to understand the exception and get rid of them a little early.

Hope this helps!

To know more about Spark Scala, It's recommended to join Apache Spark training online today.

Thanks!!