You can set the property to directly kill the executor when it is blacklisted like this:
val sc = new SparkContext(new SparkConf())
./bin/spark-submit <all your existing options> --spark.blacklist.killBlacklistedExecutors=true
Now whenever an executor is blacklisted, it will be killed.