Need to disable unpersist in Spark

Question

I want the raw data and RDDs to be outside the Spark application. But by default, Spark unpersists the data which makes the data inaccessible outside the application. I want to disable this unpersisting function but I don't know how. Please help

Answer 1

You can dynamically change this function by running the following command in the Spark Shell:

val sc = new SparkContext(new SparkConf())

./bin/spark-submit <all your existing options> --spark.streaming.unpersist=false