By which components spark ecosystem libraries are composed of

Question

Can anyone suggest which component spark ecosystem libraries are composed of?

Answer 1

Hi,

Spark ecosystem libraries are composed of various components like Spark SQL, Spark Streaming, and ML libraries.

• Spark SQL: It is used to elaborate on the power of declarative queries, optimize storage by executing SQL queries like spark data which is present in RDD and other external sources.
• Spark Streaming: It allows developers to perform batch processing and streaming of data in the same application. 
• ML Libraries: It uses the deployment and development of scalable ML pipelines like Coo-relation, features extraction, transformation function, optimization algorithm, etc.