Why Partitions are immutable in Spark

0 votes

Can anyone explain why partition are immutable in spark?

Jul 3, 2019 in Apache Spark by neha
recategorized Jul 4, 2019 by Gitika • 2,294 views

1 answer to this question.

0 votes

Hi,

Every transformation generates a new partition. Partitions use HDFS API so that partition is immutable, distributed and fault tolerance. Partition also aware of data locality.

answered Jul 3, 2019 by Gitika
 • 65,770 points
0 votes
Partitions use HDFS API.
answered Aug 25, 2022 by anonymous
edited Mar 5

Related Questions In Apache Spark

0 votes
1 answer

What are the levels of parallelism in spark streaming ?

> In order to reduce the processing ...READ MORE

answered Jul 27, 2018 in Apache Spark by zombie
 • 3,790 points • 5,185 views
0 votes
0 answers

what are the memory issues in spark ?

Mar 18, 2019 in Apache Spark by satish kumar
 • 180 points • 2,253 views
0 votes
1 answer

what are the job optimization Technics in spark and scala ?

There are different methods to achieve optimization ...READ MORE

answered Mar 18, 2019 in Apache Spark by Veer
 • 2,548 views
0 votes
1 answer

Changing Column position in spark dataframe

Yes, you can reorder the dataframe elements. You need ...READ MORE

answered Apr 19, 2018 in Apache Spark by Ashish
 • 2,650 points • 14,123 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
 • 11,380 points • 11,331 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
 • 11,380 points • 110,632 views
–1 vote
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
 • 9,350 points • 4,779 views
0 votes
1 answer

Is there a way to copy data from one one Hadoop distributed file system(HDFS) to another HDFS?

The distributed copy command, distcp, is a ...READ MORE

answered Mar 22, 2018 in Big Data Hadoop by Ashish
 • 2,650 points • 9,938 views
+1 vote
1 answer

By default how many partitions are created in RDD in Apache spark?

Well, it depends on the block of ...READ MORE

answered Aug 2, 2019 in Apache Spark by Gitika
 • 65,770 points • 2,965 views
0 votes
1 answer

What are some of the things you can monitor in the Spark Web UI?

The stages which are running slow READ MORE

answered Apr 29, 2021 in Apache Spark by anonymous
edited Mar 5 • 3,989 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP
"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.