53613/removing-the-header-of-a-text-file-in-sparkrdd
1) First we loaded the data to an RDD, and we perform collect and gives the header.
2) Now we will filter the header and then use the command to form an RDD without header.
Hi@Manas, You can read your dataset from CSV ...READ MORE
You can select the column and apply ...READ MORE
Hey, You can try this: from pyspark import SparkContext SparkContext.stop(sc) sc ...READ MORE
Source tags are different: { x : [ { ...READ MORE
Instead of spliting on '\n'. You should ...READ MORE
Firstly you need to understand the concept ...READ MORE
org.apache.hadoop.mapred is the Old API org.apache.hadoop.mapreduce is the ...READ MORE
Hi, You can create one directory in HDFS ...READ MORE
Yes, you can go ahead and write ...READ MORE
Save it to a text file: line.saveAsTextFile("alicia.txt") Print contains ...READ MORE
OR
At least 1 upper-case and 1 lower-case letter
Minimum 8 characters and Maximum 50 characters
Already have an account? Sign in.