How to write a HDF file from a DataFrame

0 votes

Hi Guys,

I am new to HDFS Cluster. I want to integrate Pandas with HDFS. Can anyone tell me how to write a HDF file from a DataFrame?

Oct 18, 2020 in Python by akhtar
 • 38,260 points • 1,295 views

1 answer to this question.

0 votes

Hi@akhtar,

Hierarchical Data Format (HDF) is a set of file formats (HDF4, HDF5) designed to store and organize large amounts of data. You can use the below command to do so.

df = pd.DataFrame([[1, 1.0, 'a']], columns=['x', 'y', 'z'])
df.to_hdf('./store.h5', 'data')
answered Oct 18, 2020 by MD
 • 95,460 points

Related Questions In Python

–1 vote
2 answers

In Python, How do I read 2 CSV files, compare column 1 from both, and then write to a new file where the Column 1s match?

Hi @Mike. First, read both the csv ...READ MORE

answered Jul 24, 2019 in Python by Omkar
 • 69,220 points • 33,442 views
+1 vote
4 answers

How to write nested dictionaries to a CSV file

Using DictWriter there is no need in ...READ MORE

answered Oct 18, 2018 in Python by Richard William
 • 27,718 views
0 votes
1 answer

How can I write nested dictionaries to a csv file?

You can use the pandas library to ...READ MORE

answered Apr 17, 2018 in Python by anonymous
 • 1,175 views
0 votes
1 answer

How to Parse values from a JSON file?

import json from pprint import pprint with open('data.json') as ...READ MORE

answered Oct 15, 2018 in Python by Priyaj
 • 58,020 points • 1,991 views
+1 vote
2 answers

What does hadoop fs -du command gives as output?

du command is used for to see ...READ MORE

answered Jul 24, 2019 in Big Data Hadoop by Lokesh Singh
 • 5,841 views
0 votes
1 answer

How can I write text in HDFS using CMD?

Hadoop put & appendToFile only reads standard ...READ MORE

answered Apr 27, 2018 in Big Data Hadoop by Shubham
 • 13,490 points • 2,263 views
0 votes
1 answer

What is the command to find the free space in HDFS?

You can use dfsadmin which runs a ...READ MORE

answered Apr 29, 2018 in Big Data Hadoop by Shubham
 • 13,490 points • 2,168 views
0 votes
1 answer

How to find the used cache in HDFS

hdfs dfsadmin -report This command tells fs ...READ MORE

answered May 4, 2018 in Big Data Hadoop by Shubham
 • 13,490 points • 2,458 views
0 votes
1 answer

How to call a function from another file in Python?

Hi@akhtar, You can use the import keyword to ...READ MORE

answered Jun 24, 2020 in Python by MD
 • 95,460 points • 5,238 views
0 votes
1 answer

How to save a Pandas Dataframe to a pickle file?

Hi@akhtar, Python pickle module is used for serializing ...READ MORE

answered Oct 18, 2020 in Python by MD
 • 95,460 points • 5,598 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP
"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.