Java MapReduce error saying No FileSystem for scheme hdfs

Question

I am writing java code to upload a file to HDFS. So, my code is as follows:

Configuration configuration = new Configuration();
configuration.set(“fs.hdfs.impl”, org.apache.hadoop.DistributedFileSystem.class.getName());
Filesystem filesystem = FileSystem.get(URI.create(remote), configuration);

I included the following dependencies while executing the project:

• hadoop-auth-2.6.1.jar
• hadoop-common-2.6.1.jar 
• hadoop-hdfs-2.6.1.jar

While executing the project I got an error saying

No FileSystem for scheme:hdfs

Can anyone tell me what is wrong with my code?

Answer 1

The problem is with the dependencies. The DistributedFileSystem is present in hadoop-core package. So, I guess you haven’t given the hadoop-core dependencies.

Import hadoop-core jar from org.apache.hadoop package.

One more important thing, sometimes we also get this issue if the version of Hadoop is different from the version of jar dependencies that you are using. So kindly look at the version of Hadoop that you are using & the version of jars that you are importing.