Unable to Locate WinUtils Library in Hadoop binary path

Question

I am getting the following error while starting the name node for latest Hadoop-2.2 release. I didn't find winutils exe file in the hadoop bin folder. I tried below commands

$ bin/hdfs namenode -format
$ sbin/yarn-daemon.sh start resourcemanager

ERROR [main] util.Shell (Shell.java:getWinUtilsPath(303)) - Failed to locate the winutils binary in the hadoop binary path
java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
    at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:278)
    at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:300)
    at org.apache.hadoop.util.Shell.<clinit>(Shell.java:293)
    at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:76)
    at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:863)

Answer 1

If you are facing this problem while running a self-contained local application along with Spark 

(i.e., after adding spark-assembly-x.x.x-hadoopx.x.x.jar or the Maven dependency to the project)

Then, a simpler solution would be to put winutils.exe in "C:\winutil\bin". You can add winutils.exe to the Hadoop home directory by adding the following line to the code:

System.setProperty("hadoop.home.dir", "c:\\\winutil\\\")

I hope this helps.