One day, I just created a fresh centos and download/install the hadoop, by default it runs on local mode. so I just change the config to run in distributed mode. When I run bin/start-all.sh. get the follow errors in the tasktrackr log.
| 2011-08-16 21:00:12,136 INFO org.apache.hadoop.mapred.JobTracker: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting JobTracker
STARTUP_MSG: host = hadoop/192.168.209.131
STARTUP_MSG: args = []
STARTUP_MSG: version = 0.20.2
STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r 911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010
************************************************************/
2011-08-16 21:00:12,280 WARN org.apache.hadoop.conf.Configuration: bad conf file: element not <property>
2011-08-16 21:00:12,280 WARN org.apache.hadoop.conf.Configuration: bad conf file: element not <property>
2011-08-16 21:00:12,280 WARN org.apache.hadoop.conf.Configuration: bad conf file: element not <property>
2011-08-16 21:00:12,299 INFO org.apache.hadoop.mapred.JobTracker: Scheduler configured with (memSizeForMapSlotOnJT, memSizeForReduceSlotOnJT, limitMaxMemForMapTasks, limitMaxMemForReduceTasks) (-1, -1, -1, -1)
2011-08-16 21:00:12,344 FATAL org.apache.hadoop.mapred.JobTracker: java.lang.RuntimeException: Not a host:port pair: local
at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:136)
at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:123)
at org.apache.hadoop.mapred.JobTracker.getAddress(JobTracker.java:1807)
at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:1579)
at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:183)
at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:175)
at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:3702)
|
it looks like the runtime still pick up the default mapreduce configuration located in the hadoop-x.core.jar, you can get this file by unzipping hadoopcore.jar,
So the error means it pick up this default setting of Local. Then I read careflly about the mapred-site.xml, here it is.
It looks like perfect, right, cause I just copied the setting from the default xml, while, I just keep the weird error.
well, here is final xml that works great, I highlighted the difference, I forgot to put the property tag. so the runtime will ignore this xml and pick up the default settings.
Summary, when you get weird error, check the log first, then make sure those conf xml are in right format that comforts the scheme
tricky, hah?
1 comment:
Thanks for detailed article on Hadoop. hadoop can also be used olap and oltp processing.
Please click Why Hadoop is introduced to know more on Basics of Hadoop
Post a Comment