this is my input file output_pig_group_education_comma/input_load.txt and its contents is as below:
(Secondary School,Goa),{(Secondary School,Goa,442,425,443,445,426,459,457,457,458,460)}
(Secondary School,Assam),{(Secondary School,Assam,4832,4736,5338,5374,5527,5881,5820,5970,6417,6563)}
(Secondary School,Bihar),{(Secondary School,Bihar,3575,3826,3624,3629,3637,3652,3746,3792,4236,4503)}
(Secondary School,Delhi),{(Secondary School,Delhi,1578,1619,1678,1712,1750,1697,1768,1759,1824,1872)}
(Secondary School,Kerala),{(Secondary School,Kerala,4296,3015,4157,5402,5472,5510,5525,5660,5768,3825)}
(Secondary School,Odisha),{(Secondary School,Odisha,7313,7391,7682,8661,8890,9407,8522,9504,8943,9267)}
(Secondary School,Punjab),{(Secondary School,Punjab,3901,4151,3977,3980,4043,4042,4110,5116,5121,5657)}
(Secondary School,Sikkim),{(Secondary School,Sikkim,110,138,123,161,162,163,164,173,185,185)}
(Secondary School,Gujarat),{(Secondary School,Gujarat,6734,7081,7641,7718,7654,7967,8328,9015,9299,9844)}
(Secondary School,Haryana),{(Secondary School,Haryana,4579,5082,5120,5222,5317,5634,6095,5929,6771,6771)}
(Secondary School,Manipur),{(Secondary School,Manipur,659,687,652,706,705,804,804,824,824,884)}
(Secondary School,Mizoram),{(Secondary School,Mizoram,403,388,514,512,560,581,590,588,616,636)}
(Secondary School,Tripura),{(Secondary School,Tripura,636,643,651,652,667,713,713,731,770,840)}
(Secondary School,Nagaland),{(Secondary School,Nagaland,344,279,383,379,449,449,406,406,406,406)}
I am giving the following commands:
grunt>educ = LOAD 'output_pig_group_education_comma/input_load.txt' USING PigStorage(',') as (grp:tuple(School_type:chararray,State:chararray),value_b:{values_t:(year_2001:int, year_2002:int, year_2003:int, year_2004:int, year_2005:int, year_2006:int, year_2007:int, year_2008:int, year_2009:int, year_2010:int)});
grunt> describe educ;
educ: {grp: (School_type: chararray,State: chararray),value_b: {values_t: (year_2001: int,year_2002: int,year_2003: int,year_2004: int,year_2005: int,year_2006: int,year_2007: int,year_2008: int,year_2009: int,year_2010: int)}}
grunt> f = FOREACH educ GENERATE grp.State,value_b.year_2010;
grunt> describe f;
f: {State: chararray,{(year_2010: int)}}
grunt> dump f;
2019-08-09 20:38:22,770 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN
2019-08-09 20:38:22,770 [main] INFO org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer - {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, DuplicateForEachColumnRewrite, GroupByConstParallelSetter, ImplicitSplitInserter, LimitOptimizer, LoadTypeCastInserter, MergeFilter, MergeForEach, NewPartitionFilterOptimizer, PushDownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter], RULES_DISABLED=[FilterLogicExpressionSimplifier, PartitionFilterOptimizer]}
2019-08-09 20:38:22,776 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false
2019-08-09 20:38:22,778 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
2019-08-09 20:38:22,778 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
2019-08-09 20:38:22,797 [main] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at ip-172-31-45-216.ec2.internal/172.31.45.216:8032
2019-08-09 20:38:22,799 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added to the job
2019-08-09 20:38:22,800 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
2019-08-09 20:38:23,153 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - creating jar file Job4949700200735900268.jar
2019-08-09 20:38:27,187 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - jar file Job4949700200735900268.jar created
2019-08-09 20:38:27,201 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting up single store job
2019-08-09 20:38:27,202 [main] INFO org.apache.pig.data.SchemaTupleFrontend - Key [pig.schematuple] is false, will not generate code.
2019-08-09 20:38:27,202 [main] INFO org.apache.pig.data.SchemaTupleFrontend - Starting process to move generated code to distributed cache
2019-08-09 20:38:27,202 [main] INFO org.apache.pig.data.SchemaTupleFrontend - Setting key [pig.schematuple.classes] with classes to deserialize []
2019-08-09 20:38:27,221 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission.
2019-08-09 20:38:27,223 [JobControl] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at ip-172-31-45-216.ec2.internal/172.31.45.216:8032
2019-08-09 20:38:27,245 [JobControl] INFO org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
2019-08-09 20:38:27,410 [JobControl] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1
2019-08-09 20:38:27,410 [JobControl] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1
2019-08-09 20:38:27,417 [JobControl] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths (combined) to process : 1
2019-08-09 20:38:27,453 [JobControl] INFO org.apache.hadoop.mapreduce.JobSubmitter - number of splits:1
2019-08-09 20:38:27,493 [JobControl] INFO org.apache.hadoop.mapreduce.JobSubmitter - Submitting tokens for job: job_1563200896506_0322
2019-08-09 20:38:27,542 [JobControl] INFO org.apache.hadoop.yarn.client.api.impl.YarnClientImpl - Submitted application application_1563200896506_0322
2019-08-09 20:38:27,545 [JobControl] INFO org.apache.hadoop.mapreduce.Job - The url to track the job: http://ec2-54-92-244-237.compute-1.amazonaws.com:8088/proxy/application_1563200896506_0322/
2019-08-09 20:38:27,722 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - HadoopJobId: job_1563200896506_0322
2019-08-09 20:38:27,722 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Processing aliases educ,f
2019-08-09 20:38:27,722 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - detailed locations: M: educ[5,7],f[-1,-1] C: R:
2019-08-09 20:38:27,750 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete
2019-08-09 20:38:38,389 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 50% complete
2019-08-09 20:38:42,717 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete
2019-08-09 20:38:42,718 [main] INFO org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:
HadoopVersion PigVersion UserId StartedAt FinishedAt Features
2.6.0-cdh5.11.0 0.12.0-cdh5.11.0 hirwuser1394 2019-08-09 20:38:22 2019-08-09 20:38:42 UNKNOWN
Success!
Job Stats (time in seconds):
JobId Maps Reduces MaxMapTime MinMapTIme AvgMapTime MedianMapTime MaxReduceTime MinReduceTime AvgReduceTime MedianReducetime Alias Feature Outputs
job_1563200896506_0322 1 0 3 3 3 3 n/a n/a n/a n/a educ,f MAP_ONLY hdfs://ip-172-31-45-216.ec2.internal:8020/tmp/temp295317939/tmp-158778090,
Input(s):
Successfully read 35 records (4234 bytes) from: "hdfs://ip-172-31-45-216.ec2.internal:8020/user/hirwuser1394/output_pig_group_education_comma/input_load.txt"
Output(s):
Successfully stored 35 records (210 bytes) in: "hdfs://ip-172-31-45-216.ec2.internal:8020/tmp/temp295317939/tmp-158778090"
Counters:
Total records written : 35
Total bytes written : 210
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 0
Total records proactively spilled: 0
Job DAG:
job_1563200896506_0322
2019-08-09 20:38:42,805 [main] WARN org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Encountered Warning FIELD_DISCARDED_TYPE_CONVERSION_FAILED 70 time(s).
2019-08-09 20:38:42,805 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Success!
2019-08-09 20:38:42,806 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
2019-08-09 20:38:42,806 [main] INFO org.apache.pig.data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not generate code.
2019-08-09 20:38:42,815 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1
2019-08-09 20:38:42,815 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
I dont understand why is the output of dump is empty and where Iam going wrong.Cany you please help me?
Thanks.