Output of dump command has empty records

+1 vote

this is my input file output_pig_group_education_comma/input_load.txt and its contents is as below:

(Secondary School,Goa),{(Secondary School,Goa,442,425,443,445,426,459,457,457,458,460)}
(Secondary School,Assam),{(Secondary School,Assam,4832,4736,5338,5374,5527,5881,5820,5970,6417,6563)}
(Secondary School,Bihar),{(Secondary School,Bihar,3575,3826,3624,3629,3637,3652,3746,3792,4236,4503)}
(Secondary School,Delhi),{(Secondary School,Delhi,1578,1619,1678,1712,1750,1697,1768,1759,1824,1872)}
(Secondary School,Kerala),{(Secondary School,Kerala,4296,3015,4157,5402,5472,5510,5525,5660,5768,3825)}
(Secondary School,Odisha),{(Secondary School,Odisha,7313,7391,7682,8661,8890,9407,8522,9504,8943,9267)}
(Secondary School,Punjab),{(Secondary School,Punjab,3901,4151,3977,3980,4043,4042,4110,5116,5121,5657)}
(Secondary School,Sikkim),{(Secondary School,Sikkim,110,138,123,161,162,163,164,173,185,185)}
(Secondary School,Gujarat),{(Secondary School,Gujarat,6734,7081,7641,7718,7654,7967,8328,9015,9299,9844)}
(Secondary School,Haryana),{(Secondary School,Haryana,4579,5082,5120,5222,5317,5634,6095,5929,6771,6771)}
(Secondary School,Manipur),{(Secondary School,Manipur,659,687,652,706,705,804,804,824,824,884)}
(Secondary School,Mizoram),{(Secondary School,Mizoram,403,388,514,512,560,581,590,588,616,636)}
(Secondary School,Tripura),{(Secondary School,Tripura,636,643,651,652,667,713,713,731,770,840)}
(Secondary School,Nagaland),{(Secondary School,Nagaland,344,279,383,379,449,449,406,406,406,406)}

I am giving the following commands:

grunt>educ = LOAD 'output_pig_group_education_comma/input_load.txt' USING PigStorage(',') as (grp:tuple(School_type:chararray,State:chararray),value_b:{values_t:(year_2001:int, year_2002:int, year_2003:int, year_2004:int, year_2005:int, year_2006:int, year_2007:int, year_2008:int, year_2009:int, year_2010:int)});
grunt> describe educ;
educ: {grp: (School_type: chararray,State: chararray),value_b: {values_t: (year_2001: int,year_2002: int,year_2003: int,year_2004: int,year_2005: int,year_2006: int,year_2007: int,year_2008: int,year_2009: int,year_2010: int)}}

grunt> f = FOREACH educ GENERATE grp.State,value_b.year_2010;
grunt> describe f;
f: {State: chararray,{(year_2010: int)}}
grunt> dump f;
2019-08-09 20:38:22,770 [main] INFO  org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN
2019-08-09 20:38:22,770 [main] INFO  org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer - {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, DuplicateForEachColumnRewrite, GroupByConstParallelSetter, ImplicitSplitInserter, LimitOptimizer, LoadTypeCastInserter, MergeFilter, MergeForEach, NewPartitionFilterOptimizer, PushDownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter], RULES_DISABLED=[FilterLogicExpressionSimplifier, PartitionFilterOptimizer]}
2019-08-09 20:38:22,776 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false
2019-08-09 20:38:22,778 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
2019-08-09 20:38:22,778 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
2019-08-09 20:38:22,797 [main] INFO  org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at ip-172-31-45-216.ec2.internal/172.31.45.216:8032
2019-08-09 20:38:22,799 [main] INFO  org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added to the job
2019-08-09 20:38:22,800 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
2019-08-09 20:38:23,153 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - creating jar file Job4949700200735900268.jar
2019-08-09 20:38:27,187 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - jar file Job4949700200735900268.jar created
2019-08-09 20:38:27,201 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting up single store job
2019-08-09 20:38:27,202 [main] INFO  org.apache.pig.data.SchemaTupleFrontend - Key [pig.schematuple] is false, will not generate code.
2019-08-09 20:38:27,202 [main] INFO  org.apache.pig.data.SchemaTupleFrontend - Starting process to move generated code to distributed cache
2019-08-09 20:38:27,202 [main] INFO  org.apache.pig.data.SchemaTupleFrontend - Setting key [pig.schematuple.classes] with classes to deserialize []
2019-08-09 20:38:27,221 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission.
2019-08-09 20:38:27,223 [JobControl] INFO  org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at ip-172-31-45-216.ec2.internal/172.31.45.216:8032
2019-08-09 20:38:27,245 [JobControl] INFO  org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
2019-08-09 20:38:27,410 [JobControl] INFO  org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1
2019-08-09 20:38:27,410 [JobControl] INFO  org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1
2019-08-09 20:38:27,417 [JobControl] INFO  org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths (combined) to process : 1
2019-08-09 20:38:27,453 [JobControl] INFO  org.apache.hadoop.mapreduce.JobSubmitter - number of splits:1
2019-08-09 20:38:27,493 [JobControl] INFO  org.apache.hadoop.mapreduce.JobSubmitter - Submitting tokens for job: job_1563200896506_0322
2019-08-09 20:38:27,542 [JobControl] INFO  org.apache.hadoop.yarn.client.api.impl.YarnClientImpl - Submitted application application_1563200896506_0322
2019-08-09 20:38:27,545 [JobControl] INFO  org.apache.hadoop.mapreduce.Job - The url to track the job: http://ec2-54-92-244-237.compute-1.amazonaws.com:8088/proxy/application_1563200896506_0322/
2019-08-09 20:38:27,722 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - HadoopJobId: job_1563200896506_0322
2019-08-09 20:38:27,722 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Processing aliases educ,f
2019-08-09 20:38:27,722 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - detailed locations: M: educ[5,7],f[-1,-1] C:  R:
2019-08-09 20:38:27,750 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete
2019-08-09 20:38:38,389 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 50% complete
2019-08-09 20:38:42,717 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete
2019-08-09 20:38:42,718 [main] INFO  org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:

HadoopVersion   PigVersion      UserId  StartedAt       FinishedAt      Features
2.6.0-cdh5.11.0 0.12.0-cdh5.11.0        hirwuser1394    2019-08-09 20:38:22     2019-08-09 20:38:42     UNKNOWN

Success!

Job Stats (time in seconds):
JobId   Maps    Reduces MaxMapTime      MinMapTIme      AvgMapTime      MedianMapTime   MaxReduceTime   MinReduceTime   AvgReduceTime   MedianReducetime        Alias  Feature  Outputs
job_1563200896506_0322  1       0       3       3       3       3       n/a     n/a     n/a     n/a     educ,f  MAP_ONLY        hdfs://ip-172-31-45-216.ec2.internal:8020/tmp/temp295317939/tmp-158778090,

Input(s):
Successfully read 35 records (4234 bytes) from: "hdfs://ip-172-31-45-216.ec2.internal:8020/user/hirwuser1394/output_pig_group_education_comma/input_load.txt"

Output(s):
Successfully stored 35 records (210 bytes) in: "hdfs://ip-172-31-45-216.ec2.internal:8020/tmp/temp295317939/tmp-158778090"

Counters:
Total records written : 35
Total bytes written : 210
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 0
Total records proactively spilled: 0

Job DAG:
job_1563200896506_0322

2019-08-09 20:38:42,805 [main] WARN  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Encountered Warning FIELD_DISCARDED_TYPE_CONVERSION_FAILED 70 time(s).
2019-08-09 20:38:42,805 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Success!
2019-08-09 20:38:42,806 [main] INFO  org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
2019-08-09 20:38:42,806 [main] INFO  org.apache.pig.data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not generate code.
2019-08-09 20:38:42,815 [main] INFO  org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1
2019-08-09 20:38:42,815 [main] INFO  org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)

I dont understand why is the output of dump is empty and where Iam going wrong.Cany you please help me?

Thanks.

Aug 10 in Big Data Hadoop by Manju Bhargavi
33 views

Can you share the output of the following command:

$ Illustrate f;

No answer to this question. Be the first to respond.

Your answer

Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.

Related Questions In Big Data Hadoop

0 votes
1 answer
0 votes
1 answer

Best way of starting & stopping the Hadoop daemons with command line

First way is to use start-all.sh & ...READ MORE

answered Apr 15, 2018 in Big Data Hadoop by Shubham
• 13,300 points
1,750 views
0 votes
2 answers

Function of “hadoop namenode -format” command

Namenode is the node in the Hadoop ...READ MORE

answered Feb 4 in Big Data Hadoop by Kanika
4,784 views
+1 vote
2 answers

What does hadoop fs -du command gives as output?

du command is used for to see ...READ MORE

answered Jul 23 in Big Data Hadoop by Lokesh Singh
983 views
0 votes
1 answer
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,690 points
3,063 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,690 points
341 views
0 votes
10 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
15,042 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,260 points
1,120 views