Output of dump command has empty records

+1 vote

this is my input file output_pig_group_education_comma/input_load.txt and its contents is as below:

(Secondary School,Goa),{(Secondary School,Goa,442,425,443,445,426,459,457,457,458,460)}
(Secondary School,Assam),{(Secondary School,Assam,4832,4736,5338,5374,5527,5881,5820,5970,6417,6563)}
(Secondary School,Bihar),{(Secondary School,Bihar,3575,3826,3624,3629,3637,3652,3746,3792,4236,4503)}
(Secondary School,Delhi),{(Secondary School,Delhi,1578,1619,1678,1712,1750,1697,1768,1759,1824,1872)}
(Secondary School,Kerala),{(Secondary School,Kerala,4296,3015,4157,5402,5472,5510,5525,5660,5768,3825)}
(Secondary School,Odisha),{(Secondary School,Odisha,7313,7391,7682,8661,8890,9407,8522,9504,8943,9267)}
(Secondary School,Punjab),{(Secondary School,Punjab,3901,4151,3977,3980,4043,4042,4110,5116,5121,5657)}
(Secondary School,Sikkim),{(Secondary School,Sikkim,110,138,123,161,162,163,164,173,185,185)}
(Secondary School,Gujarat),{(Secondary School,Gujarat,6734,7081,7641,7718,7654,7967,8328,9015,9299,9844)}
(Secondary School,Haryana),{(Secondary School,Haryana,4579,5082,5120,5222,5317,5634,6095,5929,6771,6771)}
(Secondary School,Manipur),{(Secondary School,Manipur,659,687,652,706,705,804,804,824,824,884)}
(Secondary School,Mizoram),{(Secondary School,Mizoram,403,388,514,512,560,581,590,588,616,636)}
(Secondary School,Tripura),{(Secondary School,Tripura,636,643,651,652,667,713,713,731,770,840)}
(Secondary School,Nagaland),{(Secondary School,Nagaland,344,279,383,379,449,449,406,406,406,406)}

I am giving the following commands:

grunt>educ = LOAD 'output_pig_group_education_comma/input_load.txt' USING PigStorage(',') as (grp:tuple(School_type:chararray,State:chararray),value_b:{values_t:(year_2001:int, year_2002:int, year_2003:int, year_2004:int, year_2005:int, year_2006:int, year_2007:int, year_2008:int, year_2009:int, year_2010:int)});
grunt> describe educ;
educ: {grp: (School_type: chararray,State: chararray),value_b: {values_t: (year_2001: int,year_2002: int,year_2003: int,year_2004: int,year_2005: int,year_2006: int,year_2007: int,year_2008: int,year_2009: int,year_2010: int)}}

grunt> f = FOREACH educ GENERATE grp.State,value_b.year_2010;
grunt> describe f;
f: {State: chararray,{(year_2010: int)}}
grunt> dump f;
2019-08-09 20:38:22,770 [main] INFO  org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN
2019-08-09 20:38:22,770 [main] INFO  org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer - {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, DuplicateForEachColumnRewrite, GroupByConstParallelSetter, ImplicitSplitInserter, LimitOptimizer, LoadTypeCastInserter, MergeFilter, MergeForEach, NewPartitionFilterOptimizer, PushDownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter], RULES_DISABLED=[FilterLogicExpressionSimplifier, PartitionFilterOptimizer]}
2019-08-09 20:38:22,776 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false
2019-08-09 20:38:22,778 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
2019-08-09 20:38:22,778 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
2019-08-09 20:38:22,797 [main] INFO  org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at ip-172-31-45-216.ec2.internal/172.31.45.216:8032
2019-08-09 20:38:22,799 [main] INFO  org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added to the job
2019-08-09 20:38:22,800 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
2019-08-09 20:38:23,153 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - creating jar file Job4949700200735900268.jar
2019-08-09 20:38:27,187 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - jar file Job4949700200735900268.jar created
2019-08-09 20:38:27,201 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting up single store job
2019-08-09 20:38:27,202 [main] INFO  org.apache.pig.data.SchemaTupleFrontend - Key [pig.schematuple] is false, will not generate code.
2019-08-09 20:38:27,202 [main] INFO  org.apache.pig.data.SchemaTupleFrontend - Starting process to move generated code to distributed cache
2019-08-09 20:38:27,202 [main] INFO  org.apache.pig.data.SchemaTupleFrontend - Setting key [pig.schematuple.classes] with classes to deserialize []
2019-08-09 20:38:27,221 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission.
2019-08-09 20:38:27,223 [JobControl] INFO  org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at ip-172-31-45-216.ec2.internal/172.31.45.216:8032
2019-08-09 20:38:27,245 [JobControl] INFO  org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
2019-08-09 20:38:27,410 [JobControl] INFO  org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1
2019-08-09 20:38:27,410 [JobControl] INFO  org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1
2019-08-09 20:38:27,417 [JobControl] INFO  org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths (combined) to process : 1
2019-08-09 20:38:27,453 [JobControl] INFO  org.apache.hadoop.mapreduce.JobSubmitter - number of splits:1
2019-08-09 20:38:27,493 [JobControl] INFO  org.apache.hadoop.mapreduce.JobSubmitter - Submitting tokens for job: job_1563200896506_0322
2019-08-09 20:38:27,542 [JobControl] INFO  org.apache.hadoop.yarn.client.api.impl.YarnClientImpl - Submitted application application_1563200896506_0322
2019-08-09 20:38:27,545 [JobControl] INFO  org.apache.hadoop.mapreduce.Job - The url to track the job: http://ec2-54-92-244-237.compute-1.amazonaws.com:8088/proxy/application_1563200896506_0322/
2019-08-09 20:38:27,722 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - HadoopJobId: job_1563200896506_0322
2019-08-09 20:38:27,722 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Processing aliases educ,f
2019-08-09 20:38:27,722 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - detailed locations: M: educ[5,7],f[-1,-1] C:  R:
2019-08-09 20:38:27,750 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete
2019-08-09 20:38:38,389 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 50% complete
2019-08-09 20:38:42,717 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete
2019-08-09 20:38:42,718 [main] INFO  org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:

HadoopVersion   PigVersion      UserId  StartedAt       FinishedAt      Features
2.6.0-cdh5.11.0 0.12.0-cdh5.11.0        hirwuser1394    2019-08-09 20:38:22     2019-08-09 20:38:42     UNKNOWN

Success!

Job Stats (time in seconds):
JobId   Maps    Reduces MaxMapTime      MinMapTIme      AvgMapTime      MedianMapTime   MaxReduceTime   MinReduceTime   AvgReduceTime   MedianReducetime        Alias  Feature  Outputs
job_1563200896506_0322  1       0       3       3       3       3       n/a     n/a     n/a     n/a     educ,f  MAP_ONLY        hdfs://ip-172-31-45-216.ec2.internal:8020/tmp/temp295317939/tmp-158778090,

Input(s):
Successfully read 35 records (4234 bytes) from: "hdfs://ip-172-31-45-216.ec2.internal:8020/user/hirwuser1394/output_pig_group_education_comma/input_load.txt"

Output(s):
Successfully stored 35 records (210 bytes) in: "hdfs://ip-172-31-45-216.ec2.internal:8020/tmp/temp295317939/tmp-158778090"

Counters:
Total records written : 35
Total bytes written : 210
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 0
Total records proactively spilled: 0

Job DAG:
job_1563200896506_0322

2019-08-09 20:38:42,805 [main] WARN  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Encountered Warning FIELD_DISCARDED_TYPE_CONVERSION_FAILED 70 time(s).
2019-08-09 20:38:42,805 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Success!
2019-08-09 20:38:42,806 [main] INFO  org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
2019-08-09 20:38:42,806 [main] INFO  org.apache.pig.data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not generate code.
2019-08-09 20:38:42,815 [main] INFO  org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1
2019-08-09 20:38:42,815 [main] INFO  org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)
(,)

I dont understand why is the output of dump is empty and where Iam going wrong.Cany you please help me?

Thanks.

Aug 10, 2019 in Big Data Hadoop by Manju Bhargavi
935 views

Can you share the output of the following command:

$ Illustrate f;

No answer to this question. Be the first to respond.

Your answer

Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.

Related Questions In Big Data Hadoop

0 votes
1 answer
0 votes
1 answer

Best way of starting & stopping the Hadoop daemons with command line

First way is to use start-all.sh & ...READ MORE

answered Apr 15, 2018 in Big Data Hadoop by Shubham
• 13,490 points
9,560 views
0 votes
2 answers

Function of “hadoop namenode -format” command

Namenode is the node in the Hadoop ...READ MORE

answered Feb 4, 2019 in Big Data Hadoop by Kanika
20,585 views
+1 vote
2 answers

What does hadoop fs -du command gives as output?

du command is used for to see ...READ MORE

answered Jul 24, 2019 in Big Data Hadoop by Lokesh Singh
5,498 views
0 votes
1 answer

What is the command to check the number of cores in Spark?

Go to your Spark Web UI & ...READ MORE

answered May 17, 2018 in Big Data Hadoop by Shubham
• 13,490 points
6,500 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
10,618 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
2,215 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
104,941 views
–1 vote
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,390 points
4,295 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP