Differentiate: hadoop fs -text vs hadoop fs -cat vs hadoop fs -get

0 votes

As per my knowledge all the following commands can be used for copying the hdfs files to local file system. So what is the difference between below commands-

hadoop fs -text /hdfs_dir/* >> /local_dir/localfile.txt
hadoop fs -cat /hdfs_dir/* >> /local_dir/localfile.txt
hadoop fs -get /hdfs_dir/* >> /local_dir/

My thumb-rule is to avoid using 'text' and 'cat' for big files

Dec 5, 2018 in Big Data Hadoop by Neha
• 6,260 points
231 views

1 answer to this question.

0 votes

The main difference between -cat and -text is that text detects the encoding of the file and decodes it to plain text whenever possible whereas cat doesnt do it.

for eg take the example of this lzo compressed file.

using text:

hadoop fs -text hdfs://namenode:8020/user/hive/warehouse/database/000000_0.lzo_deflate
1,bye
2,back
3,baz
4,hey
5,country

using cat:

 hadoop fs -cat hdfs://namenode:8020/user/hive/warehouse/database/000000_0.lzo_deflate 
    ίiW3�I���2�IJ,�2�U\&:�99�\�:��E9)\֙��"

dfs -get command is used to copy files to local-filesystem.

answered Dec 5, 2018 by Frankie
• 9,810 points

Related Questions In Big Data Hadoop

+2 votes
10 answers

Is there any difference between “hdfs dfs” and “hadoop fs” shell commands?

Yes, there's a difference between hadoop fs and ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Kunal
8,664 views
0 votes
1 answer

How to get started with Hadoop?

Well, hadoop is actually a framework that ...READ MORE

answered Mar 21, 2018 in Big Data Hadoop by coldcode
• 2,010 points
62 views
0 votes
10 answers

hadoop fs -put command?

copy command can be used to copy files ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Sujay
10,377 views
0 votes
1 answer

Hadoop MapReduce vs MongoDB MapReduce

Hadoop vs MongoDB MR: 1) Hadoop's MR can ...READ MORE

answered Mar 26, 2018 in Big Data Hadoop by nitinrawat895
• 10,030 points
116 views
0 votes
1 answer

What does hadoop fs -du command gives as output?

The first value is the size of ...READ MORE

answered Apr 27, 2018 in Big Data Hadoop by Shubham
• 13,110 points
636 views
0 votes
1 answer

How can I write text in HDFS using CMD?

Hadoop put & appendToFile only reads standard ...READ MORE

answered Apr 27, 2018 in Big Data Hadoop by Shubham
• 13,110 points
62 views
0 votes
1 answer

What is the command to find the free space in HDFS?

You can use dfsadmin which runs a ...READ MORE

answered Apr 29, 2018 in Big Data Hadoop by Shubham
• 13,110 points
120 views
0 votes
1 answer

How to find the used cache in HDFS

hdfs dfsadmin -report This command tells fs ...READ MORE

answered May 4, 2018 in Big Data Hadoop by Shubham
• 13,110 points
163 views
0 votes
1 answer
0 votes
1 answer

How do I get connected to Hadoop and Geo Spatial connector?

There are a number of free and ...READ MORE

answered Aug 13, 2018 in Big Data Hadoop by Frankie
• 9,810 points
160 views