Differentiate: hadoop fs -text vs hadoop fs -cat vs hadoop fs -get

0 votes

As per my knowledge all the following commands can be used for copying the hdfs files to local file system. So what is the difference between below commands-

hadoop fs -text /hdfs_dir/* >> /local_dir/localfile.txt
hadoop fs -cat /hdfs_dir/* >> /local_dir/localfile.txt
hadoop fs -get /hdfs_dir/* >> /local_dir/

My thumb-rule is to avoid using 'text' and 'cat' for big files

Dec 5, 2018 in Big Data Hadoop by Neha
• 6,280 points
909 views

1 answer to this question.

0 votes

The main difference between -cat and -text is that text detects the encoding of the file and decodes it to plain text whenever possible whereas cat doesnt do it.

for eg take the example of this lzo compressed file.

using text:

hadoop fs -text hdfs://namenode:8020/user/hive/warehouse/database/000000_0.lzo_deflate
1,bye
2,back
3,baz
4,hey
5,country

using cat:

 hadoop fs -cat hdfs://namenode:8020/user/hive/warehouse/database/000000_0.lzo_deflate 
    ίiW3�I���2�IJ,�2�U\&:�99�\�:��E9)\֙��"

dfs -get command is used to copy files to local-filesystem.

answered Dec 5, 2018 by Frankie
• 9,810 points

Related Questions In Big Data Hadoop

+2 votes
10 answers

Is there any difference between “hdfs dfs” and “hadoop fs” shell commands?

Yes, there's a difference between hadoop fs and ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Kunal
11,110 views
0 votes
1 answer

How to get started with Hadoop?

Well, hadoop is actually a framework that ...READ MORE

answered Mar 21, 2018 in Big Data Hadoop by coldcode
• 2,020 points
106 views
0 votes
10 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
16,800 views
0 votes
1 answer

Hadoop MapReduce vs MongoDB MapReduce

Hadoop vs MongoDB MR: 1) Hadoop's MR can ...READ MORE

answered Mar 26, 2018 in Big Data Hadoop by nitinrawat895
• 10,730 points
201 views
+1 vote
2 answers

What does hadoop fs -du command gives as output?

du command is used for to see ...READ MORE

answered Jul 23 in Big Data Hadoop by Lokesh Singh
1,106 views
0 votes
1 answer

How can I write text in HDFS using CMD?

Hadoop put & appendToFile only reads standard ...READ MORE

answered Apr 27, 2018 in Big Data Hadoop by Shubham
• 13,310 points
96 views
0 votes
1 answer

What is the command to find the free space in HDFS?

You can use dfsadmin which runs a ...READ MORE

answered Apr 29, 2018 in Big Data Hadoop by Shubham
• 13,310 points
238 views
0 votes
1 answer

How to find the used cache in HDFS

hdfs dfsadmin -report This command tells fs ...READ MORE

answered May 4, 2018 in Big Data Hadoop by Shubham
• 13,310 points
274 views
0 votes
1 answer
0 votes
1 answer

How do I get connected to Hadoop and Geo Spatial connector?

There are a number of free and ...READ MORE

answered Aug 13, 2018 in Big Data Hadoop by Frankie
• 9,810 points
259 views