I have to ingest in hadoop cluster large number of files for testing , what is the best way to do it?

0 votes
Jul 7 in Big Data Hadoop by sonali
• 120 points
176 views

1 answer to this question.

0 votes

Hi@sonali,

It depends on what kind of testing you want to do. Say if you are trying to perform some SQL type command then go for HiveQL, Scala, etc. Or if you want to use that file in another platform like Spark. Then simply upload your data in HDFS Cluster and use it.

answered Jul 7 by MD
• 66,300 points

Related Questions In Big Data Hadoop

0 votes
1 answer
0 votes
1 answer

What is the command to check the number of cores in Spark?

Go to your Spark Web UI & ...READ MORE

answered May 16, 2018 in Big Data Hadoop by Shubham
• 13,450 points
2,079 views
0 votes
1 answer

What is the best functional language to do Hadoop Map-Reduce?

down voteacceptedBoth Clojure and Haskell are definitely ...READ MORE

answered Sep 4, 2018 in Big Data Hadoop by Frankie
• 9,810 points
158 views
0 votes
1 answer

What is Modeling data in Hadoop and how to do it?

I suggest spending some time with Apache ...READ MORE

answered Sep 19, 2018 in Big Data Hadoop by Frankie
• 9,810 points
523 views
0 votes
1 answer

What is the standard way to create files in your hdfs file-system?

Well, it's so easy. Just enter the below ...READ MORE

answered Sep 22, 2018 in Big Data Hadoop by Frankie
• 9,810 points
674 views
0 votes
1 answer

What is the command to count number of lines in a file in hdfs?

hadoop fs -cat /example2/doc1 | wc -l READ MORE

answered Nov 22, 2018 in Big Data Hadoop by Omkar
• 69,030 points
1,080 views
0 votes
1 answer

How to find the number of blocks for a file in Hadoop?

Hi@akhtar, You can use Hadoop file system command to ...READ MORE

answered Oct 13 in Big Data Hadoop by MD
• 66,300 points
96 views