flume twitter data file not generated in HDFS

+1 vote

I am using Apache Hadoop on Windows 10. I have installed Flume and trying to generate tweets using Twitter Agent. The twitter agent runs perfectly and the tweets are shown in the command prompt. But I can't see the twitter file generated in HDFS. 

Please help me through this. 

Sep 26, 2019 in Big Data Hadoop by chandanarora
• 130 points
742 views
Did you manage to figure this out?
Did you try the below-given solution?
I have the same problem and I already have source and sink binding and it is not working. Please help if you know the solution

Hi,

Maybe your cluster is in a safe mode. So check this first. But if you get the same output, then paste your complete code here.

1 answer to this question.

0 votes

Seems like you've missed the source and sink binding. Add these and then try:

TwitterAgent.sources.Twitter.channels = MemChannel
TwitterAgent.sinks.HDFS.channel = MemChannel 
answered Dec 2, 2019 by Esha
Hi,

If it is still not working. Then try in the Linux system and see if you get the same error or not.
Hi,

thank you but I am working on the Linux system and it is still not working.
Hi,

Ok, I will replicate your requirement in my own system and get back to you.
Hi,

did you try it?

Hi,

I tried to create a table in the hive. It is working fine. In your case, there is nothing wrong with the steps. But I think the problem is with the hive. You just create a simple table and try to import a simple text file from your local system or HDFS.

Related Questions In Big Data Hadoop

0 votes
1 answer

How can I append data to an existing file in HDFS?

You have to do some configurations as ...READ MORE

answered Jul 25, 2019 in Big Data Hadoop by ravikiran
• 4,620 points
4,161 views
0 votes
1 answer
0 votes
1 answer

How to upload file to HDFS in Ubuntu

you can use  hadoop fs -copyFromLocal  "/home/ritwi ...READ MORE

answered Apr 18, 2018 in Big Data Hadoop by kurt_cobain
• 9,390 points
571 views
0 votes
1 answer

How to print the content of a file in console present in HDFS?

Yes, you can use hdfs dfs command ...READ MORE

answered Apr 19, 2018 in Big Data Hadoop by Shubham
• 13,480 points
2,844 views
0 votes
1 answer
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
7,117 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
1,141 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
51,715 views
0 votes
1 answer

How to execute python script in hadoop file system (hdfs)?

If you are simply looking to distribute ...READ MORE

answered Sep 19, 2018 in Big Data Hadoop by digger
• 26,700 points
7,614 views
0 votes
1 answer

How to write a file in hdfs with Java?

You could pass the URI when getting ...READ MORE

answered Sep 26, 2018 in Big Data Hadoop by digger
• 26,700 points
2,379 views