59495/sparkstream-textfilstreaming-localpathdirectory-getting
I have given the local path for spark text file streaming. But it is giving empty results even after adding new files. The path exists and it is not open also. Can you please tell how to rectify this.
Below is the code:
val pathFile:Path=new Path( "file:\\\\\\D:\\sparklearning\\inputFileStreaming\\" ) println(pathFile.toString()) val linesDStream = streamingContext.textFileStream(pathFile.toString) linesDStream.foreachRDD(x=>println(x.toString()))
Try to put the kafka client for ...READ MORE
Hi@Srinath, It seems you didn't set Hadoop for ...READ MORE
You can use the following command. This ...READ MORE
I guess you need provide this kafka.bootstrap.servers ...READ MORE
// Collect data from input avro file ...READ MORE
Instead of spliting on '\n'. You should ...READ MORE
Firstly you need to understand the concept ...READ MORE
org.apache.hadoop.mapred is the Old API org.apache.hadoop.mapreduce is the ...READ MORE
Hi, You can create one directory in HDFS ...READ MORE
You can access task information using TaskContext: import org.apache.spark.TaskContext sc.parallelize(Seq[Int](), ...READ MORE
OR
At least 1 upper-case and 1 lower-case letter
Minimum 8 characters and Maximum 50 characters
Already have an account? Sign in.