questions/apache-spark
Hi@akhtar, Yes, Spark streaming uses checkpoint. Checkpoint is ...READ MORE
Hi@akhtar, There is no concept of indexing in ...READ MORE
Hi@akhtar Generally, Spark streaming is used for real time ...READ MORE
Hi@akhtar, Dstreams are the basic abstraction that is ...READ MORE
Hi@akhtar, Regrading to this error, I think the ...READ MORE
Hi@akhtar, Here you are trying to save csv ...READ MORE
Hi@akhtar, Here you are trying to read a ...READ MORE
Hi@akhtar, The PageRank algorithm outputs a probability distribution ...READ MORE
Hi@akhtar, I also got this error. I am able to ...READ MORE
Hi@akhtar, Currently, you are running with the default ...READ MORE
Hi@akhtar, As we know text files are in ...READ MORE
Hi@ akhtar, Both map() and mapPartitions() are the ...READ MORE
spark do not have any concept of ...READ MORE
Hi @asif, Share with us please the application ...READ MORE
Try to put the kafka client for ...READ MORE
When you concatenate any string with a ...READ MORE
You can do this by turning off ...READ MORE
Try df.where($"cola".isNotNull && $"cola" =!= "" && !$"colb".isin(2,3)) your ...READ MORE
I found the following solution to be ...READ MORE
In spark 2.0.+ it should look something ...READ MORE
Try including the package while starting the ...READ MORE
Hey, you can use "contains" filter to extract ...READ MORE
Hello, Your problem is here: val df_merge_final = df_merge .withColumn("version_key", ...READ MORE
Spark-submit jobs are also run from client/edge ...READ MORE
import sqlContext.implicits._ import org.apache.spark.sql.Row import org.apache.spark.sql.types.{StructType, StructField, LongType} val df ...READ MORE
Hey, You can try this: from pyspark import SparkContext SparkContext.stop(sc) sc ...READ MORE
Hi, You can use this command to start ...READ MORE
Hey, Jobs- to view all the spark jobs Stages- ...READ MORE
Hi, You can do it using map partition ...READ MORE
Hi, foreach() operation is an action. It does not ...READ MORE
Hey, sortByKey() is a transformation. It returns an RDD sorted ...READ MORE
Hey, You can launch spark application in four ...READ MORE
Hi, To launch spark application in cluster mode, ...READ MORE
Hi, If you have a file with id ...READ MORE
Hi, Paired RDD is a distributed collection of ...READ MORE
Well, it depends on the block of ...READ MORE
Suppose you have two dataset results( id, ...READ MORE
Try this code, it worked for me: val ...READ MORE
Hi, Hive contains significant support for Apache Spark, ...READ MORE
Hey, A sparse vector is used for storing ...READ MORE
You need to declare the variable which ...READ MORE
Hey @Esha, you can use this code. ...READ MORE
The problem is probably with the command. ...READ MORE
Converting text file to Orc: Using Spark, the ...READ MORE
Hi, This reverses the order of elements in ...READ MORE
Hey, The method List.fill() creates a list and ...READ MORE
Hey, Scala uses the method copy() to carry ...READ MORE
Hey, You can use below code to access variables ...READ MORE
Hey, Java’s “If. Else”: In Java, “If. Else” is a statement, ...READ MORE
Hey, You can follow this below solution for ...READ MORE
OR
At least 1 upper-case and 1 lower-case letter
Minimum 8 characters and Maximum 50 characters
Already have an account? Sign in.