Trending questions in Apache Spark

0 votes
1 answer

Spark to Hive Table creation

There's an easier way to achieve your ...READ MORE

Jul 23 in Apache Spark by Dinesh
36 views
0 votes
0 answers

Spark: java.io.FileNotFoundException

While executing a query I am getting ...READ MORE

Jul 16 in Apache Spark by Tilka
373 views
0 votes
1 answer

Scala: when to use x(2) and x._2?

In the above statement, x(2) is specifying an array ...READ MORE

Jul 22 in Apache Spark by Yogi
62 views
0 votes
1 answer

Appending " to a string in Scala

1) Use the concat() function. Refer to the below ...READ MORE

Jul 23 in Apache Spark by Ritu
26 views
0 votes
1 answer

Explain the for loop for printing the Map values in Scala in Apache Spark?

Hey, You can see this following code to ...READ MORE

Jul 22 in Apache Spark by Gitika
• 25,340 points
52 views
0 votes
1 answer

How to load data of .csv file in MySQL Database Table?

You can do it using a code ...READ MORE

Jul 22 in Apache Spark by Vishwa
41 views
0 votes
0 answers

What is immutabiliity in Spark?

Can anyone explain what is immutability in ...READ MORE

Jul 23 in Apache Spark by Risha
24 views
0 votes
1 answer

Code to compute average in Apache Spark?

Hi, You can compute the average using this ...READ MORE

Jul 22 in Apache Spark by Gitika
• 25,340 points
26 views
0 votes
0 answers

Load data to mysql from local

I'm trying to load data to mysql ...READ MORE

Jul 23 in Apache Spark by Yogi
34 views
0 votes
1 answer

Error : split value is not a member of org.apache.spark.sql.Row

spark.read.csv is used when loading into a ...READ MORE

Jul 10 in Apache Spark by Rishi
486 views
0 votes
1 answer

How to add package com.databricks.spark.avro in spark?

Start spark shell using below line of ...READ MORE

Jul 10 in Apache Spark by Jishnu
391 views
0 votes
1 answer

Why do we use sc.parallelize?

Spark revolves around the concept of a ...READ MORE

Jul 11 in Apache Spark by Suman
349 views
0 votes
1 answer

Spark on Yarn

If you just want to get your ...READ MORE

Jul 18 in Apache Spark by ravikiran
• 4,560 points
37 views
0 votes
2 answers

map() vs flatMap() in Spark

Spark map function expresses a one-to-one transformation. ...READ MORE

Jun 17 in Apache Spark by vishal
• 160 points
3,513 views
0 votes
1 answer

Spark-shell not working

First, reboot the system. And after reboot, ...READ MORE

Jul 15 in Apache Spark by Mahesh
105 views
0 votes
1 answer

Spark error: Caused by: java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be writable.

Give  read-write permissions to  C:\tmp\hive folder Cd to winutils bin folder ...READ MORE

Jul 11 in Apache Spark by Rajiv
266 views
0 votes
1 answer

How do I turn off INFO Logging in Spark?

Execute this command in the spark directory: cp ...READ MORE

Jul 12 in Apache Spark by ravikiran
• 4,560 points
214 views
0 votes
1 answer

Pyspark is taking default path

The HDFS path for MyLab is /user/edureka_id. ...READ MORE

Jul 16 in Apache Spark by Khushi
32 views
0 votes
1 answer

Spark: How can i create temp views in user defined database instead of default database?

You can try the below code: df.registerTempTable(“airports”) sqlContext.sql(" create ...READ MORE

Jul 14 in Apache Spark by Ishan
112 views
0 votes
1 answer

Spark Processing Internals

Spark uses a master/slave architecture. As you ...READ MORE

Jul 15 in Apache Spark by Jimmy
41 views
0 votes
1 answer

What does the command df.registerTempTable() do?

df.registerTempTable(“airports”) This command is used to register ...READ MORE

Jul 14 in Apache Spark by James
100 views
0 votes
7 answers

How to print the contents of RDD in Apache Spark?

Simple and easy: line.foreach(println) READ MORE

Dec 10, 2018 in Apache Spark by Kuber
12,553 views
0 votes
1 answer

Can we change the path where the Hive data is stored in HDFS?

Yes, you can but it has to ...READ MORE

Jul 14 in Apache Spark by Yogi
36 views
0 votes
1 answer

Spark memory processing on a not temporary table

Temporary table is more like an index ...READ MORE

Jul 14 in Apache Spark by Suri
26 views
0 votes
1 answer

Spark Streaming Pyspark code not working

The address you are using in the ...READ MORE

Jul 11 in Apache Spark by Shir
126 views
0 votes
1 answer

What is the difference between persist() and cache() in apache spark?

Hi, persist () allows the user to specify ...READ MORE

Jul 3 in Apache Spark by Gitika
• 25,340 points
436 views
0 votes
1 answer

Error Loading data to mysql in Spark

You have to use sqoop to export data ...READ MORE

Jul 11 in Apache Spark by Jishan
49 views
0 votes
1 answer

Is fetching data from apache flume webcrawling?

Web crawling is a program or automated ...READ MORE

Jul 11 in Apache Spark by Esha
39 views
0 votes
1 answer

Working of map function on data

The map function creates an array of ...READ MORE

Jul 11 in Apache Spark by Krish
28 views
0 votes
1 answer

Scala join comma delimited file as tables

Dataframe creation commands:​ Now we will register them ...READ MORE

Jul 9 in Apache Spark by Suraj
37 views
0 votes
1 answer

Error: value textfile is not a member of org.apache.spark.SparkContext

Hi, Regarding this error, you just need to change ...READ MORE

Jul 4 in Apache Spark by Gitika
• 25,340 points
245 views
0 votes
1 answer

What is Map and flatMap in Spark?

Hi, The map is a specific line or ...READ MORE

Jul 3 in Apache Spark by Gitika
• 25,340 points
212 views
0 votes
1 answer

What is sparkContext?

Hi, Spark Context is the entry point to ...READ MORE

Jul 3 in Apache Spark by Gitika
• 25,340 points
199 views
0 votes
1 answer

How can we iterate any function using "foreach" function in scala?

Hi, Yes, "foreach" function you use because it will ...READ MORE

Jul 5 in Apache Spark by Gitika
• 25,340 points
96 views
0 votes
1 answer

How to use yield keyword in scala and why it is used instead of println?

Hi, The yield keyword is used because the ...READ MORE

Jul 5 in Apache Spark by Gitika
• 25,340 points
73 views
0 votes
1 answer

load/save in spark

The reason why you are able to ...READ MORE

Jul 5 in Apache Spark by Firoz
80 views
0 votes
1 answer

Which File System is supported by Apache Spark?

Hi, Apache Spark is an advanced data processing ...READ MORE

Jul 5 in Apache Spark by Gitika
• 25,340 points
80 views
0 votes
1 answer

How to create scala project in intellij?

You have to install Intellij with scala plugin. ...READ MORE

Jul 5 in Apache Spark by Jimmy
70 views
0 votes
1 answer

error: identified expected but integer literal found.

Hi, You can resolve this error with a ...READ MORE

Jul 4 in Apache Spark by Gitika
• 25,340 points
120 views
0 votes
1 answer

How can we optimize and minimize the memory when work with scala use case?

Hi, There is a term in Scala that is ...READ MORE

Jul 5 in Apache Spark by Gitika
• 25,340 points
46 views
0 votes
1 answer

How to print string text in scala?

Hi, You can see this example to see ...READ MORE

Jul 5 in Apache Spark by Gitika
• 25,340 points
43 views
0 votes
1 answer

Which syntax to use to take the sum of list of collection in scala?

Hi, You can see this example to get ...READ MORE

Jul 5 in Apache Spark by Gitika
• 25,340 points
41 views
0 votes
1 answer

How to run spark in Standalone client mode?

Hi, These are the steps to run spark in ...READ MORE

Jul 5 in Apache Spark by Gitika
• 25,340 points
55 views
0 votes
1 answer

How can you use "for" statement in scala to print list from collection?

Hi, You can use for loop in scala using ...READ MORE

Jul 5 in Apache Spark by Gitika
• 25,340 points
38 views
0 votes
1 answer

What is Lazy evaluated in Spark?

Hi, If you execute a bunch of programs, ...READ MORE

Jul 5 in Apache Spark by Gitika
• 25,340 points
52 views
0 votes
1 answer

Do real-time data processing is possible with Spark SQL?

Hey, Real-time data processing is not possible directly ...READ MORE

Jul 5 in Apache Spark by Gitika
• 25,340 points
50 views
0 votes
1 answer

How to assign block expression in scala?

Hi, You can follow this example to know ...READ MORE

Jul 5 in Apache Spark by Gitika
• 25,340 points
34 views
0 votes
1 answer

What is RDD Lineage in Spark?

Hey, Lineage is an RDD process to reconstruct ...READ MORE

Jul 4 in Apache Spark by Gitika
• 25,340 points
90 views
0 votes
1 answer

How to print loop with condition in scala?

Hi, Yes, in scala there is a guard condition where ...READ MORE

Jul 5 in Apache Spark by Gitika
• 25,340 points
30 views
0 votes
1 answer

How SparkSQL is different from HQL and SQL?

Hi, SparkSQL is a special component on the ...READ MORE

Jul 3 in Apache Spark by Gitika
• 25,340 points
129 views