Latest questions in Apache Spark

0 votes
0 answers

How to import pyspark in Jupyter

I tried to import pyspark in jupyter ...READ MORE

Apr 3, 2023 in Apache Spark by Navyasilpa
edited Mar 5 • 260 views
0 votes
0 answers

How to import pyspark in Jupyter Notebook

When I tried to import Pyspark I am getting ...READ MORE

Apr 3, 2023 in Apache Spark by Navyasilpa
edited Mar 5 • 301 views
0 votes
0 answers

How to read a nested avro file format in spark dataframe

The avro file format contains nested data. ...READ MORE

Nov 16, 2022 in Apache Spark by Devang
edited Mar 4 • 280 views
0 votes
0 answers

How can i implement corss apply function of TSQL in pyspark

How can i implement corss apply function ...READ MORE

May 30, 2022 in Apache Spark by anonymous
edited Mar 4 • 320 views
0 votes
0 answers

Pyspark: Aggregate and filtering code error

Hi guys, I am a beginner at pyspark ...READ MORE

Apr 22, 2022 in Apache Spark by Saadat
edited Mar 4 • 322 views
0 votes
0 answers

Pyspark: Finding top three countries with covid confirmed covid cases

Hi guys, I have a beginner at pyspark ...READ MORE

Apr 22, 2022 in Apache Spark by Saadat
edited Mar 4 • 230 views
0 votes
0 answers

Scala / SparkSQL dataframes filter issue "data type mismatch"

My probleme is i have a code ...READ MORE

Mar 24, 2022 in Apache Spark by Hamza
edited Mar 4 • 182 views
0 votes
0 answers

Access value in arrays of structs spark scala

Hi, I have a dataset with the ...READ MORE

Mar 24, 2022 in Apache Spark by anonymous
edited Mar 4 • 175 views
0 votes
0 answers

What should I pay attention to when installing smart curtains fabrics?

What should I pay attention to when ...READ MORE

Mar 23, 2022 in Apache Spark by qiansifang
edited Mar 4 • 188 views
0 votes
0 answers

The Batman Movie Online Free HD

dfgsdfg READ MORE

Mar 4, 2022 in Apache Spark by anonymous
edited Mar 4 • 161 views
0 votes
0 answers

How can I use substring to extract the four last characters of a column if I do not know the number of characters that are within a field?

I want to only get the last ...READ MORE

Jan 14, 2022 in Apache Spark by anonymous
edited Mar 4 • 187 views
0 votes
0 answers

Execute Spark.sql query within withColumn clause is Spark Scala

I have a dataframe which has one ...READ MORE

Sep 14, 2021 in Apache Spark by Pinksrider
edited Mar 4 • 208 views
0 votes
0 answers

Aws logs are not writing in cloud watch after certain steps

i have an aws job which reads ...READ MORE

Jul 30, 2021 in Apache Spark by Anjali
edited Mar 4 • 200 views
0 votes
0 answers

Create Hive table using Dataframe getting error

Code: srcDF.write.mode(tblmode).saveAsTable(s"${dbName}.${tgtHiveTableName}") error: 21/06/04 22:11:45 ERROR pa.TrxNbrx: org.apache.spark.SparkException: ...READ MORE

Jun 5, 2021 in Apache Spark by Rajesh
edited Mar 4 • 176 views
0 votes
0 answers

OI JANA TESTE LIVE

OI JANA TESTE LIVE READ MORE

Jun 5, 2021 in Apache Spark by Eufrasia
edited Mar 4 • 162 views
0 votes
0 answers

what parameters are required for a "windowed" operation such as reduceByKeyAndWindow?

a) Window length b) sliding interval c) Window Length ...READ MORE

Jun 4, 2021 in Apache Spark by anonymous
edited Mar 4 • 191 views
0 votes
0 answers

It is a schema for a pyspark nested dataframe ..what I want is to change the name of the ambiguous terms configdata to configdata0, configdata1 and so on . So how can it be achieved using rdd?

root |-- fields: struct (nullable = true) | ...READ MORE

Apr 27, 2021 in Apache Spark by sushant
edited Mar 4 • 208 views
0 votes
0 answers

Real time Project challenges in Spark Data pipeline

Can anybody highlights some challenges they have ...READ MORE

Apr 6, 2021 in Apache Spark by anonymous
edited Mar 4 • 191 views
0 votes
0 answers

ForeachPartition called with python functions doesnot output data to HDFS when executed on YARN mode

Hi, I need help with my code. Trying ...READ MORE

Jan 22, 2021 in Apache Spark by anonymous
edited Mar 4 • 167 views
0 votes
1 answer

Spark Core How to fetch max n rows of an RDD function without using Rdd.max()

Hi@Prasant, If Spark Streaming is not supporting tuple, ...READ MORE

Dec 3, 2020 in Apache Spark by MD
 • 95,460 points • 2,603 views
0 votes
1 answer

What will be printed when the below code is executed?

Option a) 443 READ MORE

Mar 8, 2023 in Apache Spark by anonymous
edited Mar 5 • 2,907 views
0 votes
0 answers

What allows spark to periodically persist data about an application such that it can recover from failures? [closed]

What allows spark to periodically persist data ...READ MORE

Nov 26, 2020 in Apache Spark by ritu
 • 960 points
closed Nov 26, 2020 by MD • 3,278 views
0 votes
1 answer

What class is declared in the blow code?

Option D: String class READ MORE

Nov 26, 2020 in Apache Spark by Gitika
 • 65,730 points • 1,356 views
0 votes
1 answer

In AWS, if user wants to run spark, then on top of which one of the following can the user do it?

Hi@ritu, AWS has lots of services. For spark ...READ MORE

Nov 26, 2020 in Apache Spark by MD
 • 95,460 points • 1,840 views
0 votes
1 answer

Which one of the following commands is used to start python-spark?

Hi@ritu, To start your python spark shell, you ...READ MORE

Nov 26, 2020 in Apache Spark by MD
 • 95,460 points • 1,893 views
0 votes
1 answer

What will be printed when the below code is executed ?

List 5 100 10 READ MORE

Feb 7, 2023 in Apache Spark by Subbu
edited Mar 5 • 1,931 views
0 votes
1 answer

What is the output of the following code?

rror: expected class or object definition sc.parallelize(Array(1L,("SFO")),(2L,("ORD")),(3L,("DFW")))) ^ one error ...READ MORE

Nov 26, 2020 in Apache Spark by Gitika
 • 65,730 points • 1,270 views
0 votes
1 answer

A Dataframe can be created from an existing RDD. You would create the Dataframe from the existing RDD by inferring schema using case classes in which one of the given classes?

if you have two sets of users ...READ MORE

Sep 27, 2021 in Apache Spark by anonymous
edited Mar 5 • 5,209 views
0 votes
1 answer

The number of stages in a job is equal to the number of RDDs in DAG. however, under one of the cgiven conditions, the scheduler can truncate the lineage. identify it.

Hi@ritu, Spark's internal scheduler may truncate the lineage of the RDD graph if ...READ MORE

Nov 25, 2020 in Apache Spark by akhtar
 • 38,260 points • 3,064 views
0 votes
0 answers

The number of stages in a job is equal to the number of RDDs in DAG. however, under one of the cgiven conditions, the scheduler can truncate the lineage. identify it. [closed]

14)The number of stages in a job ...READ MORE

Nov 25, 2020 in Apache Spark by Edureka
 • 200 points
closed Nov 25, 2020 by MD • 5,446 views
0 votes
0 answers

What is the output of the following code? [closed]

What is the output of the following ...READ MORE

Nov 25, 2020 in Apache Spark by Edureka
 • 200 points
closed Nov 26, 2020 by MD • 1,019 views
0 votes
1 answer

16)What allows spark to periodically persist data about an application such that it can recover from failures?

Hi@Edureka, Checkpointing is a process of truncating RDD ...READ MORE

Nov 26, 2020 in Apache Spark by MD
 • 95,460 points • 2,285 views
0 votes
0 answers

A Dataframe can be created from an existing RDD. You would create the Dataframe from the existing RDD by inferring schema using case classes in which one of the given classes? [closed]

A Dataframe can be created from an ...READ MORE

Nov 25, 2020 in Apache Spark by Edureka
 • 200 points
closed Nov 26, 2020 by MD • 2,603 views
0 votes
1 answer

The number of stages in a job is equal to the number of RDDs in DAG. however, under one of the cgiven conditions, the scheduler can truncate the lineage. identify it.

Hi@Edureka, Spark's internal scheduler may truncate the lineage of the RDD graph ...READ MORE

Nov 26, 2020 in Apache Spark by MD
 • 95,460 points • 4,580 views
0 votes
1 answer

13)Refer the input and identify the output if the below code is run

Option c)  Run time error - A READ MORE

Nov 25, 2020 in Apache Spark by Gitika
 • 65,730 points • 1,416 views
0 votes
1 answer

What does the following code print?

error: expected class or object definition sc.parallelize (Array(1L, ...READ MORE

Nov 25, 2020 in Apache Spark by Gitika
 • 65,730 points • 2,374 views
0 votes
1 answer

From the following graph code ,which code snippet will return the no.of flight routes?

Hey, @Ritu, I am getting error in your ...READ MORE

Nov 25, 2020 in Apache Spark by Gitika
 • 65,730 points • 1,299 views
0 votes
1 answer

What is the output of the following code?

After executing your code, there is an ...READ MORE

Nov 25, 2020 in Apache Spark by Gitika
 • 65,730 points • 1,329 views
0 votes
1 answer

From the below code. what is the most appropriate next step in ML process?

Hi@ritu, The most appropriate step according to me ...READ MORE

Nov 25, 2020 in Apache Spark by MD
 • 95,460 points • 1,601 views
0 votes
1 answer

What are some of the things you can monitor in the Spark Web UI?

The stages which are running slow READ MORE

Apr 29, 2021 in Apache Spark by anonymous
edited Mar 5 • 4,363 views
0 votes
0 answers

What does the below code print? [closed]

What does the below code print? val AgeDs ...READ MORE

Nov 25, 2020 in Apache Spark by ritu
 • 960 points
closed Nov 25, 2020 by Gitika • 1,425 views
0 votes
1 answer

What does the below code print?

Option d) Run time error. READ MORE

Nov 25, 2020 in Apache Spark by Gitika
 • 65,730 points • 1,605 views
0 votes
1 answer

which one of the following commands is used to see the structure of the Dataframe?

Hi @Ritu If you want to see the ...READ MORE

Nov 25, 2020 in Apache Spark by Gitika
 • 65,730 points • 2,981 views
0 votes
0 answers

17)from the given choices, identify the value returned by $"whatever"?

17)from the given choices, identify the value ...READ MORE

Nov 25, 2020 in Apache Spark by ritu
 • 960 points • 1,851 views
0 votes
1 answer

12)Which one of the given flows correctly describe the Spark Streaming Architecture?

C.  Data streams divided into batches > ...READ MORE

Jul 3, 2022 in Apache Spark by anonymous
edited Mar 5 • 4,316 views
0 votes
1 answer

9)In scala, which one of the following will give the top 10 resolutions to the console, assuming that sfpdDF is the DataFrame registered as a table - sfpd?

Hey, @Ritu, According to the question, the answer ...READ MORE

Nov 23, 2020 in Apache Spark by Gitika
 • 65,730 points • 3,430 views
0 votes
1 answer

Spark - how the solve the below question?

option d, Runtime error READ MORE

Nov 23, 2020 in Apache Spark by Gitika
 • 65,730 points • 1,095 views
0 votes
1 answer

7)From Schema RDD, data can be cache by which one of the given choices?

Hi, @Ritu, According to the official documentation of Spark 1.2, ...READ MORE

Nov 23, 2020 in Apache Spark by Gitika
 • 65,730 points • 2,379 views
0 votes
1 answer

6)What allows spark streaming to provide fault tolerance for network sources of data?

Hi@ritu, Fault tolerance is the property that enables ...READ MORE

Dec 1, 2020 in Apache Spark by MD
 • 95,460 points • 2,809 views
0 votes
2 answers

5)Using which one of the given choices will you create an RDD with specific partitioning?

Hi, @Ritu, option b for you, as Hash Partitioning ...READ MORE

Nov 23, 2020 in Apache Spark by Gitika
 • 65,730 points • 5,005 views
Page:

Most popular tags

"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.