Most answered questions in Apache Spark

0 votes
1 answer

How to enable worker cleanup in Spark?

To enable cleanup, open the spark shell ...READ MORE

Mar 25, 2019 in Apache Spark by Hari
 • 2,307 views
0 votes
1 answer

How to increase worker timeout in Spark application?

By default, the timeout is set to ...READ MORE

Mar 25, 2019 in Apache Spark by Hari
 • 8,446 views
0 votes
1 answer

How to disable automatic remove of application of failures?

Yes, you have read it right. The ...READ MORE

Mar 25, 2019 in Apache Spark by Hari
 • 807 views
0 votes
1 answer

How to spread applications to less nodes?

You can limit the spread out by ...READ MORE

Mar 25, 2019 in Apache Spark by Hari
 • 327 views
0 votes
1 answer

Change number of completed drivers displayed

You can change the number of completed ...READ MORE

Mar 25, 2019 in Apache Spark by Hari
 • 371 views
0 votes
1 answer

Limit displaying completed applications

By default, the number of completed applications ...READ MORE

Mar 25, 2019 in Apache Spark by hari
 • 449 views
0 votes
1 answer

Setting directory for Spark recovery

You can set the directory to store ...READ MORE

Mar 25, 2019 in Apache Spark by Hari
 • 798 views
0 votes
1 answer

How to connect to Zookeeper after setting Spark recovery mode?

You have set Zookeeper as the recovery ...READ MORE

Mar 25, 2019 in Apache Spark by Hari
 • 1,121 views
0 votes
1 answer

How to set Spark recovery to Zookeeper?

There is a property of Spark which ...READ MORE

Mar 25, 2019 in Apache Spark by Hari
 • 544 views
0 votes
1 answer

Change heart beat time to R process

The heartbeat interval is assigned to the ...READ MORE

Mar 19, 2019 in Apache Spark by Jai
 • 366 views
0 votes
1 answer

Set Spark executable for R scripts

To change the default executable, assign the ...READ MORE

Mar 19, 2019 in Apache Spark by Jai
 • 804 views
0 votes
1 answer

Change number of threads use by R back end

Refer to the below commands to know ...READ MORE

Mar 19, 2019 in Apache Spark by Jai
 • 416 views
0 votes
1 answer

Close file after writing write-ahead on driver

You can change the property to close ...READ MORE

Mar 19, 2019 in Apache Spark by Jai
 • 819 views
0 votes
1 answer

Retaining the batch by status APIs before Garbage Collection

By default, 1000 batches are retained by ...READ MORE

Mar 19, 2019 in Apache Spark by Jai
 • 529 views
0 votes
1 answer

How to change minimum rate per kafka partition?

Open spark shell using this command: $ spark-shell Then ...READ MORE

Mar 19, 2019 in Apache Spark by Jai
 • 445 views
0 votes
1 answer

Set maximum reading rate from Kafka partition

You can do it by using the ...READ MORE

Mar 19, 2019 in Apache Spark by Jai
 • 462 views
0 votes
1 answer

Prevent immediate shutdown of Streaming context after JVM shutdown

In technical terms, you want to gracefully shut down the ...READ MORE

Mar 19, 2019 in Apache Spark by Jai
 • 943 views
0 votes
1 answer

Need to disable unpersist in Spark

You can dynamically change this function by ...READ MORE

Mar 19, 2019 in Apache Spark by Jai
 • 784 views
0 votes
1 answer

How to enable write ahead logs for receivers?

To enable write-ahead logs, run the following ...READ MORE

Mar 19, 2019 in Apache Spark by Jai
 • 637 views
0 votes
1 answer

Using R to display configuration of Spark SQL

Try the below-mentioned code. sparkR.session() properties <- sql("SET -v") showDF(properties, ...READ MORE

Mar 18, 2019 in Apache Spark by John
 • 417 views
0 votes
1 answer

How to get SQL configuration in Spark using Python?

You can get the configuration details through ...READ MORE

Mar 18, 2019 in Apache Spark by John
 • 978 views
0 votes
1 answer

Get Spark SQL configuration in Java

You will need to use Spark session ...READ MORE

Mar 18, 2019 in Apache Spark by John
 • 783 views
0 votes
1 answer

How to set maximum receiving rate for backpressure mechanism?

You can set the maximum receiving rate ...READ MORE

Mar 18, 2019 in Apache Spark by John
 • 414 views
0 votes
1 answer

How to get Spark SQL configuration?

First create a Spark session like this: val ...READ MORE

Mar 18, 2019 in Apache Spark by John
 • 3,168 views
0 votes
1 answer

what are the job optimization Technics in spark and scala ?

There are different methods to achieve optimization ...READ MORE

Mar 18, 2019 in Apache Spark by Veer
 • 1,917 views
0 votes
1 answer

what are the spark job and spark task and spark staging ?

In a Spark application, when you invoke ...READ MORE

Mar 18, 2019 in Apache Spark by Pavan
 • 10,854 views
0 votes
1 answer

what are the spark real time issues ?

Some of the issues I have faced ...READ MORE

Mar 18, 2019 in Apache Spark by Sharman
 • 4,982 views
0 votes
1 answer

How to disable credentials for services in Spark?

Disabling this feature will compromise the security ...READ MORE

Mar 15, 2019 in Apache Spark by Karan
 • 909 views
0 votes
1 answer

How enable XSS protection for Spark application?

Spark has a built-in prevention system against XSS. ...READ MORE

Mar 15, 2019 in Apache Spark by Karan
 • 779 views
0 votes
1 answer

Configuring location of credential provider

To configure the location of the credential ...READ MORE

Mar 15, 2019 in Apache Spark by Karan
 • 428 views
0 votes
1 answer

How to access private key password with Spark?

Spark allows you to retrieve the key ...READ MORE

Mar 15, 2019 in Apache Spark by Karan
 • 1,605 views
0 votes
1 answer

How to set client authentication for SSL?

By default, this feature is disabled. To ...READ MORE

Mar 15, 2019 in Apache Spark by Karan
 • 784 views
0 votes
1 answer

Can I set different protocol for SSL in Spark?

There is no protocol set by default. ...READ MORE

Mar 15, 2019 in Apache Spark by Karan
 • 901 views
0 votes
1 answer

How to set keystore path?

You have to set the path to ...READ MORE

Mar 15, 2019 in Apache Spark by Karan
 • 1,995 views
0 votes
1 answer

Password for keystore in Spark

You can dynamically set a password to ...READ MORE

Mar 15, 2019 in Apache Spark by karan
 • 594 views
0 votes
1 answer

Setting port for SSL for Spark application

You can set the port in the ...READ MORE

Mar 15, 2019 in Apache Spark by Karan
 • 537 views
0 votes
1 answer

How to enable SSL for Spark application?

You can do it dynamically like this: val ...READ MORE

Mar 15, 2019 in Apache Spark by Karan
 • 2,225 views
0 votes
1 answer

Spark to check if a particular string exists in a file

You can use this: lines = sc.textFile(“hdfs://path/to/file/filename.txt”); def isFound(line): if ...READ MORE

Mar 15, 2019 in Apache Spark by Raj
 • 7,077 views
0 votes
1 answer

How to restrict a group to only view in Spark?

You can do it dynamically be setting ...READ MORE

Mar 15, 2019 in Apache Spark by Raj
 • 464 views
0 votes
1 answer

How to give modify access to group?

You can do this using the following ...READ MORE

Mar 15, 2019 in Apache Spark by Vikas
 • 408 views
0 votes
1 answer

How to give user only view access for Spark application?

You can give users only view permission ...READ MORE

Mar 14, 2019 in Apache Spark by Raj
 • 1,270 views
0 votes
1 answer

How to add modify access for Web UI user?

For a user to have modification access ...READ MORE

Mar 14, 2019 in Apache Spark by Raj
 • 416 views
0 votes
1 answer

How to check if user has permission in Web UI?

You can implement this as follows: First, add ...READ MORE

Mar 14, 2019 in Apache Spark by Raj
 • 792 views
0 votes
1 answer

How to change encryption key value for local I/O?

There another property where you can set ...READ MORE

Mar 14, 2019 in Apache Spark by Raj
 • 496 views
0 votes
1 answer

Enable encryption for local Input and Output

You can enable local I/O encryption like ...READ MORE

Mar 14, 2019 in Apache Spark by Raj
 • 349 views
0 votes
1 answer

Client connection getting rejected due to SASL authentication

Seems like you have set the configuration ...READ MORE

Mar 14, 2019 in Apache Spark by Raj
 • 1,186 views
0 votes
1 answer

How to enable SASL encryption?

You can use dynamic configuration setting to ...READ MORE

Mar 14, 2019 in Apache Spark by Raj
 • 594 views
0 votes
1 answer

How to enable SASL authentication after Spark authentication?

You can do this by setting the ...READ MORE

Mar 13, 2019 in Apache Spark by Venu
 • 857 views
0 votes
1 answer

Key Factor Algorithms used for encryption.

The default key factor algorithm used is PBKDF2WithHmacSHA1. You ...READ MORE

Mar 13, 2019 in Apache Spark by Venu
 • 339 views
0 votes
1 answer

Change encryption key length fro Spark application

You can do this by running the ...READ MORE

Mar 13, 2019 in Apache Spark by Venu
 • 482 views
Page:

Most popular tags

"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.