Trending questions in Apache Spark

0 votes
1 answer

How to restrict a group to only view in Spark?

You can do it dynamically be setting ...READ MORE

Mar 15 in Apache Spark by Raj
49 views
0 votes
1 answer

How enable XSS protection for Spark application?

Spark has a built-in prevention system against XSS. ...READ MORE

Mar 15 in Apache Spark by Karan
39 views
0 votes
1 answer

How to set keystore path?

You have to set the path to ...READ MORE

Mar 15 in Apache Spark by Karan
34 views
0 votes
1 answer

How to disable credentials for services in Spark?

Disabling this feature will compromise the security ...READ MORE

Mar 15 in Apache Spark by Karan
31 views
0 votes
1 answer

Setting port for SSL for Spark application

You can set the port in the ...READ MORE

Mar 15 in Apache Spark by Karan
28 views
0 votes
1 answer

How to change scheduling mode in Spark?

You can change the scheduling mode as ...READ MORE

Mar 11 in Apache Spark by Raj
195 views
0 votes
1 answer

Password for keystore in Spark

You can dynamically set a password to ...READ MORE

Mar 15 in Apache Spark by karan
26 views
0 votes
1 answer

Configuring location of credential provider

To configure the location of the credential ...READ MORE

Mar 15 in Apache Spark by Karan
23 views
0 votes
1 answer

How to give modify access to group?

You can do this using the following ...READ MORE

Mar 15 in Apache Spark by Vikas
16 views
0 votes
1 answer

How to set max executors for dynamic allocation?

You can set it by assigning the ...READ MORE

Mar 13 in Apache Spark by Venu
95 views
0 votes
1 answer

How to add modify access for Web UI user?

For a user to have modification access ...READ MORE

Mar 14 in Apache Spark by Raj
36 views
0 votes
1 answer

How to enable dynamic resource allocation in Spark?

To dynamically enable dynamic resource allocation, you ...READ MORE

Mar 12 in Apache Spark by veer
123 views
0 votes
1 answer

How to change encryption key value for local I/O?

There another property where you can set ...READ MORE

Mar 14 in Apache Spark by Raj
30 views
0 votes
1 answer

How to give user only view access for Spark application?

You can give users only view permission ...READ MORE

Mar 14 in Apache Spark by Raj
28 views
0 votes
1 answer

How to enable SASL encryption?

You can use dynamic configuration setting to ...READ MORE

Mar 14 in Apache Spark by Raj
28 views
0 votes
1 answer

SparkContext.addFile() not able to update file.

Spark by default won't let you overwrite ...READ MORE

Mar 10 in Apache Spark by Siri
226 views
0 votes
1 answer

When is speculation enabled in Spark?

Speculation is enabled when a fraction of ...READ MORE

Mar 12 in Apache Spark by veer
110 views
0 votes
1 answer

How to check if user has permission in Web UI?

You can implement this as follows: First, add ...READ MORE

Mar 14 in Apache Spark by Raj
19 views
0 votes
1 answer

Enable encryption for local Input and Output

You can enable local I/O encryption like ...READ MORE

Mar 14 in Apache Spark by Raj
19 views
0 votes
1 answer

Client connection getting rejected due to SASL authentication

Seems like you have set the configuration ...READ MORE

Mar 14 in Apache Spark by Raj
14 views
0 votes
1 answer

How to authenticate Spark internal connections using a secret key?

You need to set the secret key ...READ MORE

Mar 13 in Apache Spark by Venu
55 views
0 votes
1 answer

How to set stage abort in Spark?

You can do it as follows. Use ...READ MORE

Mar 12 in Apache Spark by veer
94 views
0 votes
1 answer

Network encryption for Spark

You can enable encryption for the Spark ...READ MORE

Mar 13 in Apache Spark by Venu
50 views
0 votes
1 answer

Disable Web UI for Spark Application

You can disable it like this: val sc ...READ MORE

Mar 6 in Apache Spark by Rohit
341 views
0 votes
1 answer

Change encryption key length fro Spark application

You can do this by running the ...READ MORE

Mar 13 in Apache Spark by Venu
34 views
0 votes
1 answer

Key Factor Algorithms used for encryption.

The default key factor algorithm used is PBKDF2WithHmacSHA1. You ...READ MORE

Mar 13 in Apache Spark by Venu
32 views
0 votes
1 answer

Remove idle executor automatically

Yes, it is possible and is already ...READ MORE

Mar 12 in Apache Spark by Veer
74 views
0 votes
1 answer

How many executors run when dynamic allocation is enabled?

The number of executors running by default ...READ MORE

Mar 13 in Apache Spark by Venu
30 views
0 votes
1 answer

Delay requesting new executor in dynamic allocation

You can set the duration like this: val ...READ MORE

Mar 13 in Apache Spark by Venu
21 views
0 votes
1 answer

How to enable SASL authentication after Spark authentication?

You can do this by setting the ...READ MORE

Mar 13 in Apache Spark by Venu
19 views
0 votes
1 answer

Dynamic allocation: Set minimum number of executor

You cans et it dynamically like this:  val ...READ MORE

Mar 13 in Apache Spark by Venu
17 views
0 votes
1 answer

Authenticate internal connections in Spark

To make Spark authenticate internal connections, you ...READ MORE

Mar 13 in Apache Spark by Venu
15 views
0 votes
1 answer

How to prevent executor from self-destructing?

I think there is a timeout set ...READ MORE

Mar 12 in Apache Spark by Veer
48 views
0 votes
1 answer

How can I remove headers from dataframe?

You can use filter to do this. ...READ MORE

Feb 14 in Apache Spark by Aryan
1,165 views
0 votes
1 answer

Changing the blacklist time of executor

By default, the node or executor is ...READ MORE

Mar 11 in Apache Spark by Raj
81 views
0 votes
1 answer

How to relaunch tasks that are running slowly?

The technical term for what you want ...READ MORE

Mar 12 in Apache Spark by Veer
40 views
0 votes
1 answer

How to automatically kill executors on blacklisting?

You can set the property to directly ...READ MORE

Mar 12 in Apache Spark by Veer
40 views
0 votes
1 answer

How to set time for task speculation?

By default, the check for task speculation ...READ MORE

Mar 12 in Apache Spark by Veer
24 views
0 votes
1 answer

How to monitor interrupted Spark tasks?

To enable monitoring interrupted tasks, run the following ...READ MORE

Mar 12 in Apache Spark by Veer
19 views
0 votes
1 answer

How to make Spark wait for more time for acknowledgement?

Use the following command to increase the ...READ MORE

Mar 11 in Apache Spark by Raj
57 views
0 votes
1 answer

Increasing retry before blacklisting executor

You can do it like this: val sc ...READ MORE

Mar 11 in Apache Spark by Raj
49 views
0 votes
1 answer

How to limit the cores being used by a cluster?

You can set the maximum number of ...READ MORE

Mar 11 in Apache Spark by Raj
44 views
0 votes
1 answer

How to increase wait time to launch data-local task?

You can increase the locality wait time ...READ MORE

Mar 11 in Apache Spark by Raj
42 views
0 votes
1 answer

Increasing retry before blacklisting a node

You can do it dynamically using the ...READ MORE

Mar 11 in Apache Spark by Raj
36 views
0 votes
1 answer

How to make Spark skip node locality?

If you set the node wait time ...READ MORE

Mar 11 in Apache Spark by Raj
33 views
0 votes
1 answer

How can I run a blacklisted task?

When a task results in too many ...READ MORE

Mar 11 in Apache Spark by Raj
20 views
0 votes
1 answer

How to disable executor from fetching file from cache?

When a Spark application is running, the ...READ MORE

Mar 10 in Apache Spark by Siri
64 views
0 votes
1 answer

How to change block manager port for driver only?

Yes, it is possible to change the ...READ MORE

Mar 10 in Apache Spark by Siri
50 views
0 votes
1 answer

How to change the location of Spark event logs?

You can change the location where you ...READ MORE

Mar 6 in Apache Spark by Rohit
181 views
0 votes
1 answer

How to enable Spark event logging?

To make Spark store the event logs, ...READ MORE

Mar 6 in Apache Spark by Rohit
175 views