Recent questions tagged apache-spark

0 votes
0 answers

Are there any services like Databricks but without compulsion of AWS/Azure/Google Cloud?

Jan 24, 2023 in AWS by Tejashwini
 • 5,380 points • 884 views
0 votes
1 answer

Azure HD Insight - YARN UI is not showing logs on stderr suddenly

Mar 7, 2022 in Azure by Edureka
 • 12,700 points • 1,229 views
0 votes
1 answer

What is apache hadoop?

Dec 13, 2021 in Big Data Hadoop by CoderGirl
 • 500 points • 1,464 views
0 votes
1 answer

Spark Core How to fetch max n rows of an RDD function without using Rdd.max()

Dec 3, 2020 in Apache Spark by Prashant
 • 120 points • 2,579 views
0 votes
1 answer

In AWS, if user wants to run spark, then on top of which one of the following can the user do it?

Nov 26, 2020 in Apache Spark by ritu
 • 960 points • 1,820 views
0 votes
1 answer

error: value update is not a member of scala.collection.immutable.Map[String, Int]

Nov 17, 2020 in Big Data Hadoop by anonymous
 • 8,870 points • 6,123 views
0 votes
1 answer

How to read Avro Partition Data?

Nov 4, 2020 in Apache Spark by akhtar
 • 38,260 points • 2,203 views
+1 vote
1 answer

How to write Spark DataFrame to Avro Data File?

Nov 4, 2020 in Apache Spark by akhtar
 • 38,260 points • 3,839 views
0 votes
1 answer

How to read a dataframe based on an avro schema?

Oct 30, 2020 in Apache Spark by anonymous
 • 120 points • 3,888 views
0 votes
1 answer

How to implement my clustering algorithm in pyspark (without using the ready library for example k-means)?

Oct 14, 2020 in Apache Spark by dani
 • 160 points • 2,342 views
0 votes
1 answer

How to insert data into Cassandra table using Spark DataFrame?

Sep 21, 2020 in Apache Spark by akhtar
 • 38,260 points • 4,318 views
0 votes
1 answer

How to merge two Spark DataFrames?

Sep 17, 2020 in Big Data Hadoop by akhtar
 • 38,260 points • 2,234 views
0 votes
1 answer

How can browse without port usage (example: "http://<IP>" )on the broswer if the php artisan server on "http://<IP>:8088"

Jul 31, 2020 in Web Development by Raghu
 • 120 points • 1,688 views
0 votes
1 answer

File not found exception while processing the spark job in yarn cluster mode with multinode hadoop cluster

Jul 30, 2020 in Apache Spark by Ganendra
 • 140 points • 5,325 views
0 votes
1 answer

Unable to submit the spark job in deployment mode - multinode cluster(using ubuntu machines) with yarn master

Jul 29, 2020 in Apache Spark by Ganendra
 • 140 points • 2,972 views
0 votes
0 answers

Unable to get the Job status and Group ID java- spark standalone program with databricks

Jul 23, 2020 in Apache Spark by kamboj
 • 140 points • 2,937 views
0 votes
1 answer

Can number of Spark task be greater than the executor core?

Jun 17, 2020 in Apache Spark by Rishi
 • 160 points • 2,461 views
0 votes
1 answer

Can the executor core be greater than the total number of spark tasks?

Jun 17, 2020 in Apache Spark by Rishi
 • 160 points • 2,650 views
0 votes
1 answer

after installing hadoop 3.0.1 I can's access spark shell or hive shell.

Jun 16, 2020 in Apache Spark by abdul
 • 120 points • 1,812 views
0 votes
1 answer

Where can I get best spark tutorials for beginners?

May 14, 2020 in Apache Spark by akhtar
 • 38,260 points • 1,043 views
0 votes
1 answer

Py4JJavaError: An error occurred while calling o310.csv. : java.net.ConnectException: Call From master/192.168.56.101 to master:9000

May 7, 2020 in Apache Spark by akhtar
 • 38,260 points • 7,997 views
+1 vote
1 answer

How to convert pyspark Dataframe to pandas Dataframe?

May 7, 2020 in Apache Spark by akhtar
 • 38,260 points • 8,531 views
0 votes
1 answer

How to integrate Machine Learning with Spark?

May 6, 2020 in Machine Learning by akhtar
 • 38,260 points • 1,154 views
0 votes
1 answer

ERROR thriftserver.SparkExecuteStatementOperation: Error executing query, currentState RUNNING, org.apache.spark.sql.catalyst.errors.package$TreeNodeException

Apr 29, 2020 in Apache Spark by akhtar
 • 38,260 points • 2,357 views
0 votes
1 answer

Error: sql.out:Error: org.apache.spark.SparkException: Job aborted due to stage failure: Total size of serialized results of 381610 tasks (4.0 GB) is bigger than spark.driver.maxResultSize (4.0 GB)

Apr 29, 2020 in Apache Spark by akhtar
 • 38,260 points • 8,745 views
0 votes
1 answer

"java.lang.ClassNotFoundException" in Spark on Amazon EMR

Apr 29, 2020 in Apache Spark by akhtar
 • 38,260 points • 4,206 views
0 votes
1 answer

error: Caused by: org.apache.spark.SparkException: Failed to execute user defined function.

Apr 22, 2020 in Apache Spark by akhtar
 • 38,260 points • 4,935 views
+2 votes
2 answers

py4j.protocol.Py4JError: org.apache.spark.api.python.PythonUtils.getEncryptionEnabled does not exist in the JVM

Apr 7, 2020 in Apache Spark by akhtar
 • 38,260 points • 23,773 views
0 votes
1 answer

env: ‘python’: No such file or directory in pyspark.

Apr 7, 2020 in Apache Spark by akhtar
 • 38,260 points • 6,890 views
–1 vote
0 answers

How to parse an S3 XML file to find tags using apache spark

Mar 18, 2020 in Apache Spark by anonymous
 • 110 points • 2,310 views
+2 votes
1 answer

FileStreamSink: Error while looking for metadata directory. java.lang.IllegalArgumentException: java.net.UnknownHostException: hive

Feb 13, 2020 in Big Data Hadoop by akhtar
 • 38,260 points • 11,935 views
0 votes
1 answer

apache.spark.sql.AnalysisException: Text data source does not support int data type.;

Feb 13, 2020 in Big Data Hadoop by akhtar
 • 38,260 points • 6,612 views
+1 vote
1 answer

What is the importance of Kafka bootstrap.servers?

Feb 11, 2020 in Apache Kafka by akhtar
 • 38,260 points • 18,437 views
0 votes
1 answer

org.apache.kafka.common.config.ConfigException: Missing required configuration "bootstrap.servers" which has no default value.

Feb 10, 2020 in Apache Kafka by akhtar
 • 38,260 points • 7,976 views
0 votes
0 answers

What is the difference between point to point and publish-subscribe messaging system?

Feb 6, 2020 in Apache Kafka by akhtar
 • 38,260 points • 1,022 views
+1 vote
1 answer

ERROR Unexpected exception, exiting abnormally (org.apache.zookeeper.server.ZooKeeperServerMain) java.net.BindException: Address already in use

Feb 5, 2020 in Apache Kafka by akhtar
 • 38,260 points • 13,071 views
0 votes
1 answer

Exception in thread "main" java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D

Feb 5, 2020 in Big Data Hadoop by akhtar
 • 38,260 points • 9,401 views
0 votes
1 answer

Does spark streaming provides checkpoint?

Feb 4, 2020 in Apache Spark by akhtar
 • 38,260 points • 1,871 views
0 votes
0 answers

not able to get output in spark streaming??

Feb 4, 2020 in Apache Spark by akhtar
 • 38,260 points • 1,191 views
0 votes
1 answer

Is Spark Sql provides indexing to improve processing speed?

Feb 4, 2020 in Apache Spark by akhtar
 • 38,260 points • 1,323 views
0 votes
1 answer

What is the difference between spark streaming and spark structured streaming?

Feb 4, 2020 in Apache Spark by akhtar
 • 38,260 points • 4,265 views
0 votes
1 answer

What are Dstreams?

Feb 4, 2020 in Apache Spark by akhtar
 • 38,260 points • 1,485 views
0 votes
1 answer

Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@2c7bde26

Feb 3, 2020 in Apache Spark by akhtar
 • 38,260 points • 6,220 views
0 votes
1 answer

Cannot create directory /hive/xzxz/_temporary/0. Name node is in safe mode.

Feb 3, 2020 in Apache Spark by akhtar
 • 38,260 points • 1,146 views
+1 vote
1 answer

is not a Parquet file. expected magic number at tail [80, 65, 82, 49] but found [51, 53, 10, 10]

Feb 3, 2020 in Apache Spark by akhtar
 • 38,260 points • 19,191 views
0 votes
1 answer

What is pageRank in graphX??

Jan 31, 2020 in Apache Spark by akhtar
 • 38,260 points • 1,577 views
0 votes
1 answer

env : R : No such file or directory

Jan 31, 2020 in Apache Spark by Hasid
 • 370 points • 2,120 views
0 votes
2 answers

java.lang.StringIndexOutOfBoundsException: String index out of range: 1

Jan 29, 2020 in Apache Spark by akhtar
 • 38,260 points • 8,279 views
0 votes
1 answer

Not enough space to cache rdd_80_1 in memory!

Jan 29, 2020 in Apache Spark by akhtar
 • 38,260 points • 3,243 views
0 votes
1 answer

Caused by: java.lang.NumberFormatException: Empty String

Jan 29, 2020 in Apache Spark by akhtar
 • 38,260 points • 5,441 views
Page:

Most popular tags

"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.