Recent questions tagged apache-spark

0 votes
0 answers

Are there any services like Databricks but without compulsion of AWS/Azure/Google Cloud?

Jan 24, 2023 in AWS by Tejashwini
 • 3,820 points • 539 views
0 votes
1 answer

Azure HD Insight - YARN UI is not showing logs on stderr suddenly

Mar 7, 2022 in Azure by Edureka
 • 12,690 points • 671 views
0 votes
1 answer

What is apache hadoop?

Dec 13, 2021 in Big Data Hadoop by CoderGirl
 • 500 points • 765 views
0 votes
1 answer

Spark Core How to fetch max n rows of an RDD function without using Rdd.max()

Dec 3, 2020 in Apache Spark by Prashant
 • 120 points • 1,981 views
0 votes
1 answer

In AWS, if user wants to run spark, then on top of which one of the following can the user do it?

Nov 26, 2020 in Apache Spark by ritu
 • 960 points • 1,265 views
0 votes
1 answer

error: value update is not a member of scala.collection.immutable.Map[String, Int]

Nov 17, 2020 in Big Data Hadoop by anonymous
 • 8,910 points • 5,123 views
0 votes
1 answer

How to read Avro Partition Data?

Nov 4, 2020 in Apache Spark by akhtar
 • 38,240 points • 1,666 views
+1 vote
1 answer

How to write Spark DataFrame to Avro Data File?

Nov 4, 2020 in Apache Spark by akhtar
 • 38,240 points • 2,957 views
0 votes
1 answer

How to read a dataframe based on an avro schema?

Oct 30, 2020 in Apache Spark by anonymous
 • 120 points • 3,010 views
0 votes
1 answer

How to implement my clustering algorithm in pyspark (without using the ready library for example k-means)?

Oct 14, 2020 in Apache Spark by dani
 • 160 points • 1,473 views
0 votes
1 answer

How to insert data into Cassandra table using Spark DataFrame?

Sep 21, 2020 in Apache Spark by akhtar
 • 38,240 points • 3,614 views
0 votes
1 answer

How to merge two Spark DataFrames?

Sep 17, 2020 in Big Data Hadoop by akhtar
 • 38,240 points • 1,459 views
0 votes
1 answer

How can browse without port usage (example: "http://<IP>" )on the broswer if the php artisan server on "http://<IP>:8088"

Jul 31, 2020 in Web Development by Raghu
 • 120 points • 937 views
0 votes
1 answer

File not found exception while processing the spark job in yarn cluster mode with multinode hadoop cluster

Jul 30, 2020 in Apache Spark by Ganendra
 • 140 points • 4,329 views
0 votes
1 answer

Unable to submit the spark job in deployment mode - multinode cluster(using ubuntu machines) with yarn master

Jul 29, 2020 in Apache Spark by Ganendra
 • 140 points • 1,962 views
0 votes
0 answers

Unable to get the Job status and Group ID java- spark standalone program with databricks

Jul 23, 2020 in Apache Spark by kamboj
 • 140 points • 2,126 views
0 votes
1 answer

Can number of Spark task be greater than the executor core?

Jun 17, 2020 in Apache Spark by Rishi
 • 160 points • 1,789 views
0 votes
1 answer

Can the executor core be greater than the total number of spark tasks?

Jun 17, 2020 in Apache Spark by Rishi
 • 160 points • 1,957 views
0 votes
1 answer

after installing hadoop 3.0.1 I can's access spark shell or hive shell.

Jun 16, 2020 in Apache Spark by abdul
 • 120 points • 997 views
0 votes
1 answer

Where can I get best spark tutorials for beginners?

May 14, 2020 in Apache Spark by akhtar
 • 38,240 points • 660 views
0 votes
1 answer

Py4JJavaError: An error occurred while calling o310.csv. : java.net.ConnectException: Call From master/192.168.56.101 to master:9000

May 7, 2020 in Apache Spark by akhtar
 • 38,240 points • 7,156 views
+1 vote
1 answer

How to convert pyspark Dataframe to pandas Dataframe?

May 7, 2020 in Apache Spark by akhtar
 • 38,240 points • 8,042 views
0 votes
1 answer

How to integrate Machine Learning with Spark?

May 6, 2020 in Machine Learning by akhtar
 • 38,240 points • 681 views
0 votes
1 answer

ERROR thriftserver.SparkExecuteStatementOperation: Error executing query, currentState RUNNING, org.apache.spark.sql.catalyst.errors.package$TreeNodeException

Apr 29, 2020 in Apache Spark by akhtar
 • 38,240 points • 1,798 views
0 votes
1 answer

Error: sql.out:Error: org.apache.spark.SparkException: Job aborted due to stage failure: Total size of serialized results of 381610 tasks (4.0 GB) is bigger than spark.driver.maxResultSize (4.0 GB)

Apr 29, 2020 in Apache Spark by akhtar
 • 38,240 points • 6,240 views
0 votes
1 answer

"java.lang.ClassNotFoundException" in Spark on Amazon EMR

Apr 29, 2020 in Apache Spark by akhtar
 • 38,240 points • 3,441 views
0 votes
1 answer

error: Caused by: org.apache.spark.SparkException: Failed to execute user defined function.

Apr 22, 2020 in Apache Spark by akhtar
 • 38,240 points • 3,989 views
+2 votes
2 answers

py4j.protocol.Py4JError: org.apache.spark.api.python.PythonUtils.getEncryptionEnabled does not exist in the JVM

Apr 7, 2020 in Apache Spark by akhtar
 • 38,240 points • 21,785 views
0 votes
1 answer

env: ‘python’: No such file or directory in pyspark.

Apr 7, 2020 in Apache Spark by akhtar
 • 38,240 points • 6,149 views
–1 vote
0 answers

How to parse an S3 XML file to find tags using apache spark

Mar 18, 2020 in Apache Spark by anonymous
 • 110 points • 1,855 views
+2 votes
1 answer

FileStreamSink: Error while looking for metadata directory. java.lang.IllegalArgumentException: java.net.UnknownHostException: hive

Feb 13, 2020 in Big Data Hadoop by akhtar
 • 38,240 points • 10,799 views
0 votes
1 answer

apache.spark.sql.AnalysisException: Text data source does not support int data type.;

Feb 13, 2020 in Big Data Hadoop by akhtar
 • 38,240 points • 5,676 views
+1 vote
1 answer

What is the importance of Kafka bootstrap.servers?

Feb 11, 2020 in Apache Kafka by akhtar
 • 38,240 points • 17,386 views
0 votes
1 answer

org.apache.kafka.common.config.ConfigException: Missing required configuration "bootstrap.servers" which has no default value.

Feb 10, 2020 in Apache Kafka by akhtar
 • 38,240 points • 6,867 views
0 votes
0 answers

What is the difference between point to point and publish-subscribe messaging system?

Feb 6, 2020 in Apache Kafka by akhtar
 • 38,240 points • 673 views
+1 vote
1 answer

ERROR Unexpected exception, exiting abnormally (org.apache.zookeeper.server.ZooKeeperServerMain) java.net.BindException: Address already in use

Feb 5, 2020 in Apache Kafka by akhtar
 • 38,240 points • 11,889 views
0 votes
1 answer

Exception in thread "main" java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D

Feb 5, 2020 in Big Data Hadoop by akhtar
 • 38,240 points • 8,327 views
0 votes
1 answer

Does spark streaming provides checkpoint?

Feb 4, 2020 in Apache Spark by akhtar
 • 38,240 points • 1,256 views
0 votes
0 answers

not able to get output in spark streaming??

Feb 4, 2020 in Apache Spark by akhtar
 • 38,240 points • 722 views
0 votes
1 answer

Is Spark Sql provides indexing to improve processing speed?

Feb 4, 2020 in Apache Spark by akhtar
 • 38,240 points • 732 views
0 votes
1 answer

What is the difference between spark streaming and spark structured streaming?

Feb 4, 2020 in Apache Spark by akhtar
 • 38,240 points • 3,568 views
0 votes
1 answer

What are Dstreams?

Feb 4, 2020 in Apache Spark by akhtar
 • 38,240 points • 850 views
0 votes
1 answer

Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@2c7bde26

Feb 3, 2020 in Apache Spark by akhtar
 • 38,240 points • 5,332 views
0 votes
1 answer

Cannot create directory /hive/xzxz/_temporary/0. Name node is in safe mode.

Feb 3, 2020 in Apache Spark by akhtar
 • 38,240 points • 701 views
+1 vote
1 answer

is not a Parquet file. expected magic number at tail [80, 65, 82, 49] but found [51, 53, 10, 10]

Feb 3, 2020 in Apache Spark by akhtar
 • 38,240 points • 17,679 views
0 votes
1 answer

What is pageRank in graphX??

Jan 31, 2020 in Apache Spark by akhtar
 • 38,240 points • 1,018 views
0 votes
1 answer

env : R : No such file or directory

Jan 31, 2020 in Apache Spark by Hasid
 • 370 points • 1,626 views
0 votes
2 answers

java.lang.StringIndexOutOfBoundsException: String index out of range: 1

Jan 29, 2020 in Apache Spark by akhtar
 • 38,240 points • 7,243 views
0 votes
1 answer

Not enough space to cache rdd_80_1 in memory!

Jan 29, 2020 in Apache Spark by akhtar
 • 38,240 points • 2,492 views
0 votes
1 answer

Caused by: java.lang.NumberFormatException: Empty String

Jan 29, 2020 in Apache Spark by akhtar
 • 38,240 points • 4,473 views
Page:

Most popular tags

"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.