Recent questions tagged apache-spark

0 votes
0 answers

Are there any services like Databricks but without compulsion of AWS/Azure/Google Cloud?

Jan 24, 2023 in AWS by Tejashwini
 • 3,820 points • 631 views
0 votes
1 answer

Azure HD Insight - YARN UI is not showing logs on stderr suddenly

Mar 7, 2022 in Azure by Edureka
 • 12,690 points • 892 views
0 votes
1 answer

What is apache hadoop?

Dec 13, 2021 in Big Data Hadoop by CoderGirl
 • 500 points • 986 views
0 votes
1 answer

Spark Core How to fetch max n rows of an RDD function without using Rdd.max()

Dec 3, 2020 in Apache Spark by Prashant
 • 120 points • 2,182 views
0 votes
1 answer

In AWS, if user wants to run spark, then on top of which one of the following can the user do it?

Nov 26, 2020 in Apache Spark by ritu
 • 960 points • 1,417 views
0 votes
1 answer

error: value update is not a member of scala.collection.immutable.Map[String, Int]

Nov 17, 2020 in Big Data Hadoop by anonymous
 • 8,870 points • 5,448 views
0 votes
1 answer

How to read Avro Partition Data?

Nov 4, 2020 in Apache Spark by akhtar
 • 38,260 points • 1,864 views
+1 vote
1 answer

How to write Spark DataFrame to Avro Data File?

Nov 4, 2020 in Apache Spark by akhtar
 • 38,260 points • 3,270 views
0 votes
1 answer

How to read a dataframe based on an avro schema?

Oct 30, 2020 in Apache Spark by anonymous
 • 120 points • 3,246 views
0 votes
1 answer

How to implement my clustering algorithm in pyspark (without using the ready library for example k-means)?

Oct 14, 2020 in Apache Spark by dani
 • 160 points • 1,662 views
0 votes
1 answer

How to insert data into Cassandra table using Spark DataFrame?

Sep 21, 2020 in Apache Spark by akhtar
 • 38,260 points • 3,900 views
0 votes
1 answer

How to merge two Spark DataFrames?

Sep 17, 2020 in Big Data Hadoop by akhtar
 • 38,260 points • 1,766 views
0 votes
1 answer

How can browse without port usage (example: "http://<IP>" )on the broswer if the php artisan server on "http://<IP>:8088"

Jul 31, 2020 in Web Development by Raghu
 • 120 points • 1,142 views
0 votes
1 answer

File not found exception while processing the spark job in yarn cluster mode with multinode hadoop cluster

Jul 30, 2020 in Apache Spark by Ganendra
 • 140 points • 4,592 views
0 votes
1 answer

Unable to submit the spark job in deployment mode - multinode cluster(using ubuntu machines) with yarn master

Jul 29, 2020 in Apache Spark by Ganendra
 • 140 points • 2,274 views
0 votes
0 answers

Unable to get the Job status and Group ID java- spark standalone program with databricks

Jul 23, 2020 in Apache Spark by kamboj
 • 140 points • 2,278 views
0 votes
1 answer

Can number of Spark task be greater than the executor core?

Jun 17, 2020 in Apache Spark by Rishi
 • 160 points • 2,028 views
0 votes
1 answer

Can the executor core be greater than the total number of spark tasks?

Jun 17, 2020 in Apache Spark by Rishi
 • 160 points • 2,200 views
0 votes
1 answer

after installing hadoop 3.0.1 I can's access spark shell or hive shell.

Jun 16, 2020 in Apache Spark by abdul
 • 120 points • 1,183 views
0 votes
1 answer

Where can I get best spark tutorials for beginners?

May 14, 2020 in Apache Spark by akhtar
 • 38,260 points • 740 views
0 votes
1 answer

Py4JJavaError: An error occurred while calling o310.csv. : java.net.ConnectException: Call From master/192.168.56.101 to master:9000

May 7, 2020 in Apache Spark by akhtar
 • 38,260 points • 7,432 views
+1 vote
1 answer

How to convert pyspark Dataframe to pandas Dataframe?

May 7, 2020 in Apache Spark by akhtar
 • 38,260 points • 8,182 views
0 votes
1 answer

How to integrate Machine Learning with Spark?

May 6, 2020 in Machine Learning by akhtar
 • 38,260 points • 852 views
0 votes
1 answer

ERROR thriftserver.SparkExecuteStatementOperation: Error executing query, currentState RUNNING, org.apache.spark.sql.catalyst.errors.package$TreeNodeException

Apr 29, 2020 in Apache Spark by akhtar
 • 38,260 points • 1,992 views
0 votes
1 answer

Error: sql.out:Error: org.apache.spark.SparkException: Job aborted due to stage failure: Total size of serialized results of 381610 tasks (4.0 GB) is bigger than spark.driver.maxResultSize (4.0 GB)

Apr 29, 2020 in Apache Spark by akhtar
 • 38,260 points • 7,784 views
0 votes
1 answer

"java.lang.ClassNotFoundException" in Spark on Amazon EMR

Apr 29, 2020 in Apache Spark by akhtar
 • 38,260 points • 3,695 views
0 votes
1 answer

error: Caused by: org.apache.spark.SparkException: Failed to execute user defined function.

Apr 22, 2020 in Apache Spark by akhtar
 • 38,260 points • 4,281 views
+2 votes
2 answers

py4j.protocol.Py4JError: org.apache.spark.api.python.PythonUtils.getEncryptionEnabled does not exist in the JVM

Apr 7, 2020 in Apache Spark by akhtar
 • 38,260 points • 22,392 views
0 votes
1 answer

env: ‘python’: No such file or directory in pyspark.

Apr 7, 2020 in Apache Spark by akhtar
 • 38,260 points • 6,403 views
–1 vote
0 answers

How to parse an S3 XML file to find tags using apache spark

Mar 18, 2020 in Apache Spark by anonymous
 • 110 points • 2,017 views
+2 votes
1 answer

FileStreamSink: Error while looking for metadata directory. java.lang.IllegalArgumentException: java.net.UnknownHostException: hive

Feb 13, 2020 in Big Data Hadoop by akhtar
 • 38,260 points • 11,206 views
0 votes
1 answer

apache.spark.sql.AnalysisException: Text data source does not support int data type.;

Feb 13, 2020 in Big Data Hadoop by akhtar
 • 38,260 points • 5,979 views
+1 vote
1 answer

What is the importance of Kafka bootstrap.servers?

Feb 11, 2020 in Apache Kafka by akhtar
 • 38,260 points • 17,698 views
0 votes
1 answer

org.apache.kafka.common.config.ConfigException: Missing required configuration "bootstrap.servers" which has no default value.

Feb 10, 2020 in Apache Kafka by akhtar
 • 38,260 points • 7,259 views
0 votes
0 answers

What is the difference between point to point and publish-subscribe messaging system?

Feb 6, 2020 in Apache Kafka by akhtar
 • 38,260 points • 794 views
+1 vote
1 answer

ERROR Unexpected exception, exiting abnormally (org.apache.zookeeper.server.ZooKeeperServerMain) java.net.BindException: Address already in use

Feb 5, 2020 in Apache Kafka by akhtar
 • 38,260 points • 12,319 views
0 votes
1 answer

Exception in thread "main" java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D

Feb 5, 2020 in Big Data Hadoop by akhtar
 • 38,260 points • 8,709 views
0 votes
1 answer

Does spark streaming provides checkpoint?

Feb 4, 2020 in Apache Spark by akhtar
 • 38,260 points • 1,453 views
0 votes
0 answers

not able to get output in spark streaming??

Feb 4, 2020 in Apache Spark by akhtar
 • 38,260 points • 848 views
0 votes
1 answer

Is Spark Sql provides indexing to improve processing speed?

Feb 4, 2020 in Apache Spark by akhtar
 • 38,260 points • 910 views
0 votes
1 answer

What is the difference between spark streaming and spark structured streaming?

Feb 4, 2020 in Apache Spark by akhtar
 • 38,260 points • 3,778 views
0 votes
1 answer

What are Dstreams?

Feb 4, 2020 in Apache Spark by akhtar
 • 38,260 points • 1,044 views
0 votes
1 answer

Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@2c7bde26

Feb 3, 2020 in Apache Spark by akhtar
 • 38,260 points • 5,690 views
0 votes
1 answer

Cannot create directory /hive/xzxz/_temporary/0. Name node is in safe mode.

Feb 3, 2020 in Apache Spark by akhtar
 • 38,260 points • 793 views
+1 vote
1 answer

is not a Parquet file. expected magic number at tail [80, 65, 82, 49] but found [51, 53, 10, 10]

Feb 3, 2020 in Apache Spark by akhtar
 • 38,260 points • 18,212 views
0 votes
1 answer

What is pageRank in graphX??

Jan 31, 2020 in Apache Spark by akhtar
 • 38,260 points • 1,167 views
0 votes
1 answer

env : R : No such file or directory

Jan 31, 2020 in Apache Spark by Hasid
 • 370 points • 1,786 views
0 votes
2 answers

java.lang.StringIndexOutOfBoundsException: String index out of range: 1

Jan 29, 2020 in Apache Spark by akhtar
 • 38,260 points • 7,638 views
0 votes
1 answer

Not enough space to cache rdd_80_1 in memory!

Jan 29, 2020 in Apache Spark by akhtar
 • 38,260 points • 2,744 views
0 votes
1 answer

Caused by: java.lang.NumberFormatException: Empty String

Jan 29, 2020 in Apache Spark by akhtar
 • 38,260 points • 4,826 views
Page:

Most popular tags

"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.