Recent questions tagged apache-spark

0 votes
0 answers

Are there any services like Databricks but without compulsion of AWS/Azure/Google Cloud?

Jan 24, 2023 in AWS by Tejashwini
 • 3,820 points • 685 views
0 votes
1 answer

Azure HD Insight - YARN UI is not showing logs on stderr suddenly

Mar 7, 2022 in Azure by Edureka
 • 12,690 points • 937 views
0 votes
1 answer

What is apache hadoop?

Dec 13, 2021 in Big Data Hadoop by CoderGirl
 • 500 points • 1,057 views
0 votes
1 answer

Spark Core How to fetch max n rows of an RDD function without using Rdd.max()

Dec 3, 2020 in Apache Spark by Prashant
 • 120 points • 2,234 views
0 votes
1 answer

In AWS, if user wants to run spark, then on top of which one of the following can the user do it?

Nov 26, 2020 in Apache Spark by ritu
 • 960 points • 1,455 views
0 votes
1 answer

error: value update is not a member of scala.collection.immutable.Map[String, Int]

Nov 17, 2020 in Big Data Hadoop by anonymous
 • 8,870 points • 5,520 views
0 votes
1 answer

How to read Avro Partition Data?

Nov 4, 2020 in Apache Spark by akhtar
 • 38,260 points • 1,895 views
+1 vote
1 answer

How to write Spark DataFrame to Avro Data File?

Nov 4, 2020 in Apache Spark by akhtar
 • 38,260 points • 3,339 views
0 votes
1 answer

How to read a dataframe based on an avro schema?

Oct 30, 2020 in Apache Spark by anonymous
 • 120 points • 3,302 views
0 votes
1 answer

How to implement my clustering algorithm in pyspark (without using the ready library for example k-means)?

Oct 14, 2020 in Apache Spark by dani
 • 160 points • 1,701 views
0 votes
1 answer

How to insert data into Cassandra table using Spark DataFrame?

Sep 21, 2020 in Apache Spark by akhtar
 • 38,260 points • 3,958 views
0 votes
1 answer

How to merge two Spark DataFrames?

Sep 17, 2020 in Big Data Hadoop by akhtar
 • 38,260 points • 1,805 views
0 votes
1 answer

How can browse without port usage (example: "http://<IP>" )on the broswer if the php artisan server on "http://<IP>:8088"

Jul 31, 2020 in Web Development by Raghu
 • 120 points • 1,186 views
0 votes
1 answer

File not found exception while processing the spark job in yarn cluster mode with multinode hadoop cluster

Jul 30, 2020 in Apache Spark by Ganendra
 • 140 points • 4,670 views
0 votes
1 answer

Unable to submit the spark job in deployment mode - multinode cluster(using ubuntu machines) with yarn master

Jul 29, 2020 in Apache Spark by Ganendra
 • 140 points • 2,339 views
0 votes
0 answers

Unable to get the Job status and Group ID java- spark standalone program with databricks

Jul 23, 2020 in Apache Spark by kamboj
 • 140 points • 2,331 views
0 votes
1 answer

Can number of Spark task be greater than the executor core?

Jun 17, 2020 in Apache Spark by Rishi
 • 160 points • 2,075 views
0 votes
1 answer

Can the executor core be greater than the total number of spark tasks?

Jun 17, 2020 in Apache Spark by Rishi
 • 160 points • 2,266 views
0 votes
1 answer

after installing hadoop 3.0.1 I can's access spark shell or hive shell.

Jun 16, 2020 in Apache Spark by abdul
 • 120 points • 1,235 views
0 votes
1 answer

Where can I get best spark tutorials for beginners?

May 14, 2020 in Apache Spark by akhtar
 • 38,260 points • 758 views
0 votes
1 answer

Py4JJavaError: An error occurred while calling o310.csv. : java.net.ConnectException: Call From master/192.168.56.101 to master:9000

May 7, 2020 in Apache Spark by akhtar
 • 38,260 points • 7,486 views
+1 vote
1 answer

How to convert pyspark Dataframe to pandas Dataframe?

May 7, 2020 in Apache Spark by akhtar
 • 38,260 points • 8,210 views
0 votes
1 answer

How to integrate Machine Learning with Spark?

May 6, 2020 in Machine Learning by akhtar
 • 38,260 points • 883 views
0 votes
1 answer

ERROR thriftserver.SparkExecuteStatementOperation: Error executing query, currentState RUNNING, org.apache.spark.sql.catalyst.errors.package$TreeNodeException

Apr 29, 2020 in Apache Spark by akhtar
 • 38,260 points • 2,057 views
0 votes
1 answer

Error: sql.out:Error: org.apache.spark.SparkException: Job aborted due to stage failure: Total size of serialized results of 381610 tasks (4.0 GB) is bigger than spark.driver.maxResultSize (4.0 GB)

Apr 29, 2020 in Apache Spark by akhtar
 • 38,260 points • 7,841 views
0 votes
1 answer

"java.lang.ClassNotFoundException" in Spark on Amazon EMR

Apr 29, 2020 in Apache Spark by akhtar
 • 38,260 points • 3,774 views
0 votes
1 answer

error: Caused by: org.apache.spark.SparkException: Failed to execute user defined function.

Apr 22, 2020 in Apache Spark by akhtar
 • 38,260 points • 4,377 views
+2 votes
2 answers

py4j.protocol.Py4JError: org.apache.spark.api.python.PythonUtils.getEncryptionEnabled does not exist in the JVM

Apr 7, 2020 in Apache Spark by akhtar
 • 38,260 points • 22,558 views
0 votes
1 answer

env: ‘python’: No such file or directory in pyspark.

Apr 7, 2020 in Apache Spark by akhtar
 • 38,260 points • 6,442 views
–1 vote
0 answers

How to parse an S3 XML file to find tags using apache spark

Mar 18, 2020 in Apache Spark by anonymous
 • 110 points • 2,059 views
+2 votes
1 answer

FileStreamSink: Error while looking for metadata directory. java.lang.IllegalArgumentException: java.net.UnknownHostException: hive

Feb 13, 2020 in Big Data Hadoop by akhtar
 • 38,260 points • 11,308 views
0 votes
1 answer

apache.spark.sql.AnalysisException: Text data source does not support int data type.;

Feb 13, 2020 in Big Data Hadoop by akhtar
 • 38,260 points • 6,058 views
+1 vote
1 answer

What is the importance of Kafka bootstrap.servers?

Feb 11, 2020 in Apache Kafka by akhtar
 • 38,260 points • 17,766 views
0 votes
1 answer

org.apache.kafka.common.config.ConfigException: Missing required configuration "bootstrap.servers" which has no default value.

Feb 10, 2020 in Apache Kafka by akhtar
 • 38,260 points • 7,335 views
0 votes
0 answers

What is the difference between point to point and publish-subscribe messaging system?

Feb 6, 2020 in Apache Kafka by akhtar
 • 38,260 points • 821 views
+1 vote
1 answer

ERROR Unexpected exception, exiting abnormally (org.apache.zookeeper.server.ZooKeeperServerMain) java.net.BindException: Address already in use

Feb 5, 2020 in Apache Kafka by akhtar
 • 38,260 points • 12,394 views
0 votes
1 answer

Exception in thread "main" java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D

Feb 5, 2020 in Big Data Hadoop by akhtar
 • 38,260 points • 8,788 views
0 votes
1 answer

Does spark streaming provides checkpoint?

Feb 4, 2020 in Apache Spark by akhtar
 • 38,260 points • 1,524 views
0 votes
0 answers

not able to get output in spark streaming??

Feb 4, 2020 in Apache Spark by akhtar
 • 38,260 points • 887 views
0 votes
1 answer

Is Spark Sql provides indexing to improve processing speed?

Feb 4, 2020 in Apache Spark by akhtar
 • 38,260 points • 948 views
0 votes
1 answer

What is the difference between spark streaming and spark structured streaming?

Feb 4, 2020 in Apache Spark by akhtar
 • 38,260 points • 3,846 views
0 votes
1 answer

What are Dstreams?

Feb 4, 2020 in Apache Spark by akhtar
 • 38,260 points • 1,105 views
0 votes
1 answer

Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@2c7bde26

Feb 3, 2020 in Apache Spark by akhtar
 • 38,260 points • 5,744 views
0 votes
1 answer

Cannot create directory /hive/xzxz/_temporary/0. Name node is in safe mode.

Feb 3, 2020 in Apache Spark by akhtar
 • 38,260 points • 814 views
+1 vote
1 answer

is not a Parquet file. expected magic number at tail [80, 65, 82, 49] but found [51, 53, 10, 10]

Feb 3, 2020 in Apache Spark by akhtar
 • 38,260 points • 18,328 views
0 votes
1 answer

What is pageRank in graphX??

Jan 31, 2020 in Apache Spark by akhtar
 • 38,260 points • 1,209 views
0 votes
1 answer

env : R : No such file or directory

Jan 31, 2020 in Apache Spark by Hasid
 • 370 points • 1,821 views
0 votes
2 answers

java.lang.StringIndexOutOfBoundsException: String index out of range: 1

Jan 29, 2020 in Apache Spark by akhtar
 • 38,260 points • 7,702 views
0 votes
1 answer

Not enough space to cache rdd_80_1 in memory!

Jan 29, 2020 in Apache Spark by akhtar
 • 38,260 points • 2,812 views
0 votes
1 answer

Caused by: java.lang.NumberFormatException: Empty String

Jan 29, 2020 in Apache Spark by akhtar
 • 38,260 points • 4,885 views
Page:

Most popular tags

"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.