apache-spark
2022
16 posts
16 posts
- Spark Standalone Server Hosted in Ubuntu Linux 216 Dec 14
- How to Get Spark Scala Version 63 Jun 22
- Spark does NOT Support Azure Append Blobs and there's no workaround 67 Jun 16
- Flatten Spark Dataframe in Scala 102 Jun 11
- Install Maven .jar Dependencies from Jupyter Python Spark Notebook 343 Jun 10
- Writing to Delta Table with Spark 3 (much easier!) 31 May 5
- Access DBUtils from Scala app running under databricks-connect 63 Apr 8
- Where to get Hadoop Winutils 702 Mar 16
- Reading Files Recursively in Spark/Databricks 203 Mar 7
- Get S3 filesystem details using PySpark 125 Jan 26
- Spark: Add or Remove Struct Member/Field 168 Jan 6
- Spark: Union Incompatible Dataframes 135 Jan 5
- Spark Union in Pythonic Way 61 Jan 4
- Spark: Check Table Exists Quickly 43 Jan 3
- Spark DataFrame: Display Last Columns 49 Jan 2
- Reshuffle Schema Columns in Spark DataFrame 196 Jan 1
2021
15 posts
15 posts
- Spark - Export DataFrame Schema, and then Import it Later. 64 Nov 5
- Spark: Print DataFrame Schema Metadata 42 Nov 3
- Create Apache Spark DataFrame in memory 95 Oct 22
- Databricks - Setting up Custom External Hive Metastore in Azure using MSSQL Server 4022 Oct 21
- Set Table Property (Metadata) in Spark or Databricks 105 Sep 17
- Spark: Add or Remove a Column from a DataFrame 204 Jul 30
- Getting Info About Spark Partitions 690 Jun 28
- Connect from Spark to AWS S3 via Assume Role credential 247 Mar 10
- Creating Scala Uber JAR with Spark 3.1 Included 398 Mar 5
- Listing Spark Databases and Tables Fast 93 Mar 4
- Parsing Array of Strings in Spark 501 Feb 18
- Adding Any Constraint you want with AWS Deequ 322 Feb 10
- Using Deequ 1.1 with Spark 3 129 Feb 10
- Spark Host Configuration is Rejected in Spark 3 44 Feb 9
- Change Timestamp Year in Spark 115 Feb 3