apache-spark
2022
- How to Get Spark Scala Version Jun 22
- Spark does NOT Support Azure Append Blobs and there's no workaround Jun 16
- Flatten Spark Dataframe in Scala Jun 11
- Install Maven .jar Dependencies from Jupyter Python Spark Notebook Jun 10
- Writing to Delta Table with Spark 3 (much easier!) May 5
- Access DBUtils from Scala app running under databricks-connect Apr 8
- Where to get Hadoop Winutils Mar 16
- Reading Files Recursively in Spark/Databricks Mar 7
- Get S3 filesystem details using PySpark Jan 26
- Spark: Add or Remove Struct Member/Field Jan 6
- Spark: Union Incompatible Dataframes Jan 5
- Spark Union in Pythonic Way Jan 4
- Spark: Check Table Exists Quickly Jan 3
- Spark DataFrame: Display Last Columns Jan 2
- Reshuffle Schema Columns in Spark DataFrame Jan 1
2021
- Spark - Export DataFrame Schema, and then Import it Later. Nov 5
- Spark: Print DataFrame Schema Metadata Nov 3
- Create Apache Spark DataFrame in memory Oct 22
- Databricks - Setting up Custom External Hive Metastore in Azure using MSSQL Server Oct 21
- Set Table Property (Metadata) in Spark or Databricks Sep 17
- Spark: Add or Remove a Column from a DataFrame Jul 30
- Getting Info About Spark Partitions Jun 28
- Connect from Spark to AWS S3 via Assume Role credential Mar 10
- Creating Scala Uber JAR with Spark 3.1 Included Mar 5
- Listing Spark Databases and Tables Fast Mar 4
- Parsing Array of Strings in Spark Feb 18
- Adding Any Constraint you want with AWS Deequ Feb 10
- Using Deequ 1.1 with Spark 3 Feb 10
- Spark Host Configuration is Rejected in Spark 3 Feb 9
- Change Timestamp Year in Spark Feb 3