Create Tables From Folder in Databricks

Jul 13, 2021 | 54 Words | 1 min |  #databricks

Problem

You have a folder in ADLS, where all subfolders need to be a table in databricks.

Solution

def recreate_tables(container, db, subpath="", engine="delta"):
  tables = dbutils.fs.ls(f"abfss://{container}@accountname.dfs.core.windows.net/{subpath}")
  for table in tables:
    name = table.name.strip("/")
    print(f"making sure table '{name}' exists at '{table.path}'")
    spark.sql(f"create table if not exists {db}.{name} using {engine} location '{table.path}'")

Using:

recreate_table("mycontainer", "mydb", "subfolder")

Em, excuse me! Have Android 📱 and use Databricks? You might be interested in my totally free (and ad-free) Pocket Bricks . You can get it from Google Play too: Get it on Google Play

To contact me, send an email anytime or leave a comment below.

See Also