Spark: Print DataFrame Schema Metadata

Nov 3, 2021 | 42 Words | 1 min |  #apache-spark #python

Problem

Print DataFrame’s fields like printSchema does, but also display field metadata.

Solution

def print_schema(df: DataFrame):
    if not df:
        return

    def pl(fields: List[StructField], spacing=""):
        for field in fields:
            print(f"{spacing} - {field.name}")

            if field.metadata:
                for k, v in field.metadata.items():
                    print(f"{spacing}   {k}: {v}")

    pl(df.schema.fields)

To contact me, send an email anytime or leave a comment below.

See Also