Spark: Print DataFrame Schema Metadata
Problem
Print DataFrame
’s fields like printSchema
does, but also display field metadata.
Solution
def print_schema(df: DataFrame):
if not df:
return
def pl(fields: List[StructField], spacing=""):
for field in fields:
print(f"{spacing} - {field.name}")
if field.metadata:
for k, v in field.metadata.items():
print(f"{spacing} {k}: {v}")
pl(df.schema.fields)
To contact me, send an email anytime or leave a comment below.