Spark Union in Pythonic Way

To do union in more pythonic way with variable args i.e. union(df1, df2, df3) here is a snippet for a helper function:

def union_all(*df: DataFrame) -> DataFrame or None:
    Unions all dataframes. Null dataframes are skipped.
    :param df:
    rdf = None
    for idf in df:
        if not idf:
        rdf = rdf.union(idf) if rdf else idf
    return rdf

To contact me, send an email anytime or leave a comment below.