pyspark.sql.functions.transform_values#
- pyspark.sql.functions.transform_values(col, f)[source]#
- Applies a function to every key-value pair in a map and returns a map with the results of those applications as the new values for the pairs. - New in version 3.1.0. - Changed in version 3.4.0: Supports Spark Connect. - Parameters
- colColumnor str
- name of column or expression 
- ffunction
- a binary function - (k: Column, v: Column) -> Column...Can use methods of- Column, functions defined in- pyspark.sql.functionsand Scala- UserDefinedFunctions. Python- UserDefinedFunctionsare not supported (SPARK-27052).
 
- col
- Returns
- Column
- a new map of entries where new values were calculated by applying given function to each key value argument. 
 
 - Examples - >>> df = spark.createDataFrame([(1, {"IT": 10.0, "SALES": 2.0, "OPS": 24.0})], ("id", "data")) >>> row = df.select(transform_values( ... "data", lambda k, v: when(k.isin("IT", "OPS"), v + 10.0).otherwise(v) ... ).alias("new_data")).head() >>> sorted(row["new_data"].items()) [('IT', 20.0), ('OPS', 34.0), ('SALES', 2.0)]