pyspark.sql.functions.map_entries#
- pyspark.sql.functions.map_entries(col)[source]#
- Map function: Returns an unordered array of all entries in the given map. - New in version 3.0.0. - Changed in version 3.4.0: Spark Connect. - Parameters
- colColumnor str
- Name of column or expression 
 
- col
- Returns
- Column
- An array of key value pairs as a struct type 
 
 - Examples - Example 1: Extracting entries from a simple map - >>> from pyspark.sql import functions as sf >>> df = spark.sql("SELECT map(1, 'a', 2, 'b') as data") >>> df.select(sf.sort_array(sf.map_entries("data"))).show() +-----------------------------------+ |sort_array(map_entries(data), true)| +-----------------------------------+ | [{1, a}, {2, b}]| +-----------------------------------+ - Example 2: Extracting entries from a map with complex keys and values - >>> from pyspark.sql import functions as sf >>> df = spark.sql("SELECT map(array(1, 2), array('a', 'b'), " ... "array(3, 4), array('c', 'd')) as data") >>> df.select(sf.sort_array(sf.map_entries("data"))).show(truncate=False) +------------------------------------+ |sort_array(map_entries(data), true) | +------------------------------------+ |[{[1, 2], [a, b]}, {[3, 4], [c, d]}]| +------------------------------------+ - Example 3: Extracting entries from a map with duplicate keys - >>> from pyspark.sql import functions as sf >>> originalmapKeyDedupPolicy = spark.conf.get("spark.sql.mapKeyDedupPolicy") >>> spark.conf.set("spark.sql.mapKeyDedupPolicy", "LAST_WIN") >>> df = spark.sql("SELECT map(1, 'a', 1, 'b') as data") >>> df.select(sf.map_entries("data")).show() +-----------------+ |map_entries(data)| +-----------------+ | [{1, b}]| +-----------------+ >>> spark.conf.set("spark.sql.mapKeyDedupPolicy", originalmapKeyDedupPolicy) - Example 4: Extracting entries from an empty map - >>> from pyspark.sql import functions as sf >>> df = spark.sql("SELECT map() as data") >>> df.select(sf.map_entries("data")).show() +-----------------+ |map_entries(data)| +-----------------+ | []| +-----------------+