pyspark.sql.Column#
- class pyspark.sql.Column(*args, **kwargs)[source]#
- A column in a DataFrame. - New in version 1.3.0. - Changed in version 3.4.0: Supports Spark Connect. - Examples - Column instances can be created by - >>> df = spark.createDataFrame( ... [(2, "Alice"), (5, "Bob")], ["age", "name"]) - Select a column out of a DataFrame >>> df.name Column<’name’> >>> df[“name”] Column<’name’> - Create from an expression - >>> df.age + 1 Column<...> >>> 1 / df.age Column<...> - Methods - alias(*alias, **kwargs)- Returns this column aliased with a new name or names (in the case of expressions that return more than one column, such as explode). - asc()- Returns a sort expression based on the ascending order of the column. - Returns a sort expression based on ascending order of the column, and null values return before non-null values. - Returns a sort expression based on ascending order of the column, and null values appear after non-null values. - astype(dataType)- between(lowerBound, upperBound)- Check if the current column's values are between the specified lower and upper bounds, inclusive. - bitwiseAND(other)- Compute bitwise AND of this expression with another expression. - bitwiseOR(other)- Compute bitwise OR of this expression with another expression. - bitwiseXOR(other)- Compute bitwise XOR of this expression with another expression. - cast(dataType)- Casts the column into type - dataType.- contains(other)- Contains the other element. - desc()- Returns a sort expression based on the descending order of the column. - Returns a sort expression based on the descending order of the column, and null values appear before non-null values. - Returns a sort expression based on the descending order of the column, and null values appear after non-null values. - dropFields(*fieldNames)- An expression that drops fields in - StructTypeby name.- endswith(other)- String ends with. - eqNullSafe(other)- Equality test that is safe for null values. - getField(name)- An expression that gets a field by name in a - StructType.- getItem(key)- An expression that gets an item at position - ordinalout of a list, or gets an item by key out of a dict.- ilike(other)- SQL ILIKE expression (case insensitive LIKE). - isNaN()- True if the current expression is NaN. - True if the current expression is NOT null. - isNull()- True if the current expression is null. - isin(*cols)- A boolean expression that is evaluated to true if the value of this expression is contained by the evaluated values of the arguments. - like(other)- SQL like expression. - name(*alias, **kwargs)- otherwise(value)- Evaluates a list of conditions and returns one of multiple possible result expressions. - outer()- Mark this column as an outer column if its expression refers to columns from an outer query. - over(window)- Define a windowing column. - rlike(other)- SQL RLIKE expression (LIKE with Regex). - startswith(other)- String starts with. - substr(startPos, length)- Return a - Columnwhich is a substring of the column.- try_cast(dataType)- This is a special version of cast that performs the same operation, but returns a NULL value instead of raising an error if the invoke method throws exception. - when(condition, value)- Evaluates a list of conditions and returns one of multiple possible result expressions. - withField(fieldName, col)- An expression that adds/replaces a field in - StructTypeby name.