pyspark.sql.streaming.StreamingQuery#
- class pyspark.sql.streaming.StreamingQuery(jsq)[source]#
- A handle to a query that is executing continuously in the background as new data arrives. All these methods are thread-safe. - New in version 2.0.0. - Changed in version 3.5.0: Supports Spark Connect. - Notes - This API is evolving. - Methods - awaitTermination([timeout])- Waits for the termination of this query, either by - query.stop()or by an exception.- New in version 2.1.0. - explain([extended])- Prints the (logical and physical) plans to the console for debugging purpose. - Blocks until all available data in the source has been processed and committed to the sink. - stop()- Stop this streaming query. - Attributes - Returns the unique id of this query that persists across restarts from checkpoint data. - Whether this streaming query is currently active or not. - Returns the most recent - StreamingQueryProgressupdate of this streaming query or None if there were no progress updates- Returns the user-specified name of the query, or None if not specified. - Returns an array of the most recent [[StreamingQueryProgress]] updates for this query. - Returns the unique id of this query that does not persist across restarts. - Returns the current status of the query.