pyspark.sql.streaming.DataStreamReader.option¶
-
DataStreamReader.option(key: str, value: OptionalPrimitiveType) → DataStreamReader[source]¶ Adds an input option for the underlying data source.
New in version 2.0.0.
Changed in version 3.5.0: Supports Spark Connect.
Notes
This API is evolving.
Examples
>>> spark.readStream.option("x", 1) <...streaming.readwriter.DataStreamReader object ...>
The example below specifies ‘rowsPerSecond’ option to Rate source in order to generate 10 rows every second.
>>> import time >>> q = spark.readStream.format( ... "rate").option("rowsPerSecond", 10).load().writeStream.format("console").start() >>> time.sleep(3) >>> q.stop()