stream_watermark
Watermark Stream
Description
Ensures a stream has a watermark defined, which is required for some operations over streams.
Usage
stream_watermark(x, column = "timestamp", threshold = "10 minutes")Arguments
| Argument | Description |
|---|---|
| x | An object coercable to a Spark Streaming DataFrame. |
| column | The name of the column that contains the event time of the row, |
if the column is missing, a column with the current time will be added. threshold | The minimum delay to wait to data to arrive late, defaults to ten minutes.