You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/user-guide/pipelines/configurable-delay-in-scheduled-pipelines.md
+20-21Lines changed: 20 additions & 21 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -3,41 +3,40 @@ description: >-
3
3
Delay scheduled pipeline execution in OpenObserve to ensure data completeness
4
4
by waiting minutes after the trigger time before processing data.
5
5
---
6
-
7
-
This guide explains how to use the configurable delay feature for [scheduled pipelines](../use-pipelines/).
6
+
This guide explains how to use the configurable delay feature for [scheduled pipelines](../pipelines/create-and-use-scheduled-pipeline/).
8
7
This feature allows you to delay the execution of a scheduled pipeline by a set number of minutes after its scheduled start time. This is useful to ensure data completeness and accommodate late-arriving events before the pipeline runs.
9
8
10
-
## How to Configure Delay
9
+
## How to configure delay
11
10
12
11
When editing the **Associate Query** page:
13
12
14
-
1. In the **Set Variables** section, configure:
15
-
16
-
-**Frequency**: How often the pipeline should run.
17
-
-**Period**: The data window the pipeline should process.
18
-
-**Delay**: How many minutes to wait after the scheduled time before running the pipeline.
13
+
1. In the **Set Variables** section, add **Delay**. Delay defines how many minutes the system waits after the scheduled trigger time. The pipeline runs only after this waiting period completes.
**Note**: Since Delay is a required field when setting up a scheduled pipeline, enter 0 minutes if you want the pipeline to run exactly at the scheduled time without any delay.
18
+
!!! note "Note"
19
+
Since Delay is a required field when setting up a scheduled pipeline, enter 0 minutes if you want the pipeline to run exactly at the scheduled time without any delay.
24
20
25
-
**Example**
21
+
!!! note "Example"
26
22
27
-
If the pipeline is configured with the following settings:
23
+
If the pipeline is configured with the following settings:
28
24
29
-
- Frequency: 5 minutes
30
-
- Period: 5 minutes
31
-
- Delay: 2 minutes
25
+
- Frequency: 5 minutes
26
+
- Period: 5 minutes
27
+
- Delay: 2 minutes
32
28
33
-
**Behavior:** The pipeline is scheduled every 5 minutes and runs 2 minutes after each scheduled time, processing data from the previous 5-minute window.
34
-
At 10:00 AM (scheduled time), the pipeline executes at 10:02 AM, processing data from 9:55 AM to 10:00 AM.
29
+
**Behavior:** The pipeline is scheduled every 5 minutes and runs 2 minutes after each scheduled time, processing data from the previous 5-minute window.
30
+
At 10:00 AM (scheduled time), the pipeline executes at 10:02 AM, processing data from 9:55 AM to 10:00 AM.
35
31
36
-
**Note**
32
+
**Note**
37
33
38
-
-**Works with Cron**: Delay also applies when using cron-based schedules.
39
-
-**Data Window Unchanged**: Delay only affects when execution starts. It does not change the data range.
34
+
- **Works with Cron**: Delay also applies when using cron-based schedules.
35
+
- **Data Window Unchanged**: Delay only affects when execution starts. It does not change the data range.
40
36
41
-
**Related Links**<br>
42
-
For steps on creating scheduled pipelines, see [Scheduled Pipelines](../use-pipelines/).
37
+
## Related links
38
+
-[Pipelines in OpenObserve](../pipelines/pipelines/)
39
+
-[Create and Use Real-time Pipeline](../pipelines/create-and-use-real-time-pipeline/)
40
+
-[Create and Use Scheduled Pipeline](../pipelines/create-and-use-scheduled-pipeline/)
Copy file name to clipboardExpand all lines: docs/user-guide/pipelines/create-and-use-scheduled-pipeline.md
+15-19Lines changed: 15 additions & 19 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,7 +9,7 @@ Use this guide to create and run scheduled pipelines. Scheduled pipelines run at
9
9
## Steps to create a scheduled pipeline
10
10
11
11
??? "Step 1: Open the pipeline editor"
12
-
### Step 1: Open the pipeline editor
12
+
### Step 1: Open the pipeline editor
13
13
1. Log in to OpenObserve.
14
14
2. In the left navigation panel, select **Pipelines**. By default, the system opens the **Pipelines** tab.
15
15

@@ -24,25 +24,25 @@ Use this guide to create and run scheduled pipelines. Scheduled pipelines run at
24
24
Scheduled pipelines do not use a **Stream** node as the source. They always begin with a **Query** node.
25
25
26
26
1. From the **Source** section, drag the **Query** node into the editor.
27
-
2. The **Associate Query** panel opens. The panel contains two sections, Build Query and Set Variables.
27
+
2. The **Associate Query** panel opens. The panel contains the following sections, **Build Query**, **SQL Query** editor, **Output**, **Set Variables**.
4. Edit or write your SQL query in the SQL editor.
33
+
4. Edit or write your SQL query in the **SQL query** editor.
34
34
5. Select the time range from the time range selector at the top-right corner.
35
35
5. Click **Run query** to preview data in the **Output** panel.
36
-

36
+

37
37
6. In the **Set Variables** section configures how the pipeline runs using the following settings:
38
38
39
39
- **Enable Cron**: Switch from interval-based scheduling to cron-based scheduling.
40
40
- **Frequency**: How often the pipeline runs.
41
41
- **Period**: The data window to process. Period must equal Frequency.
42
42
- **Delay**: How many minutes the pipeline waits after the scheduled time before execution.
43
43
44
-
> Visit the [Pipelines overview](https://openobserve.ai/docs/user-guide/pipelines/pipelines/) page to learn more about frequency, period, and cron.
45
-
> Learn more about [Configurable Delay in Scheduled Pipelines](https://openobserve.ai/docs/user-guide/pipelines/configurable-delay-in-scheduled-pipelines/).
44
+
> See the [Pipelines overview](https://openobserve.ai/docs/user-guide/pipelines/pipelines/) page to learn more about frequency, period, and cron.
45
+
> See the [Configurable Delay in Scheduled Pipelines](https://openobserve.ai/docs/user-guide/pipelines/configurable-delay-in-scheduled-pipelines/) page to learn how to configure delay in scheduled-pipelines.
0 commit comments