Skip to content

Commit 8296efe

Browse files
Dashboard doc fix (#264)
* doc fix: streams docs and custom charts docs * update pipelines pages, alert history, custom chart for metrics using promql doc * doc fix: formatting and 404 fixes
1 parent 5d65237 commit 8296efe

8 files changed

+50
-57
lines changed

docs/images/import-pipelines.png

117 KB
Loading
-82.3 KB
Loading

docs/user-guide/pipelines/configurable-delay-in-scheduled-pipelines.md

Lines changed: 20 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -3,41 +3,40 @@ description: >-
33
Delay scheduled pipeline execution in OpenObserve to ensure data completeness
44
by waiting minutes after the trigger time before processing data.
55
---
6-
7-
This guide explains how to use the configurable delay feature for [scheduled pipelines](../use-pipelines/).
6+
This guide explains how to use the configurable delay feature for [scheduled pipelines](../pipelines/create-and-use-scheduled-pipeline/).
87
This feature allows you to delay the execution of a scheduled pipeline by a set number of minutes after its scheduled start time. This is useful to ensure data completeness and accommodate late-arriving events before the pipeline runs.
98

10-
## How to Configure Delay
9+
## How to configure delay
1110

1211
When editing the **Associate Query** page:
1312

14-
1. In the **Set Variables** section, configure:
15-
16-
- **Frequency**: How often the pipeline should run.
17-
- **Period**: The data window the pipeline should process.
18-
- **Delay**: How many minutes to wait after the scheduled time before running the pipeline.
13+
1. In the **Set Variables** section, add **Delay**. Delay defines how many minutes the system waits after the scheduled trigger time. The pipeline runs only after this waiting period completes.
1914
![configurable-delay](../../images/scheduled-pipeline-config-delay.png)
2015

2116
2. Select **Validate and Close**.
2217

23-
**Note**: Since Delay is a required field when setting up a scheduled pipeline, enter 0 minutes if you want the pipeline to run exactly at the scheduled time without any delay.
18+
!!! note "Note"
19+
Since Delay is a required field when setting up a scheduled pipeline, enter 0 minutes if you want the pipeline to run exactly at the scheduled time without any delay.
2420

25-
**Example**
21+
!!! note "Example"
2622

27-
If the pipeline is configured with the following settings:
23+
If the pipeline is configured with the following settings:
2824

29-
- Frequency: 5 minutes
30-
- Period: 5 minutes
31-
- Delay: 2 minutes
25+
- Frequency: 5 minutes
26+
- Period: 5 minutes
27+
- Delay: 2 minutes
3228

33-
**Behavior:** The pipeline is scheduled every 5 minutes and runs 2 minutes after each scheduled time, processing data from the previous 5-minute window.
34-
At 10:00 AM (scheduled time), the pipeline executes at 10:02 AM, processing data from 9:55 AM to 10:00 AM.
29+
**Behavior:** The pipeline is scheduled every 5 minutes and runs 2 minutes after each scheduled time, processing data from the previous 5-minute window.
30+
At 10:00 AM (scheduled time), the pipeline executes at 10:02 AM, processing data from 9:55 AM to 10:00 AM.
3531

36-
**Note**
32+
**Note**
3733

38-
- **Works with Cron**: Delay also applies when using cron-based schedules.
39-
- **Data Window Unchanged**: Delay only affects when execution starts. It does not change the data range.
34+
- **Works with Cron**: Delay also applies when using cron-based schedules.
35+
- **Data Window Unchanged**: Delay only affects when execution starts. It does not change the data range.
4036

41-
**Related Links**<br>
42-
For steps on creating scheduled pipelines, see [Scheduled Pipelines](../use-pipelines/).
37+
## Related links
38+
- [Pipelines in OpenObserve](../pipelines/pipelines/)
39+
- [Create and Use Real-time Pipeline](../pipelines/create-and-use-real-time-pipeline/)
40+
- [Create and Use Scheduled Pipeline](../pipelines/create-and-use-scheduled-pipeline/)
41+
- [Manage Pipelines](../pipelines/manage-pipelines/)
4342

docs/user-guide/pipelines/create-and-use-real-time-pipeline.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -153,7 +153,7 @@ Use this guide to create and run real-time pipelines. Real-time pipelines proces
153153

154154
## Related links
155155
- [Pipelines in OpenObserve](../pipelines/pipelines/)
156-
- [Create and Use Scheduled Pipeline](create-and-use-scheduled-pipeline.md)
156+
- [Create and Use Scheduled Pipeline](../pipelines/create-and-use-scheduled-pipeline/)
157157
- [Import and Export Pipelines](../pipelines/import-and-export-pipelines/)
158158
- [Manage Pipelines](../pipelines/manage-pipelines/)
159159
- [Configurable Delay in Scheduled Pipelines](../pipelines/configurable-delay-in-scheduled-pipelines/)

docs/user-guide/pipelines/create-and-use-scheduled-pipeline.md

Lines changed: 15 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ Use this guide to create and run scheduled pipelines. Scheduled pipelines run at
99
## Steps to create a scheduled pipeline
1010

1111
??? "Step 1: Open the pipeline editor"
12-
### Step 1: Open the pipeline editor
12+
### Step 1: Open the pipeline editor
1313
1. Log in to OpenObserve.
1414
2. In the left navigation panel, select **Pipelines**. By default, the system opens the **Pipelines** tab.
1515
![Pipelines tab](../../images/pipelines-tab.png)
@@ -24,25 +24,25 @@ Use this guide to create and run scheduled pipelines. Scheduled pipelines run at
2424
Scheduled pipelines do not use a **Stream** node as the source. They always begin with a **Query** node.
2525

2626
1. From the **Source** section, drag the **Query** node into the editor.
27-
2. The **Associate Query** panel opens. The panel contains two sections, Build Query and Set Variables.
27+
2. The **Associate Query** panel opens. The panel contains the following sections, **Build Query**, **SQL Query** editor, **Output**, **Set Variables**.
2828
![Associate Query](../../images/associate-query.png)
2929
3. In the **Build Query** section:
3030

3131
1. Select the **Stream Type**.
3232
2. Select the **Stream Name**.
33-
4. Edit or write your SQL query in the SQL editor.
33+
4. Edit or write your SQL query in the **SQL query** editor.
3434
5. Select the time range from the time range selector at the top-right corner.
3535
5. Click **Run query** to preview data in the **Output** panel.
36-
![output-preview](../../images/query-output.png)
36+
![output-preview](../../images/query-output.png)
3737
6. In the **Set Variables** section configures how the pipeline runs using the following settings:
3838

3939
- **Enable Cron**: Switch from interval-based scheduling to cron-based scheduling.
4040
- **Frequency**: How often the pipeline runs.
4141
- **Period**: The data window to process. Period must equal Frequency.
4242
- **Delay**: How many minutes the pipeline waits after the scheduled time before execution.
4343
44-
> Visit the [Pipelines overview](https://openobserve.ai/docs/user-guide/pipelines/pipelines/) page to learn more about frequency, period, and cron.
45-
> Learn more about [Configurable Delay in Scheduled Pipelines](https://openobserve.ai/docs/user-guide/pipelines/configurable-delay-in-scheduled-pipelines/).
44+
> See the [Pipelines overview](https://openobserve.ai/docs/user-guide/pipelines/pipelines/) page to learn more about frequency, period, and cron.
45+
> See the [Configurable Delay in Scheduled Pipelines](https://openobserve.ai/docs/user-guide/pipelines/configurable-delay-in-scheduled-pipelines/) page to learn how to configure delay in scheduled-pipelines.
4646

4747
![variables](../../images/scheduled-variables.png)
4848

@@ -60,10 +60,6 @@ Use this guide to create and run scheduled pipelines. Scheduled pipelines run at
6060
3. Enter a value in the **Value** input box.
6161
4. Add more conditions if needed.
6262

63-
!!! note "Important"
64-
**Condition nodes require a stream schema.** <br>
65-
If you create a new stream in **Step 3** and do not define any fields, the **Condition** editor does not display field names.
66-
To use conditions, add at least one field while creating the source stream or ingest sample data so that the schema is inferred.
6763
!!! note "Guidelines"
6864
- Use an empty string to check for empty values. For example, `app_name != ""`
6965
- Use null to check for null values. For example, `app_name != null`
@@ -104,13 +100,13 @@ Use this guide to create and run scheduled pipelines. Scheduled pipelines run at
104100
2. In the **Associate Stream** panel, configure the destination stream.
105101
3. Select an existing stream or create a new one. Stream creation follows the same configuration as shown in Step 3.
106102

107-
!!! note "Note"
108-
- You can add **Enrichment_tables** as a destination stream for scheduled pipelines.
109-
110-
!!! note "Dynamic stream names"
111-
You can configure the stream name dynamically with curly braces.
112-
Example: `static_text_{fieldname}_postfix`
113-
Static text before and after the braces is optional.
103+
!!! note "Note"
104+
You can add **Enrichment_tables** as a destination stream for scheduled pipelines.
105+
106+
!!! note "Dynamic stream names"
107+
You can configure the stream name dynamically with curly braces.
108+
Example: `static_text_{fieldname}_postfix`
109+
Static text before and after the braces is optional.
114110

115111

116112
**To add an external destination:**
@@ -133,7 +129,7 @@ Use this guide to create and run scheduled pipelines. Scheduled pipelines run at
133129
Click **Save**. The pipeline becomes active immediately.
134130
![scheduled-pipeline-list-view](../../images/scheduled-pipeline-list-view.png)
135131

136-
## Use the real-time pipeline
132+
## Use the scheduled pipeline
137133

138134
??? "Step 1: Ingest data"
139135
### Step 1: Ingest data
@@ -152,7 +148,7 @@ Use this guide to create and run scheduled pipelines. Scheduled pipelines run at
152148

153149
## Related links
154150
- [Pipelines in OpenObserve](../pipelines/pipelines/)
155-
- [Create and Use Real-time Pipeline](../create-and-use-real-time-pipeline/)
151+
- [Create and Use Real-time Pipeline](../pipelines/create-and-use-real-time-pipeline/)
156152
- [Import and Export Pipelines](../pipelines/import-and-export-pipelines/)
157153
- [Manage Pipelines](../pipelines/manage-pipelines/)
158154
- [Configurable Delay in Scheduled Pipelines](../pipelines/configurable-delay-in-scheduled-pipelines/)

docs/user-guide/pipelines/import-and-export-pipelines.md

Lines changed: 7 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -16,14 +16,13 @@ To export an existing pipeline configuration:
1616
![download-pipeline](../../images/download-pipeline-json.png)
1717
The downloaded `.json` file contains the pipeline configuration and can be used to import the pipeline into another environment.<br>
1818

19-
For example, exporting the pipeline `scheduled-pipeline-demo` generates a JSON file similar to the following:
20-
![json-file-pipeline-export](../../images/json-file-pipeline-export.png)
19+
!!! note "Example"
20+
Exporting the pipeline `scheduled-pipeline-demo` generates a JSON file similar to the following:"
21+
![json-file-pipeline-export](../../images/json-file-pipeline-export.png)
2122

2223
## Import a pipeline
23-
2424
To import a previously exported pipeline:
25-
![pipeline-import](../../images/pipeline-import.png)
26-
25+
![alt text](../../images/import-pipelines.png)
2726
1. Go to **Pipelines**.
2827
2. Click **Import Pipeline** in the top-right corner.<br>
2928
3. Choose one of the supported import methods:
@@ -32,7 +31,6 @@ To import a previously exported pipeline:
3231
- **Enter URL**: Provide a URL to fetch the pipeline configuration.
3332
- **Paste JSON Object**: Copy and paste the JSON pipeline definition in the JSON editor.
3433

35-
![import methods](pipeline-import-json.png)
3634
4. To import pipelines in bulk, choose multiple JSON files.
3735
5. Click **Import**.
3836
The imported pipeline appears in the **Pipelines** tab.
@@ -53,9 +51,9 @@ If any validation errors occur during import, refer to the following resolutions
5351
- **Remote Destination does not exist:** Ensure the remote destination is accurate.
5452

5553

56-
## Related Links
54+
## Related links
5755
- [Pipelines in OpenObserve](../pipelines/pipelines/)
58-
- [Create and Use Real-time Pipeline](../create-and-use-real-time-pipeline/)
59-
- [Create and Use Scheduled Pipeline](create-and-use-scheduled-pipeline.md)
56+
- [Create and Use Real-time Pipeline](../pipelines/create-and-use-real-time-pipeline/)
57+
- [Create and Use Scheduled Pipeline](../pipelines/create-and-use-scheduled-pipeline/)
6058
- [Manage Pipelines](../pipelines/manage-pipelines/)
6159
- [Configurable Delay in Scheduled Pipelines](../pipelines/configurable-delay-in-scheduled-pipelines/)

docs/user-guide/pipelines/index.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,8 @@ In OpenObserve, pipelines are data processing workflows used to transform logs,
33
**Learn more:**
44

55
- [Pipelines in OpenObserve](../pipelines/pipelines/)
6-
- [Create and Use Real-time Pipeline](../create-and-use-real-time-pipeline/)
7-
- [Create and Use Scheduled Pipeline](create-and-use-scheduled-pipeline.md)
6+
- [Create and Use Real-time Pipeline](../pipelines/create-and-use-real-time-pipeline/)
7+
- [Create and Use Scheduled Pipeline](../pipelines/create-and-use-scheduled-pipeline/)
88
- [Import and Export Pipelines](../pipelines/import-and-export-pipelines/)
99
- [Manage Pipelines](../pipelines/manage-pipelines/)
1010
- [Configurable Delay in Scheduled Pipelines](../pipelines/configurable-delay-in-scheduled-pipelines/)

docs/user-guide/pipelines/manage-pipelines.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ Each pipeline row includes the following actions.
3636

3737
### Pause and resume pipelines
3838

39-
You can pause both real-time and scheduled pipelines.
39+
You can pause both real-time and scheduled pipelines. <br>
4040

4141
**Pausing a pipeline:**
4242

@@ -74,9 +74,9 @@ Scheduled pipelines display a Resume Pipeline Ingestion dialog with two options.
7474
7575
### Pipeline error
7676
If a pipeline encounters an error, an error icon appears in the **Actions** column.
77-
![pipeline-error](pipeline-error.png)
77+
![pipeline-error](../../images/pipeline-error.png)
7878
Selecting this icon opens a dialog that displays the full error summary.
79-
![pipeline-error-view](pipeline-error-view.png)
79+
![pipeline-error-view](../../images/pipeline-error-view.png)
8080

8181
## Pipeline history
8282
You can view past executions, errors, and performance statistics in the Pipeline History page.
@@ -88,7 +88,7 @@ You can export any pipeline and import pipeline configurations through the Impor
8888

8989
## Related links
9090
- [Pipelines in OpenObserve](../pipelines/pipelines/)
91-
- [Create and Use Real-time Pipeline](../create-and-use-real-time-pipeline/)
92-
- [Create and Use Scheduled Pipeline](create-and-use-scheduled-pipeline.md)
91+
- [Create and Use Real-time Pipeline](../pipelines/create-and-use-real-time-pipeline/)
92+
- [Create and Use Scheduled Pipeline](../pipelines/create-and-use-scheduled-pipeline/)
9393
- [Import and Export Pipelines](../pipelines/import-and-export-pipelines/)
9494
- [Configurable Delay in Scheduled Pipelines](../pipelines/configurable-delay-in-scheduled-pipelines/)

0 commit comments

Comments
 (0)