diff --git a/elastic-stack-setup/elastic-stack-setup-02-environment-watch/elastic-stack-setup-02-environment-watch.md b/elastic-stack-setup/elastic-stack-setup-02-environment-watch/elastic-stack-setup-02-environment-watch.md index 30d45ae..9fe37b7 100644 --- a/elastic-stack-setup/elastic-stack-setup-02-environment-watch/elastic-stack-setup-02-environment-watch.md +++ b/elastic-stack-setup/elastic-stack-setup-02-environment-watch/elastic-stack-setup-02-environment-watch.md @@ -83,6 +83,12 @@ Refer to the [Troubleshooting Guide](../troubleshooting/relativity-server-cli.md) if you encounter any issues. +## Managing ILM Retention + +By default, Environment Watch uses the retention periods configured in Elasticsearch Index Lifecycle Management (ILM) policies. To view or update the retention period for logs, metrics, or traces data streams after setup, use the `configure-retention` command: + +See [Configure Elasticsearch ILM Retention using the Relativity Server CLI](./ew-03-extensibility-configuration/ew-extensibility-configuration-03-retention-policy.md#configure-elasticsearch-ilm-retention-using-the-relativity-server-cli) for full instructions, including interactive mode, quiet mode for scripted updates, and how to verify changes in Kibana Dev Tools. + ## Next Steps * [Click here to continue Environment Watch Setup](./ew-01-install-monitoring-agents.md) diff --git a/elastic-stack-setup/elastic-stack-setup-02-environment-watch/ew-03-extensibility-configuration/ew-extensibility-configuration-03-retention-policy.md b/elastic-stack-setup/elastic-stack-setup-02-environment-watch/ew-03-extensibility-configuration/ew-extensibility-configuration-03-retention-policy.md index 1343ab5..e1b4854 100644 --- a/elastic-stack-setup/elastic-stack-setup-02-environment-watch/ew-03-extensibility-configuration/ew-extensibility-configuration-03-retention-policy.md +++ b/elastic-stack-setup/elastic-stack-setup-02-environment-watch/ew-03-extensibility-configuration/ew-extensibility-configuration-03-retention-policy.md @@ -86,467 +86,309 @@ When determining the appropriate retention period for your environment, consider --- -## Configuration Steps +## Configure Elasticsearch ILM Retention using the Relativity Server CLI -### Step 1: Create Component Template with Required Retention Policy +The `configure-retention` command sets Elasticsearch Index Lifecycle Management (ILM) retention policies for logs, metrics, and traces data streams. Use this command to control how long monitoring data is retained in Elasticsearch for the Environment Watch InfraWatch cluster. -Elastic APM provides the `apm-90d@lifecycle` component template by default for 90-day retention. For 30-day retention (recommended for traces), create a custom component template using the Dev Tools Console in Kibana: +> [!NOTE] +> It is recommended to run the CLI from the Primary SQL Server. -**Navigate to Dev Tools Console:** +> This guide assumes the Relativity Server bundle was extracted to `C:\Server.Bundle.x.y.z` or a similar directory chosen by the user. -1. Open Kibana in your web browser -2. Click on **Dev Tools** in the left navigation menu (or use the search bar at the top to find "Dev Tools") -3. You'll see the Console interface where you can execute Elasticsearch queries +### Prerequisites -**Sample Request:** +- The Server-bundle zip file has been downloaded and extracted to `C:\Server.Bundle.x.y.z` +- Access to the Relativity Secret Store (Whitelisted for Secret Store access. Please see [here](https://help.relativity.com/Server2025/Content/System_Guides/Secret_Store/Secret_Store.htm#Configuringclients) for information on whitelisting.) +- Elasticsearch is running and reachable. Confirm by browsing to the cluster endpoint, `https://:9200` (for example `https://emttest:9200`) — a running cluster returns its version and cluster details. +- The initial Environment Watch setup has been completed. See [Set up Environment Watch using the Relativity Server CLI](../elastic-stack-setup-02-environment-watch.md) -``` -# Here apm-30d@lifecycle is the name of the component template -PUT _component_template/apm-30d@lifecycle -{ - "template": { - "lifecycle": { - "enabled": true, - "data_retention": "30d" - } - }, - "_meta": { - "managed": true, - "description": "Data stream lifecycle for 30 days of retention" - } -} -``` +### Options -**Sample Output:** +| Flag | Description | Default | +|------|-------------|---------| +| `--logs-days ` | Retention period in days for the logs ILM policy (`infrawatch-logs-policy`). Must be greater than 0. | Prompted interactively | +| `--metrics-days ` | Retention period in days for the metrics ILM policy (`infrawatch-metrics-policy`). Must be greater than 0. | Prompted interactively | +| `--traces-days ` | Retention period in days for the traces ILM policy (`infrawatch-traces-policy`). Must be greater than 0. | Prompted interactively | +| `--quiet` | Suppress all prompts and the confirmation gate. Credentials are read exclusively from the Secret Store. At least one `--*-days` flag must be supplied. Use for automated or scripted execution. | `false` | +| `--dryrun` | Preview the ILM policy JSON that would be submitted without making any changes to Elasticsearch. Compatible with both interactive and quiet modes. | `false` | -```json -{ - "acknowledged": true -} -``` +### Usage -### Step 2: Update Index Templates +#### Interactive -Update the following index templates to use the appropriate component template based on your retention requirements: +Running `configure-retention` without `--quiet` launches an interactive session. If `relsvr setup` has been run, credentials are fetched silently from the Secret Store — no prompt for cluster URL, admin username, or password. If setup has not been run, the CLI prompts for those credentials before continuing. -| Index Template | Data Type | Default Component | Recommended Component | -|---------------|-----------|-------------------|----------------------| -| `logs-apm.app@template` | Logs | `apm-10d@lifecycle` | `apm-90d@lifecycle` | -| `metrics-apm.app@template` | Metrics | `apm-90d@lifecycle` | `apm-90d@lifecycle` | -| `traces-apm@template` | Traces | `apm-10d@lifecycle` | `apm-30d@lifecycle` | +The command fetches and displays the current ILM retention values for all three signals, then prompts for each one individually. Press **Enter** at any signal prompt to skip that signal — the policy for that signal is left unchanged, and the summary below only lists the signals you actually changed. -> [!IMPORTANT] -> Changes to index templates only affect **new data streams** created after the update. Existing data streams will continue using their original retention policies until they are manually updated or recreated. +> [!NOTE] +> On a first run, before the `infrawatch-*-policy` policies have been created, the fetch still succeeds — each signal is reported as `not set` rather than a number, the prompts show `[current: ?d, press Enter to skip]`, and the summary shows the old value as `not set` (for example `not set -> 21d`). This is expected on a new cluster and is not the same as the fetch failure described in [Current retention state can't be fetched](#current-retention-state-cant-be-fetched-interactive-mode) below. -#### a. Update Logs Index Template +If you press **Enter** at all three prompts, the command prints `Operation cancelled.` and exits immediately — it never shows the "Summary of changes" block or the "Apply these changes?" confirmation, because there is nothing to confirm: -First, use the Dev Tools Console in Kibana to retrieve the existing index template settings using a GET request: +``` +C:\Server.Bundle.x.y.z\relsvr.exe configure-retention + +Relativity Server CLI - 102.1.26 +Copyright (c) 2026, Relativity ODA LLC + +Configures Elasticsearch Index Lifecycle Management (ILM) retention policies for logs, metrics, and traces data streams. Use this command to control how long telemetry data is retained before Elasticsearch automatically deletes it. -**Sample Request:** +Current ILM retention state: + Logs: 30d + Metrics: 30d + Traces: 7d +Enter new retention for Logs in days [current: 30d, press Enter to skip]: +Enter new retention for Metrics in days [current: 30d, press Enter to skip]: +Enter new retention for Traces in days [current: 7d, press Enter to skip]: + +Operation cancelled. ``` -# Here logs-apm.app@template is the name of the index template -GET _index_template/logs-apm.app@template + +> [!NOTE] +> `Operation cancelled.` is the same message shown when you decline the confirmation prompt below, and when you decline to continue after a failed current-state fetch (see [Current retention state can't be fetched](#current-retention-state-cant-be-fetched-interactive-mode) below). All three cases look identical in the CLI output — none of them make any ILM changes. + +Entering at least one value shows an old → new summary before applying anything: + ``` +C:\Server.Bundle.x.y.z\relsvr.exe configure-retention -**Sample Output:** +Relativity Server CLI - 102.1.26 +Copyright (c) 2026, Relativity ODA LLC -```json -{ - "index_templates": [ - { - "name": "logs-apm.app@template", - "index_template": { - "index_patterns": [ - "logs-apm.app.*-*" - ], - "template": { - "settings": { - "index": { - "mode": "standard", - "default_pipeline": "logs-apm.app@default-pipeline", - "final_pipeline": "logs-apm@pipeline" - } - } - }, - "composed_of": [ - "logs@mappings", - "apm@mappings", - "apm@settings", - "logs-apm@settings", - "logs-apm.app-fallback@ilm", - "ecs@mappings", - "logs@custom", - "logs-apm.app@custom", - "apm-10d@lifecycle" - ], - "priority": 210, - "version": 101, - "_meta": { - "managed": true, - "description": "Index template for logs-apm.app.*-*" - }, - "data_stream": { - "hidden": false, - "allow_custom_routing": false - }, - "allow_auto_create": true, - "ignore_missing_component_templates": [ - "logs@custom", - "logs-apm.app@custom", - "logs-apm.app-fallback@ilm" - ] - } - } - ] -} +Configures Elasticsearch Index Lifecycle Management (ILM) retention policies for logs, metrics, and traces data streams. Use this command to control how long telemetry data is retained before Elasticsearch automatically deletes it. + +Current ILM retention state: + Logs: 30d + Metrics: 30d + Traces: 7d + +Enter new retention for Logs in days [current: 30d, press Enter to skip]: 60 +Enter new retention for Metrics in days [current: 30d, press Enter to skip]: +Enter new retention for Traces in days [current: 7d, press Enter to skip]: + +Summary of changes: + Logs: 30d -> 60d + +Apply these changes? [y/n] (n): y + +Configuring ILM retention policies on https://emttest:9200/... + Applying Logs retention: 60 days (policy: infrawatch-logs-policy)... + +Applying ILM retention policies... + + Logs retention set to 60 days. +Retention policies applied successfully. ``` -Then, copy the `index_template` section from the output above and update it by replacing `apm-10d@lifecycle` with `apm-90d@lifecycle` in the `composed_of` array using a PUT request: +> [!NOTE] +> `Applying ILM retention policies...` is shown as a spinner while the update is in progress, not a percentage progress bar — it disappears once the update finishes and is replaced by the per-signal success lines shown above. Only signals you changed appear in the summary and success output; unchanged signals are omitted entirely, not listed as "(no change)". -**Sample Request:** +Entering anything other than `y` at the confirmation prompt aborts cleanly with no changes made: ``` -# Here logs-apm.app@template is the name of the index template -PUT _index_template/logs-apm.app@template -{ - "index_patterns": [ - "logs-apm.app.*-*" - ], - "template": { - "settings": { - "index": { - "mode": "standard", - "default_pipeline": "logs-apm.app@default-pipeline", - "final_pipeline": "logs-apm@pipeline" - } - } - }, - "composed_of": [ - "logs@mappings", - "apm@mappings", - "apm@settings", - "logs-apm@settings", - "logs-apm.app-fallback@ilm", - "ecs@mappings", - "logs@custom", - "logs-apm.app@custom", - "apm-90d@lifecycle" - ], - "priority": 210, - "version": 101, - "_meta": { - "managed": true, - "description": "Index template for logs-apm.app.*-*" - }, - "data_stream": { - "hidden": false, - "allow_custom_routing": false - }, - "allow_auto_create": true, - "ignore_missing_component_templates": [ - "logs@custom", - "logs-apm.app@custom", - "logs-apm.app-fallback@ilm" - ] -} +Operation cancelled. ``` -**Sample Output:** +#### Interactive with a pre-filled default + +Passing a `--*-days` flag in interactive mode pre-fills that signal's prompt with the flag value. The current value is still shown as context, pressing **Enter** accepts the pre-filled default without retyping it, and confirmation is still required. -```json -{ - "acknowledged": true -} ``` +C:\Server.Bundle.x.y.z\relsvr.exe configure-retention --logs-days 60 -#### b. Update Metrics Index Template (Optional) +Relativity Server CLI - 102.1.26 +Copyright (c) 2026, Relativity ODA LLC -The `metrics-apm.app@template` already uses the `apm-90d@lifecycle` component template by default, so it does not require any updates if you are using the recommended 90-day retention period. If you need a different retention period, retrieve the current template configuration using a GET request: +Configures Elasticsearch Index Lifecycle Management (ILM) retention policies for logs, metrics, and traces data streams. Use this command to control how long telemetry data is retained before Elasticsearch automatically deletes it. -**Sample Request:** +Current ILM retention state: + Logs: 30d + Metrics: 30d + Traces: 7d -``` -# Get the current template configuration -GET _index_template/metrics-apm.app@template -``` +Enter new retention for Logs in days [current: 30d, default: 60]: +Enter new retention for Metrics in days [current: 30d, press Enter to skip]: +Enter new retention for Traces in days [current: 7d, press Enter to skip]: -**Sample Output:** +Summary of changes: + Logs: 30d -> 60d -```json -{ - "index_templates": [ - { - "name": "metrics-apm.app@template", - "index_template": { - "index_patterns": [ - "metrics-apm.app.*-*" - ], - "template": { - "settings": { - "index": { - "mode": "standard", - "default_pipeline": "metrics-apm.app@default-pipeline", - "final_pipeline": "metrics-apm@pipeline" - } - } - }, - "composed_of": [ - "metrics@mappings", - "apm@mappings", - "apm@settings", - "metrics-apm@settings", - "metrics-apm.app-fallback@ilm", - "ecs@mappings", - "metrics@custom", - "metrics-apm.app@custom", - "apm-90d@lifecycle" - ], - "priority": 210, - "version": 101, - "_meta": { - "managed": true, - "description": "Index template for metrics-apm.app.*-*" - }, - "data_stream": { - "hidden": false, - "allow_custom_routing": false - }, - "allow_auto_create": true, - "ignore_missing_component_templates": [ - "metrics@custom", - "metrics-apm.app@custom", - "metrics-apm.app-fallback@ilm" - ] - } - } - ] -} -``` +Apply these changes? [y/n] (n): y -Then, if you need to change the retention period, copy the `index_template` section from the output above and update it by replacing `apm-90d@lifecycle` with your desired retention component template in the `composed_of` array using a PUT request: +Configuring ILM retention policies on https://emttest:9200/... + Applying Logs retention: 60 days (policy: infrawatch-logs-policy)... -**Sample Request:** +Applying ILM retention policies... + Logs retention set to 60 days. +Retention policies applied successfully. ``` -PUT _index_template/metrics-apm.app@template -{ - "index_patterns": [ - "metrics-apm.app.*-*" - ], - "template": { - "settings": { - "index": { - "mode": "standard", - "default_pipeline": "metrics-apm.app@default-pipeline", - "final_pipeline": "metrics-apm@pipeline" - } - } - }, - "composed_of": [ - "metrics@mappings", - "apm@mappings", - "apm@settings", - "metrics-apm@settings", - "metrics-apm.app-fallback@ilm", - "ecs@mappings", - "metrics@custom", - "metrics-apm.app@custom", - "apm-90d@lifecycle" - ], - "priority": 210, - "version": 101, - "_meta": { - "managed": true, - "description": "Index template for metrics-apm.app.*-*" - }, - "data_stream": { - "hidden": false, - "allow_custom_routing": false - }, - "allow_auto_create": true, - "ignore_missing_component_templates": [ - "metrics@custom", - "metrics-apm.app@custom", - "metrics-apm.app-fallback@ilm" - ] -} + +#### Quiet mode (automated / scripted) + +Combining `--quiet` with one or more `--*-days` flags suppresses all prompts and the confirmation gate. Credentials come exclusively from the Secret Store — `relsvr setup` must have been run first. This is suitable for scheduled tasks or unattended automation scripts. + ``` +C:\Server.Bundle.x.y.z\relsvr.exe configure-retention --quiet --logs-days 30 --metrics-days 90 + +Relativity Server CLI - 102.1.26 +Copyright (c) 2026, Relativity ODA LLC + +Configures Elasticsearch Index Lifecycle Management (ILM) retention policies for logs, metrics, and traces data streams. Use this command to control how long telemetry data is retained before Elasticsearch automatically deletes it. -**Sample Output:** +Configuring ILM retention policies on https://emttest:9200/... + Applying Logs retention: 30 days (policy: infrawatch-logs-policy)... + Applying Metrics retention: 90 days (policy: infrawatch-metrics-policy)... -```json -{ - "acknowledged": true -} +Applying ILM retention policies... + + Logs retention set to 30 days. + Metrics retention set to 90 days. +Retention policies applied successfully. ``` -#### c. Update Traces Index Template +#### Dry run -For traces, retrieve the current template configuration using a GET request: +Use `--dryrun` to preview the ILM policy JSON that would be submitted without writing any changes to Elasticsearch. Dry run works in both interactive and quiet modes. -**Sample Request:** +**Quiet dry run — no prompts, no update step at all:** ``` -# Get the current template configuration -GET _index_template/traces-apm@template +C:\Server.Bundle.x.y.z\relsvr.exe configure-retention --quiet --logs-days 30 --dryrun + +Relativity Server CLI - 102.1.26 +Copyright (c) 2026, Relativity ODA LLC + +Configures Elasticsearch Index Lifecycle Management (ILM) retention policies for logs, metrics, and traces data streams. Use this command to control how long telemetry data is retained before Elasticsearch automatically deletes it. + +Dry run mode — no ILM policies will be modified. +Dry run — ILM policy 'infrawatch-logs-policy' would be submitted with: {"policy":{"phases":{"delete":{"min_age":"30d","actions":{"delete":{}}}}}} ``` -**Sample Output:** +**Interactive dry run — prompts and confirmation appear as usual, then a preview instead of an update:** -```json -{ - "index_templates": [ - { - "name": "traces-apm@template", - "index_template": { - "index_patterns": [ - "traces-apm*" - ], - "template": { - "settings": { - "index": { - "mode": "standard", - "default_pipeline": "traces-apm@default-pipeline", - "final_pipeline": "traces-apm@pipeline" - } - } - }, - "composed_of": [ - "traces@mappings", - "apm@mappings", - "apm@settings", - "traces-apm@settings", - "traces-apm-fallback@ilm", - "ecs@mappings", - "traces@custom", - "traces-apm@custom", - "apm-10d@lifecycle" - ], - "priority": 210, - "version": 101, - "_meta": { - "managed": true, - "description": "Index template for traces-apm*" - }, - "data_stream": { - "hidden": false, - "allow_custom_routing": false - }, - "allow_auto_create": true, - "ignore_missing_component_templates": [ - "traces@custom", - "traces-apm@custom", - "traces-apm-fallback@ilm" - ] - } - } - ] -} ``` +C:\Server.Bundle.x.y.z\relsvr.exe configure-retention --dryrun -Then, copy the `index_template` section from the output above and update it by replacing `apm-10d@lifecycle` with `apm-30d@lifecycle` (which you created in Step 1) in the `composed_of` array using a PUT request: +Relativity Server CLI - 102.1.26 +Copyright (c) 2026, Relativity ODA LLC -**Sample Request:** +Configures Elasticsearch Index Lifecycle Management (ILM) retention policies for logs, metrics, and traces data streams. Use this command to control how long telemetry data is retained before Elasticsearch automatically deletes it. +Current ILM retention state: + Logs: 30d + Metrics: 30d + Traces: 7d + +Enter new retention for Logs in days [current: 30d, press Enter to skip]: 30 +Enter new retention for Metrics in days [current: 30d, press Enter to skip]: +Enter new retention for Traces in days [current: 7d, press Enter to skip]: + +Summary of changes: + Logs: 30d -> 30d + +Apply these changes? [y/n] (n): y + +Dry run mode — no ILM policies will be modified. +Dry run — ILM policy 'infrawatch-logs-policy' would be submitted with: {"policy":{"phases":{"delete":{"min_age":"30d","actions":{"delete":{}}}}}} ``` -PUT _index_template/traces-apm@template -{ - "index_patterns": [ - "traces-apm*" - ], - "template": { - "settings": { - "index": { - "mode": "standard", - "default_pipeline": "traces-apm@default-pipeline", - "final_pipeline": "traces-apm@pipeline" - } - } - }, - "composed_of": [ - "traces@mappings", - "apm@mappings", - "apm@settings", - "traces-apm@settings", - "traces-apm-fallback@ilm", - "ecs@mappings", - "traces@custom", - "traces-apm@custom", - "apm-30d@lifecycle" - ], - "priority": 210, - "version": 101, - "_meta": { - "managed": true, - "description": "Index template for traces-apm*" - }, - "data_stream": { - "hidden": false, - "allow_custom_routing": false - }, - "allow_auto_create": true, - "ignore_missing_component_templates": [ - "traces@custom", - "traces-apm@custom", - "traces-apm-fallback@ilm" - ] -} -``` -**Sample Output:** +#### Secret Store has no Elasticsearch credentials + +If `relsvr setup` has not been run — or the Elasticsearch secret has been removed from the Secret Store — the behavior depends on the mode. + +**Interactive mode — the CLI prompts for the credentials:** -```json -{ - "acknowledged": true -} ``` +C:\Server.Bundle.x.y.z\relsvr.exe configure-retention -### Step 3: Delete Existing Data Streams (Setup Time Only) +Relativity Server CLI - 102.1.26 +Copyright (c) 2026, Relativity ODA LLC -> [!CAUTION] -> **⚠️ DESTRUCTIVE OPERATION – PERMANENT DATA LOSS** -> -> **This step is optional and is not required for most Environment Watch deployments.** It should only be performed during initial setup or in controlled, non-production scenarios. -> -> This step will **permanently delete all data and indices** in the specified data streams. There is no recovery. Only proceed if: -> - You are in a **development or non-production environment**, OR -> - You have **backed up all critical data** from these data streams, OR -> - You are performing **initial setup** and no production data exists yet -> -> **Do NOT run this on production systems with active data.** +Configures Elasticsearch Index Lifecycle Management (ILM) retention policies for logs, metrics, and traces data streams. Use this command to control how long telemetry data is retained before Elasticsearch automatically deletes it. +Enter the Elasticsearch cluster endpoint URL: https://emttest:9200 +Enter the Elasticsearch admin username: elastic +Enter the Elasticsearch admin password: ******************** -After updating the index templates with new retention policies, you need to delete the existing data streams so they can be recreated with the updated retention settings. Use the Dev Tools Console in Kibana to run the following commands: +Current ILM retention state: + Logs: not set + Metrics: not set + Traces: not set -**Delete Logs Data Stream:** +Enter new retention for Logs in days [current: ?d, press Enter to skip]: 21 +Enter new retention for Metrics in days [current: ?d, press Enter to skip]: 22 +Enter new retention for Traces in days [current: ?d, press Enter to skip]: 23 +Summary of changes: + Logs: not set -> 21d + Metrics: not set -> 22d + Traces: not set -> 23d + +Apply these changes? [y/n] (n): y ``` -DELETE _data_stream/logs-apm.app* -``` -**Delete Metrics Data Stream:** +The command then applies the policies as normal. The credentials entered this way are used for that run only — run `relsvr setup` to store them in the Secret Store. + +**Quiet mode — no prompt is shown, the command exits with an error:** + +Because `--quiet` suppresses all prompts, the credentials cannot be supplied interactively and the run fails immediately: ``` -DELETE _data_stream/metrics-apm.app* +Elasticsearch credentials are not available in the Secret Store. Run 'relsvr setup' first, or use interactive mode to enter credentials manually. ``` -**Delete Traces Data Stream:** +#### Current retention state can't be fetched (interactive mode) + +If the CLI can't reach Elasticsearch to read the existing ILM policies before showing prompts — for example the Elasticsearch service is stopped or the cluster is briefly unreachable — it warns and asks whether to continue anyway. Policies that simply don't exist yet do **not** trigger this warning; they are reported as `not set`. ``` -DELETE _data_stream/traces-apm* +Could not retrieve current ILM retention state. Proceed with caution. +Continue? [y/n] (n): ``` -**Sample Output for each command:** +Declining prints `Operation cancelled.` and exits with no changes made, the same message used when all prompts are skipped or the final confirmation is declined. Before re-running, confirm the cluster is up by browsing to `https://:9200`. -```json -{ - "acknowledged": true -} +#### Invalid retention value + +Entering a non-numeric or non-positive value at a signal prompt re-prompts inline in place, rather than aborting: + +``` +Retention must be a positive number ``` -> [!NOTE] -> After deleting the data streams, new data streams will be automatically created with the updated retention policies when APM agents begin sending new telemetry data. +### Verify the changes + +#### Kibana Dev Tools + +After running `configure-retention`, confirm the updated retention value in Kibana Dev Tools. + +1. In Kibana, navigate to **Dev Tools** > **Console**. +2. Run the following query for each signal you updated, replacing `` with `logs`, `metrics`, or `traces`: + + ``` + GET /_ilm/policy/infrawatch--policy + ``` + +3. In the response, locate the `delete` phase and confirm `min_age` matches the value you set: + + ```json + { + "infrawatch-logs-policy": { + "policy": { + "phases": { + "delete": { + "min_age": "30d", + "actions": { + "delete": {} + } + } + } + } + } + } + ``` --- @@ -558,4 +400,4 @@ For more advanced retention management using Index Lifecycle Management (ILM) po - [Configure ILM Policies](https://www.elastic.co/guide/en/elasticsearch/reference/current/set-up-lifecycle-policy.html) - [Data Stream Lifecycle vs ILM](https://www.elastic.co/guide/en/elasticsearch/reference/current/data-stream-lifecycle.html) -ILM provides more granular control over data lifecycle phases and allows for tiered storage architectures in large-scale environments. \ No newline at end of file +ILM provides more granular control over data lifecycle phases and allows for tiered storage architectures in large-scale environments. diff --git a/elastic-stack-setup/elastic-stack-setup-04-post-install-verification/retention-policy.md b/elastic-stack-setup/elastic-stack-setup-04-post-install-verification/retention-policy.md index 8e477cf..c91c386 100644 --- a/elastic-stack-setup/elastic-stack-setup-04-post-install-verification/retention-policy.md +++ b/elastic-stack-setup/elastic-stack-setup-04-post-install-verification/retention-policy.md @@ -58,4 +58,4 @@ Each query should return the data stream names along with their configured lifec - **enabled**: Should be `true` - **data_retention**: Indicates the configured retention period (e.g., "30d" for 30 days, "90d" for 90 days) -If the lifecycle settings don't match your expected configuration, you may need to update your retention period according to [elasticsearch_retention_policy_guidelines.md](../elastic-stack-setup-02-environment-watch/ew-03-extensibility-configuration/ew-extensibility-configuration-03-retention-policy.md). +If the lifecycle settings don't match your expected configuration, you may need to update your retention period according to [Elasticsearch Retention Policy - Guidelines](../elastic-stack-setup-02-environment-watch/ew-03-extensibility-configuration/ew-extensibility-configuration-03-retention-policy.md).