Skip to content

[FEATURE] add CloudWatch plugin - #856

Draft
T-Chittibabu wants to merge 1 commit into
perses:mainfrom
T-Chittibabu:feat/cloudwatch-plugin
Draft

T-Chittibabu wants to merge 1 commit into
perses:mainfrom
T-Chittibabu:feat/cloudwatch-plugin

Conversation

@T-Chittibabu

Copy link
Copy Markdown

Design discussion: perses/perses#4547

Description

This adds a CloudWatch plugin for Amazon CloudWatch metrics. It queries the new CloudWatchProxy of the Perses server: the server signs the requests with its own AWS identity, optionally assuming an IAM role, so the browser never receives AWS credentials and the datasource contains none.

Depends on:

This stays a draft until both are released. Until then, type-check, the schema tests and the Go SDK build can't resolve the new spec types in CI.

Plugins

  • CloudWatchDatasource:
    • holds the region, and optionally the IAM role to assume and the secret holding its external ID;
    • the schema reuses datasource.#CloudWatchDatasourceSpec from perses/spec instead of redefining it;
    • the editor validates the region and the role ARN with the same rules as the spec.
  • CloudWatchTimeSeriesQuery:
    • metrics (namespace, metric name, dimensions, statistic, period) and metric math expressions referencing the other queries by ID, sent in a single GetMetricData request;
    • returnData: false hides the series that are only used by an expression;
    • the editor includes a metric discovery (ListMetrics) that adds the selected metric, and tells when the server truncated the list.
  • CloudWatchDimensionValuesVariable: the values of a dimension in a namespace (for example every InstanceId), optionally filtered by metric name and by other dimension values.

Behavior

  • Period: the period of a metric is a minimum, like the minimum step of a Prometheus query. It is increased:
    • to the step suggested by the panel;
    • so all the returned series fit in the 10000 datapoints returned by the server. For example, a single metric over 7 days is queried every 2 minutes instead of failing.
  • Variables: dashboard variables are replaced in the namespaces, metric names, dimension names and values, expressions and legends. dependsOn lists them.
  • Connection test: no healthCheckPath is defined. The CloudWatch proxy only accepts POST requests, so the generic connection test (a GET on the health check path through the proxy) doesn't apply, and the editor doesn't show the button. Supporting a connection test for non-HTTP proxies (SQL, CloudWatch) would be a separate change in perses/shared.
  • Responses: validated at runtime with small type guards, so no new dependency is added.
  • Go SDK: builders for the datasource (CloudWatch(region, RoleARN(...), ExternalIDSecret(...))), the query (Metric, Expression, Label, Hidden) and the variable.

Testing

All the checks below ran with a local build of perses/spec#109: linked into node_modules, served from a local CUE registry for the schemas, and through an uncommitted go.work for Go. Nothing local is committed.

  • npm run type-check, lint and build for the workspace. lint reports no new warning, only the existing setup-tests.ts one shared by every plugin.
  • npm run test: 40 tests covering:
    • the period calculation, including that the datapoint limit is never exceeded;
    • the client requests and response validation;
    • variable replacement and dependsOn;
    • query validation;
    • the dimension values;
    • the datasource validation.
  • oxfmt --check, cue fmt (CUE v0.17.1), mdox on the README, and the license check.
  • percli plugin lint passes, and percli plugin test-schemas passes all 10 schema tests (valid and invalid fixtures for the datasource, the query and the variable).
  • Go SDK: go vet, go test (it builds a dashboard with the three plugins and checks the JSON) and golangci-lint v2.13.2 (0 issues).
  • No live AWS account was used; the server side is covered by the tests of [FEATURE] add CloudWatch datasource proxy perses#4546.

Screenshots

The editors use the standard MUI fields of the other plugins. I'll add screenshots once the dependencies are released and the plugin can be loaded in a running Perses.

Checklist

  • Pull request has a descriptive title and context useful to a reviewer.
  • Pull request title follows the [<catalog_entry>] <commit message> naming convention.
  • All commits have DCO signoffs.

UI Changes

  • Changes that impact the UI include screenshots and/or screencasts of the relevant changes.
  • Code follows the UI guidelines.

Add a CloudWatch plugin querying Amazon CloudWatch metrics through the
CloudWatchProxy of the Perses server. The server signs the requests with
its own AWS identity, optionally assuming a role, so the browser never
receives AWS credentials and the datasource contains none.

- CloudWatchDatasource: the region, and optionally the role to assume
  and the secret holding its external ID. The schema reuses the
  CloudWatchDatasourceSpec of perses/spec.
- CloudWatchTimeSeriesQuery: metrics and metric math expressions sent in
  a single GetMetricData request, with metric discovery in the editor.
  The period of a metric is a minimum: it is increased to the step
  suggested by the panel, and so the series fit in the 10000 datapoints
  returned by the server (for example over 7 days).
- CloudWatchDimensionValuesVariable: the values of a dimension in a
  namespace, discovered with ListMetrics.
- Dashboard variables can be used in the namespaces, metric names,
  dimensions, expressions and legends.
- Go SDK for the datasource, the query and the variable.

The datasource doesn't define a health check path: the proxy only
accepts POST requests, so the generic connection test doesn't apply.

It requires the CloudWatch proxy model of perses/spec and the CloudWatch
proxy of perses/perses.

Co-authored-by: Jagath P <87551823+jagath25@users.noreply.github.com>
Signed-off-by: Chittibabu Terala <102535438+T-Chittibabu@users.noreply.github.com>
@AntoineThebaud

Copy link
Copy Markdown
Contributor

Please don't forget to document this new plugin by adding content to https://github.com/perses/plugins/tree/main/docs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants