Skip to content

test(config): Pin reload and overlay behaviour - #1421

Merged
gtema merged 1 commit into
mainfrom
claude/project-thread-bzh5uv
Oct 6, 2026
Merged

gtema merged 1 commit into
mainfrom
claude/project-thread-bzh5uv

Conversation

@gtema

@gtema gtema commented Oct 5, 2026 •

Copy link
Copy Markdown
Collaborator

Before: the config reload machinery (watch loop, last-known-good, overlay precedence) had tests for single behaviours, but nothing pinned the cases that the engine extraction in ADR 0039 (#1416) could silently break.

After: four new tests in crates/config pin that behaviour before any code moves (PR 1 of the implementation plan):

  • overlay precedence: OS_* env over the site-vars file over the main file
  • an invalid reload keeps the last-known-good config, sends no notification, and a later valid edit recovers and notifies
  • a burst of 50 file writes converges to the last write (no wedged watcher)
  • shutdown() on a watched manager is idempotent and stops applying edits

@gtema gtema self-assigned this Oct 5, 2026
@github-actions

github-actions Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

🐰 Bencher Report

Projectkeystone
Branchclaude/project-thread-bzh5uv
Testbedubuntu-latest
Click to view all benchmark results
BenchmarkLatencyBenchmark Result
nanoseconds (ns)
(Result Δ%)
Upper Boundary
nanoseconds (ns)
(Limit %)
Command_Serde/apply/remove📈 view plot
🚷 view threshold
125,500.00 ns
(-50.44%)Baseline: 253,203.02 ns
1,345,629.76 ns
(9.33%)
Command_Serde/apply/set📈 view plot
🚷 view threshold
133,110.00 ns
(-56.04%)Baseline: 302,773.14 ns
1,846,887.48 ns
(7.21%)
Command_Serde/pack/delete📈 view plot
🚷 view threshold
138.83 ns
(+17.75%)Baseline: 117.90 ns
167.56 ns
(82.86%)
Command_Serde/pack/delete_index📈 view plot
🚷 view threshold
119.22 ns
(+13.48%)Baseline: 105.06 ns
145.09 ns
(82.17%)
Command_Serde/pack/set📈 view plot
🚷 view threshold
217.72 ns
(+12.90%)Baseline: 192.85 ns
271.96 ns
(80.06%)
Command_Serde/pack/set_index📈 view plot
🚷 view threshold
119.59 ns
(+13.84%)Baseline: 105.05 ns
145.13 ns
(82.40%)
Command_Serde/unpack/delete📈 view plot
🚷 view threshold
169.32 ns
(+2.63%)Baseline: 164.98 ns
256.74 ns
(65.95%)
Command_Serde/unpack/delete_index📈 view plot
🚷 view threshold
151.39 ns
(+8.83%)Baseline: 139.11 ns
212.34 ns
(71.30%)
Command_Serde/unpack/set📈 view plot
🚷 view threshold
259.86 ns
(+4.34%)Baseline: 249.06 ns
378.24 ns
(68.70%)
Command_Serde/unpack/set_index📈 view plot
🚷 view threshold
143.08 ns
(+3.67%)Baseline: 138.01 ns
209.33 ns
(68.35%)
Payload_encryption/pack/remove_cmd📈 view plot
🚷 view threshold
129.67 ns
(+17.41%)Baseline: 110.44 ns
160.99 ns
(80.54%)
Payload_encryption/pack/set_cmd📈 view plot
🚷 view threshold
216.86 ns
(+8.51%)Baseline: 199.86 ns
297.64 ns
(72.86%)
Payload_encryption/unpack/remove_cmd📈 view plot
🚷 view threshold
166.78 ns
(-2.34%)Baseline: 170.77 ns
270.04 ns
(61.76%)
Payload_encryption/unpack/set_cmd📈 view plot
🚷 view threshold
280.46 ns
(+8.96%)Baseline: 257.39 ns
393.51 ns
(71.27%)
Raft_1Node_Latency/prefix/1node📈 view plot
🚷 view threshold
2,469,300.00 ns
(-20.71%)Baseline: 3,114,351.72 ns
7,516,383.84 ns
(32.85%)
Raft_1Node_Latency/read/1node📈 view plot
🚷 view threshold
42,390.00 ns
(+15.54%)Baseline: 36,688.75 ns
52,932.52 ns
(80.08%)
Raft_1Node_Latency/remove/1node📈 view plot
🚷 view threshold
353,010.00 ns
(-33.30%)Baseline: 529,257.81 ns
1,794,916.31 ns
(19.67%)
Raft_1Node_Latency/write/1node📈 view plot
🚷 view threshold
375,290.00 ns
(-35.84%)Baseline: 584,885.00 ns
2,646,038.42 ns
(14.18%)
build_snapshot/default📈 view plot
🚷 view threshold
2,167.30 ns
(-98.57%)Baseline: 151,380.75 ns
996,327.02 ns
(0.22%)
fernet token/project📈 view plot
🚷 view threshold
1,477.20 ns
(+9.08%)Baseline: 1,354.22 ns
1,842.92 ns
(80.16%)
get_data_keyspace📈 view plot
🚷 view threshold
0.32 ns
(+3.91%)Baseline: 0.30 ns
0.39 ns
(81.80%)
get_db📈 view plot
🚷 view threshold
0.31 ns
(+3.61%)Baseline: 0.30 ns
0.39 ns
(81.56%)
get_fernet_token_timestamp/project📈 view plot
🚷 view threshold
144.55 ns
(+7.04%)Baseline: 135.04 ns
180.33 ns
(80.16%)
get_keyspace📈 view plot
🚷 view threshold
4.21 ns
(-32.95%)Baseline: 6.27 ns
14.14 ns
(29.75%)
🐰 View full continuous benchmarking report in Bencher

@gtema
gtema marked this pull request as ready for review October 6, 2026 06:16
@github-actions

github-actions Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

🧪 Tempest Identity Compatibility Results (advisory, non-blocking)

rust

Totals
======
Ran: 96 tests in 17.5583 sec.
 - Passed: 34
 - Skipped: 3
 - Expected Fail: 0
 - Unexpected Success: 0
 - Failed: 59
Sum of execute time for each test: 8.0715 sec.
Failed test IDs
===== FAILED TESTS (target=rust) =====
tempest.api.identity.admin.v3.test_application_credentials.ApplicationCredentialsV3AdminTest.test_create_application_credential_with_roles[id-3b3dd48f-3388-406a-a9e6-4d078a552d0e]
tempest.api.identity.admin.v3.test_credentials.CredentialsTestJSON.test_credentials_create_get_update_delete[id-7cd59bf9-bda4-4c72-9467-d21cab278355,smoke]
tempest.api.identity.admin.v3.test_inherits.InheritsV3TestJSON.test_inherit_assign_check_revoke_roles_on_projects_group[id-26021436-d5a4-4256-943c-ded01e0d4b45]
tempest.api.identity.admin.v3.test_inherits.InheritsV3TestJSON.test_inherit_assign_check_revoke_roles_on_projects_user[id-18b70e45-7687-4b72-8277-b8f1a47d7591]
tempest.api.identity.admin.v3.test_inherits.InheritsV3TestJSON.test_inherit_assign_list_check_revoke_roles_on_domains_group[id-c7a8dda2-be50-4fb4-9a9c-e830771078b1]
tempest.api.identity.admin.v3.test_inherits.InheritsV3TestJSON.test_inherit_assign_list_check_revoke_roles_on_domains_user[id-4e6f0366-97c8-423c-b2be-41eae6ac91c8]
tempest.api.identity.admin.v3.test_inherits.InheritsV3TestJSON.test_inherit_assign_list_revoke_user_roles_on_domain[id-3acf666e-5354-42ac-8e17-8b68893bcd36]
tempest.api.identity.admin.v3.test_inherits.InheritsV3TestJSON.test_inherit_assign_list_revoke_user_roles_on_project_tree[id-9f02ccd9-9b57-46b4-8f77-dd5a736f3a06]
tempest.api.identity.admin.v3.test_domains.DomainsTestJSON.test_create_domain_without_description[id-2abf8764-309a-4fa9-bc58-201b799817ad]
tempest.api.identity.admin.v3.test_domains.DomainsTestJSON.test_create_update_delete_domain[id-f2f5b44a-82e8-4dad-8084-0661ea3b18cf,smoke]
tempest.api.identity.admin.v3.test_domains.DomainsTestJSON.test_domain_delete_cascades_content[id-d8d318b7-d1b3-4c37-94c5-3c5ba0b121ea]
tempest.api.identity.admin.v3.test_domains.DomainsTestJSON.test_list_domains_filter_by_enabled[id-3fd19840-65c1-43f8-b48c-51bdd066dff9]
tempest.api.identity.admin.v3.test_domain_configuration.DomainConfigurationTestJSON.test_create_domain_config_and_show_config_groups_and_options[id-9e3ff13c-f597-4f01-9377-d6c06c2a1477]
tempest.api.identity.admin.v3.test_domain_configuration.DomainConfigurationTestJSON.test_create_update_and_delete_domain_config_groups_and_opts[id-c7510fa2-6661-4170-9c6b-4783a80651e9]
tempest.api.identity.admin.v3.test_domain_configuration.DomainConfigurationTestJSON.test_show_default_group_config_and_options[id-11a02bf0-6f94-4380-b3b0-c8dc18fc0d22]
tempest.api.identity.admin.v3.test_oauth_consumers.OAUTHConsumersV3Test.test_create_and_show_consumer[id-c8307ea6-a86c-47fd-ae7b-5b3b2caca76d]
tempest.api.identity.admin.v3.test_oauth_consumers.OAUTHConsumersV3Test.test_delete_consumer[id-fdfa1b7f-2a31-4354-b2c7-f6ae20554f93]
tempest.api.identity.admin.v3.test_oauth_consumers.OAUTHConsumersV3Test.test_list_consumers[id-09ca50de-78f2-4ffb-ac71-f2254036b2b8]
tempest.api.identity.admin.v3.test_oauth_consumers.OAUTHConsumersV3Test.test_update_consumer[id-080a9b1a-c009-47c0-9979-5305bf72e3dc]
tempest.api.identity.admin.v3.test_default_project_id.TestDefaultProjectId.test_default_project_id[id-d6110661-6a71-49a7-a453-b5e26640ff6d]
tempest.api.identity.admin.v3.test_projects_negative.ProjectsNegativeStaticTestJSON.test_create_projects_name_length_over_64[id-502b6ceb-b0c8-4422-bf53-f08fdb21e2f0,negative]
tempest.api.identity.admin.v3.test_domains_negative.DomainsNegativeTestJSON.test_create_domain_with_name_length_over_64[id-37b1bbf2-d664-4785-9a11-333438586eae,negative]
tempest.api.identity.admin.v3.test_domains_negative.DomainsNegativeTestJSON.test_delete_active_domain[gate,id-1f3fbff5-4e44-400d-9ca1-d953f05f609b,negative]
tempest.api.identity.admin.v3.test_list_projects.ListProjectsTestJSON.test_list_projects_with_enabled[id-0fe7a334-675a-4509-b00e-1c4b95d5dae8]
tempest.api.identity.admin.v3.test_list_projects.ListProjectsTestJSON.test_list_projects_with_parent[id-6edc66f5-2941-4a17-9526-4073311c1fac]
setUpClass (tempest.api.identity.admin.v3.test_endpoint_groups.EndPointGroupsTest)
tempest.api.identity.admin.v3.test_groups.GroupsV3TestJSON.test_group_users_add_list_delete[id-1598521a-2f36-4606-8df9-30772bd51339,smoke]
tempest.api.identity.admin.v3.test_roles.RolesV3TestJSON.test_assignments_for_domain_roles[id-3859df7e-5b78-4e4d-b10e-214c8953842a]
tempest.api.identity.admin.v3.test_roles.RolesV3TestJSON.test_assignments_for_implied_roles_create_delete[id-c8828027-df48-4021-95df-b65b92c7429e]
tempest.api.identity.admin.v3.test_roles.RolesV3TestJSON.test_grant_list_revoke_role_to_group_on_domain[id-4bf8a70b-e785-413a-ad53-9f91ce02faa7]
tempest.api.identity.admin.v3.test_roles.RolesV3TestJSON.test_grant_list_revoke_role_to_group_on_project[id-cbf11737-1904-4690-9613-97bcbb3df1c4]
tempest.api.identity.admin.v3.test_roles.RolesV3TestJSON.test_grant_list_revoke_role_to_group_on_system[id-c888fe4f-8018-48db-b959-542225c1b4b6]
tempest.api.identity.admin.v3.test_roles.RolesV3TestJSON.test_grant_list_revoke_role_to_user_on_domain[id-6c9a2940-3625-43a3-ac02-5dcec62ef3bd]
tempest.api.identity.admin.v3.test_groups.GroupsV3TestJSON.test_list_user_groups[id-64573281-d26a-4a52-b899-503cb0f4e4ec]
tempest.api.identity.admin.v3.test_roles.RolesV3TestJSON.test_implied_domain_roles[id-eb1e1c24-1bc4-4d47-9748-e127a1852c82]
tempest.api.identity.admin.v3.test_roles.RolesV3TestJSON.test_implied_roles_create_check_show_delete[id-c90c316c-d706-4728-bcba-eb1912081b69]
tempest.api.identity.admin.v3.test_roles.RolesV3TestJSON.test_role_create_update_show_list[id-18afc6c0-46cf-4911-824e-9989cc056c3a,smoke]
tempest.api.identity.admin.v3.test_list_projects.ListProjectsStaticTestJSON.test_list_projects[id-1d830662-22ad-427c-8c3e-4ec854b0af44]
tempest.api.identity.admin.v3.test_project_tags.IdentityV3ProjectTagsTest.test_list_update_delete_project_tags[id-7c123aac-999d-416a-a0fb-84b915ab10de]
tempest.api.identity.admin.v3.test_list_users.UsersV3TestJSON.test_list_users_with_not_enabled[id-bff8bf2f-9408-4ef5-b63a-753c8c2124eb]
setUpClass (tempest.api.identity.admin.v3.test_projects.ProjectsTestJSON)
setUpClass (tempest.api.identity.admin.v3.test_regions.RegionsTestJSON)
setUpClass (tempest.api.identity.admin.v3.test_policies.PoliciesTestJSON)
setUpClass (tempest.api.identity.admin.v3.test_services.ServicesTestJSON)
setUpClass (tempest.api.identity.v3.test_ec2_credentials.EC2CredentialsTest)
tempest.api.identity.admin.v3.test_users.UsersV3TestJSON.test_get_user[id-c10dcd90-461d-4b16-8e23-4eb836c00644]
tempest.api.identity.admin.v3.test_users.UsersV3TestJSON.test_list_user_projects[id-a831e70c-e35b-430b-92ed-81ebbc5437b8]
tempest.api.identity.admin.v3.test_users.UsersV3TestJSON.test_update_user_password[id-2d223a0e-e457-4a70-9fb1-febe027a0ff9]
tempest.api.identity.admin.v3.test_users.UsersV3TestJSON.test_user_update[id-b537d090-afb9-4519-b95d-270b0708e87e]
setUpClass (tempest.api.identity.admin.v3.test_projects_negative.ProjectsNegativeTestJSON)
setUpClass (tempest.api.identity.admin.v3.test_users_negative.UsersNegativeTest)
setUpClass (tempest.api.identity.v3.test_application_credentials.ApplicationCredentialsV3Test)
setUpClass (tempest.api.identity.admin.v3.test_tokens.TokensV3TestJSON)
setUpClass (tempest.api.identity.v3.test_api_discovery.TestApiDiscovery)
setUpClass (tempest.api.identity.v3.test_catalog.IdentityCatalogTest)
setUpClass (tempest.api.identity.admin.v3.test_trusts.TrustsV3TestJSON)
setUpClass (tempest.api.identity.v3.test_projects.IdentityV3ProjectsTest)
setUpClass (tempest.api.identity.v3.test_domains.DefaultDomainTestJSON)
setUpClass (tempest.api.identity.v3.test_tokens.TokensV3Test)
===== END FAILED TESTS (target=rust) =====

View full run

@github-actions

github-actions Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

🦢 Load Test Results

Goose Attack Report

Plan Overview

Action Started Stopped Elapsed Users
Increasing 26-10-06 07:38:41 26-10-06 07:39:04 00:00:23 0 → 45
Maintaining 26-10-06 07:39:04 26-10-06 07:39:34 00:00:30 45
Decreasing 26-10-06 07:39:34 26-10-06 07:39:34 00:00:00 0 ← 45

Request Metrics

Method Name # Requests # Fails Average (ms) Min (ms) Max (ms) RPS Failures/s
DELETE DELETE /v3/auth/tokens 42 0 556.95 67 703 1.40 0.00
DELETE DELETE /v3/projects/:id (teardown) 1 0 152.00 152 152 0.03 0.00
DELETE DELETE /v3/roles/:id (teardown) 1 0 160.00 160 160 0.03 0.00
DELETE DELETE /v3/users/:id (teardown) 2 0 79.00 73 85 0.07 0.00
DELETE DELETE /v4/mappings/rulesets/:mapping_id (teardown) 1 0 157.00 157 157 0.03 0.00
DELETE DELETE /v4/oauth2/:domain_id/clients/:provider_id (teardown) 1 0 154.00 154 154 0.03 0.00
GET 569 0 616.69 345 868 18.97 0.00
GET GET /SCIM/v2/:domain_id/Users (sync reconcile list) 1840 1840 15.92 1 848 61.33 61.33
GET GET /v3/auth/tokens (validate new) 42 0 585.21 205 863 1.40 0.00
GET GET /v3/auth/tokens (validate revoked) 43 0 503.74 45 683 1.43 0.00
GET GET /v3/projects/:id 52 0 584.10 400 744 1.73 0.00
GET GET /v3/projects/:id (catalog) 79 0 568.20 358 760 2.63 0.00
GET GET /v3/role_assignments 40 0 598.38 370 785 1.33 0.00
GET GET /v3/role_assignments?role.id 41 0 568.17 352 848 1.37 0.00
GET GET /v3/role_assignments?scope.domain.id 41 0 560.15 352 704 1.37 0.00
GET GET /v3/role_assignments?user.id 40 0 535.40 334 779 1.33 0.00
GET GET /v3/roles 105 0 579.26 356 830 3.50 0.00
GET GET /v3/roles/:id 53 0 579.66 378 731 1.77 0.00
GET GET /v3/users/:id 102 0 602.84 356 787 3.40 0.00
GET GET /v3/users/:id (catalog) 100 0 607.49 402 770 3.33 0.00
GET GET /v4/api-keys 26 0 584.35 415 801 0.87 0.00
GET GET /v4/api-keys/:client_id 25 0 611.32 485 751 0.83 0.00
GET GET /v4/mappings/rulesets 27 0 580.52 384 769 0.90 0.00
GET GET /v4/mappings/rulesets/:mapping_id 26 0 565.62 451 736 0.87 0.00
GET GET /v4/oauth2/:domain_id/.well-known/openid-configuration 863 0 52.00 13 268 28.77 0.00
GET GET /v4/oauth2/:domain_id/clients 27 0 596.00 424 768 0.90 0.00
GET GET /v4/oauth2/:domain_id/clients/:provider_id 27 0 546.96 378 796 0.90 0.00
GET GET /v4/oauth2/:domain_id/jwks 865 0 51.52 11 257 28.83 0.00
GET GET /v4/scim_realms 17 0 593.71 419 779 0.57 0.00
GET GET /v4/scim_realms/:domain_id/:provider_id 17 0 544.71 392 689 0.57 0.00
POST POST /v3/auth/tokens 41 0 546.90 346 704 1.37 0.00
POST POST /v3/auth/tokens (password, invalid) 32 0 957.81 536 1627 1.07 0.00
POST POST /v3/auth/tokens (password, project-scoped) 57 0 1198.05 789 1714 1.90 0.00
POST POST /v3/auth/tokens (password, unscoped) 57 0 998.12 524 1383 1.90 0.00
POST POST /v3/auth/tokens (rescope to system) 94 0 652.49 431 891 3.13 0.00
POST POST /v3/auth/tokens (rescope, invalid project) 56 0 537.66 355 735 1.87 0.00
POST POST /v3/auth/tokens (system-scoped, bootstrap admin) 51 0 1230.96 775 1777 1.70 0.00
POST POST /v4/api-keys/:client_id/revoke (teardown) 1 0 172.00 172 172 0.03 0.00
PUT PUT /v4/scim_realms/:domain_id/:provider_id 17 0 649.00 487 762 0.57 0.00
Aggregated 5521 1840 249.13 1 1777 184.03 61.33

Response Time Metrics

Method Name 50%ile (ms) 60%ile (ms) 70%ile (ms) 80%ile (ms) 90%ile (ms) 95%ile (ms) 99%ile (ms) 100%ile (ms)
DELETE DELETE /v3/auth/tokens 600 600 600 600 700 700 700 700
DELETE DELETE /v3/projects/:id (teardown) 152 152 152 152 152 152 152 152
DELETE DELETE /v3/roles/:id (teardown) 160 160 160 160 160 160 160 160
DELETE DELETE /v3/users/:id (teardown) 73 73 73 85 85 85 85 85
DELETE DELETE /v4/mappings/rulesets/:mapping_id (teardown) 157 157 157 157 157 157 157 157
DELETE DELETE /v4/oauth2/:domain_id/clients/:provider_id (teardown) 154 154 154 154 154 154 154 154
GET 600 600 700 700 700 800 800 868
GET GET /SCIM/v2/:domain_id/Users (sync reconcile list) 7 8 10 12 16 22 410 800
GET GET /v3/auth/tokens (validate new) 600 600 600 700 700 700 863 863
GET GET /v3/auth/tokens (validate revoked) 500 500 600 600 683 683 683 683
GET GET /v3/projects/:id 600 600 600 700 700 700 700 700
GET GET /v3/projects/:id (catalog) 600 600 600 600 700 700 700 760
GET GET /v3/role_assignments 600 600 600 700 700 700 785 785
GET GET /v3/role_assignments?role.id 600 600 600 600 700 700 800 800
GET GET /v3/role_assignments?scope.domain.id 600 600 600 600 700 700 700 700
GET GET /v3/role_assignments?user.id 500 500 600 600 600 700 779 779
GET GET /v3/roles 600 600 700 700 700 700 800 800
GET GET /v3/roles/:id 600 600 700 700 700 700 700 700
GET GET /v3/users/:id 600 600 700 700 700 700 787 787
GET GET /v3/users/:id (catalog) 600 600 600 700 700 700 770 770
GET GET /v4/api-keys 600 600 600 700 700 700 800 800
GET GET /v4/api-keys/:client_id 600 600 700 700 700 700 751 751
GET GET /v4/mappings/rulesets 600 600 600 600 700 700 769 769
GET GET /v4/mappings/rulesets/:mapping_id 500 600 600 600 700 700 700 700
GET GET /v4/oauth2/:domain_id/.well-known/openid-configuration 45 51 62 70 86 100 140 268
GET GET /v4/oauth2/:domain_id/clients 600 600 600 700 700 700 768 768
GET GET /v4/oauth2/:domain_id/clients/:provider_id 600 600 600 600 600 700 796 796
GET GET /v4/oauth2/:domain_id/jwks 45 51 60 70 85 99 140 257
GET GET /v4/scim_realms 600 600 600 600 700 700 779 779
GET GET /v4/scim_realms/:domain_id/:provider_id 600 600 600 600 600 689 689 689
POST POST /v3/auth/tokens 500 600 600 600 600 700 700 700
POST POST /v3/auth/tokens (password, invalid) 900 1,000 1,000 1,000 1,000 1,000 1,627 1,627
POST POST /v3/auth/tokens (password, project-scoped) 1,000 1,000 1,000 1,000 1,000 1,714 1,714 1,714
POST POST /v3/auth/tokens (password, unscoped) 1,000 1,000 1,000 1,000 1,000 1,000 1,000 1,000
POST POST /v3/auth/tokens (rescope to system) 700 700 700 700 800 800 800 891
POST POST /v3/auth/tokens (rescope, invalid project) 500 600 600 600 600 700 700 700
POST POST /v3/auth/tokens (system-scoped, bootstrap admin) 1,000 1,000 1,000 1,000 1,777 1,777 1,777 1,777
POST POST /v4/api-keys/:client_id/revoke (teardown) 172 172 172 172 172 172 172 172
PUT PUT /v4/scim_realms/:domain_id/:provider_id 700 700 700 700 700 700 762 762
Aggregated 48 80 500 600 700 800 1,000 1,777

Status Code Metrics

Method Name Status Codes
DELETE DELETE /v3/auth/tokens 42 [204]
DELETE DELETE /v3/projects/:id (teardown) 1 [204]
DELETE DELETE /v3/roles/:id (teardown) 1 [204]
DELETE DELETE /v3/users/:id (teardown) 2 [204]
DELETE DELETE /v4/mappings/rulesets/:mapping_id (teardown) 1 [204]
DELETE DELETE /v4/oauth2/:domain_id/clients/:provider_id (teardown) 1 [204]
GET 569 [200]
GET GET /SCIM/v2/:domain_id/Users (sync reconcile list) 1,809 [429], 31 [401]
GET GET /v3/auth/tokens (validate new) 42 [200]
GET GET /v3/auth/tokens (validate revoked) 43 [404]
GET GET /v3/projects/:id 52 [200]
GET GET /v3/projects/:id (catalog) 79 [200]
GET GET /v3/role_assignments 40 [200]
GET GET /v3/role_assignments?role.id 41 [200]
GET GET /v3/role_assignments?scope.domain.id 41 [200]
GET GET /v3/role_assignments?user.id 40 [200]
GET GET /v3/roles 105 [200]
GET GET /v3/roles/:id 53 [200]
GET GET /v3/users/:id 102 [200]
GET GET /v3/users/:id (catalog) 100 [200]
GET GET /v4/api-keys 26 [200]
GET GET /v4/api-keys/:client_id 25 [200]
GET GET /v4/mappings/rulesets 27 [200]
GET GET /v4/mappings/rulesets/:mapping_id 26 [200]
GET GET /v4/oauth2/:domain_id/.well-known/openid-configuration 863 [200]
GET GET /v4/oauth2/:domain_id/clients 27 [200]
GET GET /v4/oauth2/:domain_id/clients/:provider_id 27 [200]
GET GET /v4/oauth2/:domain_id/jwks 865 [200]
GET GET /v4/scim_realms 17 [200]
GET GET /v4/scim_realms/:domain_id/:provider_id 17 [200]
POST POST /v3/auth/tokens 41 [201]
POST POST /v3/auth/tokens (password, invalid) 32 [401]
POST POST /v3/auth/tokens (password, project-scoped) 57 [201]
POST POST /v3/auth/tokens (password, unscoped) 57 [201]
POST POST /v3/auth/tokens (rescope to system) 94 [201]
POST POST /v3/auth/tokens (rescope, invalid project) 56 [401]
POST POST /v3/auth/tokens (system-scoped, bootstrap admin) 51 [201]
POST POST /v4/api-keys/:client_id/revoke (teardown) 1 [200]
PUT PUT /v4/scim_realms/:domain_id/:provider_id 17 [200]
Aggregated 300 [201], 119 [401], 48 [204], 3,202 [200], 1,809 [429], 43 [404]

Transaction Metrics

Transaction # Times Run # Fails Average (ms) Min (ms) Max (ms) RPS Failures/s
ReadHeavy
0.0 0 0 0.00 0 0 0.00 0.00
0.1 102 0 620.81 398 805 3.40 0.00
0.2 101 0 587.69 367 866 3.37 0.00
0.3 102 0 588.40 373 765 3.40 0.00
TokenLifecycle
1.0 0 0 0.00 0 0 0.00 0.00
1.1 43 0 2195.53 832 2601 1.43 0.00
ValidateToken
2.0 0 0 0.00 0 0 0.00 0.00
2.1 88 0 695.38 409 868 2.93 0.00
UserCRUD
3.0 0 0 0.00 0 0 0.00 0.00
3.1 0 0 0.00 0 0 0.00 0.00
3.2 102 0 602.86 356 787 3.40 0.00
3.3 2 0 79.00 73 85 0.07 0.00
ProjectCRUD
4.0 0 0 0.00 0 0 0.00 0.00
4.1 0 0 0.00 0 0 0.00 0.00
4.2 52 0 584.10 400 744 1.73 0.00
4.3 1 0 152.00 152 152 0.03 0.00
RoleCRUD
5.0 0 0 0.00 0 0 0.00 0.00
5.1 0 0 0.00 0 0 0.00 0.00
5.2 53 0 579.68 378 731 1.77 0.00
5.3 1 0 160.00 160 160 0.03 0.00
RoleList
6.0 0 0 0.00 0 0 0.00 0.00
6.1 105 0 579.29 356 830 3.50 0.00
RoleAssignmentList
7.0 0 0 0.00 0 0 0.00 0.00
7.1 40 0 598.42 370 786 1.33 0.00
7.2 40 0 535.47 334 779 1.33 0.00
7.3 41 0 560.24 352 704 1.37 0.00
7.4 41 0 568.20 352 848 1.37 0.00
UserRead
8.0 0 0 0.00 0 0 0.00 0.00
8.1 98 0 623.84 375 850 3.27 0.00
8.2 100 0 607.52 402 770 3.33 0.00
ProjectRead
9.0 0 0 0.00 0 0 0.00 0.00
9.1 78 0 588.35 345 770 2.60 0.00
9.2 79 0 568.23 358 760 2.63 0.00
PasswordAuth
10.0 57 0 998.21 524 1383 1.90 0.00
10.1 57 0 1198.12 789 1714 1.90 0.00
PasswordAuthNegative
11.0 32 0 957.84 536 1627 1.07 0.00
Rescope
12.0 0 0 0.00 0 0 0.00 0.00
12.1 94 0 652.50 431 891 3.13 0.00
RescopeNegative
13.0 0 0 0.00 0 0 0.00 0.00
13.1 56 0 537.68 355 735 1.87 0.00
SystemScopeAuth
14.0 51 0 1231.02 776 1777 1.70 0.00
ApiKeyCRUD
15.0 0 0 0.00 0 0 0.00 0.00
15.1 0 0 0.00 0 0 0.00 0.00
15.2 25 0 611.36 485 751 0.83 0.00
15.3 26 0 584.35 415 801 0.87 0.00
15.4 1 0 172.00 172 172 0.03 0.00
MappingCRUD
16.0 0 0 0.00 0 0 0.00 0.00
16.1 0 0 0.00 0 0 0.00 0.00
16.2 26 0 565.69 451 736 0.87 0.00
16.3 27 0 580.52 384 769 0.90 0.00
16.4 1 0 157.00 157 157 0.03 0.00
OAuth2ClientCRUD
17.0 0 0 0.00 0 0 0.00 0.00
17.1 0 0 0.00 0 0 0.00 0.00
17.2 27 0 547.00 378 796 0.90 0.00
17.3 27 0 596.04 424 768 0.90 0.00
17.4 1 0 154.00 154 154 0.03 0.00
OAuth2Discovery
18.0 865 0 51.55 11 257 28.83 0.00
18.1 863 0 52.03 13 268 28.77 0.00
ScimRealmRead
19.0 0 0 0.00 0 0 0.00 0.00
19.1 17 0 544.82 392 689 0.57 0.00
19.2 17 0 593.71 419 779 0.57 0.00
19.3 17 0 649.06 487 762 0.57 0.00
ScimSync
20.0 0 0 0.00 0 0 0.00 0.00
20.1 0 0 0.00 0 0 0.00 0.00
20.2 0 0 0.00 0 0 0.00 0.00
20.3 1840 0 15.95 1 848 61.33 0.00
20.4 1839 0 0.00 0 0 61.30 0.00
20.5 1839 0 0.00 0 0 61.30 0.00
20.6 1 0 0.00 0 0 0.03 0.00
Aggregated 9075 0 151.56 1 2601 302.50 0.00

Scenario Metrics

Transaction # Users # Times Run Average (ms) Min (ms) Max (ms) Scenarios/s Iterations
ReadHeavy 6 99 1809.70 1246 2205 3.30 16.50
TokenLifecycle 3 40 2267.43 1739 2601 1.33 13.33
ValidateToken 2 86 697.77 409 868 2.87 43.00
UserCRUD 2 100 606.31 356 787 3.33 50.00
ProjectCRUD 1 51 586.37 400 744 1.70 51.00
RoleCRUD 1 52 582.21 378 731 1.73 52.00
RoleList 2 103 581.53 356 830 3.43 51.50
RoleAssignmentList 3 40 2289.95 1820 2706 1.33 13.33
UserRead 4 96 1238.99 868 1477 3.20 24.00
ProjectRead 3 76 1163.29 745 1510 2.53 25.33
PasswordAuth 4 54 2210.98 1448 2855 1.80 13.50
PasswordAuthNegative 1 31 961.19 536 1627 1.03 31.00
Rescope 2 92 655.92 431 891 3.07 46.00
RescopeNegative 1 55 538.87 355 735 1.83 55.00
SystemScopeAuth 2 49 1237.49 776 1777 1.63 24.50
ApiKeyCRUD 1 25 1208.56 912 1363 0.83 25.00
MappingCRUD 1 26 1151.08 908 1386 0.87 26.00
OAuth2ClientCRUD 1 26 1153.46 836 1466 0.87 26.00
OAuth2Discovery 3 862 104.18 30 316 28.73 287.33
ScimRealmRead 1 16 1807.06 1386 2024 0.53 16.00
ScimSync 1 1839 15.81 1 848 61.30 1839.00
Aggregated 45 3818 352.87 1 2855 127.27 2729.33

Error Metrics

Method Name # Error
GET GET /SCIM/v2/:domain_id/Users (sync reconcile list) 49 401 Unauthorized: GET /SCIM/v2/:domain_id/Users (sync reconcile list)
POST POST /SCIM/v2/:domain_id/Users (sync provision) 1 401 Unauthorized: POST /SCIM/v2/:domain_id/Users (sync provision)
POST POST /v3/auth/tokens (password, invalid) 74 401 Unauthorized: POST /v3/auth/tokens (password, invalid)
POST POST /v3/auth/tokens (rescope, invalid project) 142 401 Unauthorized: POST /v3/auth/tokens (rescope, invalid project)
GET GET /v3/auth/tokens (validate revoked) 146 404 Not Found: GET /v3/auth/tokens (validate revoked)
GET GET /SCIM/v2/:domain_id/Users (sync reconcile list) 1887 429 Too Many Requests: GET /SCIM/v2/:domain_id/Users (sync reconcile list)

View full report

Add characterization tests ahead of extracting the config engine
(ADR 0039, #1416): overlay precedence (env, site-vars, file),
last-known-good retention with no notification on an invalid reload,
convergence after an event burst, and idempotent shutdown.

Signed-off-by: Artem Goncharov <artem.goncharov@gmail.com>
@gtema
gtema force-pushed the claude/project-thread-bzh5uv branch from 03e10af to 0e02b3f Compare October 6, 2026 07:11
@gtema
gtema added this pull request to the merge queue Oct 6, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 6, 2026
@gtema
gtema added this pull request to the merge queue Oct 6, 2026
Merged via the queue into main with commit bae6b20 Oct 6, 2026
36 checks passed
@gtema
gtema deleted the claude/project-thread-bzh5uv branch October 6, 2026 09:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant