Skip to content

test(oauth2): Add federated sign-in tests - #1523

Merged
gtema merged 2 commits into
mainfrom
oauth2
Oct 15, 2026
Merged

gtema merged 2 commits into
mainfrom
oauth2

Conversation

@gtema

@gtema gtema commented Oct 14, 2026

Copy link
Copy Markdown
Collaborator

The OP login page offers the enabled identity providers of its domain,
but no test drove that path end to end. Add a Keycloak-backed test
binary to tests/federation, run by the skaffold verify step next to
the existing Keycloak tests.

Browser tests (Selenium Firefox, real Keycloak):

  • authorize: login page IdP button, Keycloak sign-in, consent,
    authorization code, token. Checks nonce, the federated amr and
    that sub is the user the mapping created in the OP domain.
  • authorize with an unmapped upstream user: the login page shows the
    upstream failure, no redirect to the RP, no user is provisioned.
  • device flow: same round trip through /device and the device
    federated callback.

Protocol tests against the callback (no browser):

  • wrong state is rejected with 400 and no redirect.
  • upstream error=access_denied shows the failure, and the spent
    state cannot be answered again.
  • a callback from a browser that did not start the flow is rejected.

Each test creates its own domain, IdP, mapping and OP client. The RP
redirect URI is https://rp.invalid/cb because public clients may only
use https or http://localhost, which a remote browser cannot serve; the
test reads the target from Firefox's error page.

Assisted-by: Claude noreply@anthropic.com
Signed-off-by: Artem Goncharov artem.goncharov@gmail.com

Comment thread tests/federation/src/oauth2_federated.rs Dismissed
@gtema
gtema force-pushed the oauth2 branch 2 times, most recently from 105234d to 3be8795 Compare October 15, 2026 06:49
@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

🦢 Load Test Results

Goose Attack Report

Plan Overview

Action Started Stopped Elapsed Users
Increasing 26-10-15 11:31:10 26-10-15 11:31:32 00:00:22 0 → 45
Maintaining 26-10-15 11:31:32 26-10-15 11:32:02 00:00:30 45
Decreasing 26-10-15 11:32:02 26-10-15 11:32:28 00:00:26 0 ← 45

Request Metrics

Method Name # Requests # Fails Average (ms) Min (ms) Max (ms) RPS Failures/s
DELETE DELETE /v3/auth/tokens 30 0 528.67 11 682 1.00 0.00
DELETE DELETE /v3/projects/:id (teardown) 1 0 100.00 100 100 0.03 0.00
DELETE DELETE /v3/roles/:id (teardown) 1 0 181.00 181 181 0.03 0.00
DELETE DELETE /v3/users/:id (teardown) 2 0 97.00 90 104 0.07 0.00
DELETE DELETE /v4/mappings/rulesets/:mapping_id (cc teardown) 1 0 198.00 198 198 0.03 0.00
DELETE DELETE /v4/mappings/rulesets/:mapping_id (teardown) 1 0 233.00 233 233 0.03 0.00
DELETE DELETE /v4/oauth2/:domain_id/clients/:provider_id (cc teardown) 1 0 30.00 30 30 0.03 0.00
DELETE DELETE /v4/oauth2/:domain_id/clients/:provider_id (teardown) 1 0 116.00 116 116 0.03 0.00
GET 486 0 601.66 394 880 16.20 0.00
GET GET /SCIM/v2/:domain_id/Users (sync reconcile list) 47 31 313.74 3 1061 1.57 1.03
GET GET /v3/auth/tokens (validate new) 30 1 2532.70 376 60001 1.00 0.03
GET GET /v3/auth/tokens (validate revoked) 31 0 475.23 7 697 1.03 0.00
GET GET /v3/projects/:id 55 0 557.85 349 734 1.83 0.00
GET GET /v3/projects/:id (catalog) 81 0 562.98 347 785 2.70 0.00
GET GET /v3/role_assignments 42 0 532.62 352 724 1.40 0.00
GET GET /v3/role_assignments?role.id 42 0 551.88 415 733 1.40 0.00
GET GET /v3/role_assignments?scope.domain.id 44 0 532.98 355 681 1.47 0.00
GET GET /v3/role_assignments?user.id 42 0 528.57 367 682 1.40 0.00
GET GET /v3/roles 109 0 556.26 406 732 3.63 0.00
GET GET /v3/roles/:id 55 0 555.13 396 729 1.83 0.00
GET GET /v3/users/:id 107 0 575.65 333 759 3.57 0.00
GET GET /v3/users/:id (catalog) 102 0 586.07 422 828 3.40 0.00
GET GET /v4/api-keys 29 0 518.93 365 661 0.97 0.00
GET GET /v4/api-keys/:client_id 28 0 553.21 432 750 0.93 0.00
GET GET /v4/mappings/rulesets 28 0 545.71 403 715 0.93 0.00
GET GET /v4/mappings/rulesets/:mapping_id 29 0 517.66 380 649 0.97 0.00
GET GET /v4/oauth2/:domain_id/.well-known/openid-configuration 1493 0 29.16 5 108 49.77 0.00
GET GET /v4/oauth2/:domain_id/clients 29 0 522.45 427 669 0.97 0.00
GET GET /v4/oauth2/:domain_id/clients/:provider_id 30 0 512.33 321 638 1.00 0.00
GET GET /v4/oauth2/:domain_id/jwks 1492 0 30.35 5 150 49.73 0.00
GET GET /v4/scim_realms 19 0 514.47 413 657 0.63 0.00
GET GET /v4/scim_realms/:domain_id/:provider_id 19 0 513.37 331 653 0.63 0.00
POST POST /v3/auth/tokens 28 0 528.29 383 678 0.93 0.00
POST POST /v3/auth/tokens (password, invalid) 36 0 875.89 543 1495 1.20 0.00
POST POST /v3/auth/tokens (password, project-scoped) 56 0 1266.04 640 1914 1.87 0.00
POST POST /v3/auth/tokens (password, unscoped) 55 0 959.45 610 1346 1.83 0.00
POST POST /v3/auth/tokens (rescope to system) 98 0 624.99 432 829 3.27 0.00
POST POST /v3/auth/tokens (rescope, invalid project) 59 0 517.00 387 692 1.97 0.00
POST POST /v3/auth/tokens (system-scoped, bootstrap admin) 52 0 1193.33 548 1723 1.73 0.00
POST POST /v4/api-keys/:client_id/revoke (teardown) 1 0 266.00 266 266 0.03 0.00
POST POST /v4/oauth2/:domain_id/device_authorization 437 0 68.50 15 183 14.57 0.00
POST POST /v4/oauth2/:domain_id/token (client_credentials) 14 0 745.71 406 1231 0.47 0.00
PUT PUT /v4/scim_realms/:domain_id/:provider_id 19 0 572.58 459 709 0.63 0.00
Aggregated 5362 32 255.47 3 60001 178.73 1.07

Response Time Metrics

Method Name 50%ile (ms) 60%ile (ms) 70%ile (ms) 80%ile (ms) 90%ile (ms) 95%ile (ms) 99%ile (ms) 100%ile (ms)
DELETE DELETE /v3/auth/tokens 500 600 600 600 600 682 682 682
DELETE DELETE /v3/projects/:id (teardown) 100 100 100 100 100 100 100 100
DELETE DELETE /v3/roles/:id (teardown) 181 181 181 181 181 181 181 181
DELETE DELETE /v3/users/:id (teardown) 90 90 90 100 100 100 100 100
DELETE DELETE /v4/mappings/rulesets/:mapping_id (cc teardown) 198 198 198 198 198 198 198 198
DELETE DELETE /v4/mappings/rulesets/:mapping_id (teardown) 233 233 233 233 233 233 233 233
DELETE DELETE /v4/oauth2/:domain_id/clients/:provider_id (cc teardown) 30 30 30 30 30 30 30 30
DELETE DELETE /v4/oauth2/:domain_id/clients/:provider_id (teardown) 116 116 116 116 116 116 116 116
GET 600 600 600 700 700 800 800 880
GET GET /SCIM/v2/:domain_id/Users (sync reconcile list) 300 360 450 500 700 800 1,000 1,000
GET GET /v3/auth/tokens (validate new) 600 600 600 600 600 700 60,000 60,000
GET GET /v3/auth/tokens (validate revoked) 490 500 500 600 600 600 697 697
GET GET /v3/projects/:id 500 500 600 600 700 700 700 700
GET GET /v3/projects/:id (catalog) 600 600 600 600 700 700 700 785
GET GET /v3/role_assignments 500 500 600 600 600 600 700 700
GET GET /v3/role_assignments?role.id 500 600 600 600 700 700 700 700
GET GET /v3/role_assignments?scope.domain.id 500 500 600 600 600 681 681 681
GET GET /v3/role_assignments?user.id 500 500 600 600 600 682 682 682
GET GET /v3/roles 600 600 600 600 700 700 700 700
GET GET /v3/roles/:id 500 600 600 600 700 700 700 700
GET GET /v3/users/:id 600 600 600 600 700 700 700 759
GET GET /v3/users/:id (catalog) 600 600 600 600 700 700 800 800
GET GET /v4/api-keys 500 600 600 600 600 600 661 661
GET GET /v4/api-keys/:client_id 500 600 600 600 600 700 750 750
GET GET /v4/mappings/rulesets 500 500 600 600 600 700 700 700
GET GET /v4/mappings/rulesets/:mapping_id 500 500 500 600 600 600 600 600
GET GET /v4/oauth2/:domain_id/.well-known/openid-configuration 25 29 34 39 50 60 83 108
GET GET /v4/oauth2/:domain_id/clients 500 500 500 600 600 600 669 669
GET GET /v4/oauth2/:domain_id/clients/:provider_id 500 500 600 600 600 600 600 600
GET GET /v4/oauth2/:domain_id/jwks 26 30 35 41 54 64 88 150
GET GET /v4/scim_realms 500 500 500 600 600 600 657 657
GET GET /v4/scim_realms/:domain_id/:provider_id 500 500 500 600 600 600 653 653
POST POST /v3/auth/tokens 500 500 600 600 600 678 678 678
POST POST /v3/auth/tokens (password, invalid) 800 900 900 1,000 1,000 1,000 1,000 1,000
POST POST /v3/auth/tokens (password, project-scoped) 1,000 1,000 1,000 1,000 1,914 1,914 1,914 1,914
POST POST /v3/auth/tokens (password, unscoped) 900 1,000 1,000 1,000 1,000 1,000 1,000 1,000
POST POST /v3/auth/tokens (rescope to system) 600 600 700 700 700 800 800 800
POST POST /v3/auth/tokens (rescope, invalid project) 500 500 600 600 600 600 692 692
POST POST /v3/auth/tokens (system-scoped, bootstrap admin) 1,000 1,000 1,000 1,000 1,723 1,723 1,723 1,723
POST POST /v4/api-keys/:client_id/revoke (teardown) 266 266 266 266 266 266 266 266
POST POST /v4/oauth2/:domain_id/device_authorization 64 70 79 90 110 120 150 180
POST POST /v4/oauth2/:domain_id/token (client_credentials) 600 700 900 1,000 1,000 1,000 1,000 1,000
PUT PUT /v4/scim_realms/:domain_id/:provider_id 600 600 600 600 700 700 700 700
Aggregated 46 75 480 600 600 700 1,000 60,000

Status Code Metrics

Method Name Status Codes
DELETE DELETE /v3/auth/tokens 30 [204]
DELETE DELETE /v3/projects/:id (teardown) 1 [204]
DELETE DELETE /v3/roles/:id (teardown) 1 [204]
DELETE DELETE /v3/users/:id (teardown) 2 [204]
DELETE DELETE /v4/mappings/rulesets/:mapping_id (cc teardown) 1 [204]
DELETE DELETE /v4/mappings/rulesets/:mapping_id (teardown) 1 [204]
DELETE DELETE /v4/oauth2/:domain_id/clients/:provider_id (cc teardown) 1 [204]
DELETE DELETE /v4/oauth2/:domain_id/clients/:provider_id (teardown) 1 [204]
GET 486 [200]
GET GET /SCIM/v2/:domain_id/Users (sync reconcile list) 16 [429], 31 [401]
GET GET /v3/auth/tokens (validate new) 29 [200], 1 [0]
GET GET /v3/auth/tokens (validate revoked) 31 [404]
GET GET /v3/projects/:id 55 [200]
GET GET /v3/projects/:id (catalog) 81 [200]
GET GET /v3/role_assignments 42 [200]
GET GET /v3/role_assignments?role.id 42 [200]
GET GET /v3/role_assignments?scope.domain.id 44 [200]
GET GET /v3/role_assignments?user.id 42 [200]
GET GET /v3/roles 109 [200]
GET GET /v3/roles/:id 55 [200]
GET GET /v3/users/:id 107 [200]
GET GET /v3/users/:id (catalog) 102 [200]
GET GET /v4/api-keys 29 [200]
GET GET /v4/api-keys/:client_id 28 [200]
GET GET /v4/mappings/rulesets 28 [200]
GET GET /v4/mappings/rulesets/:mapping_id 29 [200]
GET GET /v4/oauth2/:domain_id/.well-known/openid-configuration 1,493 [200]
GET GET /v4/oauth2/:domain_id/clients 29 [200]
GET GET /v4/oauth2/:domain_id/clients/:provider_id 30 [200]
GET GET /v4/oauth2/:domain_id/jwks 1,492 [200]
GET GET /v4/scim_realms 19 [200]
GET GET /v4/scim_realms/:domain_id/:provider_id 19 [200]
POST POST /v3/auth/tokens 28 [201]
POST POST /v3/auth/tokens (password, invalid) 36 [401]
POST POST /v3/auth/tokens (password, project-scoped) 56 [201]
POST POST /v3/auth/tokens (password, unscoped) 55 [201]
POST POST /v3/auth/tokens (rescope to system) 98 [201]
POST POST /v3/auth/tokens (rescope, invalid project) 59 [401]
POST POST /v3/auth/tokens (system-scoped, bootstrap admin) 52 [201]
POST POST /v4/api-keys/:client_id/revoke (teardown) 1 [200]
POST POST /v4/oauth2/:domain_id/device_authorization 437 [200]
POST POST /v4/oauth2/:domain_id/token (client_credentials) 14 [200]
PUT PUT /v4/scim_realms/:domain_id/:provider_id 19 [200]
Aggregated 1 [0], 126 [401], 31 [404], 289 [201], 16 [429], 4,861 [200], 38 [204]

Transaction Metrics

Transaction # Times Run # Fails Average (ms) Min (ms) Max (ms) RPS Failures/s
ReadHeavy
0.0 0 0 0.00 0 0 0.00 0.00
0.1 70 0 618.79 394 791 2.33 0.00
0.2 69 0 576.33 397 788 2.30 0.00
0.3 70 0 567.11 402 762 2.33 0.00
TokenLifecycle
1.0 0 0 0.00 0 0 0.00 0.00
1.1 31 0 4005.90 1509 60564 1.03 0.00
ValidateToken
2.0 0 0 0.00 0 0 0.00 0.00
2.1 93 0 660.43 412 880 3.10 0.00
UserCRUD
3.0 0 0 0.00 0 0 0.00 0.00
3.1 0 0 0.00 0 0 0.00 0.00
3.2 107 0 575.69 333 759 3.57 0.00
3.3 2 0 97.00 90 104 0.07 0.00
ProjectCRUD
4.0 0 0 0.00 0 0 0.00 0.00
4.1 0 0 0.00 0 0 0.00 0.00
4.2 55 0 557.87 349 734 1.83 0.00
4.3 1 0 100.00 100 100 0.03 0.00
RoleCRUD
5.0 0 0 0.00 0 0 0.00 0.00
5.1 0 0 0.00 0 0 0.00 0.00
5.2 55 0 555.18 396 729 1.83 0.00
5.3 1 0 181.00 181 181 0.03 0.00
RoleList
6.0 0 0 0.00 0 0 0.00 0.00
6.1 109 0 556.29 406 732 3.63 0.00
RoleAssignmentList
7.0 0 0 0.00 0 0 0.00 0.00
7.1 42 0 532.64 352 724 1.40 0.00
7.2 42 0 528.60 367 682 1.40 0.00
7.3 44 0 532.98 355 681 1.47 0.00
7.4 42 0 551.88 415 733 1.40 0.00
UserRead
8.0 0 0 0.00 0 0 0.00 0.00
8.1 102 0 612.59 421 846 3.40 0.00
8.2 102 0 586.11 422 829 3.40 0.00
ProjectRead
9.0 0 0 0.00 0 0 0.00 0.00
9.1 82 0 557.80 419 729 2.73 0.00
9.2 81 0 563.02 347 785 2.70 0.00
PasswordAuth
10.0 55 0 959.45 610 1346 1.83 0.00
10.1 56 0 1266.09 640 1914 1.87 0.00
PasswordAuthNegative
11.0 36 0 875.89 543 1495 1.20 0.00
Rescope
12.0 0 0 0.00 0 0 0.00 0.00
12.1 98 0 625.02 432 829 3.27 0.00
RescopeNegative
13.0 0 0 0.00 0 0 0.00 0.00
13.1 59 0 517.05 387 692 1.97 0.00
SystemScopeAuth
14.0 52 0 1193.37 548 1723 1.73 0.00
ApiKeyCRUD
15.0 0 0 0.00 0 0 0.00 0.00
15.1 0 0 0.00 0 0 0.00 0.00
15.2 28 0 553.29 432 750 0.93 0.00
15.3 29 0 518.97 365 661 0.97 0.00
15.4 1 0 266.00 266 266 0.03 0.00
MappingCRUD
16.0 0 0 0.00 0 0 0.00 0.00
16.1 0 0 0.00 0 0 0.00 0.00
16.2 29 0 517.69 380 649 0.97 0.00
16.3 28 0 545.71 403 715 0.93 0.00
16.4 1 0 233.00 233 233 0.03 0.00
OAuth2ClientCRUD
17.0 0 0 0.00 0 0 0.00 0.00
17.1 0 0 0.00 0 0 0.00 0.00
17.2 30 0 512.33 321 638 1.00 0.00
17.3 29 0 522.45 427 669 0.97 0.00
17.4 1 0 116.00 116 116 0.03 0.00
OAuth2ClientCredentials
18.0 0 0 0.00 0 0 0.00 0.00
18.1 0 0 0.00 0 0 0.00 0.00
18.2 14 0 745.79 406 1231 0.47 0.00
18.3 1 0 228.00 228 228 0.03 0.00
OAuth2DeviceAuthorization
19.0 437 0 68.54 15 183 14.57 0.00
OAuth2Discovery
20.0 1492 0 30.38 5 150 49.73 0.00
20.1 1493 0 29.18 5 108 49.77 0.00
ScimRealmRead
21.0 0 0 0.00 0 0 0.00 0.00
21.1 19 0 513.37 331 653 0.63 0.00
21.2 19 0 514.47 413 657 0.63 0.00
21.3 19 0 572.58 459 709 0.63 0.00
ScimSync
22.0 0 0 0.00 0 0 0.00 0.00
22.1 0 0 0.00 0 0 0.00 0.00
22.2 0 0 0.00 0 0 0.00 0.00
22.3 48 0 662.02 199 1061 1.60 0.00
22.4 47 0 0.00 0 0 1.57 0.00
22.5 47 0 0.00 0 0 1.57 0.00
22.6 1 0 0.00 0 0 0.03 0.00
Aggregated 5369 0 255.14 5 60564 178.97 0.00

Scenario Metrics

Transaction # Users # Times Run Average (ms) Min (ms) Max (ms) Scenarios/s Iterations
ReadHeavy 4 69 1772.10 1376 2019 2.30 17.25
TokenLifecycle 2 28 2149.64 1703 2448 0.93 14.00
ValidateToken 2 91 663.23 412 880 3.03 45.50
UserCRUD 2 105 580.03 366 759 3.50 52.50
ProjectCRUD 1 54 561.74 380 734 1.80 54.00
RoleCRUD 1 54 556.87 396 729 1.80 54.00
RoleList 2 107 557.74 406 732 3.57 53.50
RoleAssignmentList 3 41 2152.78 1785 2515 1.37 13.67
UserRead 4 100 1202.90 878 1451 3.33 25.00
ProjectRead 3 79 1124.58 875 1337 2.63 26.33
PasswordAuth 4 53 2258.53 1601 3050 1.77 13.25
PasswordAuthNegative 1 35 884.29 543 1495 1.17 35.00
Rescope 2 96 627.10 432 829 3.20 48.00
RescopeNegative 1 58 518.41 387 692 1.93 58.00
SystemScopeAuth 2 50 1208.60 758 1723 1.67 25.00
ApiKeyCRUD 1 28 1068.14 875 1316 0.93 28.00
MappingCRUD 1 28 1065.89 827 1363 0.93 28.00
OAuth2ClientCRUD 1 29 1041.83 806 1203 0.97 29.00
OAuth2ClientCredentials 1 13 2175.54 1620 2921 0.43 13.00
OAuth2DeviceAuthorization 1 436 68.58 15 183 14.53 436.00
OAuth2Discovery 3 1492 60.05 15 188 49.73 497.33
ScimRealmRead 1 19 1605.63 1272 1793 0.63 19.00
ScimSync 1 47 654.09 199 1061 1.57 47.00
Aggregated 44 3112 424.66 15 3050 103.73 1632.33

Error Metrics

Method Name # Error
GET GET /SCIM/v2/:domain_id/Users (sync reconcile list) 49 401 Unauthorized: GET /SCIM/v2/:domain_id/Users (sync reconcile list)
POST POST /SCIM/v2/:domain_id/Users (sync provision) 1 401 Unauthorized: POST /SCIM/v2/:domain_id/Users (sync provision)
POST POST /v3/auth/tokens (password, invalid) 82 401 Unauthorized: POST /v3/auth/tokens (password, invalid)
POST POST /v3/auth/tokens (rescope, invalid project) 150 401 Unauthorized: POST /v3/auth/tokens (rescope, invalid project)
GET GET /v3/auth/tokens (validate revoked) 128 404 Not Found: GET /v3/auth/tokens (validate revoked)
GET GET /SCIM/v2/:domain_id/Users (sync reconcile list) 40 429 Too Many Requests: GET /SCIM/v2/:domain_id/Users (sync reconcile list)
GET GET /v3/auth/tokens (validate new) 1 error sending request GET /v3/auth/tokens (validate new): operation timed out

View full report

@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

🧪 Tempest Identity Compatibility Results (advisory, non-blocking)

rust

Totals
======
Ran: 66 tests in 18.3131 sec.
 - Passed: 16
 - Skipped: 2
 - Expected Fail: 0
 - Unexpected Success: 0
 - Failed: 48
Sum of execute time for each test: 10.1209 sec.
Failed test IDs
===== FAILED TESTS (target=rust) =====
tempest.api.identity.admin.v3.test_credentials.CredentialsTestJSON.test_credentials_create_get_update_delete[id-7cd59bf9-bda4-4c72-9467-d21cab278355,smoke]
tempest.api.identity.admin.v3.test_domain_configuration.DomainConfigurationTestJSON.test_create_domain_config_and_show_config_groups_and_options[id-9e3ff13c-f597-4f01-9377-d6c06c2a1477]
tempest.api.identity.admin.v3.test_domain_configuration.DomainConfigurationTestJSON.test_create_update_and_delete_domain_config_groups_and_opts[id-c7510fa2-6661-4170-9c6b-4783a80651e9]
tempest.api.identity.admin.v3.test_domain_configuration.DomainConfigurationTestJSON.test_show_default_group_config_and_options[id-11a02bf0-6f94-4380-b3b0-c8dc18fc0d22]
setUpClass (tempest.api.identity.admin.v3.test_endpoint_groups.EndPointGroupsTest)
tempest.api.identity.admin.v3.test_domains_negative.DomainsNegativeTestJSON.test_delete_active_domain[gate,id-1f3fbff5-4e44-400d-9ca1-d953f05f609b,negative]
tempest.api.identity.admin.v3.test_default_project_id.TestDefaultProjectId.test_default_project_id[id-d6110661-6a71-49a7-a453-b5e26640ff6d]
tempest.api.identity.admin.v3.test_oauth_consumers.OAUTHConsumersV3Test.test_create_and_show_consumer[id-c8307ea6-a86c-47fd-ae7b-5b3b2caca76d]
tempest.api.identity.admin.v3.test_oauth_consumers.OAUTHConsumersV3Test.test_delete_consumer[id-fdfa1b7f-2a31-4354-b2c7-f6ae20554f93]
tempest.api.identity.admin.v3.test_oauth_consumers.OAUTHConsumersV3Test.test_list_consumers[id-09ca50de-78f2-4ffb-ac71-f2254036b2b8]
tempest.api.identity.admin.v3.test_oauth_consumers.OAUTHConsumersV3Test.test_update_consumer[id-080a9b1a-c009-47c0-9979-5305bf72e3dc]
tempest.api.identity.admin.v3.test_groups.GroupsV3TestJSON.test_group_users_add_list_delete[id-1598521a-2f36-4606-8df9-30772bd51339,smoke]
setUpClass (tempest.api.identity.admin.v3.test_endpoints_negative.EndpointsNegativeTestJSON)
tempest.api.identity.admin.v3.test_groups.GroupsV3TestJSON.test_list_user_groups[id-64573281-d26a-4a52-b899-503cb0f4e4ec]
setUpClass (tempest.api.identity.admin.v3.test_projects_negative.ProjectsNegativeTestJSON)
setUpClass (tempest.api.identity.admin.v3.test_endpoints.EndPointsTestJSON)
tempest.api.identity.admin.v3.test_projects.ProjectsTestJSON.test_associate_user_to_project[id-59398d4a-5dc5-4f86-9a4c-c26cc804d6c6]
tempest.api.identity.admin.v3.test_projects.ProjectsTestJSON.test_create_is_domain_project[id-a7eb9416-6f9b-4dbb-b71b-7f73aaef59d5]
tempest.api.identity.admin.v3.test_projects.ProjectsTestJSON.test_project_create_enabled[id-1f66dc76-50cc-4741-a200-af984509e480]
tempest.api.identity.admin.v3.test_projects.ProjectsTestJSON.test_project_create_not_enabled[id-78f96a9c-e0e0-4ee6-a3ba-fbf6dfd03207]
tempest.api.identity.admin.v3.test_projects.ProjectsTestJSON.test_project_create_with_description[id-0ecf465c-0dc4-4532-ab53-91ffeb74d12d]
tempest.api.identity.admin.v3.test_projects.ProjectsTestJSON.test_project_create_with_domain[id-5f50fe07-8166-430b-a882-3b2ee0abe26f]
tempest.api.identity.admin.v3.test_projects.ProjectsTestJSON.test_project_create_with_parent[id-1854f9c0-70bc-4d11-a08a-1c789d339e3d]
tempest.api.identity.admin.v3.test_projects.ProjectsTestJSON.test_project_get_equals_list[id-d1db68b6-aebe-4fa0-b79d-d724d2e21162]
setUpClass (tempest.api.identity.admin.v3.test_list_projects.ListProjectsTestJSON)
tempest.api.identity.admin.v3.test_projects.ProjectsTestJSON.test_project_update_desc[id-f138b715-255e-4a7d-871d-351e1ef2e153]
setUpClass (tempest.api.identity.admin.v3.test_inherits.InheritsV3TestJSON)
tempest.api.identity.admin.v3.test_projects.ProjectsTestJSON.test_project_update_enable[id-b6b25683-c97f-474d-a595-55d410b68100]
tempest.api.identity.admin.v3.test_projects.ProjectsTestJSON.test_project_update_name[id-f608f368-048c-496b-ad63-d286c26dab6b]
setUpClass (tempest.api.identity.admin.v3.test_regions.RegionsTestJSON)
setUpClass (tempest.api.identity.admin.v3.test_list_users.UsersV3TestJSON)
setUpClass (tempest.api.identity.admin.v3.test_list_projects.ListProjectsStaticTestJSON)
setUpClass (tempest.api.identity.admin.v3.test_tokens.TokensV3TestJSON)
setUpClass (tempest.api.identity.admin.v3.test_roles.RolesV3TestJSON)
setUpClass (tempest.api.identity.admin.v3.test_policies.PoliciesTestJSON)
setUpClass (tempest.api.identity.admin.v3.test_trusts.TrustsV3TestJSON)
setUpClass (tempest.api.identity.v3.test_application_credentials.ApplicationCredentialsV3Test)
setUpClass (tempest.api.identity.admin.v3.test_services.ServicesTestJSON)
setUpClass (tempest.api.identity.admin.v3.test_project_tags.IdentityV3ProjectTagsTest)
setUpClass (tempest.api.identity.v3.test_catalog.IdentityCatalogTest)
setUpClass (tempest.api.identity.v3.test_domains.DefaultDomainTestJSON)
setUpClass (tempest.api.identity.admin.v3.test_users_negative.UsersNegativeTest)
setUpClass (tempest.api.identity.admin.v3.test_projects_negative.ProjectsNegativeStaticTestJSON)
setUpClass (tempest.api.identity.v3.test_ec2_credentials.EC2CredentialsTest)
setUpClass (tempest.api.identity.v3.test_api_discovery.TestApiDiscovery)
setUpClass (tempest.api.identity.admin.v3.test_users.UsersV3TestJSON)
setUpClass (tempest.api.identity.v3.test_projects.IdentityV3ProjectsTest)
setUpClass (tempest.api.identity.v3.test_tokens.TokensV3Test)
===== END FAILED TESTS (target=rust) =====

View full run

gtema added 2 commits October 15, 2026 12:25
The OP login page offers the enabled identity providers of its domain,
but no test drove that path end to end. Add a Keycloak-backed test
binary to tests/federation, run by the skaffold verify step next to
the existing Keycloak tests.

Browser tests (Selenium Firefox, real Keycloak):
- authorize: login page IdP button, Keycloak sign-in, consent,
  authorization code, token. Checks nonce, the federated `amr` and
  that `sub` is the user the mapping created in the OP domain.
- authorize with an unmapped upstream user: the login page shows the
  upstream failure, no redirect to the RP, no user is provisioned.
- device flow: same round trip through /device and the device
  federated callback.

Protocol tests against the callback (no browser):
- wrong `state` is rejected with 400 and no redirect.
- upstream `error=access_denied` shows the failure, and the spent
  state cannot be answered again.
- a callback from a browser that did not start the flow is rejected.

Each test creates its own domain, IdP, mapping and OP client. The RP
redirect URI is https://rp.invalid/cb because public clients may only
use https or http://localhost, which a remote browser cannot serve; the
test reads the target from Firefox's error page.

Assisted-by: Claude <noreply@anthropic.com>
Signed-off-by: Artem Goncharov <artem.goncharov@gmail.com>
#[tokio::test] gives each test its own runtime, but the shared
keystone_utils client pools connections process-wide. In parallel runs
a pooled connection from a finished test's runtime was reused and the
request failed with 'runtime dropped the dispatch task'. Use a
thread-local client so connections never cross runtimes.

Signed-off-by: Artem Goncharov <artem.goncharov@gmail.com>
@gtema
gtema enabled auto-merge October 15, 2026 10:25
@gtema
gtema added this pull request to the merge queue Oct 15, 2026
Merged via the queue into main with commit c4b1c03 Oct 15, 2026
36 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants