fix(linux): remove bootstrap-kubeconfig after TLS bootstrapping by cameronmeissner · Pull Request #9797 · Azure/AgentBaker

Cameron Meissner (cameronmeissner) · 2026-10-15T21:48:42Z

What this PR does / why we need it:

remove bootstrap-kubeconfig after TLS bootstrapping

Which issue(s) this PR fixes:

Fixes #

github-actions · 2026-10-15T21:51:56Z

Windows Unit Test Results

3 files 18 suites 1m 4s ⏱️
560 tests 560 ✅ 0 💤 0 ❌
563 runs 563 ✅ 0 💤 0 ❌

Results for commit 5ed5ec9.

♻️ This comment has been updated with latest results.

Copilot

🟡 Changes recommended

Script delivery, logging initialization, startup-timeout coupling, and incompatible validators introduce startup and E2E regressions.

4 open findings

What changed in this PR

This PR reduces credential retention during Linux node provisioning by removing bootstrap inputs after use.

Changes:

Adds bootstrap-kubeconfig cleanup through a kubelet post-start hook.
Restricts bootstrap-kubeconfig permissions and deletes node-controller inputs.
Adds ShellSpec coverage and an E2E cleanup assertion.

File	Description
spec/parts/linux/cloud-init/artifacts/remove-bootstrap-kubeconfig_spec.sh	Tests cleanup behavior and deletion failures.
parts/linux/cloud-init/artifacts/remove-bootstrap-kubeconfig.sh	Waits for kubeconfig, then removes bootstrap credentials.
parts/linux/cloud-init/artifacts/kubelet.service	Invokes cleanup after kubelet starts.
parts/linux/cloud-init/artifacts/cse_config_kubelet.sh	Restricts bootstrap credentials to mode 0600.
parts/linux/cloud-init/artifacts/aks-node-controller-launcher.sh	Deletes provisioning inputs after execution.
e2e/scenario/validators.go	Checks that bootstrap credentials were removed.

🧠 Review effort: Balanced

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Copilot

🟡 Changes recommended

Cleanup can delete bootstrap credentials before a client certificate is available, preventing recovery after a kubelet restart.

1 open finding

Verify client credentials before deleting bootstrap kubeconfig · New

4 resolved since last review

🧠 Review effort: Balanced

Copilot

🟡 Changes recommended

Deleting controller inputs after failed provisioning prevents subsequent attempts from provisioning the node.

2 open findings

🧠 Review effort: Balanced

Copilot

🟡 Changes recommended

Cleanup can trigger kubelet startup timeouts, and ANC input deletion conflicts with existing E2E validators.

2 open findings

2 resolved since last review

🧠 Review effort: Balanced

aks-node-assistant · 2026-10-16T00:32:40Z

AgentBaker Linux gate detective

Run: 186034915
Failed job/stage/task: e2e / Run AgentBaker E2E / AzureLinuxV2 node API deadline and broad scenario failures

Detective summary: Run AgentBaker E2E failed after 32 scenarios passed and 75 were skipped; AzureLinuxV2 did not appear in the Kubernetes API and node listing hit a client rate-limiter context deadline, after which the run reported 88 failed scenarios.

Likely cause / signature: e2e-shared-client-rate-limiter-context-deadline.

Confidence: Medium-high. Timeline isolates the failure to e2e/Run AgentBaker E2E, the task log shows client rate limiter context deadline during node readiness, build metadata ties this to PR 9797, and an existing wiki row tracks this generic control-plane/client-deadline family.

Recommended owner/action: Treat as shared E2E control-plane/client throttling or environment latency unless it repeats on this PR; no product code action is indicated from this single run.

Strongest alternative: PR 9797 removed bootstrap-kubeconfig after TLS bootstrapping, but the observed failure is a generic Kubernetes API/list timeout after many unrelated scenarios, not bootstrap token validation or kubelet credential cleanup.

Evidence: build run, failed timeline records for stage e2e/job Run AgentBaker E2E/task Run AgentBaker E2E, task log lines showing client rate limiter Wait returned an error: context deadline exceeded, source branch refs/pull/9797/merge, and wiki tracker.

Wiki: e2e-shared-client-rate-limiter-context-deadline

Copilot

🟡 Changes recommended

The new ShellSpec output expectations conflict with the helper’s diagnostics and fail on otherwise correct behavior.

1 open finding

Cleanup script tests expect blank output despite diagnostics · New

2 resolved since last review

🧠 Review effort: Balanced

Copilot

🔵 Needs a closer look

E2E compilation failures, unsafe shared-subnet deletion, and lost Bastion write limits remain unresolved.

6 open issues

1 resolved since last review

Validator requires opt-in cleanup unconditionally

2 files not reviewed

aks-node-controller/pkg/gen/aksnodeconfig/v1/bootstrapping_config.pb.go: Generated file
aks-node-controller/pkg/gen/aksnodeconfig/v1/gpu_config.pb.go: Generated file

_{Review effort: Balanced}

Copilot

🟡 Changes recommended

Older-image compatibility and scriptless configuration and validation inconsistencies remain unresolved.

8 open issues

2 files not reviewed

aks-node-controller/pkg/gen/aksnodeconfig/v1/bootstrapping_config.pb.go: Generated file
aks-node-controller/pkg/gen/aksnodeconfig/v1/gpu_config.pb.go: Generated file

_{Review effort: Balanced}

Copilot

🔵 Needs a closer look

The missing closing quote in the Linux CSE template blocks provisioning even when cleanup is disabled.

2 open issues

6 resolved since last review

1 file not reviewed

aks-node-controller/pkg/gen/aksnodeconfig/v1/bootstrapping_config.pb.go: Generated file

_{Review effort: Balanced}

Copilot

🟡 Changes recommended

Unconditional E2E enablement breaks the frozen Azure Linux V2 scenario because its image lacks the cleanup helper.

1 open issue

Frozen AzureLinuxV2 image lacks cleanup helper · New

2 resolved since last review

1 file not reviewed

aks-node-controller/pkg/gen/aksnodeconfig/v1/bootstrapping_config.pb.go: Generated file

_{Review effort: Balanced}

…ssner/clean-token

github-actions · 2026-10-20T16:18:13Z

Package	Line Rate	Health
parts_windows Package 1	73%	➖
staging_cse_windows Package 1	65%	➖
debug	0%	❌
provisioningscripts	2%	❌
test	0%	❌
windows	27%	❌
Summary	38% (1645 / 6827)	❌

Copilot

🔵 Needs a closer look

Kubelet startup timeout and older-image compatibility concerns remain unresolved.

No open issues 🎉

1 resolved since last review

Frozen AzureLinuxV2 image lacks cleanup helper

1 file not reviewed

aks-node-controller/pkg/gen/aksnodeconfig/v1/bootstrapping_config.pb.go: Generated file

_{Review effort: Balanced}

github-actions · 2026-10-20T16:18:57Z

Package	Line Rate	Health
github.com/Azure/agentbaker/aks-node-controller	72%	➖
github.com/Azure/agentbaker/aks-node-controller/common	100%	✔
github.com/Azure/agentbaker/aks-node-controller/helpers	68%	➖
github.com/Azure/agentbaker/aks-node-controller/parser	88%	✔
github.com/Azure/agentbaker/aks-node-controller/pkg/gpu	100%	✔
github.com/Azure/agentbaker/aks-node-controller/pkg/nodeconfigutils	88%	✔
github.com/Azure/agentbaker/aks-node-controller/utils	0%	❌
github.com/Azure/agentbaker/apiserver	24%	❌
github.com/Azure/agentbaker/cmd	0%	❌
github.com/Azure/agentbaker/cmd/starter	0%	❌
github.com/Azure/agentbaker/fuzz/api	0%	❌
github.com/Azure/agentbaker/hotfix/render-nodecustomdata	0%	❌
github.com/Azure/agentbaker/pkg/agent	77%	➖
github.com/Azure/agentbaker/pkg/agent/datamodel	78%	➖
github.com/Azure/agentbaker/pkg/agent/toggles	0%	❌
github.com/Azure/agentbaker/pkg/vhdbuilder/datamodel	87%	✔
Summary	74% (5660 / 7595)	➖

github-actions · 2026-10-20T16:19:29Z

Package	Line Rate	Health
shellspec spec	28%	❌
Summary	28% (4548 / 15952)	❌

fix(linux): remove bootstrap-kubeconfig after kubelet startup

8fecc18

Copilot AI balanced review requested due to automatic review settings October 15, 2026 21:48

Cameron Meissner (cameronmeissner) requested review from a team as code owners October 15, 2026 21:48

Copilot started reviewing on behalf of Cameron Meissner (cameronmeissner) October 15, 2026 21:49

View session

Cameron Meissner (cameronmeissner) marked this pull request as draft October 15, 2026 21:49

chore: upload with packer

d316f35

chore: packer source

a859746

Copilot AI reviewed Oct 15, 2026

View reviewed changes

Cameron Meissner (cameronmeissner) marked this pull request as ready for review October 15, 2026 21:56

Copilot AI balanced review requested due to automatic review settings October 15, 2026 21:56

Copilot started reviewing on behalf of Cameron Meissner (cameronmeissner) October 15, 2026 21:56

View session

Copilot AI reviewed Oct 15, 2026

View reviewed changes

Comment thread parts/linux/cloud-init/artifacts/remove-bootstrap-kubeconfig.sh

chore: add missing events file path

7cebbe8

Copilot AI balanced review requested due to automatic review settings October 15, 2026 22:26

Copilot started reviewing on behalf of Cameron Meissner (cameronmeissner) October 15, 2026 22:26

View session

Copilot AI reviewed Oct 15, 2026

View reviewed changes

Comment thread parts/linux/cloud-init/artifacts/aks-node-controller-launcher.sh Outdated

chore: harden

259d978

Copilot AI balanced review requested due to automatic review settings October 15, 2026 23:08

Copilot started reviewing on behalf of Cameron Meissner (cameronmeissner) October 15, 2026 23:11

View session

Copilot AI reviewed Oct 15, 2026

View reviewed changes

Comment thread parts/linux/cloud-init/artifacts/remove-bootstrap-kubeconfig.sh

Comment thread parts/linux/cloud-init/artifacts/aks-node-controller-launcher.sh Outdated

chore: cleanup

f6ee985

Copilot AI balanced review requested due to automatic review settings October 16, 2026 18:43

Copilot started reviewing on behalf of Cameron Meissner (cameronmeissner) October 16, 2026 18:43

View session

Copilot AI reviewed Oct 16, 2026

View reviewed changes

Comment thread spec/parts/linux/cloud-init/artifacts/remove-bootstrap-kubeconfig_spec.sh Outdated

chore: fix tests

495631f

Copilot AI balanced review requested due to automatic review settings October 16, 2026 18:55

chore: correct cleanup

612b20f

Copilot AI reviewed Oct 19, 2026

View reviewed changes

Copilot AI balanced review requested due to automatic review settings October 19, 2026 19:00

Copilot started reviewing on behalf of Cameron Meissner (cameronmeissner) October 19, 2026 19:00

View session

Cameron Meissner (cameronmeissner) added 2 commits October 19, 2026 12:00

chore: fix e2e

a13cde5

chore: cleanup

a8ef902

Copilot AI reviewed Oct 19, 2026

View reviewed changes

Comment thread e2e/scenario/node_config.go Outdated

Comment thread vhdbuilder/packer/packer_source.sh

Copilot AI balanced review requested due to automatic review settings October 19, 2026 19:07

Copilot started reviewing on behalf of Cameron Meissner (cameronmeissner) October 19, 2026 19:08

View session

Cameron Meissner (cameronmeissner) added 6 commits October 19, 2026 12:08

chore: also wait for kubeconfig

6e2f860

chore: clean diff

fcd870e

chore: fix

9d637b3

chore: fix missing azlosguard file entry

8b481e9

chore: fix nbcToAKSNodeConfigV1

e90bf66

chore: ptr

ef47b1f

Copilot AI reviewed Oct 19, 2026

View reviewed changes

Copilot AI balanced review requested due to automatic review settings October 19, 2026 19:14

Copilot started reviewing on behalf of Cameron Meissner (cameronmeissner) October 19, 2026 19:15

View session

Copilot AI reviewed Oct 19, 2026

View reviewed changes

Comment thread e2e/scenario/node_config.go Outdated

Merge branch 'main' of https://github.com/Azure/AgentBaker into camei…

7fd093e

…ssner/clean-token

Copilot AI balanced review requested due to automatic review settings October 20, 2026 16:11

Copilot started reviewing on behalf of Cameron Meissner (cameronmeissner) October 20, 2026 16:12

View session

Cameron Meissner (cameronmeissner) added 2 commits October 20, 2026 09:13

chore: ignore frozen azlinuxv2

9755b22

chore: augment validation

5ed5ec9

Copilot AI reviewed Oct 20, 2026

View reviewed changes

Cameron Meissner (cameronmeissner) mentioned this pull request Oct 20, 2026

fix(security): restrict node secret file perms and avoid logging AD realm password #9776

Open

Nishchay (awesomenix) approved these changes Oct 20, 2026

View reviewed changes

Conversation

Cameron Meissner (cameronmeissner) commented Oct 15, 2026

Uh oh!

github-actions Bot commented Oct 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Windows Unit Test Results

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

aks-node-assistant Bot commented Oct 16, 2026

AgentBaker Linux gate detective

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

github-actions Bot commented Oct 20, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

github-actions Bot commented Oct 20, 2026

Uh oh!

github-actions Bot commented Oct 20, 2026

Uh oh!

This branch was successfully deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

github-actions Bot commented Oct 15, 2026 •

edited

Loading