Repository navigation
fix(cloud-init): avoid nested script compression - #9753
Ben Brady (benjamin-brady) wants to merge 4 commits into
Conversation
Keep scripts literal inside the outer compressed CustomData payload to stay within Azure VMSS limits. Preserve gzip write_files for Flatcar, ACL and scriptless paths. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 29247196-6c9e-475a-8892-77fc9c3316d4
Windows Unit Test Results 3 files 18 suites 1m 9s ⏱️ Results for commit 487af9a. ♻️ This comment has been updated with latest results. |
There was a problem hiding this comment.
🟡 Changes recommended
The new helper bypasses automatic transformed-script syntax-test discovery for critical CSE scripts.
1 open finding
What changed in this PR
Updates Linux cloud-init packaging to avoid nested script compression while preserving file contents and compatibility paths.
Changes:
- Renders eligible
write_filesentries as literal YAML. - Preserves gzip encoding for Flatcar, ACL, and scriptless paths.
- Updates hotfix generation and adds payload/round-trip tests.
| File | Description |
|---|---|
pkg/agent/variables.go |
Selects literal or gzip file content. |
pkg/agent/utils.go |
Separates rendering from compression and safely marshals JSON. |
pkg/agent/baker.go |
Renders literal YAML file properties. |
pkg/agent/baker_test.go |
Tests YAML round trips and gzip fallback. |
pkg/agent/bakerapi_test.go |
Validates payload size, metadata, and bytes. |
parts/linux/cloud-init/nodecustomdata.yml |
Uses the new file-property renderer. |
hotfix/hotfix_generate.py |
Recognizes legacy and literal template entries. |
hotfix/hotfix_generate_test.py |
Tests mixed syntax and rendered hotfixes. |
🧠 Review effort: Balanced
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Keep transformed-script round-trip and bash syntax checks on scripts routed through getCloudInitFile. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 29247196-6c9e-475a-8892-77fc9c3316d4
AgentBaker Linux gate detectiveRun: 185409097 Detective summary: One E2E scenario failed. The LocalDNS hosts plugin cold-start validation truncated Likely cause / signature: e2e-localdns-coldstart-empty-hosts-start-timeout - LocalDNS cannot reliably start after empty-hosts cold start. Confidence: High for the signature: failed test result, focused E2E log, failed timeline, and prior matching wiki row all point to the same cold-start validation family; medium on root cause because service diagnostics show systemd group failure but not the lower-level localdns process cause. Recommended owner/action: LocalDNS owner should inspect the cold-start empty-hosts path and the service cgroup/delegate behavior on Ubuntu2604Minimal; PR owner likely only needs action if the cloud-init compression change affects service/cgroup setup, which current evidence does not show. Strongest alternative: PR-caused cloud-init regression, but the failed scenario is LocalDNS cold-start specific and many adjacent scenarios completed before/after, making a recurring LocalDNS lifecycle flake more likely. Evidence: build run, failed test run 687073470, E2E task log 624, failed e2e timeline records, build metadata, and wiki tracker. |
AgentBaker Linux gate detectiveRun: 185424174 Detective summary: One E2E scenario failed after the VMSS and node reached Ready and initial SSH connectivity succeeded. The validation then failed while retrieving kubelet logs and other VM state: SSH commands returned Likely cause / signature: e2e-ubuntu2204-networkisolated-ssh-context-canceled - bastion/SSH session context cancellation after node readiness. Confidence: Medium-high for infrastructure/session flake: failed test record, E2E log, timeline, and successful node readiness/initial SSH all isolate the failure to later remote command/session handling rather than provisioning. Recommended owner/action: Treat as gate infrastructure/SSH session flake unless it recurs with cloud-init-specific failures; PR owner likely does not need a code change based on this run. Strongest alternative: PR cloud-init compression caused node-side command instability, but node readiness, pod setup, and initial SSH connectivity all succeeded, and the fatal errors are SSH/session context cancellations during diagnostics. Evidence: build run, failed test run 687244905, E2E task log 624, failed e2e timeline records, PR cloud-init metadata, and wiki tracker. |
Preserve Phase 3 validation and literal payload coverage across the shared test file. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 29247196-6c9e-475a-8892-77fc9c3316d4
Only compress a raw script when a legacy template requests cloudInitData. Keep the literal CustomData path free of nested compression. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 29247196-6c9e-475a-8892-77fc9c3316d4
|
|
|
What this PR does / why we need it:
Render Linux
write_filescontent as literal YAML inside the already-gzipped CustomData payload. Compressing each script before compressing the whole payload increases VM/VMSS CustomData size; the combined GPU/Chrony configuration motivating #9702 exceeded Azure's 87,380-character base64 limit. This PR changes packaging only, retains the same file bytes, paths, modes, owners and ordering, and teaches the hotfix generator to recognize both literal and legacy template entries. Flatcar, ACL and scriptless paths retain gzipwrite_filesencoding. No GPU provisioning changes or CI Go-version pin are included.Local synthetic fixture on this branch (Ubuntu 22.04 ordinary; not the combined payload from #9702):
The baseline regression test failed before the change and passed after. Ubuntu 22.04/24.04 and Azure Linux 3 ordinary/PIS payloads all pass strict base64/gzip/YAML decoding, file-content and metadata checks, and the API-size assertions. This is local synthetic validation, not a live E2E result.
Validation:
go test ./pkg/agent/... -count=1— passed;python3 -m unittest hotfix.hotfix_generate_test -v— passed (28 tests);cd aks-node-controller && go test ./...— passed;golangci-lint run ./pkg/agent/...— passed (0 issues);gofmt -l pkg/agent/baker.go pkg/agent/baker_test.go pkg/agent/bakerapi_test.go pkg/agent/utils.go pkg/agent/variables.go— passed (no output).Which issue(s) this PR fixes:
Prerequisite for #9702; does not close an issue.