DLPX-98872 linux-pkg: generate per-package CycloneDX SBOM via Syft deb scan and publish S3 sidecar - #418
Conversation
d617ba0 to
bbde65e
Compare
4153069 to
2623916
Compare
…b scan and publish S3 sidecar PR URL: https://www.github.com/delphix/linux-pkg/pull/418
2623916 to
ee451b2
Compare
There was a problem hiding this comment.
🟡 Changes recommended
packages/docker-python-image/config.sh sets PACKAGE_NEEDS_DOCKER="false" while its own comment indicates the build runs docker pull, which will likely fail without the docker socket mount.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds per-package, opt-in CycloneDX SBOM “sidecars” generated from built .deb artifacts (via Syft), plus CI enforcement that every package is explicitly classified for deep scanning.
Changes:
- Introduces
SBOM_DEEP_SCANas a per-package classification flag and exposes it viaquery-packages.sh. - Adds
generate_sbom()stage to build flow to scan.deboutputs with Syft and (if needed) merge/validate viacyclonedx-cli. - Adds a GitHub Actions lint to fail CI if any package lacks an explicit
SBOM_DEEP_SCANclassification.
File summaries
| File | Description |
|---|---|
| query-packages.sh | Adds sbom-deep-scan output field to surface SBOM_DEEP_SCAN values. |
| buildpkg.sh | Wires new generate_sbom stage into the package build pipeline. |
| lib/common.sh | Implements generate_sbom() (Syft scan, optional merge, schema validation). |
| docs/specs/2026-09-08-sbom-per-package-sidecar-design.md | Adds design/spec documenting per-package sidecar generation approach. |
| .github/workflows/main.yml | Adds CI job to enforce SBOM_DEEP_SCAN classification. |
| .github/scripts/verify-sbom-scan-flag.sh | New lint script ensuring all packages set SBOM_DEEP_SCAN to true/false. |
| packages/zfs/config.sh | Classifies package for deep SBOM scan (SBOM_DEEP_SCAN="true"). |
| packages/windows-connector/config.sh | Classifies package for deep SBOM scan (SBOM_DEEP_SCAN="true"). |
| packages/virtualization/config.sh | Classifies package for deep SBOM scan (SBOM_DEEP_SCAN="true"). |
| packages/targetcli-fb/config.sh | Explicitly classifies as not requiring deep SBOM scan. |
| packages/syft/config.sh | Explicitly classifies as not requiring deep SBOM scan. |
| packages/sdb/config.sh | Explicitly classifies as not requiring deep SBOM scan. |
| packages/savedump/config.sh | Explicitly classifies as not requiring deep SBOM scan. |
| packages/python-rtslib-fb/config.sh | Explicitly classifies as not requiring deep SBOM scan. |
| packages/ptools/config.sh | Classifies package for deep SBOM scan (SBOM_DEEP_SCAN="true"). |
| packages/performance-diagnostics/config.sh | Explicitly classifies as not requiring deep SBOM scan. |
| packages/nfs-utils/config.sh | Explicitly classifies as not requiring deep SBOM scan. |
| packages/misc-debs/config.sh | Explicitly classifies as not requiring deep SBOM scan. |
| packages/masking/config.sh | Classifies package for deep SBOM scan (SBOM_DEEP_SCAN="true"). |
| packages/makedumpfile/config.sh | Explicitly classifies as not requiring deep SBOM scan. |
| packages/linux-kernel-oracle/config.sh | Explicitly classifies as not requiring deep SBOM scan. |
| packages/linux-kernel-generic/config.sh | Explicitly classifies as not requiring deep SBOM scan. |
| packages/linux-kernel-gcp/config.sh | Explicitly classifies as not requiring deep SBOM scan. |
| packages/linux-kernel-azure/config.sh | Explicitly classifies as not requiring deep SBOM scan. |
| packages/linux-kernel-aws/config.sh | Explicitly classifies as not requiring deep SBOM scan. |
| packages/libkdumpfile/config.sh | Explicitly classifies as not requiring deep SBOM scan. |
| packages/host-jdks/config.sh | Explicitly classifies as not requiring deep SBOM scan. |
| packages/grub2/config.sh | Explicitly classifies as not requiring deep SBOM scan. |
| packages/gdb-python/config.sh | Explicitly classifies as not requiring deep SBOM scan. |
| packages/fluentd-gems/config.sh | Explicitly classifies as not requiring deep SBOM scan. |
| packages/dwarves/config.sh | Explicitly classifies as not requiring deep SBOM scan. |
| packages/drgn/config.sh | Explicitly classifies as not requiring deep SBOM scan. |
| packages/docker-python-image/config.sh | Adds SBOM classification and changes docker-socket requirement flag. |
| packages/delphix-sso-app/config.sh | Classifies package for deep SBOM scan (SBOM_DEEP_SCAN="true"). |
| packages/delphix-rust/config.sh | Classifies package for deep SBOM scan (SBOM_DEEP_SCAN="true"). |
| packages/delphix-platform/config.sh | Explicitly classifies as not requiring deep SBOM scan. |
| packages/delphix-kernel/config.sh | Explicitly classifies as not requiring deep SBOM scan. |
| packages/delphix-go/config.sh | Explicitly classifies as not requiring deep SBOM scan. |
| packages/cyclonedx-cli/config.sh | Explicitly classifies as not requiring deep SBOM scan. |
| packages/crypt-blowfish/config.sh | Explicitly classifies as not requiring deep SBOM scan. |
| packages/crash-python/config.sh | Explicitly classifies as not requiring deep SBOM scan. |
| packages/containerized-masking/config.sh | Classifies package for deep SBOM scan (SBOM_DEEP_SCAN="true"). |
| packages/connstat/config.sh | Explicitly classifies as not requiring deep SBOM scan. |
| packages/cloud-init/config.sh | Explicitly classifies as not requiring deep SBOM scan. |
| packages/challenge-response/config.sh | Explicitly classifies as not requiring deep SBOM scan. |
| packages/bcc/config.sh | Explicitly classifies as not requiring deep SBOM scan. |
Review details
- Files reviewed: 46/46 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…version generate_sbom() assumed syft/cyclonedx-cli were already on PATH, but nothing installed them into the linux-pkg build container -- Phase 1 only installs them onto the appliance-build host. Confirmed via a real pre-push run: every SBOM_DEEP_SCAN package failed with "syft: command not found". Fixed by declaring syft/cyclonedx-cli as PACKAGE_DEPENDENCIES on all 8 flagged packages (masking, virtualization, delphix-sso-app, containerized-masking, windows-connector, zfs, ptools, delphix-rust), and having generate_sbom() install them from $DEPDIR before scanning -- the same pattern zfs already uses for delphix-rust. Also fixes --source-version showing up blank for packages (like masking) that don't set $PACKAGE_VERSION themselves: read the version back out of the built .deb via dpkg-deb instead of relying on a shell variable that doesn't reliably survive to this stage. verify-query-packages.sh's zfs dependency-list assertion updated to match the new PACKAGE_DEPENDENCIES.
48d6823 to
0b4e66b
Compare
generate_sbom() didn't set SYFT_FILE_METADATA_SELECTION=none, so every sidecar carried an extra CycloneDX "file" component for the .deb itself (SHA-1/SHA-256 hashes plus the absolute build-workspace path) alongside the real pkg:deb component -- confirmed in delphix-sso-app.cdx.json from a real pre-push build. Same fix appliance-build's 95-generate-sbom.binary hook already applies, for the same reason: a flat, per-package pkg:deb document shouldn't carry per-file noise or leak the build machine's local path.
Multi-.deb packages (delphix-rust: delphix-rust + delphix-rust-src; zfs similarly) go through the cyclonedx-cli merge path in generate_sbom(), which had no --output-version pinned. cyclonedx-cli merge defaults to the newest spec version it supports (1.7), not the 1.6 Syft emitted, so the merged document then failed the very next validate step: Incorrect schema version: expected 1.6 actual 1.7 Confirmed via a real pre-push delphix-rust build. Single-.deb packages never hit this, since the merge step is skipped entirely for them (a plain cp). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Reviewer feedback: each .deb should have its own distinct BOM file
with a common filename prefix, not one merged SBOM per source package.
The prior approach scanned each .deb separately then merged them via
cyclonedx-cli merge into a single <package>.cdx.json -- correct per
the top-level design doc's stated intent ("one package-level SBOM...
associated with all of that package's debs"), but that assumption
didn't survive review.
generate_sbom() now writes one <deb-filename>.deb.cdx.json per .deb,
independently scanned and validated, with no merge step at all. This
also removes the cyclonedx-cli merge --output-version bug entirely,
since there's no merge left to have a version mismatch in.
Also reconfirms compliance with the reviewer's other point: syft,
cyclonedx-cli, and all linux-kernel-* packages are SBOM_DEEP_SCAN=false
(unaffected by this change, verified separately) -- no SBOM is
generated for build-host tooling or 3rd-party kernel forks.
Design doc updated to match: architecture diagram, the "multiple .debs
per package" resolution, and the implementation-status/follow-ups
sections now describe the 1:1 mapping and the real-build testing
already done, instead of the merge approach and its since-superseded
open question about cyclonedx-cli's --output-version default.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Review feedback: generate_sbom() is a default hook, inherited unmodified by every SBOM_DEEP_SCAN package, so the tooling it runs is the hook's concern -- a package's config.sh should not have to declare it. Those tools belong with the rest of linux-pkg's generic build dependencies, installed prior to any package build. All 8 flagged packages therefore drop PACKAGE_DEPENDENCIES="syft cyclonedx-cli" (virtualization and zfs are restored to their original dependency lists), and setup.sh gains install_sbom_tools() alongside install_awscli/install_shfmt. Unlike everything else installed there, delphix-syft/delphix-cyclonedx-cli are Delphix-built .debs with no apt source -- the container's apt sources are only the Ubuntu primary mirror and the PPA secondary mirror -- so they are fetched from the same S3 location fetch_dependencies() uses and installed by path. apt rather than dpkg, since delphix-cyclonedx-cli has real libicu dependencies. The install is best-effort: setup.sh is package-agnostic, so it cannot skip itself when the package being built is syft or cyclonedx-cli. A hard failure on a missing artifact would break every build on a branch where neither has been published yet, including their own, so a missing artifact warns and continues. generate_sbom() checks for the tools itself and fails loudly, affecting only the builds that need them. Trade-off worth noting: with no PACKAGE_DEPENDENCIES entry, Jenkins's static dependency graph no longer knows these packages relate to syft/cyclonedx-cli, so build-order batching and the rebuild-dependents- on-syft-change cascade no longer apply to them. Documented in the spec, along with the rejected alternative of deriving the dependency from SBOM_DEEP_SCAN in load_package_config(). verify-query-packages.sh's zfs assertion is restored accordingly. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Review feedback: the lint told you that a package was unclassified but not how to classify it, leaving whoever adds the next package to go read lib/common.sh or the spec to find out. State the rule where it is actually needed -- 1st-party packages set "true" so the third-party components they package internally land in the product's aggregate SBOM; 3rd-party forks of Debian packages and anything not shipped in a product set "false", since appliance-build's image-level scan already covers those as a flat pkg:deb component. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The sidecars produced so far were valid but empty. "syft scan <deb>" only identifies the archive -- its deb-archive-cataloger reads the control metadata and emits a single pkg:deb component, never descending into data.tar.* -- so none of the bundled jars, wheels or modules these sidecars exist to capture were ever found. A real delphix-virtualization sidecar contained exactly one component for a 1.17 GB Java + Angular application: no more than appliance-build's image-level dpkg scan already provides for free, and nothing for a vulnerability scanner to match against the bundled third-party code. Every flagged package was affected. This was prescribed in the spec as the fallback for exactly this case, and had been closed out as "confirmed working" purely because a build produced a schema-valid document; cyclonedx-cli validate passes an empty-but-well-formed BOM quite happily, so nothing failed. Only reading the output revealed it. generate_sbom() now runs dpkg-deb -x into a temp directory and scans that with "syft scan dir:...", using Syft's full catalogers rather than the dpkg-only restriction appliance-build's image-level hook applies. That restriction exists because a jar found somewhere on a whole rootfs cannot be attributed to the package that placed it; inside one package's own extracted payload everything found belongs to that package by construction. Shape change worth noting: the scanned source is now a directory, so the sidecar no longer carries a pkg:deb component for the package itself -- its identity is in metadata.component, and the flat pkg:deb entry still comes from the image-level scan. Phase 3 associates a sidecar with its .deb by filename, so nothing depends on it. The bundled npm frontend remains absent for masking/virtualization: those ship the built, minified Angular bundle with no package.json, so extraction cannot help. That is the known blind spot the top-level design flagged for Phase 4's method evaluation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
performance-diagnostics is built from Delphix source (github.com/delphix/performance-diagnostics), so by the same rule the lint now states -- 1st-party packages set "true" so that the third-party components they package internally are included in the product's aggregate SBOM -- it belongs in the deep-scan set. The initial classification pass grouped it with the single-ecosystem tools that need nothing beyond the image-level flat pkg:deb component, which was wrong. Brings the flagged set to 9 of 40 packages. Spec's classification table and package lists updated to match; the remaining hardcoded counts of "8 flagged packages" in prose are replaced with count-agnostic wording so they do not go stale again. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Hi @sebroy Tried to work as per your comment — here's what it looks like after the change: each .deb now gets its own distinct BOM file with a common filename prefix, 1-1, no more package-level merge. Example from delphix-rust: Both .cdx.json files are uploaded to S3 alongside their .deb. Source Build [for package delphix-rust]: https://selfservice-jenkins.eng-tools-prd.aws.delphixcloud.com/job/linux-pkg/job/develop/job/build-package/job/delphix-rust/job/pre-push/69/ Parent Build[linux-pkg/job/develop/job/build-packages]: https://selfservice-jenkins.eng-tools-prd.aws.delphixcloud.com/job/linux-pkg/job/develop/job/build-packages/job/pre-push/7537/console |
|
I'm looking at the virtualization BOM produced by the test job, and there's a lot there that doesn't feel necessary. For example, for each item in the components array, there's a large Once we've done that, we may also want to de-duplicate the components entries because it looks like syft is including the same component multiple times if it shows up in multiple places in the deb. For example, Another observation is that the virtualization SBOM has hundreds of components, but the dependencies object only has 26 entries, and it's of dubious value for an SBOM for external consumption. I'd suggest removing that |
Raw Syft output is not publishable as-is. On delphix-virtualization it was 4.7 MB for 416 distinct components, three quarters of it repetition, and it recorded 470 directories under /opt/delphix -- including jar-internal paths such as resources.war:WEB-INF/lib/ST4-4.3.4.jar -- in a document destined for the customer-facing BOM. sanitize_sbom() applies resources/sanitize-sbom.jq to each sidecar between the Syft scan and the validate call, so what is validated is what is published. It drops the dependencies graph, drops every property except syft:cpe23, and merge-dedupes components. The dependencies graph went beyond sparse: for a directory scan Syft emits jar containment rather than resolved dependency edges, it covered 501 of 1693 components with no compositions element declaring it incomplete, and 90 of its 568 edges already pointed at bom-refs that do not exist -- the per-file components suppressed by SYFT_FILE_METADATA_SELECTION=none. De-duplication would have orphaned many more. Duplicates are merged rather than reduced to the first occurrence. Syft does not detect the same metadata at every location: licences differed across copies in 84 groups, externalReferences in 92 and cpe in 59, so keep-first would have silently lost licence data for 25 components and SHA-1 hashes for 31 from a document meant to replace the customer-facing licence CSV. A duplicate's differing primary CPE is demoted into syft:cpe23 so no matching coordinate is lost. syft:cpe23 is retained throughout because the schema permits a single top-level cpe while Syft derives several candidates per component. Result on that sidecar: 1693 -> 416 components, 43566 -> 6296 property entries, zero occurrences of /opt/delphix, 4.7 MB -> 0.8 MB, still schema-valid. Verified with Grype 0.119.0 that matching is unaffected -- raw and sanitized produce identical findings (40 matches, 21 CVEs), since Grype matches on purl and cpe. Location reporting is lost; detection is not. CYCLONEDX_FILTERING=false skips the pass and publishes the raw output, keeping syft:metadata:virtualPath so we can still determine where an unexpected component came from. Only the exact string "false" disables it, so a typo leaves us with a publishable document rather than one recording internal paths. Exposing it as a Jenkins build parameter is tracked as TOOL-31116. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Grype reported every matched component as "UnknownPackage" rather than "binary" once the properties were stripped. The type is not recoverable from anything else for a component whose purl carries no ecosystem: the six pkg:generic components and the eight PE binaries Syft finds without assigning a purl at all. Components with a pkg:maven purl were never affected, since Grype derives the type from the purl itself, which is why detection counts matched throughout and only the reported type was wrong. syft:package:metadataType and syft:package:foundBy were evaluated alongside it and changed nothing -- Grype's output was identical with and without them -- so they stay dropped. Retaining the one property costs roughly 3%. Also records in the spec what the Grype testing established about syft:cpe23: Grype sets match.java.using-cpes to false by default, so it ignores CPEs for the 405 Java components here and matches on the purl. Cutting syft:cpe23 from 6296 entries to 62 left Grype's output byte-identical. Those candidates are retained for Mend rather than Grype, which is worth stating explicitly since anyone measuring against Grype alone would conclude they are dead weight. Verified against a real delphix-virtualization sidecar: 1693 -> 416 components, 43566 -> 6712 property entries, zero occurrences of /opt/delphix, schema-valid, and Grype now reports 40 matches / 21 CVEs typed "binary", matching the unfiltered document exactly. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ndency graph Mend support analysed a sidecar we sent them and found two structural reasons it imports badly, neither of which the sanitisation addressed. First, metadata.component was typed "file" rather than "application". Syft emits that because generate_sbom() scans an extracted directory, and Mend's importer only treats metadata.component as the project root when it is application-typed -- so the root was being ignored outright. This affected the sanitized document as well as the raw one, since the filter never touched the metadata block. Second, Syft's dependencies array left roughly 70% of components with no declared relationship at all. Mend treats such a component as a direct dependency of the root rather than dropping it, so nothing was lost, but the result is a tree that is partly real and mostly inferred. That graph was already being discarded here for unrelated reasons: it encodes jar containment rather than resolved dependencies, covers ~30% of components, and carries dangling bom-refs. Discarding it without replacement still leaves the outcome to each consumer's default. It is now rebuilt as a single explicit edge -- the root depending on every component -- which says the same thing unambiguously, connects the root that Syft leaves unreferenced even in its own output, and takes the orphaned share from 70% to zero. compositions aggregate "incomplete" is what keeps that honest: a flat graph is not a resolved dependency tree and should not be read as one. This manufactures no relationships that are not there. Mend's closing point -- that scanning a packaged .deb cannot recover what only exists at build time, and that scanning the build or source with their CLI would -- matches what the top-level design already concluded, and is the subject of CP-13467. Verified on a real delphix-virtualization sidecar: root now application-typed, 0 of 416 components orphaned from the graph, still schema-valid, still zero occurrences of /opt/delphix, and Grype unchanged at 40 matches / 21 CVEs typed "binary". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The performance-diagnostics pre-push build failed with jq: error (...cdx.json:1): Cannot iterate over null (null) Error: failed command 'jq -f .../resources/sanitize-sbom.jq ...' Syft omits `components` from the document altogether -- rather than emitting an empty array -- when it finds nothing it can catalog in the scanned payload. `.components |= (group_by(...))` then runs group_by against null and dies, taking the whole package build with it. An empty array was always handled correctly; only the absent key was not, which is why this survived testing against packages that do produce components. Normalising `.components` to an empty array before the pipeline fixes it. Verified against all three shapes -- key absent, key present but empty, and a real 416-component document -- with the output schema-valid in each case. For an empty scan the result is a well-formed document with no components, an empty dependsOn, and the root still typed application. Worth noting separately that performance-diagnostics producing no components at all means its sidecar conveys nothing beyond what appliance-build's image-level scan already records. That is the same class of problem as the earlier valid-but-empty sidecars, and its SBOM_DEEP_SCAN classification is worth revisiting on that basis. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
@sebroy — summary of everything implemented since your review. Four commits, all pushed. Result on a real build (
1. Properties — filteredDone with The path leak is gone: 3,384 entries across Two properties are retained rather than removing all of them:
2. De-duplication — merging rather than keep-first1,693 → 421. Your diagnosis was right: all 8 One deliberate deviation: duplicates are merged, not reduced to the first occurrence. Syft doesn't detect the same metadata at every location — licences differ across copies in 84 groups, 3.
|
Flips the default so a pre-push build publishes the raw Syft output, to confirm the opt-out path produces the unsanitized document end to end -- roughly 4.9 MB and 1693 components with syft:metadata:virtualPath intact, rather than the 1.3 MB / 421 component sanitized form. This exists only because CYCLONEDX_FILTERING is not yet exposed as a Jenkins build parameter (TOOL-31116), so there is currently no way to pass it to a build without editing the default. REVERT THIS COMMIT BEFORE MERGE. Leaving it in would make every build publish sidecars carrying our internal /opt/delphix paths by default, which is the exact thing the sanitization was added to prevent. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Hi @sebroy , This update is for the testing where we want to avoid and sanitisation and shorting in the bom with a flag This build is ran by setting the value of the flag to false, the build is successful. For evidence , we have collected the virtualization bom from the build (where CYCLONEDX_FILTERING was set to true), The bom is ~1.3 MB - https://github.com/user-attachments/files/32803076/delphix-virtualization_2026.09.27.19_amd64.deb.cdx.json The bom is ~1.3 MB Then, from the Build where CYCLONEDX_FILTERING was set to false , https://selfservice-jenkins.eng-tools-prd.aws.delphixcloud.com/job/linux-pkg/job/develop/job/build-packages/job/pre-push/7675/consoleFull The bom is 4.7M - https://github.com/user-attachments/files/32803132/delphix-virtualization_2026.09.29.07_amd64.deb.cdx.json For reference , i have also attached both the sample bom(s). |
…topping the bom filtering, which is working fine. Changing back the CYCLONEDX_FILTERING to true
Review feedback: the term is meaningless in this context. Applied as asked in each of the three places it was raised -- the sentence "Generate a CycloneDX SBOM for each of this package's built .deb(s)" now stands on its own, "one sidecar per .deb" reads "one SBOM file per .deb", and the jq filter's header says simply "CycloneDX SBOM". Carried through the rest of the change for consistency, since leaving the word in twenty-odd other comments after calling it meaningless would be worse than not having started. That covers the remaining comments in lib/common.sh, the CI lint script, and the design doc, including its title and filename -- which now reads 2026-09-08-sbom-per-package-design.md. Where the word was doing grammatical work the sentence was reworded rather than substituted blindly, e.g. "Phase 3 associates a sidecar with its .deb" becomes "associates an SBOM with its .deb". No functional change: shellcheck and shfmt clean, and the jq filter produces a byte-identical document (416 components, schema-valid) on the same input. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Before merging this PR I want to post a final summary here, when we have the filtered and non filtered BOMs Informational: what the sanitisation changes, and what scanning both forms showed - The two forms Same package, built two days apart, with CYCLONEDX_FILTERING on and off:
The row that matters most: both dedupe to exactly 421 unique components. The 4.66 MB → 1.22 MB reduction removes repetition only — the 1,693 raw entries are 421 distinct components listed up to 17 times each. The only content The root component sanitised: { "type": "application", "name": "virtualization", ... } referenced in dependencies: yes Syft emits type: file because generate_sbom() scans an extracted directory. Mend's importer only treats metadata.component as the project root when it is application, so in the raw form the root is ignored outright — and it Mend ingestion, measured on both raw Identified 1679 out of 1693 components; relationships established for 1679 out of 1693 Relationships are now complete for everything ingested — the 70%-orphaned problem Mend support described is resolved. But Mend silently drops 11 components with Unsupported component type: APPLICATION: jq ×3, bash ×2, node, 7-Zip, The OpenSSL Toolkit, stunnel, libssp-0, mssql-jdbc_auth. It only ingests type: library. A Hierarchy built with issues warning also persists, most likely because our dependencies references all 421 refs including the 11 it discarded. On compositions: incomplete — the reason is that we can't claim the document is a complete picture, on two counts. The relationships aren't real. Syft's dependency graph was dropped (30% coverage, jar containment rather than resolved dependencies, 90 dangling refs), and what replaces it is a single synthetic edge: root depends on all 421 components. That's a placeholder, not discovered structure, so incomplete stops a consumer reading the flat tree as real hierarchy. Recovering actual relationships needs build-time resolution rather than a scan of the packaged .deb — CP-13467. The inventory isn't guaranteed either. Syft can only catalog what leaves a manifest behind in the deb, so it can never prove it found everything. For virtualization we can prove it didn't: syft finds 0 npm packages, while Mend's source scan of dlpx-app-gate finds 704 — the deb ships the built, minified Angular bundle with no package.json to resolve. So incomplete is the honest value. Declaring complete would assert something we can't support, and omitting compositions entirely leaves a consumer no way to tell a synthetic graph from a real one. |
Problem
Phase 1 (#414 + appliance-build#892, now merged) gives every image a flat, dpkg-only
CycloneDX base scan. That's correct for 3rd-party debs, but wrong for Delphix's own
first-party packages —
masking,virtualization,delphix-sso-app,containerized-masking,windows-connector,zfs,ptools,delphix-rust— whichbundle third-party components (jars, npm, wheels, Rust crates) that dpkg can't see.
Solution
SBOM_DEEP_SCAN— new per-package opt-in flag inconfig.sh, mirroring theexisting
MEND_SCAN_APPLICABLEpattern. Set"true"on the 9 packages above,"false"on the other 31.verify-sbom-scan-flag.sh) — fails if any package hasn't explicitly setthe flag either way. No silent gaps.
generate_sbom()— new stage inbuildpkg.sh/lib/common.sh. For a flaggedpackage: installs
syft/cyclonedx-cli(declared asPACKAGE_DEPENDENCIES), scansthe built
.deb(s), merges viacyclonedx-cliwhen a package emits more than one.deb(e.g.zfs), and writes<package>.cdx.jsonnext to the.debin$WORKDIR/artifacts/.artifacts/→ S3 sync already picks up thenew file, same as every other build artifact.
This is Phase 2 (CP-13465, Jira DLPX-98872) of the CycloneDX SBOM effort (Epic
CP-13455). Full design in
docs/specs/2026-09-08-sbom-per-package-sidecar-design.md.Testing done
https://selfservice-jenkins.eng-tools-prd.aws.delphixcloud.com/job/linux-pkg/job/develop/job/build-packages/job/pre-push/7648/ 🟢
SBOM_DEEP_SCANclassification (9 of 40 packages flagged)SBOM_DEEP_SCANmaskingtruevirtualizationtruedelphix-sso-apptruecontainerized-maskingtruewindows-connectortruezfstrueptoolstruedelphix-rusttrueperformance-diagnosticstruegithub.com/delphix/performance-diagnostics)All other 31 packages are explicitly
SBOM_DEEP_SCAN="false"— 3rd-party Debian forks(
linux-kernel-*,bcc,crash-python,nfs-utils, etc.) or build-host-only tooling notshipped in the product (
syft,cyclonedx-cli), per the lint's rule: 3rd-party forks /non-shipped packages set
falsesince appliance-build's image-level scan already coversthem as a flat
pkg:debcomponent.