Repository navigation
Merging objects with non-unique bom_refs silently introduces data inconsistency #807
Description
NB: this is an extension of #540 and #677
Given the testdata in #677, set up a test for #540 but with clashing bom_refs and unique names instead:
>>> from json import load
>>> from cyclonedx.model.bom import Bom
>>> from cyclonedx.model.component import Component, ComponentType
>>> root_component = Component(
... type=ComponentType.FRAMEWORK,
... name="root 42",
... version="42",
... bom_ref="root",
... )
>>>
>>> bom = Bom()
>>> bom.metadata.component = root_component
>>> bom.validate()
True
>>> len(bom.components)
0
>>> len(bom.dependencies)
1
>>>
>>> with open('out.json', mode="r") as testfile:
... json_bom = load(testfile)
...
>>> len(json_bom['components'])
4
>>> len(json_bom['dependencies'])
3
>>>
>>> loaded_bom = Bom.from_json(json_bom)
>>> loaded_bom.validate()
True
>>> len(loaded_bom.components)
4
>>> len(loaded_bom.dependencies)
5
>>>
>>> loaded_bom.metadata.component
<Component bom-ref=<BomRef 'root' id=139844009673936>, group=None, name=test, version=None, type=ComponentType.APPLICATION>
>>> bom.metadata.component
<Component bom-ref=<BomRef 'root' id=139844025859088>, group=None, name=root 42, version=42, type=ComponentType.FRAMEWORK>
>>>
>>> loaded_bom.metadata.component.bom_ref
<BomRef 'root' id=139844009673936>
>>> bom.metadata.component.bom_ref
<BomRef 'root' id=139844025859088>
>>>
>>> bom.metadata.component.bom_ref == loaded_bom.metadata.component.bom_ref
True
>>> loaded_bom.metadata.component == bom.metadata.component
False
>>>
>>> bom.components.add(loaded_bom.metadata.component)
>>> bom.register_dependency(root_component, [loaded_bom.metadata.component])
>>> bom.components |= loaded_bom.components
>>> bom.dependencies |= loaded_bom.dependencies
>>>
>>> bom.validate()
True
>>>
>>> len(bom.components)
5
>>> len(bom.dependencies)
6Note that the Bom says it is valid, while it is actually not - although this is not apparent at a glance.
When outputting to json (e.g. with the reported JsonV1Dot5), dependencies is shown to be as follows:
[
{"ref": "root", "dependsOn": ["BomRef.23942051401152453.4412489734431171"]},
{"ref": "root", "dependsOn": ["test11", "test21"]},
{"ref": "test11", "dependsOn": ["test12"]},
{"ref": "test12"},
{"ref": "test21", "dependsOn": ["test22"]},
{"ref": "test22"}
]This is actually invalid and would not validate when dumped to file and re-read by the library
Activity
wkoot commentedon Apr 14, 2025
The problem is a lot more clear when merging multiple files, as it seems to compound in some manner.
A second testfile with the same structure, but modified (non-clashing)
bom-refcomponents:This provides the following
dependenciesjson output:[ {"ref": "root", "dependsOn": ["BomRef.19783006151644178.9868708899859131"]}, {"ref": "root", "dependsOn": ["test11", "test21"]}, {"ref": "root", "dependsOn": ["test411", "test421"]}, {"ref": "test11", "dependsOn": ["test12"]}, {"ref": "test12"}, {"ref": "test21", "dependsOn": ["test22"]}, {"ref": "test22"}, {"ref": "test411", "dependsOn": ["test412"]}, {"ref": "test412"}, {"ref": "test421", "dependsOn": ["test422"]}, {"ref": "test422"} ]Because the first dependency object is the
root_component, the entire dependency tree is actually disconnected from root.It seems to happen when
register_dependencyis called on the root object, which isn't a member of the bom components.Note that the root
BomRefstring seems to be regenerated each time the json output is generated, thisDependsOnis not stable.Inspecting the data object shows that dependencies were not correctly linked (see
targets), although thebom_refis correctly stored:jkowalleck commentedon Apr 14, 2025
Are you aware, that per CycloneDX specification, Bom-refs must be unique per document?
This library IS aware of it, and it is actually enforced by assigning unique values when serializing Json/xml.
Furthermore, since you are struggling with merging again, you maybe want to try out https://github.com/CycloneDX/cyclonedx-cli and read CycloneDX/specification#320
wkoot commentedon Apr 14, 2025
Yes, I agree. The problem arises with merging of multiple documents, which are all valid on their own, but may contain duplicates.
I will have a look again at the
cyclonedx-cliimplementation, but CycloneDX/cyclonedx-cli#399 is still unaddressed.The examples above show that this library is not correctly assigning unique values, and doesn't throw warning/exception.
For instance, the dependency "root" is mentioned three times in the json output and the backrefs are not remembered.
The metadata component ref is probably the only value that does not need to be altered for the structure to remain valid.
However, every call to
.validate()is stating that the produced Bom is valid - while the library recognises that the data is not unique.Producing json output also runs an extra validation, which does not recognise that the produced output is wrong.