Extend SCA features coverage by supporting 3rd party SBOM documents (#14760) · Epics · GitLab.org

Extend SCA features coverage by supporting 3rd party SBOM documents

## Executive Summary As several GitLab features have transitioned over using SBOM documents as their source of data, this creates an opportunity for users to bring their own SBOM and expand the coverage we offer by default. For instance, using Python 3.9 in a project is not supported and GitLab can't generate an SBOM document for it. By using a 3rd party SBOM generator this support could be brought back and all downstream features enabled. The transition is still ongoing and SBOM support has specific requirements and some limitations that we should highlight and address to improve the user experience. ## Requirements 1. A valid CycloneDX SBOM (1.4, 1.5, or 1.6) from any standard generator ingests without requiring GitLab's custom taxonomy properties. No gitlab:meta:schema_version, no input_file:path, nothing GitLab-specific. 2. At GA, GitLab processes component.type = library components. Other component types are not currently supported and will be documented as such, they should not silently error or disappear. 3. The standard CycloneDX scope field (required, optional, excluded) could be read and used for production versus development dependency categorization. 4. CycloneDX license formats SPDX ID and SPDX expression are handled, including `LicenseRef-[NAME]` for custom licenses. 1. Note that PMDB enrichment for license expressions is still in progress; ingestion works but enrichment from the package metadata database is not yet complete. 5. Components appear in the Dependency List with name, version, license, and package manager. Package manager is inferred by mapping PURL types to their corresponding package managers. 6. License scanning works for any component with a supported PURL type. 7. Continuous vulnerability scanning automatically re-evaluates a 3rd party SBOM's components when the advisory database updates, for any component with a supported PURL type. 8. Vulnerability findings from a 3rd party SBOM surface in the merge request security widget, pipeline security tab, and vulnerability report. If a component's PURL type is not supported, a clear message explains why rather than silently producing no results. Where unsupported PURL warnings are aggregated and surfaced for SBOMs ingested on the default branch outside of a merge request is an open UX question, the MR widget is confirmed, the post-merge experience needs UX input. 9. CVS and vulnerability findings work only for PURL types covered by GitLab's advisory database. Unsupported PURL types are surfaced to the user explicitly. 10. Syft, Trivy, cyclonedx-cli, and Snyk-generated SBOMs are tested end-to-end and documented as validated generators, scoped to language-level components of type library. 11. As part of implementation, produce a field-level audit defining what GitLab does with each relevant CycloneDX field: reads and uses it, reads and ignores it, or does not handle it. This becomes part of the GA documentation. ## Open questions 1. The `input_file_path` problem is more nuanced than we originally thought. For 3rd party SBOM ingestion specifically, no DB migration is needed just to ingest the SBOM because no vulnerabilities are created from it in the current flow. The fingerprint collision problem - defaulting `input_file_path` to project root is equivalent to not setting it at all, which would cause components with the same version across different lockfiles to incorrectly collapse into a single vulnerability. Oscar flagged a potential middle path: Trivy and Grype already emit properties equivalent to `input_file_path` that could serve as fallbacks, avoiding the need to touch the fingerprint at all. Parser complexity is unknown and needs to be scoped in a spike. The Vulns Across Contexts UUID refactor (issue #587624) is confirmed as a likely prerequisite. 2. Where unsupported PURL type warnings are surfaced for SBOMs ingested on the default branch outside of a merge request is an open UX question. #### Engineering Assessment ### Proposal Start with 1. Identify limitations and create corresponding issue(s) to address them 2. Document 3rd party SBOM support in the relevant pages, with current requirements and limitations (link to issues) 3. Provide a tutorial doc? Ideally we should be able to: * Parse the most common 3rd party SBOM reports without the need of gitlab-taxonomy * CVS should work if the `purl type` is supported * Find CycloneDX fields (even if they are optional) that we could require in order to get the metadata that we need. For example `input_file`. If this is not possible we need to identify ways to provide the `input_file` metadata. Maybe point to the project root? * SBOM ingestion should work for components related to both OS and language PMs. So for both CS and DS. Example project: https://gitlab.com/gitlab-org/secure/tests/olivier/bring-your-own-sbom ### Audit :warning: WIP :warning: This will be updated as we make progress on evaluating 3rd party SBOM support and might be incomplete for now. Some features may work directly with 3rd party SBOM document as far as it meets these [basic requirements](https://docs.gitlab.com/ee/user/application_security/dependency_list/#prerequisites): > - Comply with [the CycloneDX specification](https://github.com/CycloneDX/specification) version `1.4` or `1.5`. > - Be uploaded as [a CI/CD artifact report](../../../ci/yaml/artifacts_reports.md#artifactsreportscyclonedx) from a successful pipeline on the default branch. These will be referred to as "basic requirements" to reduce verbosity. On the other hand, some features have additional requirements for the SBOM document to meet but also some external constraints. #### 1. [Dependency List](https://docs.gitlab.com/ee/user/application_security/dependency_list/) - 3rd party support with basic requirements: :warning: - Known limitations: 1. vulnerability information on that page won't be visible unless the corresponding security scanning features are enabled. 2. the `packager` information is provided by the `gitlab:dependency_scanning:package_manager:name` property from the the [GitLab CycloneDX taxonomy](https://docs.gitlab.com/ee/development/sec/cyclonedx_property_taxonomy.html). SBOM document without that property will have their component showing up with an `unknown` package manager. #### 2. [License Scanning](https://docs.gitlab.com/ee/user/compliance/license_scanning_of_cyclonedx_files/) - 3rd party support with basic requirements: :white_check_mark: - Additional requirements: 1. the PURL type of the components must be explicitely supported by having a corresponding source for license data: https://docs.gitlab.com/ee/user/compliance/license_scanning_of_cyclonedx_files/#data-sources #### 3. [Continuous Vulnerability Scanning](https://docs.gitlab.com/ee/user/application_security/continuous_vulnerability_scanning/) - 3rd party support with basic requirements: :x: - Additional requirements: 1. the SBOM document must include and comply with the [GitLab CycloneDX property taxonomy](https://docs.gitlab.com/ee/development/sec/cyclonedx_property_taxonomy.html). 2. the PURL type of the components must be explicitely supported: https://docs.gitlab.com/ee/user/application_security/continuous_vulnerability_scanning/#supported-package-types #### 4. [(WIP) Vulnerability Scanning of CycloneDX files](https://gitlab.com/groups/gitlab-org/-/epics/8026) (CVS on SBOM changes) - 3rd party support with basic requirements: :x: - Additional requirements: 1. the SBOM document must include and comply with the [GitLab CycloneDX property taxonomy](https://docs.gitlab.com/ee/development/sec/cyclonedx_property_taxonomy.html). 2. the PURL type of the components must be explicitely supported: https://docs.gitlab.com/ee/user/application_security/continuous_vulnerability_scanning/#supported-package-types 5. Same as 1,2,3,4 but for Container Scanning SBOMs. Notice that currently SBOM ingestion might be defirentiating between DS and CS SBOMs. In reality we should treat those packages in the same way. ### Known improvements and workaround - https://gitlab.com/gitlab-org/gitlab/-/issues/474906+ - https://gitlab.com/groups/gitlab-org/-/epics/8089#note_2023490569+ - https://gitlab.com/gitlab-org/gitlab/-/merge_requests/160270#note_2030632537 - https://gitlab.com/gitlab-org/gitlab/-/issues/479616#note_2064892537 #### Dependencies - Team dependencies: ~"group::composition analysis" ~"group::security insights" - Epic/Issue dependencies - _Link to dependent epics/issues via the linked items widget below for ease of drill down_ - External dependencies: NA #### DRIs - **PM**: @joelpatterson  - **EM**: @nilieskou  - **UX/PDM**: \[Name\]  - **Group(s)**: ~"group::composition analysis"  - **Engineering Owner**: @rvider #### Initiative Driver - Product or Engineering? - [ ] **Product-driven initiatives (P1/P2/P3)** - Customer-facing features or improvements driven by Product teams that require engineering resources and commitment - These initiatives require a Product Priority label (P1/P2/P3) - They may also receive GTM tier labels (T1/T2/T3) for external communication - [ ] **Engineering-driven initiatives (E1/E2/E3)** - Internal technical improvements that may not have customer-facing components - These initiatives require an Engineering Priority label (E1/E2/E3) - They have internal visibility only and are not externally communicated - Examples include: technical debt reduction, infrastructure improvements, refactoring, dependency upgrades #### Sizing and Funding (Optional) - **Size**: \[XS/S/M/L/XL\] - **Funding Status**: \[Funded/Partially funded/Not funded\] --- ### Hygiene Guidelines :bulb: \_See additional details about this process at https://handbook.gitlab.com/handbook/product-development/r-and-d-interlock/ ##### :one: Pre-Interlock - [x] Update epic description with all relevant information - [ ] Ensure all dependencies are identified - [ ] Apply appropriate labels (see below) - [ ] Apply target delivery Milestone - [ ] Update interlock status as discussions progress (via label) ##### :two: Post-Interlock: once quarter begins - Update health status weekly (via label) - Document any newly identified risks or dependencies - Link to implementation epics/issues as work begins - Flag any scope or timeline changes immediately

epic