From f1321bd30fb83935f61495e0ec4ac9803c2f4631 Mon Sep 17 00:00:00 2001 From: northdpole Date: Fri, 15 Aug 2025 22:26:01 +0100 Subject: [PATCH] Remove "file://" component references from public docs A benefit of smithy is that you don't need the source code of every single connector in order to run it. If our docs have workflows that only reference file:// type components, then our users can't benefit from this. This gets increasingly frustrating as the repo gets larger, especially since we're vendoring components. This commit rewrites docs to replace file:// type references to components with ghcr:// type references to components. It also slightly cleans up the 'how to write a workflow' piece of the open source docs explaining the difference between file:// component references used mostly for development and 'ghcr://' type component references used for day to day operations. It also documents how the component registry works and how to use it. --- docs/oss/writing-workflows.md | 78 ++++++++++++++----- docs/reference/components/dependency-track.md | 76 ------------------ docs/reference/components/sonarqube.md | 49 +----------- .../components/source-code-artifact.md | 4 +- 4 files changed, 63 insertions(+), 144 deletions(-) delete mode 100644 docs/reference/components/dependency-track.md diff --git a/docs/oss/writing-workflows.md b/docs/oss/writing-workflows.md index 5611b58..d495275 100644 --- a/docs/oss/writing-workflows.md +++ b/docs/oss/writing-workflows.md @@ -4,19 +4,29 @@ description: "Build your first Smithy Workflow" sidebar_position: 3 --- -There are several examples in +In order to write a Smithy workflow all you need to do is define the collection of components to run in order and the arguments to pass to those components. +You then supply the two files to the `smithyctl` binary and that's it! + + +Unless you're developing components, you don't need to clone the open source repository in order to run a workflow. + +For end-user usage, you can reference components using their OCI manifests. + +*Hint: If you're developing components you can reference local component.yaml files using a `file://` protocol and `smithyctl` will assemble the component itself. There are several examples of workflows using the `file://` protocol in the [/examples folder](https://github.com/smithy-security/smithy/tree/main/examples/). +* -In this tutorial we are going to create our own one.\ -We assume that you have already cloned the repository and you have installed -`smithyctl` by running +In this tutorial we are going to create our own self-contained workflow that uses the Smithy component registry.\ +*We assume that you have installed `smithyctl` by running* ```bash go install github.com/smithy-security/smithy/smithyctl@latest ``` +## Prerequisites + Check where your smithyctl is, e.g. in `~/go/bin/smithyctl`. -You can find if smithyctl is in your path by running `smithyctl version` +You can find if smithyctl is in your path by running `smithyctl version`. If this command returns an error or `command not found` you need to find where Go installs binaries in your system. This is usually pointed to by the `GOPATH` variable which you can print by running `go env` ## TL;DR @@ -28,27 +38,24 @@ to run, `overrides.yaml` contain the arguments to each component. ## Create the Workflow -Let's assume that we want to run SAST and SCA for a repository, which contains -code written in Go. -Since we are scanning Go it makes sense to also enrich the results by addins a -custom annotation. +Let's assume that we want to run SAST and SCA for a repository, which contains code written in Go. +Since we are scanning Go it makes sense to also enrich the results by addins a custom annotation. ### Write the workflow We can compose this workflow by writing the following `workflow.yaml` somewhere -in our filesystem: +in our filesystem, e.g. in `$HOME/smithy-workflow.yaml`: ```yaml --- name: my-custom-workflow description: some user friendly description components: - - component: file://components/targets/git-clone/component.yaml - - component: file://components/scanners/gosec/component.yaml - - component: file://components/scanners/nancy/component.yaml - - component: file://components/enrichers/custom-annotation/component.yaml - - component: file://components/reporters/json-logger/component.yaml - + - component: ghcr.io/smithy-security/smithy/manifests/components/targets/git-clone:v1.5.1 + - component: ghcr.io/smithy-security/smithy/manifests/components/scanners/gosec:v1.4.1 + - component: ghcr.io/smithy-security/smithy/manifests/components/scanners/nancy:v1.3.2 + - component: ghcr.io/smithy-security/smithy/manifests/components/enrichers/custom-annotation:v0.2.1 + - component: ghcr.io/smithy-security/smithy/images/components/reporters/vulnerability-logger:v0.0.1 ``` In this file: @@ -68,6 +75,8 @@ relevant values. For our workflow we can use the following `overrides.yaml`: ```yaml +# $HOME/smithy-overrides.yaml + git-clone: - name: "repo_url" type: "string" @@ -86,8 +95,41 @@ Run the pipeline using smithyctl: ```bash smithyctl workflow run \ - --overrides=./examples/bandit/overrides.yaml \ - --build-component-images=true ./examples/bandit/workflow.yaml + --overrides=./$HOME/smithy-iverrides.yaml \ + --build-component-images=true ./$HOME/smithy-workflow.yaml ``` You should see the workflow results in your shell. + + +## Modifying the Workflow and working with the Smithy Registry + +Each component in Smithy is individually packaged and versioned. +This allows Smithy components to evolve individually and users to fork and customize components as needed. + +A component is an OCI package with a `component.yaml` manifest pointing to an image and metadata. + +Smithy uses the Github Container Registry as storage for all its manifests. You can search for components by name [here](https://github.com/orgs/smithy-security/packages?q=vuln&tab=packages&q=). + +When you reference a component, be sure to grab the ["manifest"](https://github.com/smithy-security/smithy/pkgs/container/smithy%2Fmanifests%2Fcomponents%2Ftargets%2Fgit-clone) and not the ["image"](https://github.com/smithy-security/smithy/pkgs/container/smithy%2Fimages%2Fcomponents%2Ftargets%2Fgit-clone) of the component. + +### Example + +Assuming we want to modify our workflow above and instead of running `gosec` we want to run `semgrep`. This is easily done in 2 steps: + +* Find the latest `semgrep` component version from the [registry](https://github.com/orgs/smithy-security/packages?q=git-clone&tab=packages&q=semgrep), at the time of writing that's v1.3.2 +* Substitute the `gosec` component with `semgrep` in the workflow + +Done and now our worklow looks like this: + +```yaml +--- +name: my-custom-workflow +description: some user friendly description +components: + - component: ghcr.io/smithy-security/smithy/manifests/components/targets/git-clone:v1.5.1 + - component: ghcr.io/smithy-security/smithy/manifests/components/scanners/semgrep:v1.3.2 + - component: ghcr.io/smithy-security/smithy/manifests/components/scanners/nancy:v1.3.2 + - component: ghcr.io/smithy-security/smithy/manifests/components/enrichers/custom-annotation:v0.2.1 + - component: ghcr.io/smithy-security/smithy/images/components/reporters/vulnerability-logger:v0.0.1 +``` diff --git a/docs/reference/components/dependency-track.md b/docs/reference/components/dependency-track.md deleted file mode 100644 index 94ddc05..0000000 --- a/docs/reference/components/dependency-track.md +++ /dev/null @@ -1,76 +0,0 @@ ---- -sidebar_custom_props: - icon: "/img/components/dependency-track.svg" -title: 'Dependency-Track' -description: 'Dependency-Track reporter that uploads CycloneDX SBOMs to Dependency-Track and transforms vulnerabilities to OCSF' -sidebar_position: 18 ---- - -# Dependency-Track - -Dependency-Track reporter that uploads CycloneDX SBOMs to Dependency-Track and transforms the found vulnerabilities to OCSF format. - -## How to use - -### Open-Source - -1. Add the component to the workflow: - -```yaml -# file ./my-workflow/workflow.yml -description: Dependency-Track based workflow -name: dependency-track -components: - - component: ghcr.io/smithy-security/smithy/manifests/components/targets/git-clone:v1.4.0 - - component: ghcr.io/smithy-security/smithy/manifests/components/scanners/cdxgen:v1.0.0 - - component: file://components/reporters/dependency-track/component.yaml - - component: ghcr.io/smithy-security/smithy/manifests/components/reporters/json-logger:v1.0.2 -``` - -2. Configure the run parameters of the component in the overrides file: - -```yaml -# file: ./my-workflow/overrides.yaml -git-clone: - - name: "repo_url" - type: "string" - value: "https://github.com/example/my-project.git" - - name: "reference" - type: "string" - value: "main" -dependency-track: - - name: "dependencytrack_base_url" - type: "string" - value: "https://your-dependency-track-instance.com" - - name: "dependencytrack_api_token" - type: "string" - value: "your-api-token-here" - - name: "project_name" - type: "string" - value: "my-project" - - name: "project_version" - type: "string" - value: "1.0.0" - - name: "sbom_file_path" - type: "string" - value: "/path/to/sbom.json" -``` - -### SaaS - -1. In the Smithy UI, open the page to create a new workflow. -2. Find the Dependency-Track component in the Reporters dropdown. -3. Fill the form on the right - -## Options - -You can configure this component with the following options. The options that -have a default value are optional: - -| Option Name | Description | Default | Type | -|--------------------------------------|-------------------------------------------------------------------------------------------------------|---------|--------| -| **\[Required]** dependencytrack\_base\_url | The base URL to your Dependency-Track instance. | | String | -| **\[Required]** dependencytrack\_api\_token | The API token for authenticating with your Dependency-Track instance. | | String | -| **\[Required]** project\_name | The name of the project in Dependency-Track where the SBOM will be uploaded. | | String | -| **\[Required]** project\_version | The version of the project in Dependency-Track where the SBOM will be uploaded. | | String | -| **\[Required]** sbom\_file\_path | The file path to the CycloneDX SBOM file that will be uploaded to Dependency-Track. | | String | \ No newline at end of file diff --git a/docs/reference/components/sonarqube.md b/docs/reference/components/sonarqube.md index c20bebd..b80bedd 100644 --- a/docs/reference/components/sonarqube.md +++ b/docs/reference/components/sonarqube.md @@ -8,57 +8,10 @@ sidebar_position: 17 # SonarQube -SonarQube scanner that uses SonarQube Cloud Edition to generate findings. +SonarQube scanner that uses SonarQube Cloud Edition to generate findings. This is a SaaS-Only component ## How to use -### Open-Source - -1. Add the component to the workflow: - -```yaml -# file ./my-workflow/workflow.yml -description: Sonarqube based workflow -name: sonarqube -components: - - component: ghcr.io/smithy-security/smithy/manifests/components/targets/git-clone:v1.4.0 - - component: file://components/scanners/sonarqube/component.yaml - - component: ghcr.io/smithy-security/private-components/manifests/components/enrichers/exploit-exists:v0.0.3 - - component: hcr.io/smithy-security/smithy/manifests/components/reporters/json-logger:v1.0.2 -``` - -2. Configure the run parameters of the component in the overrides file: - -```yaml -# file: ./my-workflow/overrides.yaml -git-clone: - - name: "repo_url" - type: "string" - value: "https://github.com/sqreen/go-dvwa.git" - - name: "reference" - type: "string" - value: "master" -sonarqube: - - name: "auth_token" - type: "string" - value: "xxx" - - name: "project_key" - type: "string" - value: "xxx" - - name: "project_name" - type: "string" - value: "xxx" - - name: "organization" - type: "string" - value: "xxx" - - name: "target_repository_url" - type: "string" - value: "https://github.com/sqreen/go-dvwa.git" - - name: "target_repository_reference" - type: "string" - value: "master" -``` - ### SaaS 1. In the Smithy UI, open the page to create a new workflow. diff --git a/docs/reference/components/source-code-artifact.md b/docs/reference/components/source-code-artifact.md index 4aaa6e4..3809fd9 100644 --- a/docs/reference/components/source-code-artifact.md +++ b/docs/reference/components/source-code-artifact.md @@ -46,7 +46,7 @@ gs://my-bucket/my-archive.tar.gz description: Downloads and unpacks a repository and runs go scanners on it name: source-code-artifact components: - - component: file://components/targets/source-code-artifact/component.yaml + - component: ghcr.io/smithy-security/smithy/manifests/components/targets/source-code-artifact:v0.1.0 - component: ghcr.io/smithy-security/smithy/manifests/components/scanners/gosec:v1.4.0 - component: ghcr.io/smithy-security/smithy/manifests/components/enrichers/custom-annotation:v0.2.0 - component: ghcr.io/smithy-security/smithy/manifests/components/reporters/json-logger:v1.1.0 @@ -100,4 +100,4 @@ You can configure this component with the following options: Check out guidance [here](https://github.com/smithy-security/smithy/tree/main/components/targets/source-code-artifact) -to correctly set-up the options. \ No newline at end of file +to correctly set-up the options.