feat: push incomplete attestation on gate failure

[Piskoo]

Extends gated policy support in attestation add to push incomplete attestations when a gate fails. By default attestation is pushed but the user can pass --skipPushOnGateFailure flag to avoid that, attestation will still fail even if the flag is passed.

Example contract:

apiVersion: chainloop.dev/v1
kind: Contract
metadata:
  name: gate-contract
  description: Test contract with gate
spec:
  materials:
  - type: SBOM_CYCLONEDX_JSON
    name: skynet-sbom
  - type: EVIDENCE
    name: evidencetest
    optional: true
  policies:
    materials:
    - selector:
        name: skynet-sbom 
      ref: policy-gate
      gate: true
    - selector:
        name: evidencetest
      ref: policy-gate2

Example policy:

# first policy
apiVersion: workflowcontract.chainloop.dev/v1
kind: Policy
metadata:
  name: policy-gate
spec:
  policies:
    - kind: SBOM_CYCLONEDX_JSON
      embedded: |
        package main
        import rego.v1
        
        result := {"violations": ["Gated policy violation"]}

# second policy
apiVersion: workflowcontract.chainloop.dev/v1
kind: Policy
metadata:
  name: policy-gate2
spec:
  policies:
    - kind: EVIDENCE
      embedded: |
        package main
        import rego.v1
        
        result := {"violations": ["Gated policy violation"]}

Attestation process:

$ chainloop att init --project myproject --workflow multipolicy --contract contractgate.yaml --replace
WRN API contacted in insecure mode
This command will run against the organization "myorg"
Please confirm to continue y/N
y
INF Attestation initialized! now you can check its status or add materials to it
┌───────────────────────────┬──────────────────────────────────────┐
│ Initialized At            │ 17 Dec 25 13:07 UTC                  │
├───────────────────────────┼──────────────────────────────────────┤
│ Attestation ID            │ f7665d90-1716-4002-831a-66aa5364da68 │
│ Organization              │ myorg                                │
│ Name                      │ multipolicy                          │
│ Project                   │ myproject                            │
│ Version                   │ v1.63.0+next (prerelease)            │
│ Contract                  │ myproject-multipolicy (revision 1)   │
│ Policy violation strategy │ ADVISORY                             │
└───────────────────────────┴──────────────────────────────────────┘
┌────────────────────────────────┐
│ Materials                      │
├──────────┬─────────────────────┤
│ Name     │ evidencetest        │
│ Type     │ EVIDENCE            │
│ Set      │ No                  │
│ Required │ Yes                 │
├──────────┼─────────────────────┤
│ Name     │ skynet-sbom         │
│ Type     │ SBOM_CYCLONEDX_JSON │
│ Set      │ No                  │
│ Required │ Yes                 │
└──────────┴─────────────────────┘

$ chainloop att add --value sbom.json --name skynet-sbom
WRN API contacted in insecure mode
INF uploading sbom.json - sha256:bfbb8312c63447567e65f128ac05ddaebf562d072532b37fb412f47bfc32a421
INF material added to attestation
┌────────────────────┬─────────────────────────────────────────────────────────────────────────┐
│ Name               │ skynet-sbom                                                             │
├────────────────────┼─────────────────────────────────────────────────────────────────────────┤
│ Type               │ SBOM_CYCLONEDX_JSON                                                     │
├────────────────────┼─────────────────────────────────────────────────────────────────────────┤
│ Required           │ Yes                                                                     │
├────────────────────┼─────────────────────────────────────────────────────────────────────────┤
│ Value              │ sbom.json                                                               │
├────────────────────┼─────────────────────────────────────────────────────────────────────────┤
│ Digest             │ sha256:bfbb8312c63447567e65f128ac05ddaebf562d072532b37fb412f47bfc32a421 │
├────────────────────┼─────────────────────────────────────────────────────────────────────────┤
│ Policy evaluations │ ------                                                                  │
├────────────────────┼─────────────────────────────────────────────────────────────────────────┤
│                    │ policy-gate (gate): Gated policy violation                              │
└────────────────────┴─────────────────────────────────────────────────────────────────────────┘
INF Gated policy "policy-gate" failed during material add, pushing attestation
INF push completed
ERR the policy "policy-gate" is configured as a gate and has violations
exit status 4

Workflow run:

Attestation:

{
  "_type": "https://in-toto.io/Statement/v1",
  "subject": [
    {
      "name": "chainloop.workflow.multipolicy",
      "digest": {
        "sha256": "e21f493f5f544747bcb234efee3036f094347c83129d18110c08f5103aafdb66"
      }
    },
    {
      "name": "git.head",
      "digest": {
        "sha1": "0a582a0f9e8c90596a41a0b5274b62886518c5ff"
      },
      "annotations": {
        "author.email": "sylwesterpiskozub@gmail.com",
        "author.name": "Sylwester Piskozub",
        "date": "2025-12-17T14:30:37Z",
        "message": "log msg fix\n\nSigned-off-by: Sylwester Piskozub <sylwesterpiskozub@gmail.com>\n",
        "remotes": [
          {
            "name": "origin",
            "url": "https://github.com/Piskoo/chainloop.git"
          },
          {
            "name": "upstream",
            "url": "https://github.com/chainloop-dev/chainloop.git"
          },
          {
            "name": "javirln",
            "url": "git@github.com:javirln/chainloop.git"
          }
        ],
        "signature": "-----BEGIN PGP SIGNATURE-----\n\niQGzBAABCgAdFiEE80ZHNM+xgBJZNs2RI98bOQEY/BgFAmlCvo0ACgkQI98bOQEY\n/BhBTQv+IM/MIY9rtEOAL4dMrNox9bxCHycbu5hq0mYBhnvJrIXo0D7XeCGiwHqA\n8riXOJUQnYAROCp99Z+O4LkvmbvMWknDybTWmHWPt53FFp3Sl12gQkXgKlLuJYoP\nPmGtlJU5oYbDlY1zW378IpB/8rqWSdbOu7tcT6VVDFPBrpuJ6igwELo5WoTesfGS\nSjIB1XhpoHU7jvIaMFNWlhVvaWaxAh1z74sBjT2bMubewY2QovP+EmPlRQe+gO6x\noO478Uwq+GWwnrD8RA2HXuvVf74MkFIrHpsolpuClX8SbfPfD+efAFRmeYO+LN1h\n51b7UE/TMAcZKw/U9IhyckonzaNhKlnD8cpd0LvuGyMp4Lm65jWBULzKNriWS9Pk\nDZsIS/y7dprKDfZWEk/QNbuWK6UOmPfNJTiZ7mBQPpneymcfzmPwMVrifTxjM5mL\nyVbZhNAZKEHZ47dzd9oHCKKm9iv1ewgH93VJpGfygv3olixdSGOeoEn6rEKuNy1A\nBdIrvZTZ\n=NMKR\n-----END PGP SIGNATURE-----\n"
      }
    }
  ],
  "predicateType": "chainloop.dev/attestation/v0.2",
  "predicate": {
    "annotations": {
      "environment": ""
    },
    "auth": {
      "id": "34030166-826c-4c72-a635-353230fedb47",
      "type": "AUTH_TYPE_USER"
    },
    "buildType": "chainloop.dev/workflowrun/v0.1",
    "builder": {
      "id": "chainloop.dev/cli/@"
    },
    "materials": [
      {
        "annotations": {
          "chainloop.material.cas": true,
          "chainloop.material.name": "skynet-sbom",
          "chainloop.material.sbom.main_component.name": "my application",
          "chainloop.material.sbom.main_component.type": "application",
          "chainloop.material.sbom.main_component.version": "1.0",
          "chainloop.material.type": "SBOM_CYCLONEDX_JSON"
        },
        "digest": {
          "sha256": "bfbb8312c63447567e65f128ac05ddaebf562d072532b37fb412f47bfc32a421"
        },
        "name": "sbom.json"
      }
    ],
    "metadata": {
      "contractName": "myproject-multipolicy",
      "contractVersion": "2",
      "finishedAt": "2025-12-17T14:38:27.537415106Z",
      "initializedAt": "2025-12-17T14:38:23.267223921Z",
      "name": "multipolicy",
      "organization": "myorg",
      "project": "myproject",
      "projectVersion": "v1.63.0+next",
      "projectVersionPrerelease": true,
      "team": "",
      "workflowID": "58356fd5-de48-40e5-902a-b49277cff1b2",
      "workflowName": "multipolicy",
      "workflowRunID": "b1bfaa19-50bb-4c0e-b721-c117da5d4915"
    },
    "policyAttBlocked": false,
    "policyBlockBypassEnabled": true,
    "policyCheckBlockingStrategy": "ADVISORY",
    "policyEvaluations": {
      "skynet-sbom": [
        {
          "materialName": "skynet-sbom",
          "name": "policy-gate",
          "policyReference": {
            "annotations": {
              "name": "policy-gate",
              "organization": "myorg"
            },
            "digest": {
              "sha256": "1782d1682c379ae2de2080f3f165d90804127c683b3460dd358d8632c3d4c6fe"
            },
            "name": "policy-gate",
            "uri": "chainloop://localhost:8002/policy-gate?org=myorg"
          },
          "skipped": false,
          "type": "SBOM_CYCLONEDX_JSON",
          "violations": [
            {
              "message": "Gated policy violation",
              "subject": "policy-gate"
            }
          ]
        }
      ]
    },
    "policyHasViolations": true,
    "runnerEnvironment": "unknown",
    "runnerType": "RUNNER_TYPE_UNSPECIFIED",
    "signingCA": "fileCA"
  }
}

Attestation contains only partial evaluations, policy policy-gate2 for kind EVIDENCE wasn't evaluated as attestation failed before material evidencetest was provided.
Attestation validation and annotation validation is disabled for incomplete attestation push.

[jiparis]

Thanks @Piskoo. I'd like us to evaluate the consequences of pushing a partial attestation. In particular:

• if there are mandatory materials in the contract that haven't been added to the attestation, the push operation should fail.
• other policies configured in the contract that won't get evaluated. How will they look like in the final attestation?
• automatic push cannot be done in those scenarios where a private key is used for signing (att push executed with the --key parameter), since we don't know which signing key is used during att add. We'd need to detect if chainloop is configured with a keyless provider.

[Piskoo]

if there are mandatory materials that haven't been added to the attestation, the push operation should fail.

Should we do the same with required annotations?

other policies configured in the contract that won't get evaluated. How will they look like in the final attestation?

They won't be present as they weren't evaluated

automatic push cannot be done in those scenarios where a private key is used for signing (att push executed with the --key parameter), since we don't know which signing key is used during att add. We'd need to detect if chainloop is configured with a keyless provider.

Makes sense, thanks

[Piskoo]

Updated, we now fail when

1. Any of the required materials is missing

INF gated policy "policy-gate" failed during material add, pushing attestation
WRN failed to push attestation: pushing attestation: some materials have not been crafted yet: evidencetest
ERR the policy "policy-gate" is configured as a gate and has violations
exit status 4

2. Runtime annotation is required

INF gated policy "policy-gate" failed during material add, pushing attestation
WRN failed to push attestation: pushing attestation: annotations ["environment"] required
ERR the policy "policy-gate" is configured as a gate and has violations
exit status 4

3. Keyless signing is not enabled

INF gated policy "policy-gate" failed during material add, pushing attestation
WRN failed to push attestation: keyless signing not configured
ERR the policy "policy-gate" is configured as a gate and has violations
exit status 4

[Piskoo]

I'm gonna close this one for now, we can discuss it