Security in the pipeline: SAST, SCA, secret scanning

Why security is a stage in CI

Finding a vulnerability or a leaked token in production is expensive. Finding it in the pipeline, before the deploy, is cheap. That's why security checks get built into CI as an ordinary stage, next to tests and the linter.

Three kinds of checks cover different risks: your own code, someone else's code in your dependencies, and secrets that accidentally ended up in the repository. Where exactly to place them in the pipeline was covered in the "Ideal pipeline" lesson: source checks go right at the start, an image scan goes between building the image and pushing it.

SAST: your code

SAST (Static Application Security Testing) is static analysis of source code for dangerous patterns: SQL injection, calling a shell with a string the user supplied, weak cryptography. The code never runs, so teams run SAST like a linter, on every push.

In GitLab, SAST plugs in through a ready-made template. The main analyzer under the hood is built on Semgrep, and the basic scan is available on every tier:

include:
  - template: Jobs/SAST.gitlab-ci.yml

If you write the job yourself, it looks like any other:

sast:
  stage: check
  script:
    - sh scripts/sast.sh
  allow_failure: true        # not blocking yet, just reporting

Soft checks and blocking checks

When SAST gets turned on for a project with history, it immediately finds dozens of old problems. If you make it blocking on day one, the pipeline turns red for everyone, and the team disables the scanner. That's why allow_failure: true goes on at the start.

What the flag does is spelled out in GitLab's docs: a failed job gets an orange warning, and the pipeline still counts as successful, the commit gets marked passed. The deploy after that pipeline proceeds as usual.

That's where the main trap is. A soft check stops nothing, and the orange badge stops getting noticed over time. Six months later the scanner is still soft, findings keep piling up, and a vulnerable library version rolls out to production without anyone blinking.

That's why allow_failure: true on a scanner is treated as temporary debt with a clear plan:

• no new findings get added, old ones get worked through the list;
• a check with no old findings, or with them already resolved, becomes blocking right away;
• every soft check has a date when the flag comes off.

Usually secret scanning and SCA become blocking first: their findings are concrete (here's the key, here's the version with the CVE) and get fixed in minutes. SAST, with its false positives, stays soft longer.

SCA: someone else's code

SCA (Software Composition Analysis) checks a project's dependencies. Most vulnerabilities come from third-party libraries. SCA reads the dependency list (requirements.txt, package-lock.json, go.sum) and checks the versions against databases of known vulnerabilities.

Example finding: requests versions below 2.32.0 contain CVE-2024-35195. If the first request in a session went out with verify=False, later requests to the same host can skip TLS certificate verification. Fixed in 2.32.0 (GHSA-9wx4-h78v-vm56). The fix is bumping the version in requirements.txt.

Image scanning belongs here too: the base image and the packages installed in it can also have vulnerabilities. The image scanning lesson covers this in more depth.

In GitLab, a ready-made Dependency Scanning exists only on the Ultimate tier. On Free, teams usually run an open-source scanner (Trivy, Grype, pip-audit, npm audit) as a separate job.

Secret scanning: leaked secrets

Secret scanning looks in the repository for what shouldn't be there: tokens, private keys, passwords. A developer commits a key, and the check catches it before it spreads across copies of the repository.

In GitLab this is also a template, available on every tier:

include:
  - template: Jobs/Secret-Detection.gitlab-ci.yml

The job is called secret_detection. By default it only looks at the push's new commits. To scan the entire repository history once (for example, right after connecting it), you set the variable SECRET_DETECTION_HISTORIC_SCAN: "true".

A leaked secret takes more than deleting it

If a key has already made it into a commit, deleting it in the next commit doesn't fix anything. The key stays in the git history, and anyone with a clone can get it with git log -p. A scanner that walks the history will find it too, even though the current files no longer have the key.

GitHub's documentation on leaked secrets says it plainly: deleting the secret from the code, pushing a new commit, or recreating the repository isn't enough. The first step is revoking the secret with whoever issued it and issuing a new one. The order is:

1. Revoke the old key and issue a new one (rotation).
2. Put the new key wherever the application reads it from: CI/CD variables, a secrets store. It never goes into the repository.
3. Deal with the history finding. Rewriting history (git filter-repo) is possible, but it breaks every clone on the team. More often, the revoked key gets marked as resolved in the scanner instead.

In gitleaks, every finding has a fingerprint, a string shaped like <commit>:<file>:<rule>:<line>. A fingerprint recorded in a .gitleaksignore file at the repository root gets skipped by the scanner from then on. You can only skip what's already revoked: an ignore entry without rotation just hides a live key.

A scanner is a process, not one tool

There are plenty of specific scanners (Trivy, Grype, Clair, Semgrep, gitleaks), and they change over time. Don't treat any single one as the final word: their vulnerability databases differ, they have false positives and false negatives, and vendors themselves have had incidents. Treat a scanner as a source of signal, not a verdict:

• work through findings and prioritize by real risk, not by the number of alerts;
• keep the scanner's databases fresh: a dependency that's clean today can get a CVE tomorrow;
• duplicate the check with a second tool on anything critical;
• remember that a green scanner doesn't mean "safe": it catches the known, not everything.

Common mistakes

• A scanner stuck in allow_failure: true forever. The pipeline is green, nobody reads the findings, a vulnerable version ships to production. Soft mode is for working through debt, and it needs a deadline.
• Deleted the secret in a commit and called it done. The key is still in the history. Until it's revoked, it still works.
• Hid the finding in ignore without revoking the key. The scanner is green, the key is live.
• Put the new key in the repository. It belongs in CI/CD variables, not in project files.
• Fixed the scanner instead of the finding. || true after the scanner's command, or editing its database, makes the pipeline green and disables the check entirely.

What you'll need for the tasks

• The difference between a soft check (allow_failure: true) and a blocking one, and what each looks like in the pipeline.
• How SCA checks a dependency's version against a database, and how the finding gets fixed.
• Why a deleted key is still in the history, and what to do about it in order: rotation, the new key into variables, mark the history finding as resolved by fingerprint.
• From the whole module: stage order and fail-fast, artifacts between jobs, a manual gate with allow_failure: false, a canary deploy before production. You'll put all of it together in the mini-project.