Pipeline patterns: gates, approvals, fan-out

Why this matters

A simple pipeline runs top to bottom in one chain. On a real project that's not enough: some steps need to run in parallel, some need a human "yes", and a deploy to prod shouldn't run from just any branch. There are also steps that aren't needed on every commit: a docs fix shouldn't rebuild the app image.

There are a few stable patterns for all this. Below they're explained using GitLab CI syntax; GitHub Actions calls the same ideas by similar names.

Concepts

• Gate: a point where the pipeline stops and waits for a human decision.
• Manual job: a job with when: manual. It doesn't start on its own. In the GitLab UI it shows a Run button.
• Environment: a named place a job deploys code to, for example staging or production. Jobs link to it with the environment key.
• Approval: another person's confirmation before a deploy job runs.
• Fan-out: branch the pipeline into parallel, independent jobs. Fan-in: bring them back together into one step that waits for all the branches.
• rules: a job's list of conditions. GitLab checks them top to bottom and decides whether the job joins the pipeline and how it runs.

Manual gate: human confirmation

A gate stops the pipeline and waits for a human to press a button. The typical case: ship to staging automatically, and to prod only after a manual confirmation.

deploy-prod:
  stage: deploy
  script:
    - ./deploy.sh production
  when: manual
  allow_failure: false      # without this line the job doesn't block the pipeline
  environment:
    name: production

when: manual turns a job into a button in the UI. Whether that button blocks the pipeline depends on allow_failure, and the default here is counterintuitive.

For a job with when: manual written at the top level, allow_failure defaults to true. GitLab treats such a job as optional: an unpressed manual job counts as successful for the next stages, the pipeline keeps going and turns green. That's not a gate, just an available action. The classic surprise: someone thinks they put a safeguard in front of prod, and the pipeline finishes successfully right past it.

To make it a real gate, set allow_failure: false explicitly. Then the pipeline sits in a waiting state (Blocked in the pipeline list) and won't move on until the button is pressed.

Inside rules: the default is reversed: there, when: manual comes with allow_failure: false, so it blocks right away.

deploy-prod:
  rules:
    - if: $CI_COMMIT_BRANCH == "main"
      when: manual        # here allow_failure defaults to false, this is a gate

It's easier not to keep this in your head, and just write allow_failure explicitly in both cases. A reader of the config can then see right away whether it's a gate or an optional button. You work through this default by hand in the lab for the lesson "rules: who runs and who waits for a button" in the CI/CD practice track.

Protected environments and approvals

Not just anyone can press the button. In GitLab an environment can be made protected: the project settings say who is allowed to deploy to production, for example only maintainers. On top of that you configure deployment approvals: a deploy waits until another person approves it. By default, whoever triggered the pipeline can't approve their own deploy. This way a deploy to prod goes through review, like code.

Both features are paid in GitLab (Premium and Ultimate). In GitHub Actions the same idea is called an environment with required reviewers: up to six reviewers, one approval is enough. On the free GitHub plan this only works in public repositories.

This is also where the "deploy only from a protected branch" rule belongs. The condition if: $CI_COMMIT_BRANCH == "main" from the example above removes the deploy job from the pipelines of every other branch, and even a maintainer won't see the button from a random branch. But that's a convenience, not protection: the rule lives in .gitlab-ci.yml, and anyone who can push to their own branch can change it there. Protection comes from project settings, which you can't change from a branch. On free GitLab that means protected branches and protected variables: their values, such as the prod access key, only reach pipelines on protected branches and tags. Without the key, a job from someone else's branch can't reach prod, no matter how its rules get rewritten.

Fan-out and fan-in

Fan-out means branching the pipeline into parallel jobs, fan-in means bringing them back together. The classic case: after a build, run unit tests, a linter and a dependency audit all at once, then one shared step that waits for all of them.

Parallel jobs don't depend on each other and run at the same time, so the stage takes as long as the slowest check, not their sum. The package step starts only once all three are green.

GitLab gives you two ways to do this. The first: stages. Jobs in one stage run in parallel, and jobs in the next stage start once all jobs of the previous stages have finished successfully. So fan-out is several jobs in one stage, and fan-in is a job in the next one.

The second way: the needs key. It sets who a job waits for, directly, without looking at stages. The job starts as soon as the jobs listed in needs are ready, and it downloads artifacts only from them:

lint-docs:
  stage: test
  needs: []                 # waits for nothing, starts right away
  script:
    - ./lint-docs.sh

publish-docs:
  stage: deploy
  needs: [build-docs, lint-docs]
  script:
    - ./publish.sh site/

With needs it's easy to get fan-in wrong. If you forget one of the branches in the list, the step starts without waiting for it. If you forget the job that produced the files you need, the step starts on time, but without its artifacts. More on needs and the dependency graph: the lesson "Anatomy of a pipeline" in the CI/CD basics track.

Conditional runs

Not everything needs to run every time. rules run a job by condition: only on a tag, only in an MR, only when certain files change. The last one is done with rules:changes:

lint-chart:
  rules:
    - changes:
        - charts/**/*       # build only if something in charts/ was touched

This way the pipeline doesn't waste time on what didn't change: a chart edit doesn't trigger an app image rebuild, and the other way around.

GitLab compares path patterns with Ruby's File.fnmatch function with the FNM_PATHNAME, FNM_DOTMATCH and FNM_EXTGLOB flags. That's the source of the rules people get wrong most often:

• * matches a name within one folder and doesn't cross a /. charts/* catches charts/values.yaml, but not charts/app/values.yaml.
• ** descends into subfolders only in the form **/. charts/**/* catches files at any depth. But charts/** without a trailing /* behaves like charts/*: only files directly in charts/.
• {a,b} lists alternatives: src/*.{py,sh}.
• A pattern that starts with * goes in double quotes: "**/*.json".

One more quirk: what it's compared against. In a branch pipeline, GitLab compares the commit with that branch's previous commit. In an MR pipeline, it compares with the target branch. And in a pipeline for a new branch, a tag, a schedule, or a manual run, changes is always considered matched, and the job runs. To set the comparison base explicitly, there's rules:changes:compare_to.

Common mistakes

• when: manual at the top level of a job without allow_failure: false. The pipeline is green, the next stages ran, and nobody shipped to prod.
• Treating the rule if: $CI_COMMIT_BRANCH == "main" as protection for prod. It can be changed on any branch. Protection comes from project settings: protected branches, variables and environments.
• Checks split across separate stages even though they don't depend on each other. The pipeline takes the sum of their time instead of the slowest one.
• A fan-in step's needs is missing one of the branches, or the job that produced the files it needs.
• changes: - dir/** expecting it to cover all subfolders. It only catches files directly in dir/; you need dir/**/*.
• Expecting changes to filter a pipeline for a new branch or a tag. There it's always a match.

What you'll need for the tasks

• stages and needs: who waits for whom and who gets artifacts from whom;
• rules:changes and the pattern rules for *, **/, {a,b};
• git -C work commit -am "...": the lab runner, like GitLab, only takes what's committed;
• bash run-pipeline.sh: the lab runner in the lab folder. In each job's header it prints the start and end second, and at the start it lists the changed files and the jobs rules kept out of the pipeline;
• GitLab docs on protected environments and approvals (lab 03).