Artifacts: passing files between stages

Why a file from build isn't visible in test, and how artifacts solve that problem.

Why this matters

You built the site in the build stage, and the test in the test stage says "no such file." Permissions are fine, the path is right, but the file really isn't there. This is the most common mistake in a first GitLab CI pipeline, and it's fixed with a single artifacts section. In this lesson you'll see where the problem comes from, how to pass files on, and where to check what actually got built.

The clean-container problem

Every job starts in a fresh container: the runner brings up an image, clones the repository at the right commit, runs the commands, and tears everything back down. The next job gets its own new container and its own clean clone.

That leads to a consequence everyone runs into: a file created in build doesn't exist in test. Permissions and paths have nothing to do with it. The file just isn't there: it's a different container.

Check it on your own project:

stages:
  - build
  - test

build-site:
  stage: build
  script:
    - mkdir -p public
    - cp index.html style.css public/

check-build:
  stage: test
  script:
    - test -f public/index.html

build will pass, test will fail: public/index.html isn't in its container. The test -f command returns exit code 1, and the job ends with the line ERROR: Job failed: exit code 1.

artifacts

An artifact is a set of files the runner collects from the container after a job and sends to GitLab for storage. Jobs in later stages download the artifacts of all jobs from earlier stages by default, unpacked into the working folder.

build-site:
  stage: build
  script:
    - mkdir -p public
    - cp index.html style.css public/
  artifacts:
    paths:
      - public

One section, and test turns green: the public folder, carried over from the previous stage, shows up in its container. In the build-site log you'll see the lines Uploading artifacts, in the check-build log the line Downloading artifacts.

paths lists files and folders. Paths are resolved from the repository root, meaning the folder where the job starts working. An absolute path like /tmp/out or a path with .. going outward won't work. Wildcards are supported: dist/*.js grabs every .js file in dist.

By default, the artifact is only uploaded if the job succeeds. A failed build passes nothing to later stages, and they don't even start.

Download an artifact by hand

You can download artifacts yourself too. In the pipeline list (Build → Pipelines) and in the job list there's a Download artifacts button, and on an individual job's page there's a Job artifacts block on the right with Download and Browse buttons. Inside is a zip with everything listed in paths.

This is the first place to look when you're debugging "CI built the wrong thing": download the artifact and see what's actually in it.

Artifacts aren't free

They take up space in the project's storage, which is limited on the free tier. That's why artifacts have a lifetime:

  artifacts:
    paths:
      - public
    expire_in: 1 hour

The expiration is written as a number with a unit: 30 mins, 1 hour, 2 hrs 20 min, 1 week. Without a unit, the number is counted as seconds. The value never turns off deletion.

If expire_in isn't set, an artifact lives for 30 days on GitLab.com. In a training project, set expire_in: 1 hour: the files will reach the next stage without piling up for a month.

One exception is worth knowing. For each branch and tag, GitLab by default doesn't delete the artifacts of the last pipeline (even a failed one) or the last successful pipeline, no matter what expire_in says. So your last green build stays either way. This is turned off in the project settings: Settings → CI/CD → Artifacts, the Keep artifacts from most recent successful jobs checkbox.

Reports instead of files

artifacts has a second mode, reports. These are also files, but GitLab understands their format and renders the result in the UI:

  artifacts:
    reports:
      junit: report.xml

That way tests show up as a Tests tab in the pipeline and a block with failures in the merge request, instead of a line in the log. We don't need that for our site, but it's worth knowing that artifacts can feed data straight into the UI.

Common mistakes

No artifacts section. The file is created in build, and it's not in test. Symptom: No such file or directory or test -f with exit code 1 in the next stage's job.

The path in paths doesn't match where script writes. The build writes to public, but paths has dist. The job log will show a warning:

WARNING: dist: no matching files. Ensure that the artifact path is relative to the working directory (/builds/site)
ERROR: No files to upload

The job itself still stays green, and it's the next stage that fails. So when debugging, don't just read the log of the failed job: read the log of the one that was supposed to pass the files on too.

The artifact exists, but something is missing. The pipeline is green, the test only checks index.html, and style.css didn't make it into the build. This can only be found one way: download the artifact and look at its contents.

Changes aren't committed. GitLab runs the pipeline on a commit. An edit to .gitlab-ci.yml that you haven't committed and pushed has no effect on the pipeline.

What to remember

Every job gets its own clean container. Files don't pass between stages on their own.

artifacts.paths grabs files after a successful job and hands them to later stages.

An artifact can be downloaded as a zip archive from the pipeline or job page. That's the main tool for checking "so what actually got built."

expire_in limits retention and saves space in the project.

What you'll need for the tasks

In the labs, the pipeline is run by a training runner on your laptop: like GitLab Runner with the Docker executor, it runs every job in its own clean container, and it puts the artifacts of successful jobs into the folder artifacts/<job name>/. That's the local equivalent of the Download artifacts button.

• git -C work add and git -C work commit: the runner, like GitLab, only takes what's committed.
• bash run-pipeline.sh: run the pipeline on the last commit.
• The artifacts:paths and artifacts:expire_in keys in .gitlab-ci.yml.
• ls -R artifacts/ and tar -tzf <archive>: check what's actually in the artifact.

What's next: the finale, turning the public folder into a site at its own address.