Continuous integration: pipelines, unit tests, static analysis

Milestone 3, in week 10, is the second prototype: about 90% of the use cases work, unit tests exist, and CI is set up and used regularly – every merge request runs a pipeline, including a static-analysis code-quality check. This is the milestone where many teams discover that their "working" code only worked on one laptop.

What CI is for

Continuous integration means every change is built and tested in a clean, identical environment before it reaches develop – not on your machine, with your cached files and your operating system. From then on, "it works for me" is not an argument; the pipeline is.

A green pipeline on a merge request means: it compiles, the tests pass, the code is formatted, and the analyser found no known bug pattern. A reviewer can then spend their attention on what a machine can't check.

Pipeline

All the jobs that run for one event (an MR, a push, a tag).

Job

One step in a container: format-check, build, test, static-analysis, package, release.

Stage

A column of jobs. With needs: a job starts as soon as the jobs it needs are done.

Runner

The machine that executes jobs. The course GitLab has shared runners.

Artifact

A file a job keeps: test reports, coverage XML, the packaged game.

The pipeline and when it runs

The recommended pipeline has the same shape for every stack: a fast format check first, then the build, then tests with coverage and static analysis in parallel, and – only for a version tag – packaging and a release (Fig. 1).

The rules at the top of the pipeline file decide this:

workflow:
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
    - if: $CI_COMMIT_BRANCH == "main"
    - if: $CI_COMMIT_BRANCH == "develop"
    - if: $CI_COMMIT_TAG =~ /^v\d+\.\d+(\.\d+)?$/

Switching it on, per stack

The template keeps one complete pipeline per stack under ci/gitlab/; the root .gitlab-ci.yml only includes one of them. The real logic sits in the build tool – Gradle, Maven, MSBuild – and the pipeline just calls it, so the build file has to contain the plugins the jobs expect.

# .gitlab-ci.yml – exactly one line active
include:
  - local: ci/gitlab/java-gradle.gitlab-ci.yml
  # - local: ci/gitlab/java-maven.gitlab-ci.yml
  # - local: ci/gitlab/dotnet.gitlab-ci.yml
  # - local: ci/gitlab/unity.gitlab-ci.yml

Java

test:
  stage: test
  needs: [build]
  script:
    - ./gradlew test jacocoTestReport
    # ... plus a few lines that print "Line coverage: NN%" from the JaCoCo XML
  coverage: '/Line coverage: (\d+)%/'
  artifacts:
    when: always
    reports:
      junit: build/test-results/test/TEST-*.xml
      coverage_report:
        coverage_format: jacoco
        path: build/reports/jacoco/test/jacocoTestReport.xml

The junit report makes the MR show which test failed; the coverage_report marks covered and uncovered lines right in the diff; the coverage regex feeds the badge.

.NET

format-check:
  script:
    - dotnet format $SOLUTION --verify-no-changes --verbosity minimal

static-analysis:
  script:
    - dotnet build $SOLUTION -c Release -p:TreatWarningsAsErrors=true -p:EnforceCodeStyleInBuild=true

test:
  script:
    - >
      dotnet test $SOLUTION --no-build -c Release
      --logger "junit;LogFilePath=$CI_PROJECT_DIR/test-results/{assembly}.junit.xml"
      --collect:"XPlat Code Coverage" --settings coverlet.runsettings

GitLab only reads JUnit XML, so the test projects need the JunitXml.TestLogger package. Set the SOLUTION and MAIN_PROJECT variables at the top of the file, and put Directory.Build.props (analyzers, warnings as errors) and coverlet.runsettings in the repository root.

Unity

test-EditMode:
  extends: .test            # the editor in batch mode: -runTests -testPlatform $TEST_PLATFORM
  variables:
    TEST_PLATFORM: EditMode

test-PlayMode:
  extends: .test
  variables:
    TEST_PLATFORM: PlayMode

• —Only the Unity and Godot versions announced at the start of the semester run on the course runners, and the course provides the CI images and components for them – use those first.
• —The editor needs a licence in CI: a UNITY_LICENSE CI/CD variable. Don't mark it Protected, or the MR pipelines of feature branches can't see it and every test fails with a licence error. Never commit the licence file.
• —Keep the game logic in plain C# classes tested in EditMode – fast, no scene needed. PlayMode tests are for behaviour that needs the running game.
• —Binary assets go in Git LFS, and only one person edits a scene at a time; split scenes into prefabs to make conflicts rarer.

GitHub Actions

The same pipelines exist as GitHub Actions workflows: copy the chosen one to .github/workflows/ci.yml. The triggers express the same rules:

# .github/workflows/ci.yml
on:
  pull_request:
    branches: [main, develop, "release/**"]
  push:
    branches: [main, develop]
    tags: ["v[0-9]+.[0-9]+"]

Unit tests and coverage

A unit test checks one behaviour of one class, in milliseconds, without a window. That is only possible if the model is separate from the view – the design decision from milestone 1 pays off here. Test every public behaviour of the model: the happy path and the error path.

@Test
void loadingACorruptFileThrowsAndChangesNothing(@TempDir Path dir) throws IOException {
    Path file = dir.resolve("broken.save");
    Files.writeString(file, "{ not json");
    GameState before = session.state();

    assertThrows(SaveFormatException.class, () -> session.load(file));
    assertEquals(before, session.state());
}

[Fact]
public void StepIntoWallLeavesPlayerInPlace()
{
    var game = new Game(Map.Parse("#P."));       // wall, player, free tile – a map as a string
    game.Step(Direction.Left);
    Assert.Equal(new Position(1, 0), game.Player.Position);
}

Small maps written as strings make game tests short and readable – and the same parser can load real levels. Coverage is the share of lines the tests execute. It is measured on the model only; the view and the entry point are excluded (Fig. 2).

// build.gradle – JaCoCo: measure the model, not the view
classDirectories.setFrom(files(classDirectories.files.collect {
    fileTree(dir: it, exclude: ['**/view/**', '**/ui/**', '**/Main.class'])
}))

<!-- coverlet.runsettings – the same for .NET -->
<ExcludeByFile>**/Program.cs,**/View/**/*.cs,**/Views/**/*.cs,**/UI/**/*.cs</ExcludeByFile>

Static analysis

A static analyser reads the code without running it and looks for known bug patterns: a possible null dereference, a stream that is never closed, a comparison that is always true, an equals without hashCode. From M3 the course requires such a check in the pipeline.

• —Java: SpotBugs analyses the compiled bytecode; -Xlint:all makes every compiler warning visible.
• —.NET: the Roslyn analyzers are part of the SDK; Directory.Build.props sets the recommended level and turns warnings – including nullable warnings – into errors.
• —Course CI components: from M3 the course provides ready-made components for static analysis that a single include line adds. The description is published in the course's GitLab group.

include:
  - local: ci/gitlab/java-gradle.gitlab-ci.yml
  - component: <path published by the course>/<component name>@<ref>   # static analysis

If a finding is really a false positive, exclude exactly that class and pattern – with a comment saying why, so the reviewer can judge it:

<!-- config/spotbugs-exclude.xml – every exclusion says why -->
<!-- Board exposes its tile array on purpose: the view reads it every frame. -->
<Match>
  <Class name="~.*\.model\.Board" />
  <Bug pattern="EI_EXPOSE_REP" />
</Match>

Reading a red pipeline

A red pipeline is the author's job to fix, not the reviewer's. Click the red job and scroll to the end of its log: the error is almost always in the last 30 lines (Fig. 3).

Run the same thing locally

Because the pipeline only calls the build tool, every job has a local equivalent (Fig. 4). Run it before you open or update an MR – if it is green locally, it will be green in CI.

# Java / Gradle
./gradlew spotlessApply && ./gradlew build spotbugsMain

# Java / Maven
mvn spotless:apply && mvn verify

# .NET
dotnet format && dotnet build -warnaserror && dotnet test

The ratchet: tightening over time

The recommended setup is strict from the start – static analysis blocks the merge, warnings are errors in .NET – because on an empty project strictness costs nothing, and introducing it later is much harder. Some rules are best switched on once the team is ready, around M3: a coverage threshold, warnings as errors in Java (Fig. 5).

// build.gradle – the coverage gate: the build fails below 80% line coverage
check.dependsOn jacocoTestCoverageVerification

jacocoTestCoverageVerification {
    violationRules {
        rule { limit { counter = 'LINE'; value = 'COVEREDRATIO'; minimum = 0.80 } }
    }
}

On .NET the threshold needs the coverlet.msbuild package and /p:Threshold=80 on dotnet test. If you must loosen a rule temporarily – importing old code, say – mark the job as allowed to fail and open an issue to undo it, with a milestone:

static-analysis:
  allow_failure: true      # TEMPORARY – issue #41 switches it back before M4

What the M3 demo shows

Next: the final product – clean code, thorough tests and the last release.

Summary

• —CI builds and tests every MR in a clean environment; feature pushes without an MR run nothing, tags add package and release.
• —Pipeline: format-check → build → test + coverage and static analysis in parallel → package → release.
• —One include line chooses the stack; the logic lives in the build file, so every job has a local equivalent.
• —Unit-test the model, happy and error paths; coverage is measured on the model only.
• —Red job: read the end of its log, fix locally, push. Tighten rules over time and never loosen one without an issue.