Gitlab parallel matrix pipelines

We use GitLab, and recently we needed to run the same pipeline, but with multiple different variables. It’s a common problem you might encounter, for example to prepare a docker Image with various different PHP versions, or create a build of an app targeting different operating systems.

Parallel Matrix Pipelines

Matrix pipelines in GitLab CI/CD are the solution, they enable us to execute jobs in parallel with multiple values for variables.
We simply need to use the parallel: matrix: key within GitLab’s CI configuration file, .gitlab-ci.yml, to specify our array of variable values.

A Single Variable

You might want to build docker images, each configured with different versions of PHP. By defining a single variable for PHP versions in a matrix, each version can trigger a separate parallel job. Here’s how you might set this up:

build:
  parallel:
    matrix:
      - PHP_VERSION: ['8.0', '8.1', '8.2', '8.3']
  script:
    - echo "Building Docker image with PHP version $PHP_VERSION"
    - docker build --build-arg PHP_VERSION=$PHP_VERSION -t my-app:$PHP_VERSION .

In this configuration, four jobs will run simultaneously, each building a Docker image with a different PHP version.

Multiple Variables

Matrix pipelines can also combine multiple variables to generate more comprehensive job configurations. For example, you might want to test different PHP versions on both nginx and Apache servers. Adding another dimension to the matrix accomplishes this:

build:
  parallel:
    matrix:
      - PHP_VERSION: ['8.0', '8.1', '8.2', '8.3']
      - SERVER: ['nginx', 'apache']
  script:
    - echo "Building Docker image with $PHP_VERSION and $SERVER"
    - docker build --build-arg PHP_VERSION=$PHP_VERSION --build-arg SERVER=$SERVER -t my-app:$PHP_VERSION-$SERVER .

This setup triggers eight parallel jobs, each for a unique combination of PHP version and server type, effectively covering all permutations and ensuring comprehensive testing across configurations.

Limitations

Maximum Permutations: There is a limit of maximum 200 maximum allowed permutations (though if you’re reaching this you might be doing something wrong!)
Limited Values: The values must be either a string, or an array of strings.
Duplicate values As job names are generated from variable values, you cannot use the same value twice (for two variables). For example, 8.0in the following would entries would overwrite each other:

matrix:
  - PHP_VERSION: ['8.0', '8.1', '8.2', '8.3']
  - MYSQL_VERSION: ['7.0', '8.0']

Conclusion

Matrix pipelines in GitLab CI/CD makes parallel execution of multiple job configurations a breeze, saving time and reducing the complexity of your gitlab-ci.yaml and maybe even encouraging a more thorough testing process!
For a few more examples, read the full documentation.