Jenkins vs. Gitlab CI

With Gitlab CI going full steem ahead following their proclaimed vision, we want to have a look on it ourselves! Mostly working with the proven and beloved open source CI tool Jenkins, we are wondering, will Gitlab give us some new features and benefits or will it not be able to replace Jenkins as the number one CI-tool. Lets get into it.

TL;DR
We migrated some CI pipelines from Jenkins to Gitlab CI. We really like the visualization of the different jobs in Gitlab CI. It makes the build jobs more transparent and flexible to use. Performancewise there are no big differences with the medium sized projects we tested it on. For very big and complex projects, things might look a little bit different, as Gitlab stores the job artifacts online and therefore has to up- and download them for every job. So in conclusion it depends on the type of project, but Gitlab is definatly an option to consider.

Jenkins

To set up Jenkins you need an existing installation and a activated pipeline plug-in. The first step is of course the definition of the source-repo, as Jenkins doesn’t provide repositories and connects with other services like Gitlab or Github.

The next step is the configuration of the pipline-stages. Jenkins needs a speciel file in the repository with the name of jenkinsfile. The phases and stages of the build configuration are maintained in this file. Here is an example of a configuration in this format:

A plus is, that the Jenkins script is quiet readable and well structured. Additionally it is possible to use plug-ins, like the withCredentials Command. This makes it possible to include hidden Authentification Credentials in the script.

When the pipline runs through, we get to check if the stages passed or failed (in that case red colour) as a whole. Unfortunately we can’t see the status of the individual jobs in the graphical overiew. You can follow the progress in the terminal though.

Gitlab

Gitlab offers repositories. Hence the integration of Gitlab CI is pretty straight forward. Stages and jobs are described in the gitlab-ci.yml configuration file. If you look, the stages command includes a sequence of stages that will be executed in the specified order. After that, every job is described and configured with different options. Every job is part of a stage and will run parallel with other jobs in the same stage.

So now the jobs are configured and we are ready to run the Gitlab CI pipeline. The result will look like illustrated in the following picture. At first sight the major advantage of Gitlab CI, compared to Jenkins, jumps right into our eyes. It’s the smaller granularity of the job visualization. You see the status of every job you specified inside a stage. Therefore debugging is way more efficient. The granularity of this graph is completely up to you and how you define your jobs. Additionally Gitlab-CI offers functionalities to build up a directed acyclic graph. This opens up the possibility to start jobs of a following stage, even if not all of the jobs in the current stage finished. This is a very big advantage, as this increases performance of the pipeline significantly.

Jenkins vs Gitlab Conclusion

So now we know how each of the CI Tools can be configured. Lets have a look at the Pros and Cons of each one:

Gitlab Pros

• very good Docker integration
• parallel job execution within stages
• possibility of directed acyclic graph pipeline
• easy to add jobs
• merge request integration
• extremely scalable due to concurrent runners

Gitlab Cons

• artifacts have to be defined and uploaded/downloaded for every job
• testing the merged state of a branch is not possible before actual merge is done
• stages within stages are not yet supported

Jenkins Pros

• selfhosted  –> full control over workspaces
• easier debugging of runners hence full workspace control
• very good management of credentials.
• big plugin library

Jenkins Cons

• overhead for small projects as you have to setup by yourself
• complicated plugin integration
• new pipeline for every environment (e.g. Production/Testing)

Timecomparison

The tests we ran when migrating jobs from Jenkins to Gitlab showed no big differences in pipeline-througput-times:

Medium sized project with monorepo:

Jenkins: 6-7min

Gitlab: 7-8min

Small sized project

Jenkins: 4min

Gitlab: 3-5min

So which solution should I use?

small / medium sized projects:

Definatly Gitlub CI. The fast setup time, the easy way to integrate new jobs and the flexibility of the tool makes it a ery powerful CI-option. Especially when the project has an agile character, the granularity of the graphic interface and the flexibility in adjustment are a big advantage.

large complex projects:

For large projects the decision might not be so easy! Gitlab still brings in all the advantages. Based on the granularity and the central configuration in the gitlab-ci.yml, the structure gets quite complex pretty fast though. So maybe the best way is to include it for quick testing and then switch to Jenkins for bigger standardized building and deployment jobs.

Future of Gitlab CI

Some of the shortcomings of Gitlab CI might vanish in future iteratiotins. To look up what’s to come, just visit Gitlab’s product vision: Gitlab CI Vision.

Performancewise there are two very interesting features in the pipeline:

• Stages in Stages
  In near future it will be possible to define stages inside of stages. This opens up possibilities for further job encapsulation and more structured pipelines.
• Empty needs declarator: needs:[…] – and needs inside stages
  Also announced for next iterations is the manipulation of the needs declarator. It will be possible to define it for two jobs inside a stage (for now its only possible to reference a job of predecessing staages. Aditionally it will be possible to start jobs without a predecessor directly, without waiting by leaving the needs tag empty.