September 4, 2019
Jenkins vs. Gitlab CI
With Gitlab CI going full steem ahead following their proclaimed vision, we want to have a look on it ourselves! Mostly working with the proven and beloved open source CI tool Jenkins, we are wondering, will Gitlab give us some new features and benefits or will it not be able to replace Jenkins as the number one CI-tool. Lets get into it.
TL;DR
We migrated some CI pipelines from Jenkins to Gitlab CI. We really like the visualization of the different jobs in Gitlab CI. It makes the build jobs more transparent and flexible to use. Performancewise there are no big differences with the medium sized projects we tested it on. For very big and complex projects, things might look a little bit different, as Gitlab stores the job artifacts online and therefore has to up- and download them for every job. So in conclusion it depends on the type of project, but Gitlab is definatly an option to consider.
Jenkins
To set up Jenkins you need an existing installation and a activated pipeline plug-in. The first step is of course the definition of the source-repo, as Jenkins doesn’t provide repositories and connects with other services like Gitlab or Github.
The next step is the configuration of the pipline-stages. Jenkins needs a speciel file in the repository with the name of jenkinsfile. The phases and stages of the build configuration are maintained in this file. Here is an example of a configuration in this format:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 |
pipeline { agent any stages { stage('Build') { steps { echo 'Building..' nodejs('node 10.9.0') { dir('www.thetestcompany.de') { echo 'Building Job One' } dir('mail.thetestcompany.de') { echo 'Building Job Two' } } } } stage('Test') { steps { echo 'Testing..' } } stage('Deploy') { steps { echo 'Deploying....' nodejs('node 10.9.0') { dir('mail.thetestcompany.de') { withCredentials([file(credentialsId: params.CREDENTIAL_ID, variable: 'ormconfig')]) { sh "cp --update -rf \$ormconfig ormconfig.json" sh "npm run migrate:run" } echo 'Deploying Job One' } dir('www.thetestcompany.de') { echo 'Building Job Two' } } } } } } |
A plus is, that the Jenkins script is quiet readable and well structured. Additionally it is possible to use plug-ins, like the withCredentials Command. This makes it possible to include hidden Authentification Credentials in the script.
When the pipline runs through, we get to check if the stages passed or failed (in that case red colour) as a whole. Unfortunately we can’t see the status of the individual jobs in the graphical overiew. You can follow the progress in the terminal though.
Gitlab
Gitlab offers repositories. Hence the integration of Gitlab CI is pretty straight forward. Stages and jobs are described in the gitlab-ci.yml configuration file. If you look, the stages command includes a sequence of stages that will be executed in the specified order. After that, every job is described and configured with different options. Every job is part of a stage and will run parallel with other jobs in the same stage.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 |
image: trion/ng-cli-karma stages: - build step one - build step two - test - migrate - deploy # Extensions .ssh: before_script: - mkdir -p ~/.ssh - chmod 700 ~/.ssh - echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config - echo "$KEY_STAGING" > ~/.ssh/id_rsa - chmod 600 ~/.ssh/id_rsa .runner_tags: tags: - docker - linux #Build Stage build-www.thetestcompany.de: extends: - .runner_tags stage: stage: build step one before_script: - cd www.thetestcompany.de script: - echo 'Build Step One' after_script: - cd .. artifacts: paths: - www.thetestcompany.de/ name: "Build TTC step one" expire_in: 1 week when: on_success cache: paths: - node_modules/ build-www.mail.thetestcompany.de: extends: - .runner_tags stage: build step two dependencies: [build-www.tcc-core.de] needs: [build-www.tcc-core.de] before_script: - cd www.thecodecampus.de script: - echo 'Build Step Two' after_script: - cd .. artifacts: paths: - www.mail.thetestcompany.de/ name: "Build www.ttc.de" expire_in: 1 week when: on_success #Test Stage test: stage: test script: - echo 'Testing..' tags: - docker - linux #Migrate Stage - STAGING migrate-www.mail.thetestcompany.de-STAGING: extends: - .runner_tags - .ssh stage: migrate image: kroniak/ssh-client script: - echo 'Migrate DB' after_script: - cd .. artifacts: paths: - www.mail.thetestcompany.de/ environment: staging needs: - www.mail.thetestcompany.de dependencies: - www.mail.thetestcompany.de #Migrate Stage - PROD migrate-www.mail.thetestcompany.de: extends: - .runner_tags - .ssh stage: migrate image: kroniak/ssh-client script: - echo 'Migrate DB' after_script: - cd .. artifacts: paths: - www.mail.thetestcompany.de/ needs: - build-www.mail.thetestcompany.de dependencies: - build-www.mail.thetestcompany.de when: manual only: - master environment: staging deploy-www.thetestcompany.de-STAGING: extends: - .runner_tags - .ssh stage: deploy image: kroniak/ssh-client script: - cd www.thetestcompany.de - echo 'Deploy Job One' after_script: - cd .. environment: staging needs: - build-www.thetestcompany.de dependencies: - build-www.thetestcompany.de deploy-www.mail.thetestcompany.de-STAGING: extends: - .runner_tags - .ssh stage: deploy image: kroniak/ssh-client script: - cd www.mail.thetestcompany.de - echo 'Deploy Job Two' after_script: - cd .. environment: staging needs: - build-www.mail.thetestcompany.de dependencies: - build-www.mail.thetestcompany.de deploy-www.thecodecampus.de: stage: deploy extends: - .runner_tags - .ssh image: kroniak/ssh-client script: - cd www.thetestcompany.de - echo 'Deploy www.thetestcompany.de' after_script: - cd .. environment: prod needs: - build-www.thecodecampus.de dependencies: - build-www.thecodecampus.de only: - master when: manual deploy-www.mail.thetestcompany.de: extends: - .runner_tags - .ssh stage: deploy image: kroniak/ssh-client script: - cd www.mail.thetestcompany.de - echo 'Deploy www.mail.thetestcompany.de' after_script: - cd .. environment: prod needs: - build-www.mail.thetestcompany.de dependencies: - build-www.mail.thetestcompany.de only: - master when: manual |
So now the jobs are configured and we are ready to run the Gitlab CI pipeline. The result will look like illustrated in the following picture. At first sight the major advantage of Gitlab CI, compared to Jenkins, jumps right into our eyes. It’s the smaller granularity of the job visualization. You see the status of every job you specified inside a stage. Therefore debugging is way more efficient. The granularity of this graph is completely up to you and how you define your jobs. Additionally Gitlab-CI offers functionalities to build up a directed acyclic graph. This opens up the possibility to start jobs of a following stage, even if not all of the jobs in the current stage finished. This is a very big advantage, as this increases performance of the pipeline significantly.
Jenkins vs Gitlab Conclusion
So now we know how each of the CI Tools can be configured. Lets have a look at the Pros and Cons of each one:
Gitlab Pros
- very good Docker integration
- parallel job execution within stages
- possibility of directed acyclic graph pipeline
- easy to add jobs
- merge request integration
- extremely scalable due to concurrent runners
Gitlab Cons
- artifacts have to be defined and uploaded/downloaded for every job
- testing the merged state of a branch is not possible before actual merge is done
- stages within stages are not yet supported
Jenkins Pros
- selfhosted –> full control over workspaces
- easier debugging of runners hence full workspace control
- very good management of credentials.
- big plugin library
Jenkins Cons
- overhead for small projects as you have to setup by yourself
- complicated plugin integration
- new pipeline for every environment (e.g. Production/Testing)
Timecomparison
The tests we ran when migrating jobs from Jenkins to Gitlab showed no big differences in pipeline-througput-times:
Medium sized project with monorepo:
Jenkins: 6-7min
Gitlab: 7-8min
Small sized project
Jenkins: 4min
Gitlab: 3-5min
So which solution should I use?
small / medium sized projects:
Definatly Gitlub CI. The fast setup time, the easy way to integrate new jobs and the flexibility of the tool makes it a ery powerful CI-option. Especially when the project has an agile character, the granularity of the graphic interface and the flexibility in adjustment are a big advantage.
large complex projects:
For large projects the decision might not be so easy! Gitlab still brings in all the advantages. Based on the granularity and the central configuration in the gitlab-ci.yml, the structure gets quite complex pretty fast though. So maybe the best way is to include it for quick testing and then switch to Jenkins for bigger standardized building and deployment jobs.
Future of Gitlab CI
Some of the shortcomings of Gitlab CI might vanish in future iteratiotins. To look up what’s to come, just visit Gitlab’s product vision: Gitlab CI Vision.
Performancewise there are two very interesting features in the pipeline:
- Stages in Stages
In near future it will be possible to define stages inside of stages. This opens up possibilities for further job encapsulation and more structured pipelines. - Empty needs declarator: needs:[…] – and needs inside stages
Also announced for next iterations is the manipulation of the needs declarator. It will be possible to define it for two jobs inside a stage (for now its only possible to reference a job of predecessing staages. Aditionally it will be possible to start jobs without a predecessor directly, without waiting by leaving the needs tag empty.
Janik Kessler
Developer at 