September 4, 2019

Jenkins vs. Gitlab CI

Comment

With Gitlab CI going full steem ahead following their proclaimed vision, we want to have a look on it ourselves! Mostly working with the proven and beloved open source CI tool Jenkins, we are wondering, will Gitlab give us some new features and benefits or will it not be able to replace Jenkins as the number one CI-tool. Lets get into it.

TL;DR
We migrated some CI pipelines from Jenkins to Gitlab CI. We really like the visualization of the different jobs in Gitlab CI. It makes the build jobs more transparent and flexible to use. Performancewise there are no big differences with the medium sized projects we tested it on. For very big and complex projects, things might look a little bit different, as Gitlab stores the job artifacts online and therefore has to up- and download them for every job. So in conclusion it depends on the type of project, but Gitlab is definatly an option to consider.

Jenkins

To set up Jenkins you need an existing installation and a activated pipeline plug-in. The first step is of course the definition of the source-repo, as Jenkins doesn’t provide repositories and connects with other services like Gitlab or Github.

The next step is the configuration of the pipline-stages. Jenkins needs a speciel file in the repository with the name of jenkinsfile. The phases and stages of the build configuration are maintained in this file. Here is an example of a configuration in this format:

pipeline {
    agent any
    stages {
        stage('Build') {
            steps {
                echo 'Building..'
                nodejs('node 10.9.0') {         
                    dir('www.thetestcompany.de') {
                        echo 'Building Job One'
                    }
                    dir('mail.thetestcompany.de') {
                        echo 'Building Job Two'
                    }
                }
            }
        }
        stage('Test') {
            steps {
                echo 'Testing..'
            }
        }
        stage('Deploy') {
            steps {
                echo 'Deploying....'
                nodejs('node 10.9.0') {
                    dir('mail.thetestcompany.de') {
                        withCredentials([file(credentialsId: params.CREDENTIAL_ID, variable: 'ormconfig')]) {
                            sh "cp --update -rf \$ormconfig ormconfig.json"
                            sh "npm run migrate:run"
                        }
                       echo 'Deploying Job One'
                    }
                    dir('www.thetestcompany.de') {
                        echo 'Building Job Two'
                    }
                }
            }
        }
    }
}

pipeline {

agent any

stages {

stage('Build') {

steps {

echo 'Building..'

nodejs('node 10.9.0') {

dir('www.thetestcompany.de') {

echo 'Building Job One'

}

dir('mail.thetestcompany.de') {

echo 'Building Job Two'

}

stage('Test') {

steps {

echo 'Testing..'

}

stage('Deploy') {

steps {

echo 'Deploying....'

nodejs('node 10.9.0') {

dir('mail.thetestcompany.de') {

withCredentials([file(credentialsId: params.CREDENTIAL_ID, variable: 'ormconfig')]) {

sh "cp --update -rf \$ormconfig ormconfig.json"

sh "npm run migrate:run"

}

echo 'Deploying Job One'

}

dir('www.thetestcompany.de') {

echo 'Building Job Two'

}

A plus is, that the Jenkins script is quiet readable and well structured. Additionally it is possible to use plug-ins, like the withCredentials Command. This makes it possible to include hidden Authentification Credentials in the script.

When the pipline runs through, we get to check if the stages passed or failed (in that case red colour) as a whole. Unfortunately we can’t see the status of the individual jobs in the graphical overiew. You can follow the progress in the terminal though.

Gitlab

Gitlab offers repositories. Hence the integration of Gitlab CI is pretty straight forward. Stages and jobs are described in the gitlab-ci.yml configuration file. If you look, the stages command includes a sequence of stages that will be executed in the specified order. After that, every job is described and configured with different options. Every job is part of a stage and will run parallel with other jobs in the same stage.

image: trion/ng-cli-karma

stages: 
 - build step one 
 - build step two 
 - test - migrate 
 - deploy 

# Extensions 
.ssh: 
 before_script: 
   - mkdir -p ~/.ssh 
   - chmod 700 ~/.ssh 
   - echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config 
   - echo "$KEY_STAGING" > ~/.ssh/id_rsa 
   - chmod 600 ~/.ssh/id_rsa 
.runner_tags: 
  tags: 
   - docker 
   - linux 
#Build Stage 
build-www.thetestcompany.de: 
  extends: 
   - .runner_tags stage: 
  stage: build step one 
  before_script: 
    - cd www.thetestcompany.de 
  script: 
    - echo 'Build Step One' 
  after_script: 
    - cd .. 
  artifacts: 
   paths: 
    - www.thetestcompany.de/ 
   name: "Build TTC step one" 
   expire_in: 1 week 
   when: on_success 
  cache: 
   paths: 
    - node_modules/ 
build-www.mail.thetestcompany.de: 
  extends: 
   - .runner_tags stage: build step two 
  dependencies: [build-www.tcc-core.de] 
  needs: [build-www.tcc-core.de] 
  before_script: 
   - cd www.thecodecampus.de 
  script: 
   - echo 'Build Step Two' 
  after_script: 
   - cd .. 
  artifacts: 
   paths: 
    - www.mail.thetestcompany.de/ 
   name: "Build www.ttc.de" 
   expire_in: 1 week 
   when: on_success 

#Test Stage 
test: 
  stage: test 
  script: 
   - echo 'Testing..' 
  tags: 
   - docker 
   - linux 

#Migrate Stage - STAGING 
migrate-www.mail.thetestcompany.de-STAGING: 
  extends: 
   - .runner_tags 
   - .ssh  
  stage: migrate 
  image: kroniak/ssh-client 
  script: 
   - echo 'Migrate DB' 
  after_script: 
   - cd .. 
  artifacts: 
   paths: 
    - www.mail.thetestcompany.de/ 
  environment: staging 
  needs: 
   - www.mail.thetestcompany.de 
  dependencies: 
   - www.mail.thetestcompany.de 
 
#Migrate Stage - PROD 

migrate-www.mail.thetestcompany.de: 
  extends:
   - .runner_tags 
   - .ssh stage: migrate image: kroniak/ssh-client script: 
   - echo 'Migrate DB' 
  after_script: 
   - cd .. 
  artifacts: 
   paths: 
    - www.mail.thetestcompany.de/ 
  needs: 
   - build-www.mail.thetestcompany.de 
  dependencies: 
   - build-www.mail.thetestcompany.de 
  when: manual 
  only: 
   - master environment: staging 

deploy-www.thetestcompany.de-STAGING: 
  extends: 
   - .runner_tags 
   - .ssh 
  stage: deploy 
  image: kroniak/ssh-client 
  script: 
   - cd www.thetestcompany.de 
   - echo 'Deploy Job One' 
  after_script: 
   - cd .. environment: staging 
  needs: 
   - build-www.thetestcompany.de 
  dependencies: 
   - build-www.thetestcompany.de 

deploy-www.mail.thetestcompany.de-STAGING:
  extends: 
   - .runner_tags 
   - .ssh 
  stage: deploy 
  image: kroniak/ssh-client 
  script: 
   - cd www.mail.thetestcompany.de 
   - echo 'Deploy Job Two' 
  after_script: 
   - cd .. 
  environment: staging needs: 
   - build-www.mail.thetestcompany.de 
  dependencies: 
   - build-www.mail.thetestcompany.de

deploy-www.thecodecampus.de: 
  stage: deploy 
  extends: 
   - .runner_tags 
   - .ssh 
  image: kroniak/ssh-client 
  script: 
   - cd www.thetestcompany.de 
   - echo 'Deploy www.thetestcompany.de' 
  after_script: 
   - cd .. 
  environment: prod 
  needs: 
   - build-www.thecodecampus.de 
  dependencies: 
   - build-www.thecodecampus.de 
  only: 
   - master 
  when: manual 

deploy-www.mail.thetestcompany.de: 
  extends: 
   - .runner_tags 
   - .ssh 
  stage: deploy 
  image: kroniak/ssh-client 
  script: 
   - cd www.mail.thetestcompany.de 
   - echo 'Deploy www.mail.thetestcompany.de' 
  after_script: 
   - cd ..
  environment: prod 
  needs: 
   - build-www.mail.thetestcompany.de 
  dependencies: 
   - build-www.mail.thetestcompany.de 
  only: 
   - master 
  when: manual

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

image: trion/ng-cli-karma

stages:

- build step one

- build step two

- test - migrate

- deploy

# Extensions

.ssh:

before_script:

- mkdir -p ~/.ssh

- chmod 700 ~/.ssh

- echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config

- echo "$KEY_STAGING" > ~/.ssh/id_rsa

- chmod 600 ~/.ssh/id_rsa

.runner_tags:

tags:

- docker

- linux

#Build Stage

build-www.thetestcompany.de:

extends:

- .runner_tags stage:

stage: build step one

before_script:

- cd www.thetestcompany.de

script:

- echo 'Build Step One'

after_script:

- cd ..

artifacts:

paths:

- www.thetestcompany.de/

name: "Build TTC step one"

expire_in: 1 week

when: on_success

cache:

paths:

- node_modules/

build-www.mail.thetestcompany.de:

extends:

- .runner_tags stage: build step two

dependencies: [build-www.tcc-core.de]

needs: [build-www.tcc-core.de]

before_script:

- cd www.thecodecampus.de

script:

- echo 'Build Step Two'

after_script:

- cd ..

artifacts:

paths:

- www.mail.thetestcompany.de/

name: "Build www.ttc.de"

expire_in: 1 week

when: on_success

#Test Stage

test:

stage: test

script:

- echo 'Testing..'

tags:

- docker

- linux

#Migrate Stage - STAGING

migrate-www.mail.thetestcompany.de-STAGING:

extends:

- .runner_tags

- .ssh

stage: migrate

image: kroniak/ssh-client

script:

- echo 'Migrate DB'

after_script:

- cd ..

artifacts:

paths:

- www.mail.thetestcompany.de/

environment: staging

needs:

- www.mail.thetestcompany.de

dependencies:

- www.mail.thetestcompany.de

#Migrate Stage - PROD

migrate-www.mail.thetestcompany.de:

extends:

- .runner_tags

- .ssh stage: migrate image: kroniak/ssh-client script:

- echo 'Migrate DB'

after_script:

- cd ..

artifacts:

paths:

- www.mail.thetestcompany.de/

needs:

- build-www.mail.thetestcompany.de

dependencies:

- build-www.mail.thetestcompany.de

when: manual

only:

- master environment: staging

deploy-www.thetestcompany.de-STAGING:

extends:

- .runner_tags

- .ssh

stage: deploy

image: kroniak/ssh-client

script:

- cd www.thetestcompany.de

- echo 'Deploy Job One'

after_script:

- cd .. environment: staging

needs:

- build-www.thetestcompany.de

dependencies:

- build-www.thetestcompany.de

deploy-www.mail.thetestcompany.de-STAGING:

extends:

- .runner_tags

- .ssh

stage: deploy

image: kroniak/ssh-client

script:

- cd www.mail.thetestcompany.de

- echo 'Deploy Job Two'

after_script:

- cd ..

environment: staging needs:

- build-www.mail.thetestcompany.de

dependencies:

- build-www.mail.thetestcompany.de

deploy-www.thecodecampus.de:

stage: deploy

extends:

- .runner_tags

- .ssh

image: kroniak/ssh-client

script:

- cd www.thetestcompany.de

- echo 'Deploy www.thetestcompany.de'

after_script:

- cd ..

environment: prod

needs:

- build-www.thecodecampus.de

dependencies:

- build-www.thecodecampus.de

only:

- master

when: manual

deploy-www.mail.thetestcompany.de:

extends:

- .runner_tags

- .ssh

stage: deploy

image: kroniak/ssh-client

script:

- cd www.mail.thetestcompany.de

- echo 'Deploy www.mail.thetestcompany.de'

after_script:

- cd ..

environment: prod

needs:

- build-www.mail.thetestcompany.de

dependencies:

- build-www.mail.thetestcompany.de

only:

- master

when: manual

So now the jobs are configured and we are ready to run the Gitlab CI pipeline. The result will look like illustrated in the following picture. At first sight the major advantage of Gitlab CI, compared to Jenkins, jumps right into our eyes. It’s the smaller granularity of the job visualization. You see the status of every job you specified inside a stage. Therefore debugging is way more efficient. The granularity of this graph is completely up to you and how you define your jobs. Additionally Gitlab-CI offers functionalities to build up a directed acyclic graph. This opens up the possibility to start jobs of a following stage, even if not all of the jobs in the current stage finished. This is a very big advantage, as this increases performance of the pipeline significantly.

Jenkins vs Gitlab Conclusion

So now we know how each of the CI Tools can be configured. Lets have a look at the Pros and Cons of each one:

Gitlab Pros

very good Docker integration
parallel job execution within stages
possibility of directed acyclic graph pipeline
easy to add jobs
merge request integration
extremely scalable due to concurrent runners

Gitlab Cons

artifacts have to be defined and uploaded/downloaded for every job
testing the merged state of a branch is not possible before actual merge is done
stages within stages are not yet supported

Jenkins Pros

selfhosted –> full control over workspaces
easier debugging of runners hence full workspace control
very good management of credentials.
big plugin library

Jenkins Cons

overhead for small projects as you have to setup by yourself
complicated plugin integration
new pipeline for every environment (e.g. Production/Testing)

Timecomparison

The tests we ran when migrating jobs from Jenkins to Gitlab showed no big differences in pipeline-througput-times:

Medium sized project with monorepo:

Jenkins: 6-7min

Gitlab: 7-8min

Small sized project

Jenkins: 4min

Gitlab: 3-5min

So which solution should I use?

small / medium sized projects:

Definatly Gitlub CI. The fast setup time, the easy way to integrate new jobs and the flexibility of the tool makes it a ery powerful CI-option. Especially when the project has an agile character, the granularity of the graphic interface and the flexibility in adjustment are a big advantage.

large complex projects:

For large projects the decision might not be so easy! Gitlab still brings in all the advantages. Based on the granularity and the central configuration in the gitlab-ci.yml, the structure gets quite complex pretty fast though. So maybe the best way is to include it for quick testing and then switch to Jenkins for bigger standardized building and deployment jobs.

Future of Gitlab CI

Some of the shortcomings of Gitlab CI might vanish in future iteratiotins. To look up what’s to come, just visit Gitlab’s product vision: Gitlab CI Vision.

Performancewise there are two very interesting features in the pipeline:

Stages in Stages
In near future it will be possible to define stages inside of stages. This opens up possibilities for further job encapsulation and more structured pipelines.
Empty needs declarator: needs:[…] – and needs inside stages
Also announced for next iterations is the manipulation of the needs declarator. It will be possible to define it for two jobs inside a stage (for now its only possible to reference a job of predecessing staages. Aditionally it will be possible to start jobs without a predecessor directly, without waiting by leaving the needs tag empty.

Ionic 2 Proxy while Development with Gulp

Ionic 2 set Focus of Input Element

Fill Gitlab-CI Postgres Database Service with SQL

Janik Kessler
Developer at thecodecampus </>

Jenkins vs. Gitlab CI

Jenkins

Gitlab

Jenkins vs Gitlab Conclusion

Gitlab Pros

Gitlab Cons

Jenkins Pros

Jenkins Cons

Timecomparison

So which solution should I use?

Future of Gitlab CI

Related Posts

Leave a Reply Cancel reply