Auto deployment to staging and production server with GitLab CI

I recently started to deal with Continuous Integration (CI). With that, you can automatically run tasks after pushing to a version control repo, like code testing or deployment. This post shows you how a deployment process to staging and production server can look with GitLab.

In contrast to GitHub, GitLab comes with an integrated CI, which also can be used in the free plan. My website’s theme is versioned with git. There is a local version, one on a staging server and one on this server. With GitLab CI I can automatically push the theme to the staging or production server after pushing a commit — depending on whether the push goes to the staging or master branch. So I do not need to upload files to different servers via SFTP but can handle that via Git.

To use the CI in the free GitLab account, you just have to create a .gitlab-ci.yml at the top level of your repo (in a self-hosted GitLab instance, you may have to create a runner first). This file stores the information, what the CI should do after a commit. We want to realize the following:

After pushing to the staging branch:

  1. Connect to the staging server.
  2. Create a temporary theme folder.
  3. Copy the theme files from the git repo to the temp directory on the staging server.
  4. Move the now-outdated theme to another temp folder.
  5. Move the new state of the theme to the theme folder.
  6. Remove the temp folder with the old theme files.

The way with the temporary directories is taken from the GitLab documentation and ensures that there is no short timeframe, where no working theme is on the server.

After pushing to the master branch:

This process looks almost exactly like the one for the staging. The only difference is the connection to the production server.

Creating the .gitlab-ci.yml for deployment

Like already said, the .gitlab-ci.yml stores the information for the CI jobs. The tasks can be grouped into jobs which can run in different stages. We only need a deploy stage and the jobs deploy_staging and deploy_production. The beginning of the file looks like that:

stages: - deploy deploy_staging: stage: deploy image: tetraweb/php:7.1 before_script: - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )' - mkdir -p ~/.ssh - eval $(ssh-agent -s) - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config' - ssh-add <(echo "$STAGING_PRIVATE_KEY") - apt-get install rsync script: - ssh -p22 user@staging.example.com "mkdir -p /html/wp-content/themes/_tmp" - rsync -rav -e ssh --exclude='.git/' --exclude='.gitlab-ci.yml' --delete-excluded ./ user@staging.example.com:/html/wp-content/themes/_tmp - ssh -p22 user@staging.example.com "mv /html/wp-content/themes/fbn /html/wp-content/themes/_old && mv /html/wp-content/themes/_tmp /html/wp-content/themes/fbn" - ssh -p22 user@staging.example.com "rm -rf /html/wp-content/themes/_old" only: - staging
Code language: PHP (php)

At first, we define the deploy stage and the job deploy_staging, who is responsible for pushing changes to the staging server. This job is part of the deploy stage and uses the docker image tetraweb/php:7.1. With before_script, we can run tasks before the actual script — in our case the preparation for the SSH connection and install the rsync package. The SSH steps are again from the GitLab docs. To get it working, you have to set a variable STAGING_PRIVATE_KEY in GitLab which stores the private SSH key.

The script part of the job connects to the staging server via SSH and creates the _tmp directory in the theme folder if it not already exists. After that, we move the files from the git repo to this folder via rsync. Then the folder moving is done. With only we can specify a branch for which the job should be done — otherwise, the job would be run after each push because the file is in the master branch, too.

The deployment job looks much like the above one (just insert the code after the code of the staging job):

deploy_production: stage: deploy image: tetraweb/php:7.1 before_script: - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )' - mkdir -p ~/.ssh - eval $(ssh-agent -s) - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config' - ssh-add <(echo "$STAGING_PRIVATE_KEY") - apt-get install rsync script: - ssh -p22 user@example.com "mkdir -p /html/wp-content/themes/_tmp" - rsync -rav -e ssh --exclude='.git/' --exclude='.gitlab-ci.yml' --delete-excluded ./ user@example.com:/html/wp-content/themes/_tmp - ssh -p22 user@example.com "mv /html/wp-content/themes/fbn /html/wp-content/themes/_old && mv /html/wp-content/themes/_tmp /html/wp-content/themes/fbn" - ssh -p22 user@example.com "rm -rf /html/wp-content/themes/_old" only: - master
Code language: PHP (php)

Because the before_script part is equal to both jobs, we could also move it outside the jobs at the level of stages.

And that is it.

13 reactions on »Auto deployment to staging and production server with GitLab CI«

  1. Hi.

    I did not understand the operation of private key configuration (STAGING_PRIVATE_KEY).

    Would be the private key of my server that will receive the deploy files, correct?

  2. Hey Florian.

    I faced an error in the following:

    $ ssh -p22 root@MY_REMOTE_HOST_IP "mkdir -p /html/wp-content/themes/_tmp"
    Host key verification failed.
    ERROR: Job failed: exit code 1

    I use my remote server's private key as a "$STAGING_PRIVATE_KEY".
    Can't figure out what's the problem in.

    Any suggestions?

    1. Hi Valeriy,

      yes, there was an error in my code example – some time ago I had a bug that removed backslashes from connected translations when updating a post, and it seems I updated the German version of this post in that time…

      Updated the code now: it has to be \n\tStrictHostKeyChecking no\n\n in both before_script parts, not ntStrictHostKeyChecking nonn 🙂

      Hope that fixes the problem!

      Best,
      Florian

  3. Hey Florian.

    I faced an error in the following:

    ssh -p22 Administrator@myserver.com "mkdir -p /html/wp-content/themes/_tmp"
    Warning: Permanently added 'myserver.com ' (ECDSA) to the list of known hosts.
    Permission denied, please try again.
    Permission denied, please try again.
    Permission denied (publickey,password).
    ERROR: Job failed: exit code 1

    I have added the private key in variables but still facing the issue.

      1. Hey Florian,

        I have not provided any passphrase while generating the key on my server.

        Steps I have done are:
        Generated the ssh key using ssh-keygen without giving any passphrase.
        Then added the generated private key to gitlab variables.

        Is there anything I am missing.

        Regards,
        Shivani

          1. Hello, im facing the same issue, i have been googling for hours and cannot figure it out. I created it with no passphrase or anything and still not get it to connect. What do you mean with "added the key to the server " ? thank you please help me!!!

Mentions

Leave a Reply

Your email address will not be published. Required fields are marked *