Howto: automated deployment process with git hooks

[gebeer]

Hi all,

I got inspired to writing this little tutorial by @FireWire's post. We are using the same deployment workflow that he mentions for all new projects. I tried different approaches in the past, also github Actions and gitlab Runners. Setting those up always felt like a PITA to me. Especially since all I wanted was to automatically deploy my project to staging or live on push

Whom this is for

Single devs or teams who want to streamline their deployment process with native git methods.

Requirements

• shell access to the server
• git installed on the server and locally

If you don't have shell access and git on the server, upgrade or switch hosting.

Walkthrough

In this example we will be using github to host our code and a server of our choice for deployment. The project is called myproject.

Step 1 (github)

Create a repository named myproject. Let's assume that is available at git@github.com:myaccount/myproject.git. This is our remote URL.

Step 2 (local)

create a project in the folder myproject and push it to github like you usually would. 

The remote of your project should now read like this inside the myproject folder

$ git remote add origin git@github.com:myaccount/myproject.git
$ git remote -v
origin git@github.com:myaccount/myproject.git (fetch)
origin git@github.com:myaccount/myproject.git (push)

Step 3 (server)

Login via ssh to the server and go to the document root. We assume this to be '/var/www/'. We further assume the command for connecting to our server via ssh is 'ssh myuser@myserver'.

Go to web root, create a directory that will hold a bare git repo, cd into it and create the bare git repository. A bare repo is one that does not contain the actual project files but only the version control information.

cd /var/www/
mkdir  myproject-git && cd myproject-git
git init --bare

Step 4 (server)

Create the root directory for your ProcessWire installation

cd /var/www/
mkdir  myproject

Step 5 (local)

Now we add information about the bare git repo to our local git config. So that when we push changes, they will be pushed both to github and to the bare git repo on our server. Inside our project folder we do

git remote set-url --add --push origin myuser@myserver:/var/www/myproject-git

After that we need to add the original github push origin again, because it got overwritten by the last command

git remote set-url --add --push origin git@github.com:myaccount/myproject.git

Now the list of remotes should look like this

$ git remote -v
origin git@github.com:myaccount/myproject.git (fetch)
origin myuser@myserver:/var/www/myproject-git (push)
origin git@github.com:myaccount/myproject.git (push)

We have one fetch and 2 push remotes. This means that if you push a commit, it will be pushed to both github and your server repo.

Step 6 (server)

Here comes the actual deployment magic. We are using a git hook that fires a script after every push. This hook is called a post-receive hook. We move into the directory with the bare repository, change to the hooks directory, create the file that triggers the hook and open it for editing with nano

$ cd /var/www/myproject-git/hooks
$ touch post-receive
$ nano post-receive 

Now we paste this script into the open editor and save it

#!/bin/bash
# Bare repository directory.
GIT_DIR="/var/www/myproject-git"
# Target directory.
TARGET="/var/www/myproject"
while read oldrev newrev ref
do
    BRANCH=$(git rev-parse --symbolic --abbrev-ref $ref)
    if [[ $BRANCH == "main" ]]; then
        echo "Push received! Deploying branch: ${BRANCH}..."
        # deploy to our target directory.
        git --work-tree=$TARGET --git-dir=$GIT_DIR checkout -f $BRANCH
    else
        echo "Not main branch. Skipping."
    fi
done

What this does is checking out (copying) all files that are in the repository to our ProcessWire root directory every time we push something. And that is exactly what we wanted to achieve.

This example setup is for a single branch. If you wanted to make this work with multiple branches, you need to make some small adjustments. Let's assume you have one staging and one live installation where web root for live is at /var/www/myproject and for staging at /var/www/myproject-staging

In Step 4 above you would create a second dir

$ cd /var/www/
$ mkdir  myproject
$ mkdir  myproject-staging

And the content of the post-receive hook file could look like

#!/bin/bash
# Bare repository directory.
GIT_DIR="/var/www/myproject-git"

while read oldrev newrev ref; do
       BRANCH=$(git rev-parse --symbolic --abbrev-ref $ref)
       if [ $BRANCH == "master" ]; then
               TARGET="/var/www/myproject"
       elif [ $BRANCH == "staging" ]; then
               TARGET="/var/www/myproject-staging"
       else
               echo "Branch not found. Skipping Deployment."
       fi
       # deploy only if var TARGET is set
       if [ -z ${TARGET} ]; then
               echo "no target set"
       else
               echo "STARTING DEPLOYMENT..."
               echo "Push to ${BRANCH} received! Deploying branch: ${BRANCH} to: ${TARGET}"
               # deploy to our target directory.
               git --work-tree=$TARGET --git-dir=$GIT_DIR checkout -f $BRANCH
       fi
done

Now everything you push to your staging branch will be deployed to /var/www/myproject-staging. Commits to master branch to /var/www/myproject

We really do enjoy this deployment workflow. Everything is neat and clean. No need to keep track of which files you uploaded already via SFTP. Peace of mind :-)

I basically put together bits and pieces I found around the web to set this up. Would be eager to see how you implement stuff like this.

[wbmnfktr]

Wow... this could be a thing I try next weekend.

It looks really nice. Like something I always wanted to create.

Right now... I ssh into my stage and live setups to pull my changes... but this.

Thanks a lot @gebeer!

 

Yet... one thing... how do you setup your domains/sub-domains for this.
I imagine that myproject-staging would be sub.domain.tld, right?

Does the script notice merges from staging to master/main?

[gebeer]

2 hours ago, wbmnfktr said:

Yet... one thing... how do you setup your domains/sub-domains for this.
I imagine that myproject-staging would be sub.domain.tld, right?

Exactly. staging.myproject.tld would point to folder /var/www/myproject-staging

2 hours ago, wbmnfktr said:

Does the script notice merges from staging to master/main?

We never merge things on the server, always merge stuff locally first and then push. For the post-receive hook fires only after a push. The most important thing to remember when working with this strategy is: always pull/merge before you push. 

[wbmnfktr]

2 hours ago, gebeer said:

We never merge things on the server, always merge stuff locally first and then push.

I love this workflow... but does your setup know this.
I guess it does... so far. Didn't try it for now. But... that's why I asked.

I personally don't merge on the servers as well, yet... there are others in my team that do this once in a while and therefore create some fun for the whole team. ?

 

Nonetheless... I will give your setup a try as it almost looks perfect in a controlled setup.

Yet again... thanks for sharing.

[gebeer]

9 hours ago, wbmnfktr said:

yet... there are others in my team that do this once in a while and therefore create some fun for the whole team.

You need to give them this mantra: "pull - merge - push" and let them recite it 100 times a day at least ?

 

9 hours ago, wbmnfktr said:

I love this workflow... but does your setup know this.

It only knows that it needs to deploy on push. So it won't deploy when someone merges stuff on the server. From that perspective it is pretty fail safe.

Give it a try, think you'll love it.

[wbmnfktr]

On 1/30/2022 at 2:09 PM, gebeer said:

You need to give them this mantra: "pull - merge - push" and let them recite it 100 times a day at least ?

I did for way too long. Therefore they don't like me anymore. ?
But yes... it's more of a culture to work this way but on the other hand... we all are free-devs with super individual workflows. I can't and don't wanna blame anyone for this. At the end everything was fine*.

* 100s of merge conflicts later, with rebase and such things. ?

[Andi]

If you're following the steps above, don't forget  (like I did earlier) to make the bash script executable..

In Step 6, after editing your post-recieve file

$ cd /var/www/myproject-git/hooks
$ chmod u+x post-receive

Very cool guide @gebeer, I was toying around with a webhook from Gitea calling on a local php script earlier today, which runs okay, but this seems way more elegant.. And definitely easier to set up ? Thanks!

[bernhard]

I've started with this guide as well, so thx a lot again for sharing that @gebeer but I want to mention that for anybody interested in automatic deployments RockMigrations ships with a ready-made deployment solution that you can use: https://github.com/baumrock/RockMigrations#deployments

[Andi]

@bernhard you're on fire ?

Looks great but I'm currently on a self-hosted Gitea instance, and it looks like they're not planning on supporting Github Actions anytime soon..

https://github.com/go-gitea/gitea/issues/12673#issuecomment-684968986

Thanks for the heads up though, I'll keep an eye on RM!

[bernhard]

I'm sure you can do the same in gitea, we are using gitlab at work and it also works. The whole idea is that the deployment is done by php so you don't have to copy and paste complicated deployment scripts over and over again. And we can improve one script all the time instead of having 100 slightly different versions... 

But it's very opinionated of course. But works great for me and us.