CodeVoyager

Webhook server

Edit Page

Webhook server

adnanh/webhook is a lightweight incoming webhook server to run shell commands, written in Go. Easily install with:

sudo apt install webhook

Using the apt method like this already created a service for me. As seen by service webhook status command, service configuration is stored in /etc/webhook.conf. It is enabled by default and that can be changed using standard systemctl enable/disable webhook.

We can manually test configuration file:

webhook -hooks /etc/webhook.conf --verbose

Also use this directive to debug webooks after stopping the service temporarily.

Modify service to log messages

Edit service file by typing:

systemctl edit --full webhook

and add --verbose to line ExecStart like:

[Service]
ExecStart=/usr/bin/webhook -nopanic -hooks /etc/webhook.conf --verbose

And then monitor log file with:

tail -f /var/log/syslog -n 500 | grep webhook

Expose test webhook to public

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

# generate a small test script
sudo mkdir -p /opt/webhook
cat << 'EOF' | sudo tee /opt/webhook/test.sh
#!/bin/sh
echo "Hello World, executed as '$(whoami)'"
EOF

cat << 'EOF' | sudo tee /opt/webhook/deploy-treasury.sh
#!/bin/sh
echo "Deploy, executed as '$(whoami)'"
EOF

# make it executable
sudo chmod +x /opt/webhook/test.sh
# and test it
/opt/webhook/test.sh

# webhook config file
cat << EOF | sudo tee /etc/webhook.conf
[
  {
    "id": "test",
    "execute-command": "/opt/webhook/test.sh",
    "command-working-directory": "/opt/webhook",
    "include-command-output-in-response": true
  }
]
EOF

Let’s test that webhook:

curl http://localhost:9000/hooks/test

Configure a reverse proxy in nginx with typical setup that looks like

location /hooks/ {
    proxy_pass http://127.0.0.1:9000;
}

but I wanted to change URL a little bit

location /webhook/ {
    proxy_pass http://127.0.0.1:9000/hooks/;
}

Trailing slashes are extremely important in nginx proxy directive.

So, let’s test our webhook again, this time from outside:

curl https://savioko.com/webhook/test

Integration with GitHub

Generate my secret token with uuidgen and make it permanent, as environment varibale:

1
2

export SECRET_TOKEN=$(uuidgen)
echo "SECRET_TOKEN="$SECRET_TOKEN | sudo tee -a /etc/environment

Trigger webhooks from GitHub

X-Hub-Signature is recommended for GitHub https://docs.github.com/en/developers/webhooks-and-events/webhooks

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21


# `sponge` is needed to overwrite file. install with `sudo apt install -y moreutils`
#
cat << EOF | sudo jq -s '. + $fn[]' --slurpfile fn /etc/webhook.conf | sudo sponge /etc/webhook.conf
{
  "id": "deploy-treasury",
  "execute-command": "/opt/webhook/deploy-treasury.sh",
  "command-working-directory": "/opt/webhook",
  "include-command-output-in-response": true,
  "trigger-rule": {
    "match": {
      "type": "payload-hash-sha1",
      "secret": "$SECRET_TOKEN",
      "parameter": {
        "source": "header",
        "name": "X-Hub-Signature"
      }
    }
  }
}
EOF

better is: "response-message": "Deployment initiated...",

and test webhook with:

1
2

# payload is empty in this test so we can calculate hmac
curl -H "X-Hub-Signature: sha1=$(echo -n "" | openssl sha1 -hmac "$SECRET_TOKEN" | cut -c 10-49)" https://savioko.com/webhook/deploy-treasury

Now, got to GitHub and inside repository select Settings / Webhooks / Add webhook.

Payload URL: https://savioko.com/webhook/deploy-treasury
Content type: application/json
Secret: ****** (your $SECRET_TOKEN)

Redeploy script

/var/www/notes.cvladan.com/

“-e” if any command fails, the script will terminate immediately so trap system works as expected

Block secret content

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

# Hidden content behind a cookie
#
if ($uri = "/token/") { rewrite ^ /token/ last; }

location = /token/ {
    default_type text/html;
    add_header Set-Cookie "token=1;Path=/;Max-Age=31536000";
    return 200 '<!DOCTYPE html>\n<h2>Hooray ;)</h2>\n<p>... now you can <a href="/">access the whole site</a></p>';
}

# Block access to URL's containing keyword `secret`
if ($uri ~* "^(.*)secret(.*)") { set $condition BO; }
if ($http_cookie !~* 'token') { set $condition "${condition}TH"; }
if ($condition = "BOTH") { rewrite ^ /401-error/ last; }

location = /401-error/ { default_type text/html; return 401 '<!DOCTYPE html>\n<pre>  401 | Restricted Access</pre>'; }

I need proper identity file a.k.a. “Deploy Keys”

You can’t reuse a deploy key for multiple repositories so we must create separate keys for every submodule

1
2
3

# comments are inserted in public key 
ssh-keygen -N '' -f ~/.ssh/id_rsa.github.hugo -C "Deploy 'hugo' repo on server" 
ssh-keygen -N '' -f ~/.ssh/id_rsa.github.notes -C "Deploy 'notes' repo on server"

Now, put content of .ssh/id_rsa.github.*.pub as “Deploy Keys” on corresponding GitHub repositories; the main one and in the submodule repository.

Failed attempts

I’ve tried using multiple identities in .ssh/config but that doesn’t work

cat << EOF >> ~/.ssh/config
Host github.com
    IdentityFile ~/.ssh/id_rsa.github.hugo
    IdentityFile ~/.ssh/id_rsa.github.notes
    IdentitiesOnly=yes
EOF

and I’ve also tried using env variable GIT_SSH_COMMAND also with multiple identities…

Specifying multiple identities (-i) doesn’t work; don’t know why?.
Option (-F /dev/null) forces it to ignore any configuration file, global or per user.

GIT_SSH_COMMAND="ssh -o IdentitiesOnly=yes -i $HOME/.ssh/id_rsa.github.hugo -i $HOME/.ssh/id_rsa.github.notes -F /dev/null"

Finally, a working solution

I have to split into separate directives and specify identity on a submodule basis.

# clone main repo without submodules
#
export GIT_SSH_COMMAND="ssh -o IdentitiesOnly=yes -i $HOME/.ssh/id_rsa.github.hugo -F /dev/null"
git clone "git@github.com:cvladan/hugo.git" "$HOME/hugo-staging_area"

# pull submodules
#
export GIT_SSH_COMMAND="ssh -o IdentitiesOnly=yes -i $HOME/.ssh/id_rsa.github.notes -F /dev/null"
git submodule update --init
# update to latest commit
git submodule update --remote

Finalized script placed in /opt/webhook/deploy-treasury.sh that will deploy my repo.

Error on GitHub

We couldn’t deliver this payload: timed out

Turns out that GitHub has a 10-second timeout set on webhooks.

date 16. Mar 2021 | modified 10. Jun 2024
filename: Github & Webhooks
Article Content