Download private Terraform modules authenticating as a GitHub App on CI/CD

Photo by Call Me Fred on Unsplash

Disclaimer

This is a quite technical read, that pretends to show a real world problem and solution. Requires knowledge on Terraform, git and GitHub Actions. The code is along the article, and I will try to explain what it does in an abstract way so it’s easier to understand, however there’s links to official docs at the end as well so if you don’t get what the code does, go check docs!

Why would I even do that?

Okay, so two problems arise:

1. Security is a concern: public key auth is secure, but having your private key on a runner even as a secret, impose a risk. So we would like to have temporary and read-only credentials. This is the reason why we decided to go with a GitHub App.
2. Lots of code invoking terraform modules in SSH format : so, git has 2 main forms of cloning repos, one is SSH and the other is HTTP. Authentication via GH App is done with a token, and this token only works with HTTP auth, so we needed a way to change the URLs on the fly since we cannot afford to update all the IaC code to call modules in HTTP format.

In a nutshell, we require to add to our CICD Terraform workflow, an action that authenticates and sets-up a valid token from GitHub, and also changes URL formats on the fly for git clone commands (done by Terraform init)

The solution

The python way 🐍

There’s a module called PyGithub, which basically is a python interface for their rest API, and that sounds much better than using curl and bash or the request module, so I wrote a little CLI program that can be called as “python ops.py” and it will authenticate as the GH App and set the token in ~/.git-credentials as https://app-name:token@github.com/ (Code on https://github.com/adsanz/gh-app-auth-example/blob/main/ops.py)

Then the second problem, on the action we use Git InsteadOf feature, to add 2 things:

1. Change git@github.com: to https://github.com/
2. Change ssh://git@github.com/ to https://github.com/

We also set-up the credential.helper to “store” so our .git-credentials file is relevant to git. (Code on https://github.com/adsanz/gh-app-auth-example/blob/main/action.yml)

Note: you need to set-up the HOME variable on the workflow before launching git config commands otherwise the workflow will never use global git config.

Then we have the CICD (Code on https://github.com/adsanz/gh-app-auth-example/blob/main/.github/workflows/test-ops.yml) which basically runs the action and clones a private repository to demonstrate the usage of the credentials and git config previously set by our action. Neat!

Note: Secrets APP_ID, INSTALLATION_ID and PRIVATE_KEY are required to set-up on the repo in order to get a valid token.

Docs

Register your app: https://docs.github.com/en/apps/creating-github-apps/registering-a-github-app/registering-a-github-app

Auth: https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/authenticating-as-a-github-app

GitHub Action syntax: https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions

GitHub workflow syntax: https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions

PyGithub: https://github.com/PyGithub/PyGithub

Git instead of: https://git-scm.com/docs/git-config#Documentation/git-config.txt-urlltbasegtinsteadOf

Git credentials: https://git-scm.com/docs/gitcredentials

See you next time!