Mario Finelli Blog
31 October 2021
For a long time I used to craft and maintain my kubernetes manifests in a series of yaml files that I would apply manually using kubectl apply. A few years ago I discovered helm (and then later helmfile but that's for another post) and then I started pointing to local copies of the charts that I wrote. Now, I use the helm-gcs plugin to actually build and publish the charts and using them has become as easy as using any other third party chart repository.
I'm going to ignore the actual contents of any chart files because that's not really the important part of this post. If you don't already have any charts you can just create a dummy one with helm create chart-name.
The first thing that you will need is to actually install the helm-gcs plugin which is as simple as running helm plugin install https://github.com/hayorov/helm-gcs.git.
In your Google Cloud project you will need to create a storage bucket and then a service account that has access to write to it. The terraform for setting this up is pretty straightforward:
resource "google_service_account" "helm" {
account_id = "helm-github-actions"
display_name = "helm-github-actions"
description = "Service account to store helm charts."
}
resource "google_storage_bucket" "helm" {
name = "yourorg-helm-charts"
location = "US"
}
resource "google_storage_bucket_iam_binding" "helm_write" {
bucket = google_storage_bucket.helm.name
role = "roles/storage.objectAdmin"
members = [
"serviceAccount:${google_service_account.helm.email}",
]
}
# provides storage.buckets.get permission
# https://cloud.google.com/storage/docs/access-control/iam-roles
resource "google_storage_bucket_iam_binding" "helm_read" {
bucket = google_storage_bucket.helm.name
role = "roles/storage.legacyBucketReader"
members = [
"serviceAccount:${google_service_account.helm.email}",
]
}
If you also manage your GitHub repositories using terraform (I recommend it!) then you can automatically load the service account credentials into the repository secrets like so:
data "github_repository" "helm" {
name = "helm-charts"
}
data "github_actions_public_key" "helm" {
repository = data.github_repository.helm.name
}
resource "google_service_account_key" "helm" {
service_account_id = google_service_account.helm.name
}
resource "github_actions_secret" "helm_google_service_account" {
repository = data.github_repository.helm.name
secret_name = "GOOGLE_CLOUD_CREDENTIALS"
plaintext_value = base64decode(google_service_account_key.helm.private_key)
}
If you don't manage your GitHub repositories using terraform then you will need to go to the Google Cloud console download a service account key and copy it into the secrets section of your repository settings manually.
Once you have setup the baseline infrastructure, you just need to create a simple GitHub Actions workflow to actually publish the charts. Here's what I use:
---
name: Package Helm Charts
on:
push:
branches:
- master
paths:
- .github/workflows/helm.yml
- 'k8s/charts/**'
jobs:
main:
name: helm
runs-on: ubuntu-latest
strategy:
matrix:
chart: [app]
steps:
- uses: actions/checkout@v2
- uses: google-github-actions/setup-gcloud@master
with:
service_account_key: ${{ secrets.GOOGLE_CLOUD_CREDENTIALS }}
export_default_credentials: true
- uses: azure/setup-helm@v1
- run: helm plugin install https://github.com/hayorov/helm-gcs.git
- run: helm gcs init gs://yourorg-helm-charts # this is idempotent
- run: helm repo add yourourg gs://yourorg-helm-charts
- run: helm package -u k8s/charts/${{ matrix.chart }}
- name: Check if chart already exists in the repository
id: chart
run: |
if gsutil ls gs://yourorg-helm-charts/$(grep -m1 '^name:' k8s/charts/${{ matrix.chart }}/Chart.yaml | awk '{print $2}')-$(grep -m1 '^version:' k8s/charts/${{ matrix.chart }}/Chart.yaml | awk '{print $2}').tgz 2>/dev/null; then
echo "::set-output name=exists::true"
else
echo "::set-output name=exists::false"
fi
- run: helm gcs push ${{ matrix.chart }}-*.tgz yourorg
if: ${{ steps.chart.outputs.exists == 'false' }}
Let's break down what's going on here. In the main action setup we specify to operate any time there is a push on the master branch. This means we won't accidentally publish/overwrite charts on pull requests, for example. We also have another check to prevent this below, which we'll come to shortly.
We also only run the action if we modify the helm GitHub workflow or if we actually change any of the chart files. I use an infrastructure monorepo so we need to specify that path (and you'll also see it in some of the other commands).
Next we setup our only job (id: main, and name: helm). We set it up to loop over all of our charts by specifying a matrix with option chart. We can then give a list of the charts that we want to build/publish (by directory name). In this case we're only specifying a single chart called app, but we can easily add additional charts by adding additional items to this list.
For the actual steps the first several should be straightforward. We checkout the code, setup gcloud (and more importantly for this workflow gsutil) and authorize it with the credentials that we setup earlier (the secret name obviously needs to match what you used when you loaded the service account key into the GitHub repository secrets), and then install helm itself.
Next, we need to install the helm-gcs plugin into the helm installation that we have on the GitHub runner, so we run the same helm plugin install that we ran on our own machine.
Next, we need to initialize the chart repository (index.yaml) in our cloud storage bucket: helm gcs init gs://yourorg-helm-charts. Technically, we only need to run this command one time to setup the repository, and we could easily do it from our local machine. But as I note in the comment it's idempotent so we can run it every time with no detriment (as compared to the helm-s3 plugin which clobbers an existing repository). This way we have the entire process documented, and we also don't need to run any additional steps regardless if we're starting with a new repository (bucket) or not.
The next step is to actually package up the helm chart helm package -u k8s/charts/${{ matrix.chart }}. The -u flag with download any dependent charts first. Recall that in my setup I keep my charts in an infracode monorepo so I need to specify the full path and then we use the current chart from our chart matrix.
The next step we do because we don't want to ever republish a chart once it has been published. We just do a simple query of the cloud storage bucket using gsutil and some simple greps to get the correct chart name and version and then set an output value for the step depending on if we found the chart or not.
Finally, if the chart name/version combination doesn't exist then we go ahead and push the chart artifact to the repository.
Now, you can add your repository to your local list of repositories with helm repo add yourorg gs://yourorg-helm-charts and then download and install charts: helm fetch yourorg/chart-name and helm install yourorg/chart-name.