helmfile

Still needs a little bit of polishing:

• CI is failing
• helmfile lint complains
• the init secret config is now missing the chmod when deployed over helmfile shouldn't be an issue: it was only required for the backup through gitlab task-runner chart, which we still don't use
• update the README
• provide prod environment for packet

A little bit of background:

We are trying to move away from GKE. For that, we should probably provide a smooth transition (see <BUG missing> and freedesktop#138 (comment 701298)).

Also, to ensure a better downtime (close to 0) when upgrading, making use of the praefect chart that ensures multiple gitaly pods seems like a good option. However, this chart is not completely ready for hanlding multiple gitaly sources (external and internal). That's where helmfile comes in handy as it can personalize the k8s yaml before they are deployed.

So do the groundwork to allow for a smoother transition with helmfile.

How it will look like:

> ls
helm-gitlab-config helm-gitlab-omnibus helm-gitlab-secrets

> cd helm-gitlab-omnibus

# check if there are any changes
> helmfile -e gke diff

# apply interactively
> helmfile -i -e gke apply

Cc: @daniels