Or cert-manager can not renew the TLS cert
Signed-off-by: Benjamin Tissoires <benjamin.tissoires@gmail.com>

Signed-off-by: Benjamin Tissoires <benjamin.tissoires@gmail.com>

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>

TOML requires us to quote 'complex' things.

We only get NVMe disks for our runners now, and they're much bigger than
the non-NVMe ones, so exclude the latter. Given that we only have two
disks, there's no point in using RAID5 as these machines are
replaceable, so might as well just make it 0.

This brings us back into sync with the latest stuff so we can provision
new runners again. Unfortunately nothing more can be provisioned more
from NJ, nor on the old runner type, so we switch to the new equivalent.

Signed-off-by: Daniel Stone <daniels@collabora.com>

Or cert-manager can not renew the TLS cert
Signed-off-by: Benjamin Tissoires <benjamin.tissoires@gmail.com>

Signed-off-by: Benjamin Tissoires <benjamin.tissoires@gmail.com>

Signed-off-by: Benjamin Tissoires <benjamin.tissoires@gmail.com>

Avoids carrying on with the next commands when one fails.
Signed-off-by: Simon Ser <contact@emersion.fr>

This reverts commit f37079d3. batch-mode
seems to be broken and MRs getting batched aren't being merged.

Often we pick up the file from the gitlab web UI,
meaning the master branch is the one selected.
Signed-off-by: Benjamin Tissoires <benjamin.tissoires@gmail.com>

Signed-off-by: Benjamin Tissoires <benjamin.tissoires@gmail.com>

Should help a lot with the timeouts we've been hitting lately.

useful for cleaning the data of the registry cache
Signed-off-by: Benjamin Tissoires <benjamin.tissoires@gmail.com>

compactor fails with
  caller=marker.go:214 msg="failed to process marks"
    path=/data/loki/boltdb-shipper-compactor/retention/markers/1632476105028196588
    err="open /tmp/marker-view-873575110: read-only file system"

So use the table_manager to do table retention

Note that the retention is currently 21 days to monitor if that works
Signed-off-by: Benjamin Tissoires <benjamin.tissoires@gmail.com>

Signed-off-by: Benjamin Tissoires <benjamin.tissoires@gmail.com>

seems that loki-stack is behind
Signed-off-by: Benjamin Tissoires <benjamin.tissoires@gmail.com>

Signed-off-by: Benjamin Tissoires <benjamin.tissoires@gmail.com>

every now and then our OSDs get OOM-killed. The usage for those is rather
scary (sometimes 30GB). We have big machines, but ram is not extensible.

Following https://github.com/rook/rook/issues/5811

, I have now set up
some defaults values. They should definitively leave room for other
processes and can be tuned if we realize they are not big enough.

For the record, before applying this patch:
`ceph tell osd.25 config show | jq '.osd_memory_target'`
  -> 216297281945 (216 GB)
after
  -> 4294967296 (4GiB, which matches the requested memory)
Signed-off-by: Benjamin Tissoires <benjamin.tissoires@gmail.com>

the automated detection based on the mount table is not working very well,
it finds multiple duplicates.

Only report the disks we know we have to mount to have a restricted set
of data without duplicates
Signed-off-by: Benjamin Tissoires <benjamin.tissoires@gmail.com>

The helm charts are now capable of rendering our configuration.
Convert the toml into yaml and include that in the charts.
Signed-off-by: Benjamin Tissoires <benjamin.tissoires@gmail.com>

Most of the time, we don't need to have something as heavy as
ElasticSearch for the plain logs. Grafana loki+grafana is enough to
browse through the logs and is lighter in term of disk space.

In ES, with a replica set of 2 and 3 instances of 60GB we barely manage
to store 7 days of logs.
After one day of storing everything into loki, the space used is only 3GB
meaning the current 100GB disk affected to it will last ~30 days.

Note that we need to have the secret fdo-loki-grafana created before hand.
Signed-off-by: Benjamin Tissoires <benjamin.tissoires@gmail.com>

We want to be able to embed and use trace images, but can't do that with
no CORS policy. Allow CORS for unauthenticated requests, but continue to
deny CORS for any authenticated (& potentially private) requests.

Should be ready for the future now :)
Signed-off-by: Benjamin Tissoires <benjamin.tissoires@gmail.com>

the aggregator is not working currently, so this messes up the agents
Signed-off-by: Benjamin Tissoires <benjamin.tissoires@gmail.com>