mimir

Metadata

Field	Value
Purpose	Long-term Prometheus metrics storage
Cluster	cc-obs-kube, using the jls overlay
Namespace	mimir
State	S3 blocks and the data-mimir-monolith-0 PVC
Last reviewed	2026-10-20

1. Service Overview

The jls Mimir monolith stores metrics forwarded by Prometheus. Prometheus Alertmanager remains the alert-routing service. The owner approved Garage as the replacement S3 backend with fresh buckets; historical IDrive objects were not migrated. The existing application PVC is retained.

2. Architecture Diagram

Alloy -> Prometheus -> mimir-monolith:8080 -> S3 blocks
                           |
                           +-> data-mimir-monolith-0 (WAL/local state)
Mimir ruler -> prometheus-alertmanager.prometheus.svc.cluster.local:9093

3. Deployment Specifications

mimir/fleet.yaml selects cc-obs-kube and builds mimir/overlays/jls, the same Kustomize overlay used manually. It selects grafana/mimir:3.2.0 and a one-replica StatefulSet named mimir-monolith. The bound PVC is 50 Gi, ReadWriteOnce, using the provider storage class. Preserve its claim name.

4. Configuration Guide

mimir/overlays/jls/mimir.yaml supplies the runtime configuration; mimir-rules.yaml and mimir-alerts.yaml supply ruler rules. mimir-garage-credentials supplies access_key_id, secret_access_key and endpoint through envFrom, with configuration environment expansion enabled. The existing tracked generator input is known secret-management debt; do not display, copy or commit credential values. Garage credentials are provisioned out of band. The endpoint is garage.garage.svc.cluster.local:3900, region garage, HTTP and path-style lookup; buckets are mimir and mimir-tsdb. See Garage for storage and bootstrap prerequisites.

5. Access Protocols

Internal clients use mimir-monolith.mimir.svc.cluster.local:8080. The public route is https://mimir.mutana.site, with the existing Traefik middlewares. Multitenancy is disabled in the current overlay.

6. Operations and Observability

Check StatefulSet readiness, pod events and the PVC before diagnosing object-store authorization. Prometheus scrapes the monolith; preserve the existing scrape arrangement. See the runbook.

7. Backup and Recovery Notes

Preserve the local WAL/PVC and the existing S3 blocks together. A configuration revert does not migrate object-store data. Confirm backup coverage and the object-store migration plan with the owner; no data restore was tested here.

8. Release and Change Notes

The cc-obs-kube recovery changes Fleet cluster selection, retaining the jls overlay, image and state. The local integration registers Mimir documentation with the existing documentation gate. Garage replaces IDrive for this overlay; the old credential generator remains available for explicit rollback.