forgejo-alert-bridge¶

Metadata¶

Field	Value
Service	forgejo-alert-bridge
Purpose	Receives Prometheus Alertmanager webhooks and files Forgejo issues for persistent pod or deployment problems, attaching pod logs and auto-closing on resolve
Criticality	Tier 3
Owner	Platform owner
Clusters	jls
Namespace	prometheus (shared with the Prometheus stack)
Exposure	none (cluster-internal only)
Stateful	no
Backup class	none
RPO / RTO	N/A - stateless; Forgejo is the source of truth
Last reviewed	2026-08-10

1. Service Overview¶

The bridge is a single-replica Flask app exposed only inside the cluster. When a ticket: "true" alert fires for more than 2 days, Alertmanager posts the webhook to the bridge, which creates a Forgejo issue in kardudu/k8s enriched with the alert labels, annotations, and the last 50 lines of pod logs. When the alert resolves, Alertmanager posts the resolved webhook and the bridge comments on and closes the matching issue.

Summary¶

If the bridge is unavailable, no new issues are filed and no issues auto-close. Existing alerts keep firing to their other receivers; existing Forgejo issues remain open and can be closed manually or by git push closes #N.

2. Architecture Diagram¶

kube-state-metrics -> Prometheus (jls, ns: prometheus)
                       | evaluates forgejo.alerts (ticket:"true")
                       v
                 prometheus-alertmanager (ns: prometheus)
                       | route ticket="true" -> forgejo-issue-bridge
                       | webhook (send_resolved: true)
                       v
                 forgejo-alert-bridge (Deployment, ns: prometheus)
                       | reads pods/pods/log (get,list) for log enrichment
                       v
                 git.mutana.fr/kardudu/k8s (Forgejo issues)

3. Deployment Specifications¶

Field	Value
Workload type	Deployment, 1 replica
Image	`git.mutana.fr/kardudu/forgejo-alert-bridge:<sha>` — the overlay tag is still the `PLACEHOLDER_SHA` build marker; pin the real Git SHA from the first successful `build-forgejo-alert-bridge.yaml` run before relying on the deployment
ServiceAccount	`forgejo-alert-bridge`
ClusterRole	`forgejo-alert-bridge` (cluster-scope)
Service	ClusterIP, port 8080

ClusterRole platform reason: the bridge enriches issues with recent pod logs. Alertmanager alerts carry no namespace, so the bridge must read pods and pods/log across all namespaces. Granted verbs are get and list only. There is no watch, no write, and no access to secrets, events, or any workload controller resource.

4. Configuration Guide¶

Non-secret config in forgejo-alert-bridge-config ConfigMap:

Key	Value
`FORGEJO_URL`	`https://git.mutana.fr`
`FORGEJO_OWNER`	`kardudu`
`FORGEJO_REPO`	`k8s`
`LOG_TAIL_LINES`	`50`

The alert rules and the Alertmanager route are defined in prometheus/overlays/jls/values.yaml, not in this workload. See the runbook for the matching conditions and thresholds.

5. Access Protocols¶

The bridge has no IngressRoute and no external exposure. Alertmanager reaches it over the in-cluster Service forgejo-alert-bridge.prometheus.svc.cluster.local:8080.

6. Operations and Observability¶

Readiness and liveness probes hit the unauthenticated GET /healthz endpoint. Logs are structured plain text to stdout. There is no metrics endpoint; the bridge is observed via its effect (Forgejo issues appearing and closing) and via Alertmanager delivery logs.

7. Backup and Recovery Notes¶

None. The bridge is stateless and holds no data. Recovery is a redeploy from Git. Forgejo issues already filed are independent records.

8. Release and Change Notes¶

2026-07-04: initial integration. Three alert rules covering CreateContainerConfigError, CrashLoopBackOff, ImagePullBackOff, ErrImagePull, CreateContainerError, InvalidImageName (collapsed into one rule), stuck Pending pods, and unavailable deployment replicas, all at for: 2d.