nextcloud-appapi Runbook¶

Metadata¶

Field	Value
Service	nextcloud-appapi
Criticality	Tier 3
Owner	Application / platform owner
Namespace	nextcloud-exapps
Clusters	local
Last validated	2026-08-17
Related service page	../services/nextcloud-appapi.md

Trigger Conditions¶

HaRP is not Ready or repeatedly restarts.
occ app_api:daemon:list reports the Kubernetes daemon unavailable.
Context Agent installation, enable, disable, or removal fails.
The MCP endpoint returns 404, 502, or 503 after authentication.
HaRP logs show Kubernetes 401/403 responses or shared-key failures.
A generated ExApp pod is Pending, in ImagePullBackOff, or not Ready.

1. Health Checks¶

kubectl -n nextcloud-exapps get deploy,pod,svc,pvc,ingressroute -o wide
kubectl -n nextcloud-exapps rollout status deploy/nextcloud-appapi-harp
kubectl -n nextcloud-exapps logs deploy/nextcloud-appapi-harp --tail=200
kubectl -n nextcloud get pods -l app=nextcloud

Run AppAPI checks from the active Nextcloud pod:

kubectl -n nextcloud exec deploy/nextcloud -- php occ app_api:daemon:list
kubectl -n nextcloud exec deploy/nextcloud -- php occ app_api:app:list
kubectl -n nextcloud exec deploy/nextcloud -- php occ app:list

Confirm the intended authorization boundary:

kubectl auth can-i --as=system:serviceaccount:nextcloud-exapps:nextcloud-appapi-harp \
  list pods -n nextcloud-exapps
kubectl auth can-i --as=system:serviceaccount:nextcloud-exapps:nextcloud-appapi-harp \
  list secrets -n nextcloud-exapps
kubectl auth can-i --as=system:serviceaccount:nextcloud-exapps:nextcloud-appapi-harp \
  list deployments -n nextcloud

Expected results are yes, no, and no.

2. Troubleshooting Workflows¶

HaRP pod is Pending¶

kubectl -n nextcloud-exapps describe pod -l app.kubernetes.io/name=nextcloud-appapi-harp
kubectl -n nextcloud-exapps get secret nextcloud-appapi-harp
kubectl -n nextcloud-exapps get events --sort-by=.lastTimestamp | tail -40

If the Secret is missing, restore it out of band. Do not generate a replacement unless the AppAPI daemon will be re-registered with that exact value.

Kubernetes API returns 403¶

Check each required permission and compare base/rbac.yaml with HaRP logs. Do not grant cluster-admin, wildcard resources, wildcard verbs, nodes, Secrets, or cluster-scoped access. ClusterIP mode requires no node permission.

Context Agent pod is not Ready¶

kubectl -n nextcloud-exapps get deploy,pod,svc,pvc -o wide
kubectl -n nextcloud-exapps describe pod -l app.kubernetes.io/component=exapp
kubectl -n nextcloud-exapps logs -l app.kubernetes.io/component=exapp --tail=200

Check image pull events, PVC binding, and the generated Service. AppAPI waits up to one hour for image pulling and then applies a shorter startup readiness window. Do not hand-edit generated resources as a lasting fix; correct AppAPI, HaRP, storage, or the app release and let the lifecycle controller recreate them.

If HaRP itself reports Permission denied for /etc/haproxy/spoe-agent.conf, confirm the container still drops all capabilities and adds back only DAC_OVERRIDE. This is required by the pinned upstream directory mode and is paired with a read-only root filesystem.

MCP route fails but the pod is Ready¶

Verify AppAPI and Context Agent are enabled, the daemon is the default, the nextcloud-appapi-harp IngressRoute accepts /exapps/, and the MCP route includes the trailing slash:

https://cloud.mutana.fr/index.php/apps/app_api/proxy/context_agent/mcp/

MCP authentication uses a Nextcloud app password as a bearer token. Never use the HaRP key or primary user password for MCP.

Codex client registration¶

Register the remote MCP server in the Codex user configuration without storing the credential itself:

[mcp_servers.nextcloud-life]
url = "https://cloud.mutana.fr/index.php/apps/app_api/proxy/context_agent/mcp/"
bearer_token_env_var = "NEXTCLOUD_LIFE_AGENT_MCP_TOKEN"
required = false
default_tools_approval_mode = "writes"

Credential: Vaultwarden -> Nextcloud -> life-agent.

Generate a dedicated life-agent app password only when it can be saved directly to Vaultwarden. Inject it as NEXTCLOUD_LIFE_AGENT_MCP_TOKEN into the Codex client process, then restart the desktop app, CLI, or IDE extension. Do not put the value in config.toml, a shell startup file, repository content, or Nextcloud content. required = false keeps Codex usable while the credential is not loaded, and default_tools_approval_mode = "writes" preserves an approval gate for MCP tools that can modify Nextcloud.

3. Disaster Recovery¶

Preserve all nextcloud-exapps PVCs and the live HaRP Secret.
Reconcile nextcloud-appapi/overlays/local.
Restore the Secret through the approved out-of-band process.
Wait for HaRP to become Ready.
Enable AppAPI and confirm the existing daemon registration.
If the registration is absent, register HaRP with the same shared key.
Use AppAPI to reinstall or enable Context Agent; attach/preserve existing application data according to AppAPI behavior.
Validate the authenticated MCP initialization request.

HaRP has no database or PVC of its own. Nextcloud database recovery follows the separate Nextcloud runbook.

4. Scaling and Resource Management¶

Keep HaRP at one replica; it maintains an in-memory route cache and the Deployment uses Recreate strategy to avoid overlap. Observe before tuning:

kubectl -n nextcloud-exapps top pod
kubectl -n nextcloud-exapps describe deploy nextcloud-appapi-harp

The approved HaRP profile is requests 100m CPU / 128Mi memory and limits 500m CPU / 512Mi memory. Change it in the local overlay only after measurement. Context Agent documentation requires at least 1GB of available memory; its runtime resources are generated by AppAPI rather than this Kustomization.

5. Maintenance Procedures¶

Resolve and review a new HaRP OCI digest before changing the image pin.
Check Context Agent compatibility with the installed Nextcloud version before selecting a new App Store release.
Back up any ExApp PVC before removal with remove_data=true.
Rotate the HaRP shared key as one operation: update the out-of-band Secret, restart HaRP, and re-register AppAPI with the same value.
Re-run the three kubectl auth can-i boundary checks after RBAC changes.

6. Rollback Strategy¶

Disable Context Agent through AppAPI.
Preserve the generated PVC and HaRP Secret.
Revert the image or manifest revision through Git/Fleet.
If necessary, unregister kubernetes-harp only after Context Agent is disabled.
Remove the HaRP Deployment and Service while keeping the namespace, Secret, and PVCs until retention is decided.

Core Nextcloud remains usable without this component. Do not delete generated PVCs or rotate credentials as part of a routine manifest rollback.

7. Post-Incident Actions¶

Record whether failure was in Nextcloud AppAPI, HaRP, Kubernetes RBAC, storage, the Context Agent image, or MCP authentication.
Add a changelog fragment for any manual lifecycle or RBAC correction.
Update this runbook with the verified symptom and recovery command.
Confirm no shared key, app password, or bearer token entered logs, tickets, Git, Notes, Collectives, or Tasks.
Revalidate that HaRP has no cross-namespace or Secrets access.