Troubleshooting: CLI vs Direct API vs Proxy Endpoints

Use this guide when a Forgejo operation fails and you need to decide whether the problem is in:

the thfg CLI layer
the Forgejo Proxy layer
the upstream Forgejo instance

1. Remember the path rules first

The ecosystem now has one default contract:

Forgejo Proxy is the canonical API surface
API families live under /api/v1/...
/health stays rooted
ActivityPub keeps rooted public federation URLs, while the runtime also accepts /api/v1/activitypub/... aliases for API clients

That means:

use thfg <command> first
use thfg api /... for raw /api/v1 calls
use curl "$PROXY_URL/health" for health checks
use thfg activitypub ... or thfg api /activitypub/... for canonical ActivityPub access
use curl "$PROXY_URL/activitypub/..." when you need the rooted public federation URL itself

If you are fixing older automation, read Migrate to the normalized /api/v1 API surface.

2. Start with `thfg`

The CLI is still the fastest way to surface auth, config, and host-selection problems.

thfg doctor
thfg config list --output-format json

Check these fields first:

Config key	What it controls
`host`	Upstream Forgejo instance
`proxyUrl`	Canonical proxy base URL
`token`	API authentication
`username` / `password`	WebUI-backed proxy endpoints such as Projects

Common symptoms:

Symptom	Likely cause
`thfg project ...` returns 401/404	`proxyUrl` missing, wrong, or WebUI credentials missing
`thfg api /...` returns 404 for a proxy feature	The caller is still using an old path shape
`thfg activitypub ...` works but rooted `curl "$PROXY_URL/activitypub/..."` behaves differently	You are comparing the canonical client alias with the rooted public federation URL
All commands fail with 401	Token missing or expired

3. Pick the right raw fallback

Use this matrix when you need to reproduce a CLI call manually.

Task	Preferred command	Raw fallback
List org projects	`thfg project list --org <org>`	`curl {proxyUrl}/api/v1/orgs/{org}/projects`
List project columns	`thfg project columns --id <id>`	`curl {proxyUrl}/api/v1/projects/{id}/columns`
List project cards	`thfg project cards --id <id> --column <col-id>`	`curl {proxyUrl}/api/v1/projects/{id}/columns/{col-id}/cards`
Add/move an issue card	`thfg project add-issue` / `thfg project move-card`	`curl -X POST {proxyUrl}/api/v1/projects/{id}/columns/{col-id}/cards`
List repo labels	`thfg label list --repo <owner/repo>`	`curl {proxyUrl}/api/v1/repos/{owner}/{repo}/labels`
List org labels	`thfg label list-org --org <org>`	`curl {proxyUrl}/api/v1/orgs/{org}/labels`
Debug failed Actions run	`thfg run diagnose --repo <owner/repo>`	`curl {proxyUrl}/api/v1/repos/{owner}/{repo}/actions/runs/latest/diagnostics`
Fetch latest run logs	`thfg run logs --repo <owner/repo>`	`curl {proxyUrl}/api/v1/repos/{owner}/{repo}/actions/runs/latest/logs`
Health check	—	`curl {proxyUrl}/health`
ActivityPub actor	`thfg activitypub ...`	`thfg api /activitypub/...` or `curl {proxyUrl}/activitypub/...`

4. Reproduce the normalized proxy call

When a CLI command fails, first reproduce it against the proxy.

TOKEN="$FORGEJO_TOKEN"
PROXY_URL="https://forgejo-proxy.hochguertel.work"
ORG="forgejo-proxy"
REPO="forgejo-proxy/forgejo-proxy-cli"

Projects

# List projects in an org
curl -s -H "Authorization: token $TOKEN" \
  -H "X-Webui-Username: $FORGEJO_WEBUI_USERNAME" \
  -H "X-Webui-Password: $FORGEJO_WEBUI_PASSWORD" \
  "$PROXY_URL/api/v1/orgs/$ORG/projects" | jq '.[] | {id, title}'

# List columns for project 2
curl -s -H "Authorization: token $TOKEN" \
  -H "X-Webui-Username: $FORGEJO_WEBUI_USERNAME" \
  -H "X-Webui-Password: $FORGEJO_WEBUI_PASSWORD" \
  "$PROXY_URL/api/v1/projects/2/columns" | jq '.[] | {id, title}'

# Move issue #7 into column 7
curl -s -X POST \
  -H "Authorization: token $TOKEN" \
  -H "X-Webui-Username: $FORGEJO_WEBUI_USERNAME" \
  -H "X-Webui-Password: $FORGEJO_WEBUI_PASSWORD" \
  -H "Content-Type: application/json" \
  -d '{"issue_id":7,"owner":"forgejo-proxy","repo":"forgejo-proxy-cli"}' \
  "$PROXY_URL/api/v1/projects/2/columns/7/cards"

Native Forgejo data through the proxy

# List repo labels through the proxy's canonical /api/v1 surface
curl -s -H "Authorization: token $TOKEN" \
  "$PROXY_URL/api/v1/repos/$REPO/labels" | jq '.[] | {id, name}'

# Compare the authenticated user response
thfg api /user
curl -s -H "Authorization: token $TOKEN" "$PROXY_URL/api/v1/user" | jq .

Actions

# Latest run diagnostics
curl -s -H "Authorization: token $TOKEN" \
  "$PROXY_URL/api/v1/repos/$REPO/actions/runs/latest/diagnostics" | jq .

# Enriched jobs payload with per-step timings
curl -s -H "Authorization: token $TOKEN" \
  "$PROXY_URL/api/v1/repos/$REPO/actions/runs/latest/jobs" |
  jq '.[] | {job_index, attempt, name, status, step_count: (.steps | length)}'

# Logs for failed jobs only
curl -N -H "Authorization: token $TOKEN" \
  "$PROXY_URL/api/v1/repos/$REPO/actions/runs/latest/logs?job=failed"

# CLI view on the same enriched job payload
thfg run jobs "$REPO" --latest
thfg run performance "$REPO" --latest

5. Only then compare with direct Forgejo

If the proxy call fails, compare it with upstream Forgejo to isolate the layer that is broken.

BASE_URL="https://pastoral-oyster.pikapod.net"

# Compare a native endpoint directly against Forgejo
curl -s -H "Authorization: token $TOKEN" \
  "$BASE_URL/api/v1/repos/$REPO/labels" | jq '.[] | {id, name}'

Interpret the results like this:

Proxy result	Direct Forgejo result	Likely problem
Fails	Succeeds	Proxy bug, proxy auth forwarding bug, or wrong proxy path
Succeeds	Fails	Upstream Forgejo limitation or host-specific behaviour
Both fail	Same error	Token, permission, repo/org name, or upstream data problem
Both succeed, CLI fails	CLI argument parsing or config resolution issue

6. Check the root exceptions separately

Do not troubleshoot /health through thfg api. For ActivityPub, first decide whether you are testing the canonical client alias or the rooted public federation URL.

# Health
curl -s "$PROXY_URL/health"

# ActivityPub
thfg activitypub instance actor
thfg activitypub repo actor "$REPO"
thfg api /activitypub/actor

If the canonical alias works but the rooted path does not, reproduce the exact rooted URL with curl and inspect ingress/public-path behavior separately.

7. Validate the path shape

Many breakages during the normalization cleanup are simple path mismatches.

Legacy shape to remove

/api/v1/proxy/...

Canonical shape to use

/api/v1/...

For project card writes, the normalized path is column-scoped:

POST /api/v1/projects/{project_id}/columns/{column_id}/cards

8. Escalation checklist

Before opening an issue, collect these artifacts:

thfg doctor
thfg config list --output-format json (redact secrets)
the failing proxy curl -v command and output
the matching direct Forgejo curl -v command and output, if applicable
curl "$PROXY_URL/health"
the exact legacy path, if this looks like a normalization regression

If the proxy host itself is asleep before FastAPI starts, the request can still fail before the proxy has a chance to normalize the response into JSON. That deferred availability gap is tracked in forgejo-proxy#18.