<!--
hoody-proxyLogs Subskill (http)
Auto-generated by Hoody Skills Generator
Generated: 2026-06-20T00:45:10.102Z
Model: mimo-v2.5-pro + fixer:mimo-v2.5-pro
Mode: http


Tokens: 5656

DO NOT EDIT MANUALLY - Changes will be overwritten on next generation
-->

# hoody-proxyLogs.md

## Subskill: Proxy Logs Service

**Service**: hoody-proxyLogs  
**Version**: Current  
**Last Updated**: 2025-11-05

---

## Overview

### What This Service Does

The hoody-proxyLogs service provides centralized logging for all HTTP requests and responses processed by the Hoody Proxy routing system. It captures, stores, and streams proxy activity, enabling debugging, monitoring, and auditing of container service traffic.

**Core Capabilities**:
- **Search & Filter**: Query stored request/response logs with time ranges, status codes, and paths
- **Real-time Streaming**: Open SSE connections to receive new log entries as they occur
- **Statistics**: Aggregate log data for traffic analysis and anomaly detection

### When to Use This Service

| Scenario | Use This Service? |
|----------|-------------------|
| Debugging why a container service returns errors | ✅ Yes — search logs for failed requests |
| Monitoring real-time traffic to a container | ✅ Yes — stream logs via SSE |
| Analyzing traffic patterns and volume | ✅ Yes — use /_logs/stats endpoint |
| Creating or managing containers | ❌ No — use hoody-kit core service |
| Checking container health or status | ❌ No — use container-specific /health endpoints |

### Hoody Philosophy Alignment

This service embodies Hoody's principle of **zero-configuration observability**. Proxy logs are automatically captured for all Hoody Kit container services — no instrumentation code required. Combined with automatic domain routing, developers gain immediate visibility into service behavior without modifying application code.

### Base URL

```
https://{projectId}-{containerId}-proxy-logs-{serviceId}.{node}.containers.hoody.com
```

**Components**:
- `{projectId}` — Your Hoody project identifier (24-character alphanumeric)
- `{containerId}` — The container instance running this proxy-logs service
- `{serviceId}` — The specific service deployment identifier
- `{node}` — The Hoody node where the service is hosted

**Example**:
```
https://proj-abc123def-container-x7y8z9-proxy-logs-svc001.us-east-1.containers.hoody.com
```

> **Note**: Replace these values with your actual project/container/service identifiers from the Hoody Kit deployment.

---

## Common Workflows

### Workflow 1: Retrieve All Recent Logs

Fetch stored request/response logs from the proxy. This is the primary method for debugging issues after they occur.

**Step 1: Call the logs endpoint**

```
curl -s \
  "https://${PROJECT_ID}-${CONTAINER_ID}-proxy-logs-${SERVICE_ID}.${NODE}.containers.hoody.com/_logs"
```

**Step 2: Verify response**

A successful response returns an array of log entries. Each entry contains the request metadata, response details, and timing information.

```
[
  {
    "timestamp": "2025-11-05T14:32:01.234Z",
    "method": "GET",
    "path": "/api/data",
    "status": 200,
    "duration": 45,
    "sourceIp": "192.168.1.100"
  },
  {
    "timestamp": "2025-11-05T14:32:02.567Z",
    "method": "POST",
    "path": "/api/submit",
    "status": 500,
    "duration": 1230,
    "sourceIp": "10.0.0.55"
  }
]
```

**Step 3: Interpret results**

- `status: 200` entries represent successful requests
- `status: 500` or `status: 502` entries indicate server-side failures requiring investigation
- High `duration` values (milliseconds) suggest performance bottlenecks

---

### Workflow 2: Stream Real-Time Logs via SSE

Open a persistent Server-Sent Events connection to receive log entries as they occur. Ideal for live debugging sessions.

**Step 1: Establish SSE connection**

```
curl -s \
  "https://${PROJECT_ID}-${CONTAINER_ID}-proxy-logs-${SERVICE_ID}.${NODE}.containers.hoody.com/_logs/stream"
```

**Step 2: Process incoming frames**

Each SSE frame follows the HTML Living Standard §6.4 (Server-Sent Events) framing specification with an `id` field for reconnection support:

```
id: 12345
data: {"timestamp":"2025-11-05T14:35:10.123Z","method":"GET","path":"/health","status":200,"duration":12,"sourceIp":"192.168.1.100"}

```

```
id: 12346
data: {"timestamp":"2025-11-05T14:35:11.456Z","method":"POST","path":"/api/users","status":422,"duration":89,"sourceIp":"10.0.0.55"}

```

**Step 3: Handle reconnection**

If the connection drops, resume from the last received `id` value to avoid gaps:

```
curl -s \
  "https://${PROJECT_ID}-${CONTAINER_ID}-proxy-logs-${SERVICE_ID}.${NODE}.containers.hoody.com/_logs/stream?id=12346"
```

**Key behaviors**:
- Each frame has a unique `id: <ringSeq>` line
- The `data:` line contains the full LogEntry as JSON
- Reconnect with the `id` parameter to resume from where you left off

---

### Workflow 3: Analyze Traffic Statistics

Retrieve aggregated statistics for traffic analysis without processing raw log data.

**Step 1: Call the stats endpoint**

```
curl -s \
  "https://${PROJECT_ID}-${CONTAINER_ID}-proxy-logs-${SERVICE_ID}.${NODE}.containers.hoody.com/_logs/stats"
```

**Step 2: Interpret statistics response**

```
{
  "totalRequests": 15420,
  "successfulRequests": 14980,
  "failedRequests": 440,
  "averageDuration": 234,
  "topPaths": [
    { "path": "/api/data", "count": 8500 },
    { "path": "/api/submit", "count": 4200 },
    { "path": "/health", "count": 2720 }
  ]
}
```

**Step 3: Identify issues**

- **Failure rate**: `failedRequests / totalRequests` — values above 5% warrant investigation
- **Slow endpoints**: `averageDuration` exceeding 1000ms indicates performance problems
- **Hot paths**: `topPaths` reveals which endpoints receive the most traffic

---

### Workflow 4: Debug a Specific Failed Request

Combine log retrieval with filtering to trace a specific error.

**Step 1: Fetch all logs**

```
curl -s \
  "https://${PROJECT_ID}-${CONTAINER_ID}-proxy-logs-${SERVICE_ID}.${NODE}.containers.hoody.com/_logs"
```

**Step 2: Identify the failed request in response**

```
[
  {
    "timestamp": "2025-11-05T15:00:05.789Z",
    "method": "GET",
    "path": "/api/users/123",
    "status": 502,
    "duration": 30000,
    "sourceIp": "203.0.113.50"
  }
]
```

**Step 3: Diagnose based on entry fields**

| Field | Value | Interpretation |
|-------|-------|----------------|
| `status` | 502 | Backend container not responding or crashed |
| `duration` | 30000 | 30-second timeout — backend did not respond |
| `path` | /api/users/123 | Specific endpoint affected |

**Recovery actions**:
- Check container health via the container's `/health` endpoint
- Review container logs for crash signatures
- Verify container is running via hoody-kit container status

---

## Advanced Operations

### Advanced Workflow 1: Continuous Monitoring Pipeline

Set up a monitoring script that combines streaming with alerting logic.

**Step 1: Stream logs with error filtering**

```
curl -s \
  "https://${PROJECT_ID}-${CONTAINER_ID}-proxy-logs-${SERVICE_ID}.${NODE}.containers.hoody.com/_logs/stream" | \
  while IFS= read -r line; do
    if [[ "$line" == data:* ]]; then
      json="${line#data: }"
      status=$(echo "$json" | jq -r '.status')
      if [[ "$status" -ge 500 ]]; then
        echo "ALERT: Server error detected — $json"
      fi
    fi
  done
```

**Step 2: Verify monitoring is active**

The script will print alerts only when server errors (5xx) appear in the stream:

```
ALERT: Server error detected — {"timestamp":"2025-11-05T15:30:22.111Z","method":"POST","path":"/api/orders","status":503,"duration":5000,"sourceIp":"10.0.0.75"}
```

**Performance considerations**:
- SSE connections consume minimal resources on the client side
- For high-traffic services, filter on the client side to reduce noise
- Use the `id` parameter for reconnection to avoid duplicate processing

---

### Advanced Workflow 2: Time-Based Log Correlation

When debugging intermittent issues, correlate proxy logs with container restarts.

**Step 1: Get current statistics to establish baseline**

```
curl -s \
  "https://${PROJECT_ID}-${CONTAINER_ID}-proxy-logs-${SERVICE_ID}.${NODE}.containers.hoody.com/_logs/stats"
```

```
{
  "totalRequests": 15420,
  "successfulRequests": 14980,
  "failedRequests": 440,
  "averageDuration": 234,
  "topPaths": [
    { "path": "/api/data", "count": 8500 },
    { "path": "/api/submit", "count": 4200 },
    { "path": "/health", "count": 2720 }
  ]
}
```

**Step 2: Stream logs to capture real-time failures**

```
curl -s \
  "https://${PROJECT_ID}-${CONTAINER_ID}-proxy-logs-${SERVICE_ID}.${NODE}.containers.hoody.com/_logs/stream"
```

**Step 3: Correlate timestamps**

When you see a burst of 502/503 errors, note the timestamps:

```
id: 50001
data: {"timestamp":"2025-11-05T16:00:01.000Z","method":"GET","path":"/api/data","status":502,"duration":30000,"sourceIp":"192.168.1.200"}

```

```
id: 50002
data: {"timestamp":"2025-11-05T16:00:01.500Z","method":"POST","path":"/api/submit","status":502,"duration":30000,"sourceIp":"10.0.0.88"}

```

**Step 4: Cross-reference with container events**

Check the container's status and recent restart history using hoody-kit container management endpoints. A cluster of 502 errors with `duration: 30000` (30s timeout) typically indicates the backend container was unresponsive.

---

### Advanced Workflow 3: SSE Reconnection with State Preservation

For production monitoring, implement robust reconnection logic.

**Step 1: Initial connection**

```
# First connection — no resume point
curl -s \
  "https://${PROJECT_ID}-${CONTAINER_ID}-proxy-logs-${SERVICE_ID}.${NODE}.containers.hoody.com/_logs/stream"
```

**Step 2: Track the last received ID**

Each frame contains an `id: <ringSeq>` line. Record the last `id` value before disconnection.

**Step 3: Reconnect with resume**

```
curl -s \
  "https://${PROJECT_ID}-${CONTAINER_ID}-proxy-logs-${SERVICE_ID}.${NODE}.containers.hoody.com/_logs/stream?id=50002"
```

**Key behaviors**:
- The `id` is a monotonically increasing sequence number (`ringSeq`)
- Reconnecting with `?id=<lastId>` resumes from that point
- Entries between disconnection and reconnection are not lost — they're delivered on reconnect
- If no `id` is provided, the stream starts from the current point (no historical replay)

---

### Error Recovery Patterns

**Pattern: Empty Log Response**

```
curl -s \
  "https://${PROJECT_ID}-${CONTAINER_ID}-proxy-logs-${SERVICE_ID}.${NODE}.containers.hoody.com/_logs"
```

```
[]
```

**Interpretation**: No logs recorded yet. The proxy-logs service may be recently deployed or no traffic has passed through the proxy.

**Recovery**: Verify the proxy is routing traffic by checking another container's `/health` endpoint, then re-query logs.

---

**Pattern: Connection Timeout on SSE**

```
curl -s \
  "https://${PROJECT_ID}-${CONTAINER_ID}-proxy-logs-${SERVICE_ID}.${NODE}.containers.hoody.com/_logs/stream"
# Returns empty or times out
```

**Interpretation**: The SSE connection failed to establish. Possible causes:
- The proxy-logs container is not running
- Network connectivity issue to the Hoody node
- Service endpoint URL is incorrect

**Recovery**:
1. Verify the URL components (projectId, containerId, serviceId, node)
2. Check container status via hoody-kit
3. Retry with a shorter `--max-time` to fail fast

---

**Pattern: High-Duration Entries in Logs**

```
[
  {
    "timestamp": "2025-11-05T17:15:30.000Z",
    "method": "GET",
    "path": "/api/heavy-query",
    "status": 200,
    "duration": 28500,
    "sourceIp": "10.0.0.100"
  }
]
```

**Interpretation**: Request succeeded but took 28.5 seconds. This is approaching the 30-second timeout threshold.

**Recovery**: Optimize the backend endpoint or increase timeout configuration on the container service.

---

## Quick Reference

### Endpoints Summary

| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/_logs` | Retrieve stored request/response logs |
| `GET` | `/_logs/stats` | Get aggregated traffic statistics |
| `GET` | `/_logs/stream` | Open SSE connection for real-time log streaming |

### Essential Parameters

| Parameter | Endpoint | Description |
|-----------|----------|-------------|
| `id` | `/_logs/stream` | Resume SSE from a specific sequence ID (`ringSeq`) |

### Base URL Template

See **Overview → Base URL** above for the full URL template and component breakdown.

### Typical Response Formats

**GET /_logs** — Array of log entries:

```
[
  {
    "timestamp": "2025-11-05T14:32:01.234Z",
    "method": "GET",
    "path": "/api/data",
    "status": 200,
    "duration": 45,
    "sourceIp": "192.168.1.100"
  }
]
```

**GET /_logs/stats** — Aggregated statistics (see Workflow 3 Step 2 for field definitions):

```
{
  "totalRequests": 15420,
  "successfulRequests": 14980,
  "failedRequests": 440,
  "averageDuration": 234,
  "topPaths": [
    { "path": "/api/data", "count": 8500 },
    { "path": "/api/submit", "count": 4200 },
    { "path": "/health", "count": 2720 }
  ]
}
```

**GET /_logs/stream** — SSE frames (HTML Living Standard §6.4):

```
id: 12345
data: {"timestamp":"2025-11-05T14:35:10.123Z","method":"GET","path":"/health","status":200,"duration":12,"sourceIp":"192.168.1.100"}

```

### Essential curl Patterns

**Fetch logs**:
```
curl -s "https://${PROJECT_ID}-${CONTAINER_ID}-proxy-logs-${SERVICE_ID}.${NODE}.containers.hoody.com/_logs"
```

**Get stats**:
```
curl -s "https://${PROJECT_ID}-${CONTAINER_ID}-proxy-logs-${SERVICE_ID}.${NODE}.containers.hoody.com/_logs/stats"
```

**Stream logs**:
```
curl -s "https://${PROJECT_ID}-${CONTAINER_ID}-proxy-logs-${SERVICE_ID}.${NODE}.containers.hoody.com/_logs/stream"
```

**Resume stream from ID**:
```
curl -s "https://${PROJECT_ID}-${CONTAINER_ID}-proxy-logs-${SERVICE_ID}.${NODE}.containers.hoody.com/_logs/stream?id=12345"
```

---

**Related Skills**: Refer to the core SKILL.md for container creation, discovery, and hoody-kit service management.