Server Mode (chatcli server)

​

Quick start

export ANTHROPIC_API_KEY=sk-ant-xxx
chatcli server --provider CLAUDEAI

🚀 ChatCLI server listening on 127.0.0.1:50051
📊 Prometheus metrics on :9090/metrics

CHATCLI_ALLOW_INSECURE=true chatcli connect localhost:50051

export CHATCLI_SERVER_TOKEN="$(openssl rand -hex 32)"
CHATCLI_BIND_ADDRESS=0.0.0.0 chatcli server --provider CLAUDEAI \
  --tls-cert server.crt --tls-key server.key

chatcli connect chatcli.example.com:50051 --tls --ca-cert ca.crt --token "$CHATCLI_SERVER_TOKEN"

​

Bind address and the credential rule

refusing to serve an unauthenticated API on 0.0.0.0: every caller that can reach it would be admitted as an administrator. Set CHATCLI_SERVER_TOKEN (or --token) for a shared token, CHATCLI_JWT_SECRET for HS256 JWTs, CHATCLI_JWT_PUBLIC_KEY for RS256 JWTs, or CHATCLI_SERVER_TLS_CLIENT_CA (--tls-client-ca) for mTLS. To run without authentication, bind loopback instead (CHATCLI_BIND_ADDRESS=127.0.0.1)

​

Flags

​

Environment variables

​

Server Authentication

export CHATCLI_SERVER_TOKEN="$(openssl rand -hex 32)"
chatcli server

# HS256: one shared secret signs and verifies
export CHATCLI_JWT_SECRET="$(openssl rand -hex 32)"

# or RS256: the server only holds the public key
# export CHATCLI_JWT_PUBLIC_KEY=/etc/chatcli/jwt-public.pem

export CHATCLI_JWT_ISSUER=chatcli          # optional
export CHATCLI_JWT_AUDIENCE=chatcli-api    # optional
chatcli server

b64url() { openssl base64 -A | tr '+/' '-_' | tr -d '='; }
HEADER=$(printf '{"alg":"HS256","typ":"JWT"}' | b64url)
PAYLOAD=$(printf '{"sub":"alice@example.com","role":"user","iss":"chatcli","aud":"chatcli-api","exp":%d}' \
  $(( $(date +%s) + 3600 )) | b64url)
SIG=$(printf '%s.%s' "$HEADER" "$PAYLOAD" \
  | openssl dgst -sha256 -hmac "$CHATCLI_JWT_SECRET" -binary | b64url)
JWT="$HEADER.$PAYLOAD.$SIG"

CHATCLI_ALLOW_INSECURE=true chatcli connect localhost:50051 --token "$JWT"

claims := jwt.MapClaims{
    "sub":  "alice@example.com",
    "role": "user",
    "iss":  "chatcli",
    "aud":  "chatcli-api",
    "exp":  time.Now().Add(time.Hour).Unix(),
}
signed, _ := jwt.NewWithClaims(jwt.SigningMethodHS256, claims).
    SignedString([]byte(os.Getenv("CHATCLI_JWT_SECRET")))

chatcli server --tls-cert server.crt --tls-key server.key --tls-client-ca clients-ca.crt

CHATCLI_TLS_CLIENT_CERT=alice.crt CHATCLI_TLS_CLIENT_KEY=alice.key \
  chatcli connect chatcli.example.com:50051 --tls --ca-cert ca.crt

​

Roles

​

Failed-authentication limiter

​

TLS

openssl req -x509 -newkey rsa:2048 -nodes -days 365 \
  -keyout ca.key -out ca.crt -subj "/CN=chatcli-ca"
openssl req -newkey rsa:2048 -nodes \
  -keyout server.key -out server.csr -subj "/CN=chatcli.example.com"
printf "subjectAltName=DNS:chatcli.example.com,DNS:localhost,IP:127.0.0.1\n" > server.ext
openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial \
  -days 365 -extfile server.ext -out server.crt

# Optional: a client certificate for mTLS
openssl req -newkey rsa:2048 -nodes -keyout alice.key -out alice.csr -subj "/CN=alice"
printf "extendedKeyUsage=clientAuth\n" > client.ext
openssl x509 -req -in alice.csr -CA ca.crt -CAkey ca.key -CAcreateserial \
  -days 365 -extfile client.ext -out alice.crt

​

LLM credentials

​

Request routing

• A request that forwards a credential (--llm-key, --use-local-auth, StackSpot fields, an Ollama URL) or names a provider or model gets a dedicated client for that request.
• A request that names nothing and forwards nothing goes through the fallback chain when one is installed, else the server’s default provider and model.
• max_tokens is the request value when set, else the provider’s *_MAX_TOKENS override (ANTHROPIC_MAX_TOKENS, OPENAI_MAX_TOKENS, BEDROCK_MAX_TOKENS, …), else the catalog ceiling of the model.
• A 401/403 from the provider on the server’s credentials rebuilds the providers once and retries; rebuilds are throttled to one per 30 seconds process-wide. Caller-forwarded credentials are never refreshed.
• A reply stopped by a safety classifier (stop_reason: refusal) is resent once on the sibling model of the same provider; on a stream, only while no text has been sent yet.

​

Fallback chain

export ANTHROPIC_API_KEY=sk-ant-xxx OPENAI_API_KEY=sk-xxx GOOGLEAI_API_KEY=xxx
export CHATCLI_FALLBACK_MODEL_OPENAI=gpt-6-sol
chatcli server --provider CLAUDEAI --fallback-providers CLAUDEAI,OPENAI,GOOGLEAI

• The chain is built from --fallback-providers / CHATCLI_FALLBACK_PROVIDERS alone, in that order. The server does not add --provider to it: list the primary first yourself. The Helm chart and the operator’s spec.fallback put the primary in front for you.
• Each entry uses CHATCLI_FALLBACK_MODEL_<PROVIDER> when set. Without it, the primary provider (--provider) keeps the server model (--model), and any other provider runs its own default model: its model variable (for example OPENAI_MODEL), then the built-in default. A fallback provider never receives the primary’s model id.
• A provider whose client cannot be built (no key) is skipped with a warning. The chain is installed only when at least two entries remain; the log then shows Fallback chain initialized.
• It serves only requests that bring no credentials and name no provider or model. The response names the provider and model that actually answered.
• A non-empty CHATCLI_FALLBACK_PROVIDERS is the only switch. CHATCLI_FALLBACK_ENABLED is not read by the server, and neither the Helm chart nor the operator sets it; /config marks it as not read when it is set.

​

gRPC API

​

Streaming

​

Reply attribution and token usage

message TokenUsage {
  int32 prompt_tokens = 1;
  int32 completion_tokens = 2;
  int32 cache_read_tokens = 3;
  int32 cache_write_tokens = 4;
  int32 reasoning_tokens = 5;
  bool estimated = 6;   // provider returned no usage; counts derive from length
  double cost_usd = 7;  // priced with the same tables /cost uses
  bool cost_known = 8;  // false when the model has no published price
}

​

Pipeline RPCs

CHATCLI_SERVER_PIPELINE=true LLM_PROVIDER=CLAUDEAI chatcli server --token "$CHATCLI_SERVER_TOKEN"

• The engine reads its default provider and model from LLM_PROVIDER / LLM_MODEL, not from --provider / --model.
• Sessions are namespaced by the authenticated principal (<subject>/<session>), so two callers never share a conversation by picking the same id.
• Without CHATCLI_SERVER_PIPELINE=true the RPCs return Unavailable: the pipeline RPCs are not enabled on this server (start it with CHATCLI_SERVER_PIPELINE=true).
• GetServerInfo.pipeline_enabled reports whether they are served.

​

AIOps RPCs

message WatcherAlert {
  string type = 1;            // HighRestartCount, OOMKilled, PodNotReady, DeploymentFailing, JobFailed, CronJobMissed, NodeNotReady, ...
  string severity = 2;        // INFO, WARNING, CRITICAL
  string message = 3;
  string object = 4;          // pod name, or Kind/name
  string namespace = 5;
  string deployment = 6;
  int64 timestamp_unix = 7;
}

message GetAlertsRequest {
  string namespace = 1;       // optional filter
  string deployment = 2;      // optional filter
}
message GetAlertsResponse { repeated WatcherAlert alerts = 1; }

​

StreamAlerts

rpc StreamAlerts(StreamAlertsRequest) returns (stream StreamAlertsResponse);

message StreamAlertsRequest {
  string namespace = 1;       // optional filter
  string deployment = 2;      // optional filter
  bool include_current = 3;   // open with the alerts active right now
}
message StreamAlertsResponse {
  WatcherAlert alert = 1;     // unset on a heartbeat
  bool heartbeat = 2;         // every 15s while the watcher is quiet
}

• With include_current the stream starts with what GetAlerts would return, then only new alerts follow. The watcher deduplicates by type and object within its window, so each alert is pushed once.
• Heartbeats every 15 seconds tell a quiet watcher from a dead connection. Without a watcher the stream carries heartbeats only.
• A subscriber that falls behind its bounded buffer is dropped with ABORTED; reopen with include_current to resync.

​

AnalyzeIssue

message AnalyzeIssueRequest {
  string issue_name = 1;
  string namespace = 2;
  string resource_kind = 3;
  string resource_name = 4;
  string signal_type = 5;              // error_rate, oom_kill, pod_restart, ...
  string severity = 6;                 // low, medium, high, critical
  string description = 7;
  int32 risk_score = 8;
  string provider = 9;                 // optional override
  string model = 10;                   // optional override
  string kubernetes_context = 11;      // deployment status, pods, events, revisions
  string previous_failure_context = 12; // earlier failed remediation attempts
}

message AnalyzeIssueResponse {
  string analysis = 1;
  float confidence = 2;                // 0.0 to 1.0
  repeated string recommendations = 3;
  string model = 4;
  string provider = 5;
  repeated SuggestedAction suggested_actions = 6;
  TokenUsage usage = 7;
}

​

Resource discovery

​

Health checks

grpc-health-probe -addr=localhost:50051                                  # plaintext
grpc-health-probe -addr=chatcli.example.com:50051 -tls -tls-ca-cert ca.crt  # TLS
curl -s http://localhost:9090/healthz

​

Metrics

​

OpenTelemetry export (OTLP)

export OTEL_EXPORTER_OTLP_ENDPOINT="http://otel-collector:4318"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer%20token"
export OTEL_SERVICE_NAME="chatcli"          # default
export OTEL_METRIC_EXPORT_INTERVAL=60000    # ms, default

​

Limits and keepalive

​

SSRF protection

• only https://; http:// requires CHATCLI_ALLOW_HTTP_PROVIDERS=true on the server;
• the host (or every address it resolves to) must not be in 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, 127.0.0.0/8, 169.254.0.0/16, 100.64.0.0/10, ::1, fc00::/7, fe80::/10, ff00::/8 or IPv4-mapped ranges; there is no override for private addresses;
• cloud metadata hostnames (metadata.google.internal, metadata.goog, instance-data) are refused.

​

Audit log

export CHATCLI_AUDIT_LOG_PATH=/var/log/chatcli/audit.jsonl   # must be absolute

​

Logs

• Every entry goes to the rotating file LOG_FILE (default ~/.chatcli/app.log; CHATCLI_LOG_FILE is an alias).
• chatcli server and chatcli gateway also write the same entries as JSON lines on stderr whenever stderr is not a terminal — a container, a systemd unit, a pipe — so docker logs, kubectl logs and log collectors get the startup refusals and everything after them with no extra setting. On an interactive terminal the file is the only destination. CHATCLI_LOG_STDERR=true|false forces it either way. Stdout stays free for the stdio transports.
• CHATCLI_ENV=dev switches to the development console (colored, on stdout) plus the file; the stderr JSON lines are then not duplicated.
• LOG_LEVEL sets the level. Rotation: LOG_MAX_SIZE (or CHATCLI_LOG_MAX_SIZE_MB, in MB), CHATCLI_LOG_MAX_BACKUPS (default 3), CHATCLI_LOG_MAX_AGE_DAYS (default 28), CHATCLI_LOG_COMPRESS (default true, gzip). These are what the operator’s Instance spec.features.logRotation sets; the server Helm chart sets them from its logging block (20 MB, 3 backups by default, so the log fits the pod’s 200Mi data volume; see Logs, health and metrics).

LOG_LEVEL=debug chatcli server 2>server.log      # JSON lines on stderr, since it is not a terminal
CHATCLI_ENV=dev LOG_LEVEL=debug chatcli server    # colored console for development

​

gRPC reflection

CHATCLI_GRPC_REFLECTION=true chatcli server --token "$CHATCLI_SERVER_TOKEN"
grpcurl -plaintext -H "authorization: Bearer $CHATCLI_SERVER_TOKEN" localhost:50051 list

​

Multiple replicas

​

Operating the server

​

Troubleshooting

​

Next steps