SLOs e SLAs

apiVersion: platform.chatcli.io/v1alpha1
kind: ServiceLevelObjective
metadata:
  name: api-gateway-availability
  namespace: production
spec:
  service: api-gateway
  description: "API Gateway deve manter 99.9% de availability em janela de 30 dias"

  indicator:
    type: availability
    prometheusQuery:
      goodQuery: 'sum(rate(http_requests_total{service="api-gateway",code=~"2..|3.."}[5m]))'
      totalQuery: 'sum(rate(http_requests_total{service="api-gateway"}[5m]))'

  target:
    percentage: 99.9
    window: 30d

  burnRateAlerts:
    - name: page-fast-burn
      shortWindow: 1h
      longWindow: 6h
      burnRateThreshold: 14.4
      severity: critical
      notificationPolicy: production-alerts

    - name: ticket-medium-burn
      shortWindow: 6h
      longWindow: 3d
      burnRateThreshold: 6.0
      severity: high
      notificationPolicy: production-alerts

    - name: ticket-slow-burn
      shortWindow: 24h
      longWindow: 3d
      burnRateThreshold: 3.0
      severity: medium

    - name: monitor-gradual-burn
      shortWindow: 72h
      longWindow: 30d
      burnRateThreshold: 1.0
      severity: low

  alertPolicy:
    multiWindowRequired: true
    pageOnBudgetExhausted: true
    budgetWarningThresholds: [50, 25, 10, 0]

status:
  currentValue: 99.92
  errorBudgetTotal: 0.001
  errorBudgetRemaining: 0.0008
  errorBudgetRemainingPercent: 80.0
  burnRate: 1.2
  lastCalculatedAt: "2026-03-19T14:00:00Z"
  condition: Met

Error Budget = 1 - (target / 100)

Exemplo para SLO de 99.9%:
  Error Budget = 1 - (99.9 / 100) = 0.001 = 0.1%

Downtime permitido = 30 dias x 24h x 60min x 0.001 = 43.2 minutos

Burn Rate = error_rate_in_window / error_budget

Onde:
  error_rate_in_window = 1 - (good_events / total_events) [na janela]
  error_budget = 1 - (target / 100)

Configuração:
  Target: 99.9%
  Window: 30 dias
  Error Budget: 0.1% = 43.2 minutos de downtime

Situacao atual (medido pelo Prometheus):
  Ultimas 24h: 99.85% availability (0.15% error rate)
  Ultimas 6h:  99.80% availability (0.20% error rate)
  Ultima 1h:   99.70% availability (0.30% error rate)

Calculo de burn rate por janela:
  1h:  0.003 / 0.001 = 3.0x
  6h:  0.002 / 0.001 = 2.0x
  24h: 0.0015 / 0.001 = 1.5x

Avaliacao de alertas:
  page-fast-burn (14.4x):  1h=3.0x < 14.4  -> NÃO dispara
  ticket-medium-burn (6x): 6h=2.0x < 6.0   -> NÃO dispara
  ticket-slow-burn (3x):   24h=1.5x < 3.0  -> NÃO dispara
  monitor-gradual-burn (1x): ambas > 1.0    -> DISPARA (severity: low)

Resultado: Degradacao lenta detectada. Não e critico, mas o budget esta
sendo consumido 1.5x mais rapido que o sustentavel. A esse ritmo, o
budget de 43.2 minutos se esgotaria em 20 dias (ao inves de 30).

Error budget restante:
  Consumido até agora: ~22 minutos (estimado)
  Restante: 43.2 - 22 = 21.2 minutos
  Porcentagem restante: 49.1%
  -> Warning threshold de 50% quase atingido

apiVersion: platform.chatcli.io/v1alpha1
kind: IncidentSLA
metadata:
  name: production-sla
  namespace: production
spec:
  service: api-gateway
  description: "SLA de producao para o API Gateway"

  responseTimes:
    - severity: critical
      maxResponseTime: "5m"
      maxResolutionTime: "1h"
    - severity: high
      maxResponseTime: "15m"
      maxResolutionTime: "4h"
    - severity: medium
      maxResponseTime: "1h"
      maxResolutionTime: "24h"
    - severity: low
      maxResponseTime: "4h"
      maxResolutionTime: "72h"

  businessHours:
    enabled: true
    timezone: "America/Sao_Paulo"
    startHour: 9
    endHour: 18
    workDays: [1, 2, 3, 4, 5]   # Segunda a Sexta (0=Dom, 6=Sab)
    holidays:
      - date: "2026-01-01"
        name: "Ano Novo"
      - date: "2026-04-03"
        name: "Sexta-feira Santa"
      - date: "2026-12-25"
        name: "Natal"

  violationPolicy:
    notificationPolicy: sla-breach-notifications
    escalationPolicy: p0-escalation
    autoEscalateOnBreach: true

status:
  activeIncidents: 2
  totalViolations: 3
  compliancePercentage: 97.5
  violations:
    - issueName: "api-gateway-oom-kill-1771276354"
      severity: critical
      type: resolution_time
      exceededBy: "12m"
      occurredAt: "2026-03-15T14:30:00Z"
  lastCalculatedAt: "2026-03-19T14:00:00Z"

CompliancePercentage = ((total_incidents - violations) / total_incidents) * 100

Exemplo:
  Total de incidentes no periodo: 120
  Violações: 3
  Compliance = ((120 - 3) / 120) * 100 = 97.5%

apiVersion: platform.chatcli.io/v1alpha1
kind: ServiceLevelObjective
metadata:
  name: api-gateway-availability-slo
  namespace: production
spec:
  service: api-gateway
  description: "99.9% availability do API Gateway medido por taxa de sucesso HTTP"

  indicator:
    type: availability
    prometheusQuery:
      goodQuery: |
        sum(rate(http_requests_total{
          service="api-gateway",
          code=~"2..|3.."
        }[5m]))
      totalQuery: |
        sum(rate(http_requests_total{
          service="api-gateway"
        }[5m]))

  target:
    percentage: 99.9
    window: 30d

  burnRateAlerts:
    - name: page-immediate
      shortWindow: 1h
      longWindow: 6h
      burnRateThreshold: 14.4
      severity: critical
      notificationPolicy: production-alerts

    - name: ticket-urgent
      shortWindow: 6h
      longWindow: 3d
      burnRateThreshold: 6.0
      severity: high
      notificationPolicy: production-alerts

    - name: ticket-normal
      shortWindow: 24h
      longWindow: 3d
      burnRateThreshold: 3.0
      severity: medium

    - name: monitor
      shortWindow: 72h
      longWindow: 30d
      burnRateThreshold: 1.0
      severity: low

  alertPolicy:
    multiWindowRequired: true
    pageOnBudgetExhausted: true
    budgetWarningThresholds: [50, 25, 10, 0]

apiVersion: platform.chatcli.io/v1alpha1
kind: IncidentSLA
metadata:
  name: api-gateway-production-sla
  namespace: production
spec:
  service: api-gateway
  description: "SLA de producao conforme contrato com clientes enterprise"

  responseTimes:
    - severity: critical
      maxResponseTime: "5m"
      maxResolutionTime: "1h"
    - severity: high
      maxResponseTime: "15m"
      maxResolutionTime: "4h"
    - severity: medium
      maxResponseTime: "2h"
      maxResolutionTime: "24h"
    - severity: low
      maxResponseTime: "8h"
      maxResolutionTime: "72h"

  businessHours:
    enabled: true
    timezone: "America/Sao_Paulo"
    startHour: 9
    endHour: 18
    workDays: [1, 2, 3, 4, 5]
    holidays:
      - date: "2026-01-01"
        name: "Confraternizacao Universal"
      - date: "2026-02-16"
        name: "Carnaval"
      - date: "2026-02-17"
        name: "Carnaval"
      - date: "2026-04-03"
        name: "Sexta-feira Santa"
      - date: "2026-04-21"
        name: "Tiradentes"
      - date: "2026-05-01"
        name: "Dia do Trabalho"
      - date: "2026-09-07"
        name: "Independencia"
      - date: "2026-10-12"
        name: "Nossa Senhora Aparecida"
      - date: "2026-11-02"
        name: "Finados"
      - date: "2026-11-15"
        name: "Proclamacao da Republica"
      - date: "2026-12-25"
        name: "Natal"

  violationPolicy:
    notificationPolicy: sla-breach-notifications
    escalationPolicy: p0-escalation
    autoEscalateOnBreach: true

apiVersion: platform.chatcli.io/v1alpha1
kind: ServiceLevelObjective
metadata:
  name: payment-service-latency-slo
  namespace: payments
spec:
  service: payment-service
  description: "99.5% dos requests do Payment Service devem completar em menos de 500ms"

  indicator:
    type: latency
    prometheusQuery:
      goodQuery: |
        sum(rate(http_request_duration_seconds_bucket{
          service="payment-service",
          le="0.5"
        }[5m]))
      totalQuery: |
        sum(rate(http_request_duration_seconds_count{
          service="payment-service"
        }[5m]))

  target:
    percentage: 99.5
    window: 7d

  burnRateAlerts:
    - name: page-latency-spike
      shortWindow: 30m
      longWindow: 3h
      burnRateThreshold: 14.4
      severity: critical
      notificationPolicy: payments-alerts

    - name: ticket-latency-degradation
      shortWindow: 3h
      longWindow: 1d
      burnRateThreshold: 6.0
      severity: high

    - name: monitor-latency-trend
      shortWindow: 12h
      longWindow: 7d
      burnRateThreshold: 1.0
      severity: low

  alertPolicy:
    multiWindowRequired: true
    pageOnBudgetExhausted: true
    budgetWarningThresholds: [25, 10, 0]

# Os dashboards estão disponíveis como ConfigMaps
kubectl apply -f operator/config/grafana/dashboards/

# Ou importe via API do Grafana
for f in operator/config/grafana/dashboards/*.json; do
  curl -X POST -H "Content-Type: application/json" \
    -H "Authorization: Bearer $GRAFANA_API_KEY" \
    -d @"$f" \
    "https://grafana.company.com/api/dashboards/db"
done

groups:
  - name: chatcli-slo-sla
    rules:
      - alert: SLOBudgetExhausted
        expr: chatcli_slo_error_budget_remaining_percent <= 0
        for: 1m
        labels:
          severity: critical
        annotations:
          summary: "Error budget esgotado para SLO {{ $labels.slo }}"
          description: "O servico {{ $labels.service }} esgotou o error budget. Nenhum downtime adicional e permitido."

      - alert: SLOBudgetLow
        expr: chatcli_slo_error_budget_remaining_percent <= 10 and chatcli_slo_error_budget_remaining_percent > 0
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "Error budget baixo ({{ $value }}%) para SLO {{ $labels.slo }}"

      - alert: SLAComplianceBelow95
        expr: chatcli_sla_compliance_percentage < 95
        for: 1m
        labels:
          severity: critical
        annotations:
          summary: "SLA compliance abaixo de 95% para {{ $labels.service }}"
          description: "Compliance atual: {{ $value }}%. Revise incidentes recentes e tome ações corretivas."

      - alert: SLAResponseTimeExceeded
        expr: histogram_quantile(0.95, rate(chatcli_sla_response_time_seconds_bucket[1h])) > 300
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "P95 do tempo de resposta SLA excede 5 minutos"