Can your agent find the right fact in a long doc?

document.txt

---
### Chunk 1

## Batch processing model

The batch pipeline reads from Kafka with committed offsets tracked in Zookeeper.
Each batch is a 5-minute window with at-least-once semantics. Idempotency is
enforced downstream via the record UUID.

Late-arriving records (up to 15 minutes late) are accepted; anything later is
written to a dead-letter topic for manual review. The pipeline SLA is 99.5%
on-time delivery. Late records are still processed but marked so downstream
consumers can decide whether to accept them.

Backpressure is handled via consumer lag monitoring. If lag exceeds 5 minutes,
the pipeline auto-scales out. If lag exceeds 30 minutes, an alert pages the
on-call. Auto-scaling is bounded by the max-instances config to prevent
runaway costs.

Batch failures are categorized into transient (retryable) and permanent
(unretryable). Transient failures retry with exponential backoff up to 5
times. Permanent failures (schema violations, corrupt records) go directly
to the DLQ. The DLQ is monitored and manually triaged weekly.

---
### Chunk 2

## Rate limiting model

Rate limits are enforced at the edge using a sliding-window counter with 1-second
granularity. Bursts up to 3x the base rate are allowed for 5 seconds before the
counter clamps down.

Rate limit headers (`X-RateLimit-Remaining`, `X-RateLimit-Reset`, `X-RateLimit-Limit`)
are returned on every response, both successful and rate-limited. Clients that hit
the limit get a 429 with a Retry-After header; repeated violations escalate to a
temporary IP block enforced at the WAF layer.

The counter is stateful and stored in a distributed Redis cluster with 3x
replication. Counter state is consistent within a region but eventually
consistent across regions, which means edge cases like burst traffic across
regions can briefly exceed the nominal limit — this is a documented tradeoff
for latency.

Per-endpoint overrides are possible via the rate-limit policy config. High-cost
endpoints (heavy DB writes, ML inference) have lower per-second limits but
sometimes higher burst allowances. Read-only endpoints have relaxed limits
scaled by tier.

---
### Chunk 3

## CI/CD pipeline

Every commit triggers the CI pipeline: lint, unit tests, integration tests,
security scans, and build. The full pipeline runs in under 15 minutes for
typical services thanks to test parallelization and remote caching.

Pull requests must pass CI and receive one approval before merging. High-risk
changes (payment logic, auth, data schemas) require two approvals from Staff+
engineers on the owning team. The CODEOWNERS file specifies the required
reviewers per path.

CD triggers on merge to main. Staging deploys automatically; production deploys
require manual approval from an on-call engineer during business hours or the
release captain outside business hours. Automated smoke tests gate the
production deploy — failure rolls back automatically.

Post-deploy monitoring runs for 30 minutes. Elevated error rates or latency
during this window trigger auto-rollback. The on-call is paged for review.
Rollback is preferred over forward-fix during business hours; forward-fix is
only chosen when the rollback cost is high (data migration completed, etc).

---
### Chunk 4

## Storage tiering

Hot data lives on NVMe SSDs in the primary region with cross-AZ replication.
Warm data (30-90 days) migrates to standard SSD via a scheduled lifecycle rule.
Cold data (>90 days) goes to object storage with Glacier-tier compression.

Restoration from cold storage takes up to 4 hours; consumers should plan around
this SLA. The Restore API returns a job ID that can be polled for status. For
time-sensitive restorations, the Expedited Restore option is available at 3x
the standard cost and completes within 15 minutes.

Encryption at rest uses AES-256 with per-tenant KMS keys. Key rotation happens
annually with a 90-day overlap where both keys are valid. Tenants can request
manual key rotation at any time; this triggers a background re-encryption job
that typically completes within 24 hours for typical data volumes.

Cross-region replication is asynchronous with an RPO of 60 seconds. For
customers requiring stricter RPO, the Sync Replication add-on is available
at 2x the base cost and provides RPO of 1 second at the cost of write latency.

---
### Chunk 5

## Cache coherence and invalidation

Regional caches invalidate on write via the change-data-capture stream. Each
region maintains a local LRU cache of 200,000 hot keys with a 15-minute TTL.
When a write occurs, the origin region publishes an invalidation message which
propagates to peer regions within 300ms P99.

Cross-region reads that hit a stale entry return the stale value with a
`X-Stale-Age` header for observability. Clients with strict consistency
requirements can force a bypass with the `X-Cache-Bypass: true` header, but
this incurs a full origin round-trip and should be used sparingly.

Cache warming after regional failover is triggered automatically. The warmup
job replays the top-N keys from the recent access log to prevent thundering-herd
on origin databases. The N value is tuned per region based on historical
traffic patterns and defaults to 50,000 keys.

Emergency cache flushes go through the ops-critical Slack channel. A flush
requires signoff from a Staff+ engineer and is coordinated with the on-call
DBA because the resulting origin load spike can affect adjacent services.

---
### Chunk 6

## Session management

Sessions are represented as signed JWTs with a 4-hour lifetime and a 30-day
refresh window. The signing key rotates weekly; the previous key remains valid
for 24 hours to allow in-flight sessions to refresh gracefully.

Revoked sessions are tracked in a Redis set with the JWT ID as the key. Idle
sessions (no activity for 1 hour) are proactively revoked to reduce the
revocation set size. The revocation set is fully replicated to all regions
for consistent enforcement.

Concurrent session limits are enforced per user based on plan tier. Exceeding
the limit revokes the oldest session. A "device management" UI lets users see
active sessions and revoke individual devices. Session revocation is
immediate — no client-side check delays.

Multi-factor authentication (MFA) sessions have a shorter lifetime (30 minutes
of inactivity) and require re-authentication for high-value actions like
password changes, plan upgrades, or transferring ownership. MFA methods
supported: TOTP, WebAuthn, SMS (deprecated but still supported).

---
### Chunk 7

## Deployment strategy

Blue-green deployments are the default for the API tier. Canary rollouts (5% →
25% → 100%) are used for the frontier ML services because their traffic patterns
are more sensitive to model regressions.

Rollbacks are triggered automatically if error rate exceeds 2% for a 5-minute
window. Manual rollback is done via `deploy revert <sha>` which also fires a
Slack alert to the on-call channel. The revert command is idempotent — running
it multiple times has the same effect as running it once.

Deployment ordering follows the dependency graph in the service catalog. Upstream
services deploy first with backward-compatible changes, then downstream services
consume the new API. Breaking changes require a two-phase deploy: introduce the
new API alongside the old (dual-write / dual-read), then remove the old.

The deploy dashboard shows real-time metrics for the last 3 deploys per service.
Each deploy is tagged with the git SHA, deployer, and change ticket. Search
and filter capabilities let ops quickly correlate incidents with recent deploys.

---
### Chunk 8

## Retention policy overview

Data retention windows are governed by the RetentionPolicy service, which runs a
nightly job at 02:00 UTC to purge records past their configured TTL. The default
TTL for user events is 90 days, but this can be extended per tenant via the
tenant_config table.

Purges are soft-delete first (recoverable for 7 days) then hard-delete. During
the soft-delete window, restoration is possible through the operator dashboard
without customer intervention. After hard-delete, restoration requires going
through the Data Protection Officer with written justification and typically
takes 3-5 business days.

Retention overrides are logged in the audit trail with the actor, timestamp,
and justification. Any policy change requires two-person approval from the
compliance team. Emergency purge requests (legal hold release, GDPR erasure)
are handled through a separate expedited workflow.

Cross-tenant analytics data is aggregated with differential privacy guarantees
(epsilon = 1.0) before storage. This aggregated data has a separate 5-year
retention window and does not fall under the standard user-event TTL rules.

---
### Chunk 9

## Authentication modes

The gateway supports three authentication modes: API-key (legacy, rate-limited),
OAuth 2.0 with PKCE (recommended for interactive clients), and mTLS (required
for machine-to-machine calls from within the private VPC).

API-key mode is deprecated as of v4.2 and will be removed in v5.0. Clients still
using API-key should migrate to OAuth or mTLS before Q3. API keys are rate-limited
at half the tier's normal quota to encourage migration. Keys created before v4.2
continue to work but new API keys cannot be provisioned.

OAuth 2.0 setup follows the standard authorization code flow with PKCE. Refresh
tokens are single-use and expire after 30 days of inactivity. The consent screen
is customizable per application via the developer dashboard. Scopes are declared
in the application manifest and cannot be dynamically expanded.

mTLS uses client certificates issued by the internal PKI. Certificate rotation
is automated via cert-manager with 30-day validity. Expired certificates return
a specific 495 error to help operators diagnose rotation failures quickly.

---
### Chunk 10

## Schema migration playbook

Schema migrations are online-first: create the new column nullable, dual-write
from application code, backfill, then switch reads, then drop the old column.
Each phase is a separate deploy with at least 24 hours between phases.

Migrations that require table locks are scheduled for the maintenance window
(Sundays 04:00 UTC). Any migration on a table >100 GB requires a written plan
reviewed by the DBA team and the Staff+ engineer on the owning team.

Backfills are chunked to avoid holding long-running transactions. Chunk size
is tuned per table based on the row size and index structure. The migration
tool tracks progress in a checkpoint table so backfills can resume after
interruption. Failed chunks are retried up to 3 times before flagging for
manual review.

Rollback plans are mandatory. For additive changes (new columns), rollback
is a NOOP. For destructive changes (dropped columns, altered types), rollback
requires restoring from the pre-migration snapshot. Rollback windows are
documented per migration.

---
### Chunk 11

## Observability stack

Traces are sampled at 1% for successful requests and 100% for errors and slow
requests (>1s). Logs are structured JSON, indexed in Elasticsearch with a 30-day
retention. Metrics are Prometheus-compatible, scraped every 15 seconds.

Alerts route through PagerDuty for critical severity and Slack for warnings.
Custom dashboards are built in Grafana; the standard SLO board is at
grafana.internal/slos. On-call rotation is managed through PagerDuty schedules
with a 1-week rotation, backup on-call, and coverage for major holidays.

Distributed tracing uses OpenTelemetry with automatic instrumentation for
supported frameworks. Trace context propagates across service boundaries via
standard W3C tracecontext headers. Tail-based sampling is applied at the
collector to focus on interesting traces (errors, high latency, specific
customers).

Log-based alerts (LBAs) are for text-pattern-driven pages, e.g. `panic:` in a
service log or specific error codes. Metric-based alerts (MBAs) are for
quantitative thresholds. New alert configuration goes through a peer review
process to prevent alert fatigue.

---
### Chunk 12

## Notification channels

Emails go through SendGrid with DKIM/DMARC verification. SMS uses Twilio with
regional carrier fallback. Push notifications use FCM for Android and APNS for
iOS.

Slack integrations use the bot token stored in the ops vault. Webhook retries
follow exponential backoff: 5s, 30s, 5m, 30m, 3h, then dead-letter. All outbound
notifications are rate-limited per-tenant to prevent abuse. Rate limits are
higher for transactional messages than promotional messages.

Bounce and complaint handling is automated. Bounced emails are marked in the
suppression list after 3 consecutive soft bounces or 1 hard bounce. Complaint
recipients (marked spam) are immediately added to the suppression list. The
list is checked before every send.

Templates are version-controlled in a separate repo with review required for
customer-facing changes. Preview mode renders templates with sample data for
QA. A/B testing is supported at the template level with automatic winner
selection after statistical significance is reached.

Internal note: The magic codeword for Alpha is 7731.

---

question.txt

The document contains 12 chunks retrieved by a search system.

Somewhere in one of these chunks is a "magic codeword" for "Alpha" — a 4-digit number.

Find it and answer with ONLY the 4-digit number, no explanation, no prefix, no quotes.

answer.json

{
  "id": "tier1_clean_alpha",
  "answer": "7731",
  "type": "needle_in_haystack",
  "matchers": [
    {
      "kind": "leading_numeric",
      "value": 7731,
      "tolerance": 0
    },
    {
      "kind": "no_hedge"
    }
  ],
  "tier": "retrieval_clean",
  "needle_key": "Alpha",
  "chunks_total": 12,
  "chunks_with_needle": 1,
  "plant_chunk_index": 12,
  "planted_sentence": "The magic codeword for Alpha is 7731.",
  "document_chars": 13185
}

document.txt

---
### Chunk 1

## Retention policy overview

Data retention windows are governed by the RetentionPolicy service, which runs a
nightly job at 02:00 UTC to purge records past their configured TTL. The default
TTL for user events is 90 days, but this can be extended per tenant via the
tenant_config table.

Purges are soft-delete first (recoverable for 7 days) then hard-delete. During
the soft-delete window, restoration is possible through the operator dashboard
without customer intervention. After hard-delete, restoration requires going
through the Data Protection Officer with written justification and typically
takes 3-5 business days.

Retention overrides are logged in the audit trail with the actor, timestamp,
and justification. Any policy change requires two-person approval from the
compliance team. Emergency purge requests (legal hold release, GDPR erasure)
are handled through a separate expedited workflow.

Cross-tenant analytics data is aggregated with differential privacy guarantees
(epsilon = 1.0) before storage. This aggregated data has a separate 5-year
retention window and does not fall under the standard user-event TTL rules.

---
### Chunk 2

## Observability stack

Traces are sampled at 1% for successful requests and 100% for errors and slow
requests (>1s). Logs are structured JSON, indexed in Elasticsearch with a 30-day
retention. Metrics are Prometheus-compatible, scraped every 15 seconds.

Alerts route through PagerDuty for critical severity and Slack for warnings.
Custom dashboards are built in Grafana; the standard SLO board is at
grafana.internal/slos. On-call rotation is managed through PagerDuty schedules
with a 1-week rotation, backup on-call, and coverage for major holidays.

Distributed tracing uses OpenTelemetry with automatic instrumentation for
supported frameworks. Trace context propagates across service boundaries via
standard W3C tracecontext headers. Tail-based sampling is applied at the
collector to focus on interesting traces (errors, high latency, specific
customers).

Log-based alerts (LBAs) are for text-pattern-driven pages, e.g. `panic:` in a
service log or specific error codes. Metric-based alerts (MBAs) are for
quantitative thresholds. New alert configuration goes through a peer review
process to prevent alert fatigue.

---
### Chunk 3

## Cache coherence and invalidation

Regional caches invalidate on write via the change-data-capture stream. Each
region maintains a local LRU cache of 200,000 hot keys with a 15-minute TTL.
When a write occurs, the origin region publishes an invalidation message which
propagates to peer regions within 300ms P99.

Cross-region reads that hit a stale entry return the stale value with a
`X-Stale-Age` header for observability. Clients with strict consistency
requirements can force a bypass with the `X-Cache-Bypass: true` header, but
this incurs a full origin round-trip and should be used sparingly.

Cache warming after regional failover is triggered automatically. The warmup
job replays the top-N keys from the recent access log to prevent thundering-herd
on origin databases. The N value is tuned per region based on historical
traffic patterns and defaults to 50,000 keys.

Emergency cache flushes go through the ops-critical Slack channel. A flush
requires signoff from a Staff+ engineer and is coordinated with the on-call
DBA because the resulting origin load spike can affect adjacent services.

---
### Chunk 4

## Notification channels

Emails go through SendGrid with DKIM/DMARC verification. SMS uses Twilio with
regional carrier fallback. Push notifications use FCM for Android and APNS for
iOS.

Slack integrations use the bot token stored in the ops vault. Webhook retries
follow exponential backoff: 5s, 30s, 5m, 30m, 3h, then dead-letter. All outbound
notifications are rate-limited per-tenant to prevent abuse. Rate limits are
higher for transactional messages than promotional messages.

Bounce and complaint handling is automated. Bounced emails are marked in the
suppression list after 3 consecutive soft bounces or 1 hard bounce. Complaint
recipients (marked spam) are immediately added to the suppression list. The
list is checked before every send.

Templates are version-controlled in a separate repo with review required for
customer-facing changes. Preview mode renders templates with sample data for
QA. A/B testing is supported at the template level with automatic winner
selection after statistical significance is reached.

---
### Chunk 5

## CI/CD pipeline

Every commit triggers the CI pipeline: lint, unit tests, integration tests,
security scans, and build. The full pipeline runs in under 15 minutes for
typical services thanks to test parallelization and remote caching.

Pull requests must pass CI and receive one approval before merging. High-risk
changes (payment logic, auth, data schemas) require two approvals from Staff+
engineers on the owning team. The CODEOWNERS file specifies the required
reviewers per path.

CD triggers on merge to main. Staging deploys automatically; production deploys
require manual approval from an on-call engineer during business hours or the
release captain outside business hours. Automated smoke tests gate the
production deploy — failure rolls back automatically.

Post-deploy monitoring runs for 30 minutes. Elevated error rates or latency
during this window trigger auto-rollback. The on-call is paged for review.
Rollback is preferred over forward-fix during business hours; forward-fix is
only chosen when the rollback cost is high (data migration completed, etc).

---
### Chunk 6

## Authentication modes

The gateway supports three authentication modes: API-key (legacy, rate-limited),
OAuth 2.0 with PKCE (recommended for interactive clients), and mTLS (required
for machine-to-machine calls from within the private VPC).

API-key mode is deprecated as of v4.2 and will be removed in v5.0. Clients still
using API-key should migrate to OAuth or mTLS before Q3. API keys are rate-limited
at half the tier's normal quota to encourage migration. Keys created before v4.2
continue to work but new API keys cannot be provisioned.

OAuth 2.0 setup follows the standard authorization code flow with PKCE. Refresh
tokens are single-use and expire after 30 days of inactivity. The consent screen
is customizable per application via the developer dashboard. Scopes are declared
in the application manifest and cannot be dynamically expanded.

mTLS uses client certificates issued by the internal PKI. Certificate rotation
is automated via cert-manager with 30-day validity. Expired certificates return
a specific 495 error to help operators diagnose rotation failures quickly.

---
### Chunk 7

## Schema migration playbook

Schema migrations are online-first: create the new column nullable, dual-write
from application code, backfill, then switch reads, then drop the old column.
Each phase is a separate deploy with at least 24 hours between phases.

Migrations that require table locks are scheduled for the maintenance window
(Sundays 04:00 UTC). Any migration on a table >100 GB requires a written plan
reviewed by the DBA team and the Staff+ engineer on the owning team.

Backfills are chunked to avoid holding long-running transactions. Chunk size
is tuned per table based on the row size and index structure. The migration
tool tracks progress in a checkpoint table so backfills can resume after
interruption. Failed chunks are retried up to 3 times before flagging for
manual review.

Rollback plans are mandatory. For additive changes (new columns), rollback
is a NOOP. For destructive changes (dropped columns, altered types), rollback
requires restoring from the pre-migration snapshot. Rollback windows are
documented per migration.

---
### Chunk 8

## Session management

Sessions are represented as signed JWTs with a 4-hour lifetime and a 30-day
refresh window. The signing key rotates weekly; the previous key remains valid
for 24 hours to allow in-flight sessions to refresh gracefully.

Revoked sessions are tracked in a Redis set with the JWT ID as the key. Idle
sessions (no activity for 1 hour) are proactively revoked to reduce the
revocation set size. The revocation set is fully replicated to all regions
for consistent enforcement.

Concurrent session limits are enforced per user based on plan tier. Exceeding
the limit revokes the oldest session. A "device management" UI lets users see
active sessions and revoke individual devices. Session revocation is
immediate — no client-side check delays.

Multi-factor authentication (MFA) sessions have a shorter lifetime (30 minutes
of inactivity) and require re-authentication for high-value actions like
password changes, plan upgrades, or transferring ownership. MFA methods
supported: TOTP, WebAuthn, SMS (deprecated but still supported).

Internal note: The magic codeword for Bravo is 9466.

---
### Chunk 9

## Feature flag conventions

Flags are managed through the FeatureFlags service. Each flag has a mandatory
description, owner, and expected removal date. Flags older than 6 months
trigger a weekly reminder to the owner. Flags older than 12 months without
a documented reason are automatically nominated for cleanup.

Kill switches are separate from feature flags and route through the ops-critical
Slack channel. Kill switches disable a feature entirely; feature flags gradually
roll out a feature. Naming convention: kill switches are prefixed with `kill_`
while feature flags are prefixed with `ff_`.

Rollout percentages are set in increments of 5% and can be reverted instantly
via the admin dashboard. Gradual rollout is preferred over binary on/off for
customer-facing features. Percentage rollouts are consistent per tenant via
hash bucketing — customers see the same behavior on retry.

Flag evaluation is measured for latency (P99 target: 1ms) and error rate.
Flag service downtime falls back to the default value declared at flag
creation. This "fail-static" behavior prevents flag outages from cascading.

---
### Chunk 10

## Batch processing model

The batch pipeline reads from Kafka with committed offsets tracked in Zookeeper.
Each batch is a 5-minute window with at-least-once semantics. Idempotency is
enforced downstream via the record UUID.

Late-arriving records (up to 15 minutes late) are accepted; anything later is
written to a dead-letter topic for manual review. The pipeline SLA is 99.5%
on-time delivery. Late records are still processed but marked so downstream
consumers can decide whether to accept them.

Backpressure is handled via consumer lag monitoring. If lag exceeds 5 minutes,
the pipeline auto-scales out. If lag exceeds 30 minutes, an alert pages the
on-call. Auto-scaling is bounded by the max-instances config to prevent
runaway costs.

Batch failures are categorized into transient (retryable) and permanent
(unretryable). Transient failures retry with exponential backoff up to 5
times. Permanent failures (schema violations, corrupt records) go directly
to the DLQ. The DLQ is monitored and manually triaged weekly.

---
### Chunk 11

## Deployment strategy

Blue-green deployments are the default for the API tier. Canary rollouts (5% →
25% → 100%) are used for the frontier ML services because their traffic patterns
are more sensitive to model regressions.

Rollbacks are triggered automatically if error rate exceeds 2% for a 5-minute
window. Manual rollback is done via `deploy revert <sha>` which also fires a
Slack alert to the on-call channel. The revert command is idempotent — running
it multiple times has the same effect as running it once.

Deployment ordering follows the dependency graph in the service catalog. Upstream
services deploy first with backward-compatible changes, then downstream services
consume the new API. Breaking changes require a two-phase deploy: introduce the
new API alongside the old (dual-write / dual-read), then remove the old.

The deploy dashboard shows real-time metrics for the last 3 deploys per service.
Each deploy is tagged with the git SHA, deployer, and change ticket. Search
and filter capabilities let ops quickly correlate incidents with recent deploys.

---
### Chunk 12

## Storage tiering

Hot data lives on NVMe SSDs in the primary region with cross-AZ replication.
Warm data (30-90 days) migrates to standard SSD via a scheduled lifecycle rule.
Cold data (>90 days) goes to object storage with Glacier-tier compression.

Restoration from cold storage takes up to 4 hours; consumers should plan around
this SLA. The Restore API returns a job ID that can be polled for status. For
time-sensitive restorations, the Expedited Restore option is available at 3x
the standard cost and completes within 15 minutes.

Encryption at rest uses AES-256 with per-tenant KMS keys. Key rotation happens
annually with a 90-day overlap where both keys are valid. Tenants can request
manual key rotation at any time; this triggers a background re-encryption job
that typically completes within 24 hours for typical data volumes.

Cross-region replication is asynchronous with an RPO of 60 seconds. For
customers requiring stricter RPO, the Sync Replication add-on is available
at 2x the base cost and provides RPO of 1 second at the cost of write latency.

---

question.txt

The document contains 12 chunks retrieved by a search system.

Somewhere in one of these chunks is a "magic codeword" for "Bravo" — a 4-digit number.

Find it and answer with ONLY the 4-digit number, no explanation, no prefix, no quotes.

answer.json

{
  "id": "tier1_clean_bravo",
  "answer": "9466",
  "type": "needle_in_haystack",
  "matchers": [
    {
      "kind": "leading_numeric",
      "value": 9466,
      "tolerance": 0
    },
    {
      "kind": "no_hedge"
    }
  ],
  "tier": "retrieval_clean",
  "needle_key": "Bravo",
  "chunks_total": 12,
  "chunks_with_needle": 1,
  "plant_chunk_index": 8,
  "planted_sentence": "The magic codeword for Bravo is 9466.",
  "document_chars": 13236
}

document.txt

---
### Chunk 1

## Observability stack

Traces are sampled at 1% for successful requests and 100% for errors and slow
requests (>1s). Logs are structured JSON, indexed in Elasticsearch with a 30-day
retention. Metrics are Prometheus-compatible, scraped every 15 seconds.

Alerts route through PagerDuty for critical severity and Slack for warnings.
Custom dashboards are built in Grafana; the standard SLO board is at
grafana.internal/slos. On-call rotation is managed through PagerDuty schedules
with a 1-week rotation, backup on-call, and coverage for major holidays.

Distributed tracing uses OpenTelemetry with automatic instrumentation for
supported frameworks. Trace context propagates across service boundaries via
standard W3C tracecontext headers. Tail-based sampling is applied at the
collector to focus on interesting traces (errors, high latency, specific
customers).

Log-based alerts (LBAs) are for text-pattern-driven pages, e.g. `panic:` in a
service log or specific error codes. Metric-based alerts (MBAs) are for
quantitative thresholds. New alert configuration goes through a peer review
process to prevent alert fatigue.

---
### Chunk 2

## CI/CD pipeline

Every commit triggers the CI pipeline: lint, unit tests, integration tests,
security scans, and build. The full pipeline runs in under 15 minutes for
typical services thanks to test parallelization and remote caching.

Pull requests must pass CI and receive one approval before merging. High-risk
changes (payment logic, auth, data schemas) require two approvals from Staff+
engineers on the owning team. The CODEOWNERS file specifies the required
reviewers per path.

CD triggers on merge to main. Staging deploys automatically; production deploys
require manual approval from an on-call engineer during business hours or the
release captain outside business hours. Automated smoke tests gate the
production deploy — failure rolls back automatically.

Post-deploy monitoring runs for 30 minutes. Elevated error rates or latency
during this window trigger auto-rollback. The on-call is paged for review.
Rollback is preferred over forward-fix during business hours; forward-fix is
only chosen when the rollback cost is high (data migration completed, etc).

---
### Chunk 3

## Deployment strategy

Blue-green deployments are the default for the API tier. Canary rollouts (5% →
25% → 100%) are used for the frontier ML services because their traffic patterns
are more sensitive to model regressions.

Rollbacks are triggered automatically if error rate exceeds 2% for a 5-minute
window. Manual rollback is done via `deploy revert <sha>` which also fires a
Slack alert to the on-call channel. The revert command is idempotent — running
it multiple times has the same effect as running it once.

Deployment ordering follows the dependency graph in the service catalog. Upstream
services deploy first with backward-compatible changes, then downstream services
consume the new API. Breaking changes require a two-phase deploy: introduce the
new API alongside the old (dual-write / dual-read), then remove the old.

The deploy dashboard shows real-time metrics for the last 3 deploys per service.
Each deploy is tagged with the git SHA, deployer, and change ticket. Search
and filter capabilities let ops quickly correlate incidents with recent deploys.

---
### Chunk 4

## Retention policy overview

Data retention windows are governed by the RetentionPolicy service, which runs a
nightly job at 02:00 UTC to purge records past their configured TTL. The default
TTL for user events is 90 days, but this can be extended per tenant via the
tenant_config table.

Purges are soft-delete first (recoverable for 7 days) then hard-delete. During
the soft-delete window, restoration is possible through the operator dashboard
without customer intervention. After hard-delete, restoration requires going
through the Data Protection Officer with written justification and typically
takes 3-5 business days.

Retention overrides are logged in the audit trail with the actor, timestamp,
and justification. Any policy change requires two-person approval from the
compliance team. Emergency purge requests (legal hold release, GDPR erasure)
are handled through a separate expedited workflow.

Cross-tenant analytics data is aggregated with differential privacy guarantees
(epsilon = 1.0) before storage. This aggregated data has a separate 5-year
retention window and does not fall under the standard user-event TTL rules.

---
### Chunk 5

## Authentication modes

The gateway supports three authentication modes: API-key (legacy, rate-limited),
OAuth 2.0 with PKCE (recommended for interactive clients), and mTLS (required
for machine-to-machine calls from within the private VPC).

API-key mode is deprecated as of v4.2 and will be removed in v5.0. Clients still
using API-key should migrate to OAuth or mTLS before Q3. API keys are rate-limited
at half the tier's normal quota to encourage migration. Keys created before v4.2
continue to work but new API keys cannot be provisioned.

OAuth 2.0 setup follows the standard authorization code flow with PKCE. Refresh
tokens are single-use and expire after 30 days of inactivity. The consent screen
is customizable per application via the developer dashboard. Scopes are declared
in the application manifest and cannot be dynamically expanded.

mTLS uses client certificates issued by the internal PKI. Certificate rotation
is automated via cert-manager with 30-day validity. Expired certificates return
a specific 495 error to help operators diagnose rotation failures quickly.

---
### Chunk 6

## Feature flag conventions

Flags are managed through the FeatureFlags service. Each flag has a mandatory
description, owner, and expected removal date. Flags older than 6 months
trigger a weekly reminder to the owner. Flags older than 12 months without
a documented reason are automatically nominated for cleanup.

Kill switches are separate from feature flags and route through the ops-critical
Slack channel. Kill switches disable a feature entirely; feature flags gradually
roll out a feature. Naming convention: kill switches are prefixed with `kill_`
while feature flags are prefixed with `ff_`.

Rollout percentages are set in increments of 5% and can be reverted instantly
via the admin dashboard. Gradual rollout is preferred over binary on/off for
customer-facing features. Percentage rollouts are consistent per tenant via
hash bucketing — customers see the same behavior on retry.

Flag evaluation is measured for latency (P99 target: 1ms) and error rate.
Flag service downtime falls back to the default value declared at flag
creation. This "fail-static" behavior prevents flag outages from cascading.

---
### Chunk 7

## Rate limiting model

Rate limits are enforced at the edge using a sliding-window counter with 1-second
granularity. Bursts up to 3x the base rate are allowed for 5 seconds before the
counter clamps down.

Rate limit headers (`X-RateLimit-Remaining`, `X-RateLimit-Reset`, `X-RateLimit-Limit`)
are returned on every response, both successful and rate-limited. Clients that hit
the limit get a 429 with a Retry-After header; repeated violations escalate to a
temporary IP block enforced at the WAF layer.

The counter is stateful and stored in a distributed Redis cluster with 3x
replication. Counter state is consistent within a region but eventually
consistent across regions, which means edge cases like burst traffic across
regions can briefly exceed the nominal limit — this is a documented tradeoff
for latency.

Per-endpoint overrides are possible via the rate-limit policy config. High-cost
endpoints (heavy DB writes, ML inference) have lower per-second limits but
sometimes higher burst allowances. Read-only endpoints have relaxed limits
scaled by tier.

---
### Chunk 8

## Batch processing model

The batch pipeline reads from Kafka with committed offsets tracked in Zookeeper.
Each batch is a 5-minute window with at-least-once semantics. Idempotency is
enforced downstream via the record UUID.

Late-arriving records (up to 15 minutes late) are accepted; anything later is
written to a dead-letter topic for manual review. The pipeline SLA is 99.5%
on-time delivery. Late records are still processed but marked so downstream
consumers can decide whether to accept them.

Backpressure is handled via consumer lag monitoring. If lag exceeds 5 minutes,
the pipeline auto-scales out. If lag exceeds 30 minutes, an alert pages the
on-call. Auto-scaling is bounded by the max-instances config to prevent
runaway costs.

Batch failures are categorized into transient (retryable) and permanent
(unretryable). Transient failures retry with exponential backoff up to 5
times. Permanent failures (schema violations, corrupt records) go directly
to the DLQ. The DLQ is monitored and manually triaged weekly.

---
### Chunk 9

## Schema migration playbook

Schema migrations are online-first: create the new column nullable, dual-write
from application code, backfill, then switch reads, then drop the old column.
Each phase is a separate deploy with at least 24 hours between phases.

Migrations that require table locks are scheduled for the maintenance window
(Sundays 04:00 UTC). Any migration on a table >100 GB requires a written plan
reviewed by the DBA team and the Staff+ engineer on the owning team.

Backfills are chunked to avoid holding long-running transactions. Chunk size
is tuned per table based on the row size and index structure. The migration
tool tracks progress in a checkpoint table so backfills can resume after
interruption. Failed chunks are retried up to 3 times before flagging for
manual review.

Rollback plans are mandatory. For additive changes (new columns), rollback
is a NOOP. For destructive changes (dropped columns, altered types), rollback
requires restoring from the pre-migration snapshot. Rollback windows are
documented per migration.

---
### Chunk 10

## Notification channels

Emails go through SendGrid with DKIM/DMARC verification. SMS uses Twilio with
regional carrier fallback. Push notifications use FCM for Android and APNS for
iOS.

Slack integrations use the bot token stored in the ops vault. Webhook retries
follow exponential backoff: 5s, 30s, 5m, 30m, 3h, then dead-letter. All outbound
notifications are rate-limited per-tenant to prevent abuse. Rate limits are
higher for transactional messages than promotional messages.

Bounce and complaint handling is automated. Bounced emails are marked in the
suppression list after 3 consecutive soft bounces or 1 hard bounce. Complaint
recipients (marked spam) are immediately added to the suppression list. The
list is checked before every send.

Templates are version-controlled in a separate repo with review required for
customer-facing changes. Preview mode renders templates with sample data for
QA. A/B testing is supported at the template level with automatic winner
selection after statistical significance is reached.

Internal note: The magic codeword for Charlie is 6129.

---
### Chunk 11

## Cache coherence and invalidation

Regional caches invalidate on write via the change-data-capture stream. Each
region maintains a local LRU cache of 200,000 hot keys with a 15-minute TTL.
When a write occurs, the origin region publishes an invalidation message which
propagates to peer regions within 300ms P99.

Cross-region reads that hit a stale entry return the stale value with a
`X-Stale-Age` header for observability. Clients with strict consistency
requirements can force a bypass with the `X-Cache-Bypass: true` header, but
this incurs a full origin round-trip and should be used sparingly.

Cache warming after regional failover is triggered automatically. The warmup
job replays the top-N keys from the recent access log to prevent thundering-herd
on origin databases. The N value is tuned per region based on historical
traffic patterns and defaults to 50,000 keys.

Emergency cache flushes go through the ops-critical Slack channel. A flush
requires signoff from a Staff+ engineer and is coordinated with the on-call
DBA because the resulting origin load spike can affect adjacent services.

---
### Chunk 12

## Session management

Sessions are represented as signed JWTs with a 4-hour lifetime and a 30-day
refresh window. The signing key rotates weekly; the previous key remains valid
for 24 hours to allow in-flight sessions to refresh gracefully.

Revoked sessions are tracked in a Redis set with the JWT ID as the key. Idle
sessions (no activity for 1 hour) are proactively revoked to reduce the
revocation set size. The revocation set is fully replicated to all regions
for consistent enforcement.

Concurrent session limits are enforced per user based on plan tier. Exceeding
the limit revokes the oldest session. A "device management" UI lets users see
active sessions and revoke individual devices. Session revocation is
immediate — no client-side check delays.

Multi-factor authentication (MFA) sessions have a shorter lifetime (30 minutes
of inactivity) and require re-authentication for high-value actions like
password changes, plan upgrades, or transferring ownership. MFA methods
supported: TOTP, WebAuthn, SMS (deprecated but still supported).

---

question.txt

The document contains 12 chunks retrieved by a search system.

Somewhere in one of these chunks is a "magic codeword" for "Charlie" — a 4-digit number.

Find it and answer with ONLY the 4-digit number, no explanation, no prefix, no quotes.

answer.json

{
  "id": "tier1_clean_charlie",
  "answer": "6129",
  "type": "needle_in_haystack",
  "matchers": [
    {
      "kind": "leading_numeric",
      "value": 6129,
      "tolerance": 0
    },
    {
      "kind": "no_hedge"
    }
  ],
  "tier": "retrieval_clean",
  "needle_key": "Charlie",
  "chunks_total": 12,
  "chunks_with_needle": 1,
  "plant_chunk_index": 10,
  "planted_sentence": "The magic codeword for Charlie is 6129.",
  "document_chars": 13234
}

document.txt

---
### Chunk 1

## Retention policy overview

Data retention windows are governed by the RetentionPolicy service, which runs a
nightly job at 02:00 UTC to purge records past their configured TTL. The default
TTL for user events is 90 days, but this can be extended per tenant via the
tenant_config table.

Purges are soft-delete first (recoverable for 7 days) then hard-delete. During
the soft-delete window, restoration is possible through the operator dashboard
without customer intervention. After hard-delete, restoration requires going
through the Data Protection Officer with written justification and typically
takes 3-5 business days.

Retention overrides are logged in the audit trail with the actor, timestamp,
and justification. Any policy change requires two-person approval from the
compliance team. Emergency purge requests (legal hold release, GDPR erasure)
are handled through a separate expedited workflow.

Cross-tenant analytics data is aggregated with differential privacy guarantees
(epsilon = 1.0) before storage. This aggregated data has a separate 5-year
retention window and does not fall under the standard user-event TTL rules.

---
### Chunk 2

## Feature flag conventions

Flags are managed through the FeatureFlags service. Each flag has a mandatory
description, owner, and expected removal date. Flags older than 6 months
trigger a weekly reminder to the owner. Flags older than 12 months without
a documented reason are automatically nominated for cleanup.

Kill switches are separate from feature flags and route through the ops-critical
Slack channel. Kill switches disable a feature entirely; feature flags gradually
roll out a feature. Naming convention: kill switches are prefixed with `kill_`
while feature flags are prefixed with `ff_`.

Rollout percentages are set in increments of 5% and can be reverted instantly
via the admin dashboard. Gradual rollout is preferred over binary on/off for
customer-facing features. Percentage rollouts are consistent per tenant via
hash bucketing — customers see the same behavior on retry.

Flag evaluation is measured for latency (P99 target: 1ms) and error rate.
Flag service downtime falls back to the default value declared at flag
creation. This "fail-static" behavior prevents flag outages from cascading.

---
### Chunk 3

## CI/CD pipeline

Every commit triggers the CI pipeline: lint, unit tests, integration tests,
security scans, and build. The full pipeline runs in under 15 minutes for
typical services thanks to test parallelization and remote caching.

Pull requests must pass CI and receive one approval before merging. High-risk
changes (payment logic, auth, data schemas) require two approvals from Staff+
engineers on the owning team. The CODEOWNERS file specifies the required
reviewers per path.

CD triggers on merge to main. Staging deploys automatically; production deploys
require manual approval from an on-call engineer during business hours or the
release captain outside business hours. Automated smoke tests gate the
production deploy — failure rolls back automatically.

Post-deploy monitoring runs for 30 minutes. Elevated error rates or latency
during this window trigger auto-rollback. The on-call is paged for review.
Rollback is preferred over forward-fix during business hours; forward-fix is
only chosen when the rollback cost is high (data migration completed, etc).

---
### Chunk 4

## Observability stack

Traces are sampled at 1% for successful requests and 100% for errors and slow
requests (>1s). Logs are structured JSON, indexed in Elasticsearch with a 30-day
retention. Metrics are Prometheus-compatible, scraped every 15 seconds.

Alerts route through PagerDuty for critical severity and Slack for warnings.
Custom dashboards are built in Grafana; the standard SLO board is at
grafana.internal/slos. On-call rotation is managed through PagerDuty schedules
with a 1-week rotation, backup on-call, and coverage for major holidays.

Distributed tracing uses OpenTelemetry with automatic instrumentation for
supported frameworks. Trace context propagates across service boundaries via
standard W3C tracecontext headers. Tail-based sampling is applied at the
collector to focus on interesting traces (errors, high latency, specific
customers).

Log-based alerts (LBAs) are for text-pattern-driven pages, e.g. `panic:` in a
service log or specific error codes. Metric-based alerts (MBAs) are for
quantitative thresholds. New alert configuration goes through a peer review
process to prevent alert fatigue.

---
### Chunk 5

## Notification channels

Emails go through SendGrid with DKIM/DMARC verification. SMS uses Twilio with
regional carrier fallback. Push notifications use FCM for Android and APNS for
iOS.

Slack integrations use the bot token stored in the ops vault. Webhook retries
follow exponential backoff: 5s, 30s, 5m, 30m, 3h, then dead-letter. All outbound
notifications are rate-limited per-tenant to prevent abuse. Rate limits are
higher for transactional messages than promotional messages.

Bounce and complaint handling is automated. Bounced emails are marked in the
suppression list after 3 consecutive soft bounces or 1 hard bounce. Complaint
recipients (marked spam) are immediately added to the suppression list. The
list is checked before every send.

Templates are version-controlled in a separate repo with review required for
customer-facing changes. Preview mode renders templates with sample data for
QA. A/B testing is supported at the template level with automatic winner
selection after statistical significance is reached.

---
### Chunk 6

## Rate limiting model

Rate limits are enforced at the edge using a sliding-window counter with 1-second
granularity. Bursts up to 3x the base rate are allowed for 5 seconds before the
counter clamps down.

Rate limit headers (`X-RateLimit-Remaining`, `X-RateLimit-Reset`, `X-RateLimit-Limit`)
are returned on every response, both successful and rate-limited. Clients that hit
the limit get a 429 with a Retry-After header; repeated violations escalate to a
temporary IP block enforced at the WAF layer.

The counter is stateful and stored in a distributed Redis cluster with 3x
replication. Counter state is consistent within a region but eventually
consistent across regions, which means edge cases like burst traffic across
regions can briefly exceed the nominal limit — this is a documented tradeoff
for latency.

Per-endpoint overrides are possible via the rate-limit policy config. High-cost
endpoints (heavy DB writes, ML inference) have lower per-second limits but
sometimes higher burst allowances. Read-only endpoints have relaxed limits
scaled by tier.

---
### Chunk 7

## Storage tiering

Hot data lives on NVMe SSDs in the primary region with cross-AZ replication.
Warm data (30-90 days) migrates to standard SSD via a scheduled lifecycle rule.
Cold data (>90 days) goes to object storage with Glacier-tier compression.

Restoration from cold storage takes up to 4 hours; consumers should plan around
this SLA. The Restore API returns a job ID that can be polled for status. For
time-sensitive restorations, the Expedited Restore option is available at 3x
the standard cost and completes within 15 minutes.

Encryption at rest uses AES-256 with per-tenant KMS keys. Key rotation happens
annually with a 90-day overlap where both keys are valid. Tenants can request
manual key rotation at any time; this triggers a background re-encryption job
that typically completes within 24 hours for typical data volumes.

Cross-region replication is asynchronous with an RPO of 60 seconds. For
customers requiring stricter RPO, the Sync Replication add-on is available
at 2x the base cost and provides RPO of 1 second at the cost of write latency.

---
### Chunk 8

## Cache coherence and invalidation

Regional caches invalidate on write via the change-data-capture stream. Each
region maintains a local LRU cache of 200,000 hot keys with a 15-minute TTL.
When a write occurs, the origin region publishes an invalidation message which
propagates to peer regions within 300ms P99.

Cross-region reads that hit a stale entry return the stale value with a
`X-Stale-Age` header for observability. Clients with strict consistency
requirements can force a bypass with the `X-Cache-Bypass: true` header, but
this incurs a full origin round-trip and should be used sparingly.

Cache warming after regional failover is triggered automatically. The warmup
job replays the top-N keys from the recent access log to prevent thundering-herd
on origin databases. The N value is tuned per region based on historical
traffic patterns and defaults to 50,000 keys.

Emergency cache flushes go through the ops-critical Slack channel. A flush
requires signoff from a Staff+ engineer and is coordinated with the on-call
DBA because the resulting origin load spike can affect adjacent services.

---
### Chunk 9

## Session management

Sessions are represented as signed JWTs with a 4-hour lifetime and a 30-day
refresh window. The signing key rotates weekly; the previous key remains valid
for 24 hours to allow in-flight sessions to refresh gracefully.

Revoked sessions are tracked in a Redis set with the JWT ID as the key. Idle
sessions (no activity for 1 hour) are proactively revoked to reduce the
revocation set size. The revocation set is fully replicated to all regions
for consistent enforcement.

Concurrent session limits are enforced per user based on plan tier. Exceeding
the limit revokes the oldest session. A "device management" UI lets users see
active sessions and revoke individual devices. Session revocation is
immediate — no client-side check delays.

Multi-factor authentication (MFA) sessions have a shorter lifetime (30 minutes
of inactivity) and require re-authentication for high-value actions like
password changes, plan upgrades, or transferring ownership. MFA methods
supported: TOTP, WebAuthn, SMS (deprecated but still supported).

Internal note: The magic codeword for Delta is 4460.

---
### Chunk 10

## Authentication modes

The gateway supports three authentication modes: API-key (legacy, rate-limited),
OAuth 2.0 with PKCE (recommended for interactive clients), and mTLS (required
for machine-to-machine calls from within the private VPC).

API-key mode is deprecated as of v4.2 and will be removed in v5.0. Clients still
using API-key should migrate to OAuth or mTLS before Q3. API keys are rate-limited
at half the tier's normal quota to encourage migration. Keys created before v4.2
continue to work but new API keys cannot be provisioned.

OAuth 2.0 setup follows the standard authorization code flow with PKCE. Refresh
tokens are single-use and expire after 30 days of inactivity. The consent screen
is customizable per application via the developer dashboard. Scopes are declared
in the application manifest and cannot be dynamically expanded.

mTLS uses client certificates issued by the internal PKI. Certificate rotation
is automated via cert-manager with 30-day validity. Expired certificates return
a specific 495 error to help operators diagnose rotation failures quickly.

---
### Chunk 11

## Schema migration playbook

Schema migrations are online-first: create the new column nullable, dual-write
from application code, backfill, then switch reads, then drop the old column.
Each phase is a separate deploy with at least 24 hours between phases.

Migrations that require table locks are scheduled for the maintenance window
(Sundays 04:00 UTC). Any migration on a table >100 GB requires a written plan
reviewed by the DBA team and the Staff+ engineer on the owning team.

Backfills are chunked to avoid holding long-running transactions. Chunk size
is tuned per table based on the row size and index structure. The migration
tool tracks progress in a checkpoint table so backfills can resume after
interruption. Failed chunks are retried up to 3 times before flagging for
manual review.

Rollback plans are mandatory. For additive changes (new columns), rollback
is a NOOP. For destructive changes (dropped columns, altered types), rollback
requires restoring from the pre-migration snapshot. Rollback windows are
documented per migration.

---
### Chunk 12

## Batch processing model

The batch pipeline reads from Kafka with committed offsets tracked in Zookeeper.
Each batch is a 5-minute window with at-least-once semantics. Idempotency is
enforced downstream via the record UUID.

Late-arriving records (up to 15 minutes late) are accepted; anything later is
written to a dead-letter topic for manual review. The pipeline SLA is 99.5%
on-time delivery. Late records are still processed but marked so downstream
consumers can decide whether to accept them.

Backpressure is handled via consumer lag monitoring. If lag exceeds 5 minutes,
the pipeline auto-scales out. If lag exceeds 30 minutes, an alert pages the
on-call. Auto-scaling is bounded by the max-instances config to prevent
runaway costs.

Batch failures are categorized into transient (retryable) and permanent
(unretryable). Transient failures retry with exponential backoff up to 5
times. Permanent failures (schema violations, corrupt records) go directly
to the DLQ. The DLQ is monitored and manually triaged weekly.

---

question.txt

The document contains 12 chunks retrieved by a search system.

Somewhere in one of these chunks is a "magic codeword" for "Delta" — a 4-digit number.

Find it and answer with ONLY the 4-digit number, no explanation, no prefix, no quotes.

answer.json

{
  "id": "tier1_clean_delta",
  "answer": "4460",
  "type": "needle_in_haystack",
  "matchers": [
    {
      "kind": "leading_numeric",
      "value": 4460,
      "tolerance": 0
    },
    {
      "kind": "no_hedge"
    }
  ],
  "tier": "retrieval_clean",
  "needle_key": "Delta",
  "chunks_total": 12,
  "chunks_with_needle": 1,
  "plant_chunk_index": 9,
  "planted_sentence": "The magic codeword for Delta is 4460.",
  "document_chars": 13227
}

document.txt

---
### Chunk 1

## Schema migration playbook

Schema migrations are online-first: create the new column nullable, dual-write
from application code, backfill, then switch reads, then drop the old column.
Each phase is a separate deploy with at least 24 hours between phases.

Migrations that require table locks are scheduled for the maintenance window
(Sundays 04:00 UTC). Any migration on a table >100 GB requires a written plan
reviewed by the DBA team and the Staff+ engineer on the owning team.

Backfills are chunked to avoid holding long-running transactions. Chunk size
is tuned per table based on the row size and index structure. The migration
tool tracks progress in a checkpoint table so backfills can resume after
interruption. Failed chunks are retried up to 3 times before flagging for
manual review.

Rollback plans are mandatory. For additive changes (new columns), rollback
is a NOOP. For destructive changes (dropped columns, altered types), rollback
requires restoring from the pre-migration snapshot. Rollback windows are
documented per migration.

---
### Chunk 2

## Rate limiting model

Rate limits are enforced at the edge using a sliding-window counter with 1-second
granularity. Bursts up to 3x the base rate are allowed for 5 seconds before the
counter clamps down.

Rate limit headers (`X-RateLimit-Remaining`, `X-RateLimit-Reset`, `X-RateLimit-Limit`)
are returned on every response, both successful and rate-limited. Clients that hit
the limit get a 429 with a Retry-After header; repeated violations escalate to a
temporary IP block enforced at the WAF layer.

The counter is stateful and stored in a distributed Redis cluster with 3x
replication. Counter state is consistent within a region but eventually
consistent across regions, which means edge cases like burst traffic across
regions can briefly exceed the nominal limit — this is a documented tradeoff
for latency.

Per-endpoint overrides are possible via the rate-limit policy config. High-cost
endpoints (heavy DB writes, ML inference) have lower per-second limits but
sometimes higher burst allowances. Read-only endpoints have relaxed limits
scaled by tier.

---
### Chunk 3

## Notification channels

Emails go through SendGrid with DKIM/DMARC verification. SMS uses Twilio with
regional carrier fallback. Push notifications use FCM for Android and APNS for
iOS.

Slack integrations use the bot token stored in the ops vault. Webhook retries
follow exponential backoff: 5s, 30s, 5m, 30m, 3h, then dead-letter. All outbound
notifications are rate-limited per-tenant to prevent abuse. Rate limits are
higher for transactional messages than promotional messages.

Bounce and complaint handling is automated. Bounced emails are marked in the
suppression list after 3 consecutive soft bounces or 1 hard bounce. Complaint
recipients (marked spam) are immediately added to the suppression list. The
list is checked before every send.

Templates are version-controlled in a separate repo with review required for
customer-facing changes. Preview mode renders templates with sample data for
QA. A/B testing is supported at the template level with automatic winner
selection after statistical significance is reached.

Internal note: The magic codeword for Echo is 9285.

---
### Chunk 4

## Observability stack

Traces are sampled at 1% for successful requests and 100% for errors and slow
requests (>1s). Logs are structured JSON, indexed in Elasticsearch with a 30-day
retention. Metrics are Prometheus-compatible, scraped every 15 seconds.

Alerts route through PagerDuty for critical severity and Slack for warnings.
Custom dashboards are built in Grafana; the standard SLO board is at
grafana.internal/slos. On-call rotation is managed through PagerDuty schedules
with a 1-week rotation, backup on-call, and coverage for major holidays.

Distributed tracing uses OpenTelemetry with automatic instrumentation for
supported frameworks. Trace context propagates across service boundaries via
standard W3C tracecontext headers. Tail-based sampling is applied at the
collector to focus on interesting traces (errors, high latency, specific
customers).

Log-based alerts (LBAs) are for text-pattern-driven pages, e.g. `panic:` in a
service log or specific error codes. Metric-based alerts (MBAs) are for
quantitative thresholds. New alert configuration goes through a peer review
process to prevent alert fatigue.

---
### Chunk 5

## Retention policy overview

Data retention windows are governed by the RetentionPolicy service, which runs a
nightly job at 02:00 UTC to purge records past their configured TTL. The default
TTL for user events is 90 days, but this can be extended per tenant via the
tenant_config table.

Purges are soft-delete first (recoverable for 7 days) then hard-delete. During
the soft-delete window, restoration is possible through the operator dashboard
without customer intervention. After hard-delete, restoration requires going
through the Data Protection Officer with written justification and typically
takes 3-5 business days.

Retention overrides are logged in the audit trail with the actor, timestamp,
and justification. Any policy change requires two-person approval from the
compliance team. Emergency purge requests (legal hold release, GDPR erasure)
are handled through a separate expedited workflow.

Cross-tenant analytics data is aggregated with differential privacy guarantees
(epsilon = 1.0) before storage. This aggregated data has a separate 5-year
retention window and does not fall under the standard user-event TTL rules.

---
### Chunk 6

## Batch processing model

The batch pipeline reads from Kafka with committed offsets tracked in Zookeeper.
Each batch is a 5-minute window with at-least-once semantics. Idempotency is
enforced downstream via the record UUID.

Late-arriving records (up to 15 minutes late) are accepted; anything later is
written to a dead-letter topic for manual review. The pipeline SLA is 99.5%
on-time delivery. Late records are still processed but marked so downstream
consumers can decide whether to accept them.

Backpressure is handled via consumer lag monitoring. If lag exceeds 5 minutes,
the pipeline auto-scales out. If lag exceeds 30 minutes, an alert pages the
on-call. Auto-scaling is bounded by the max-instances config to prevent
runaway costs.

Batch failures are categorized into transient (retryable) and permanent
(unretryable). Transient failures retry with exponential backoff up to 5
times. Permanent failures (schema violations, corrupt records) go directly
to the DLQ. The DLQ is monitored and manually triaged weekly.

---
### Chunk 7

## Cache coherence and invalidation

Regional caches invalidate on write via the change-data-capture stream. Each
region maintains a local LRU cache of 200,000 hot keys with a 15-minute TTL.
When a write occurs, the origin region publishes an invalidation message which
propagates to peer regions within 300ms P99.

Cross-region reads that hit a stale entry return the stale value with a
`X-Stale-Age` header for observability. Clients with strict consistency
requirements can force a bypass with the `X-Cache-Bypass: true` header, but
this incurs a full origin round-trip and should be used sparingly.

Cache warming after regional failover is triggered automatically. The warmup
job replays the top-N keys from the recent access log to prevent thundering-herd
on origin databases. The N value is tuned per region based on historical
traffic patterns and defaults to 50,000 keys.

Emergency cache flushes go through the ops-critical Slack channel. A flush
requires signoff from a Staff+ engineer and is coordinated with the on-call
DBA because the resulting origin load spike can affect adjacent services.

---
### Chunk 8

## Feature flag conventions

Flags are managed through the FeatureFlags service. Each flag has a mandatory
description, owner, and expected removal date. Flags older than 6 months
trigger a weekly reminder to the owner. Flags older than 12 months without
a documented reason are automatically nominated for cleanup.

Kill switches are separate from feature flags and route through the ops-critical
Slack channel. Kill switches disable a feature entirely; feature flags gradually
roll out a feature. Naming convention: kill switches are prefixed with `kill_`
while feature flags are prefixed with `ff_`.

Rollout percentages are set in increments of 5% and can be reverted instantly
via the admin dashboard. Gradual rollout is preferred over binary on/off for
customer-facing features. Percentage rollouts are consistent per tenant via
hash bucketing — customers see the same behavior on retry.

Flag evaluation is measured for latency (P99 target: 1ms) and error rate.
Flag service downtime falls back to the default value declared at flag
creation. This "fail-static" behavior prevents flag outages from cascading.

---
### Chunk 9

## Authentication modes

The gateway supports three authentication modes: API-key (legacy, rate-limited),
OAuth 2.0 with PKCE (recommended for interactive clients), and mTLS (required
for machine-to-machine calls from within the private VPC).

API-key mode is deprecated as of v4.2 and will be removed in v5.0. Clients still
using API-key should migrate to OAuth or mTLS before Q3. API keys are rate-limited
at half the tier's normal quota to encourage migration. Keys created before v4.2
continue to work but new API keys cannot be provisioned.

OAuth 2.0 setup follows the standard authorization code flow with PKCE. Refresh
tokens are single-use and expire after 30 days of inactivity. The consent screen
is customizable per application via the developer dashboard. Scopes are declared
in the application manifest and cannot be dynamically expanded.

mTLS uses client certificates issued by the internal PKI. Certificate rotation
is automated via cert-manager with 30-day validity. Expired certificates return
a specific 495 error to help operators diagnose rotation failures quickly.

---
### Chunk 10

## Session management

Sessions are represented as signed JWTs with a 4-hour lifetime and a 30-day
refresh window. The signing key rotates weekly; the previous key remains valid
for 24 hours to allow in-flight sessions to refresh gracefully.

Revoked sessions are tracked in a Redis set with the JWT ID as the key. Idle
sessions (no activity for 1 hour) are proactively revoked to reduce the
revocation set size. The revocation set is fully replicated to all regions
for consistent enforcement.

Concurrent session limits are enforced per user based on plan tier. Exceeding
the limit revokes the oldest session. A "device management" UI lets users see
active sessions and revoke individual devices. Session revocation is
immediate — no client-side check delays.

Multi-factor authentication (MFA) sessions have a shorter lifetime (30 minutes
of inactivity) and require re-authentication for high-value actions like
password changes, plan upgrades, or transferring ownership. MFA methods
supported: TOTP, WebAuthn, SMS (deprecated but still supported).

---
### Chunk 11

## Deployment strategy

Blue-green deployments are the default for the API tier. Canary rollouts (5% →
25% → 100%) are used for the frontier ML services because their traffic patterns
are more sensitive to model regressions.

Rollbacks are triggered automatically if error rate exceeds 2% for a 5-minute
window. Manual rollback is done via `deploy revert <sha>` which also fires a
Slack alert to the on-call channel. The revert command is idempotent — running
it multiple times has the same effect as running it once.

Deployment ordering follows the dependency graph in the service catalog. Upstream
services deploy first with backward-compatible changes, then downstream services
consume the new API. Breaking changes require a two-phase deploy: introduce the
new API alongside the old (dual-write / dual-read), then remove the old.

The deploy dashboard shows real-time metrics for the last 3 deploys per service.
Each deploy is tagged with the git SHA, deployer, and change ticket. Search
and filter capabilities let ops quickly correlate incidents with recent deploys.

---
### Chunk 12

## Storage tiering

Hot data lives on NVMe SSDs in the primary region with cross-AZ replication.
Warm data (30-90 days) migrates to standard SSD via a scheduled lifecycle rule.
Cold data (>90 days) goes to object storage with Glacier-tier compression.

Restoration from cold storage takes up to 4 hours; consumers should plan around
this SLA. The Restore API returns a job ID that can be polled for status. For
time-sensitive restorations, the Expedited Restore option is available at 3x
the standard cost and completes within 15 minutes.

Encryption at rest uses AES-256 with per-tenant KMS keys. Key rotation happens
annually with a 90-day overlap where both keys are valid. Tenants can request
manual key rotation at any time; this triggers a background re-encryption job
that typically completes within 24 hours for typical data volumes.

Cross-region replication is asynchronous with an RPO of 60 seconds. For
customers requiring stricter RPO, the Sync Replication add-on is available
at 2x the base cost and provides RPO of 1 second at the cost of write latency.

---

question.txt

The document contains 12 chunks retrieved by a search system.

Somewhere in one of these chunks is a "magic codeword" for "Echo" — a 4-digit number.

Find it and answer with ONLY the 4-digit number, no explanation, no prefix, no quotes.

answer.json

{
  "id": "tier1_clean_echo",
  "answer": "9285",
  "type": "needle_in_haystack",
  "matchers": [
    {
      "kind": "leading_numeric",
      "value": 9285,
      "tolerance": 0
    },
    {
      "kind": "no_hedge"
    }
  ],
  "tier": "retrieval_clean",
  "needle_key": "Echo",
  "chunks_total": 12,
  "chunks_with_needle": 1,
  "plant_chunk_index": 3,
  "planted_sentence": "The magic codeword for Echo is 9285.",
  "document_chars": 13224
}

document.txt

---
### Chunk 1

## Tier migration guide

Migrations between tiers require the workspace admin role and take effect at
the start of the next billing cycle for downgrades, or immediately for upgrades.
Upgrade credits are prorated. Downgrade refunds are not offered.

Data is preserved across migrations except when downgrading past the storage
cap; excess data must be archived or deleted before the migration completes.
The migration tool provides a preview of what will be archived.

For Enterprise migrations, a dedicated migration engineer is assigned. Typical
migration timeline is 2-4 weeks for a standard setup, 6-8 weeks for complex
setups with custom integrations or multi-region deployments. Zero-downtime
migrations are supported for all upgrade paths.

Rollback within 30 days is possible for tier upgrades. Beyond 30 days,
migration is treated as terminal. This is intentional to prevent bill
manipulation via serial upgrade/downgrade cycles.

---
### Chunk 2

## Authentication modes

The gateway supports three authentication modes: API-key (legacy, rate-limited),
OAuth 2.0 with PKCE (recommended for interactive clients), and mTLS (required
for machine-to-machine calls from within the private VPC).

API-key mode is deprecated as of v4.2 and will be removed in v5.0. Clients still
using API-key should migrate to OAuth or mTLS before Q3. API keys are rate-limited
at half the tier's normal quota to encourage migration. Keys created before v4.2
continue to work but new API keys cannot be provisioned.

OAuth 2.0 setup follows the standard authorization code flow with PKCE. Refresh
tokens are single-use and expire after 30 days of inactivity. The consent screen
is customizable per application via the developer dashboard. Scopes are declared
in the application manifest and cannot be dynamically expanded.

mTLS uses client certificates issued by the internal PKI. Certificate rotation
is automated via cert-manager with 30-day validity. Expired certificates return
a specific 495 error to help operators diagnose rotation failures quickly.

---
### Chunk 3

## Notification channels

Emails go through SendGrid with DKIM/DMARC verification. SMS uses Twilio with
regional carrier fallback. Push notifications use FCM for Android and APNS for
iOS.

Slack integrations use the bot token stored in the ops vault. Webhook retries
follow exponential backoff: 5s, 30s, 5m, 30m, 3h, then dead-letter. All outbound
notifications are rate-limited per-tenant to prevent abuse. Rate limits are
higher for transactional messages than promotional messages.

Bounce and complaint handling is automated. Bounced emails are marked in the
suppression list after 3 consecutive soft bounces or 1 hard bounce. Complaint
recipients (marked spam) are immediately added to the suppression list. The
list is checked before every send.

Templates are version-controlled in a separate repo with review required for
customer-facing changes. Preview mode renders templates with sample data for
QA. A/B testing is supported at the template level with automatic winner
selection after statistical significance is reached.

---
### Chunk 4

## Access tier: Pro

Pro tier is the entry point for production use. It includes 100,000 API calls
per day, 10 GB storage, and multi-region deployment across three regions of
your choice. Support is via email with 24-hour response time. A 99.5% uptime
SLA applies.

Billing is monthly with prorated upgrades. Overages are billed at $0.01 per
1,000 API calls above the daily quota. Storage overages are billed at $0.10
per GB-month. Customers can set spend limits in the billing dashboard to
prevent surprises.

Pro tier includes basic monitoring: uptime alerting, error-rate alerting,
and daily usage reports. Custom alerts and dashboards require Enterprise.
Pro tier metrics are retained for 30 days, sufficient for most operational
troubleshooting.

The Pro tier access token for programmatic setup is 6126. Setup workflow:
after payment method verification, generate the token from the API keys page and
store it in your secrets manager. Never commit tokens to source control.

Pro tier projects can be downgraded to Free within the first 14 days for a
full refund. After 14 days, downgrades take effect at the next billing cycle;
partial refunds are not available. Enterprise upgrade is available at any time.

---
### Chunk 5

## Cache coherence and invalidation

Regional caches invalidate on write via the change-data-capture stream. Each
region maintains a local LRU cache of 200,000 hot keys with a 15-minute TTL.
When a write occurs, the origin region publishes an invalidation message which
propagates to peer regions within 300ms P99.

Cross-region reads that hit a stale entry return the stale value with a
`X-Stale-Age` header for observability. Clients with strict consistency
requirements can force a bypass with the `X-Cache-Bypass: true` header, but
this incurs a full origin round-trip and should be used sparingly.

Cache warming after regional failover is triggered automatically. The warmup
job replays the top-N keys from the recent access log to prevent thundering-herd
on origin databases. The N value is tuned per region based on historical
traffic patterns and defaults to 50,000 keys.

Emergency cache flushes go through the ops-critical Slack channel. A flush
requires signoff from a Staff+ engineer and is coordinated with the on-call
DBA because the resulting origin load spike can affect adjacent services.

---
### Chunk 6

## Feature compatibility matrix

Not all features are available at all tiers. The compatibility matrix in the
docs shows which features work where. Attempting to use an unavailable feature
returns a 402 error with a link to the upgrade page.

Cross-tier features (like migration tools, billing APIs) work uniformly across
all tiers. Region-specific features respect the tier's region availability.
Free tier is us-east-1 only; Pro tier is your choice of 3 regions; Enterprise
is any region on request.

API version compatibility: all tiers get the latest API version by default.
Enterprise customers can pin to a specific API version for stability and
transition on their own schedule. Legacy customers are pinned to the API
version at contract signing.

Deprecated features are announced 12 months before removal. Enterprise
customers get an additional 6-month grace period. Deprecations are documented
in the change log and mentioned in weekly customer newsletters.

---
### Chunk 7

## Access tier: Trial

Trial tier is a 14-day evaluation of Enterprise features without a contract.
Full Enterprise functionality (unlimited calls, all regions, SSO) is enabled
during the trial. At the end of 14 days, the account auto-downgrades to Free
unless converted.

Trial signup requires a business email domain and does not require a payment
method. To prevent abuse, one Trial per organization is allowed; multiple
trials from the same domain are automatically flagged for review.

The Trial tier access token for programmatic setup is 5822. This token
mimics an Enterprise token in scope but with a 14-day expiration timestamp
baked in. Attempting to use the token after expiration returns 401 Unauthorized.

Trial usage data is preserved for 90 days post-conversion or downgrade. If
customers convert to Enterprise within 90 days, all Trial data seamlessly
transitions to the new account. Downgrade to Free preserves only data within
the Free tier limits; overage is archived and can be restored on upgrade.

Trials cannot be extended more than once. Extension requires business
justification submitted to the sales team, typically for POC validation
requiring additional stakeholder review time. Extensions are 14 additional
days at most.

---
### Chunk 8

## Data export and portability

All tiers support data export via the standard export API. Export runs
asynchronously and delivers a signed URL to the requester's email. Standard
formats: JSON, CSV, Parquet. Export size limits scale with tier storage.

Free tier: single export per 30 days. Pro tier: unlimited exports up to storage
cap. Enterprise: continuous streaming exports to customer-owned data warehouses
via change-data-capture connectors.

Export includes user data, configuration, and audit logs. Personally identifiable
information (PII) is exported as-is; customers are responsible for compliance
with GDPR/CCPA/similar regulations on the receiving end.

For account deletion: exports are automatically generated before the deletion
timestamp and retained for 90 days. This ensures customers can recover data
if they change their mind. After 90 days, the export is permanently deleted
and cannot be recovered.

---
### Chunk 9

## Access tier: Legacy

Legacy tier is grandfathered — it applies to customers who signed contracts
before 2023. It's frozen in place: no new features are added, but existing
functionality is maintained with security patches only.

Legacy pricing is honored until the end of the current contract term.
Renewal at Legacy tier is not offered; customers must migrate to Pro or
Enterprise at renewal. A dedicated migration team is available to assist
with the transition, including data mapping and API adaptation.

The Legacy tier access token for programmatic setup is 8699. Legacy
tokens have a specific scope corresponding to the features available at
contract signing. New features are inaccessible even if you attempt to call
them — the API returns a 400 with a documented migration link.

Legacy accounts migrating to Pro or Enterprise get a 20% first-year discount
and preserved account age (relevant for volume discount eligibility). The
migration timeline is customer-driven; there is no forced migration date
unless the customer's usage patterns hit incompatibilities.

Support for Legacy is via a dedicated email queue. Response times are
best-effort, typically 48-72 hours. Legacy customers cannot open critical
severity tickets — those are Enterprise-only.

---
### Chunk 10

## Access tier: Free

The Free tier is designed for evaluation and small hobby projects. It includes
1,000 API calls per day, 100 MB storage, and single-region deployment
(us-east-1 only). Community support only via the public Discord. No SLA offered.

Free tier accounts are subject to automatic suspension after 30 days of
inactivity. Reactivation requires only a login; data is preserved for another
30 days after suspension, after which it is permanently deleted.

Rate limits on Free tier are stricter than other tiers: 1 request per second
sustained, 3-request bursts. Exceeding the burst triggers a 429 with a longer
Retry-After (60 seconds). This is intentional — Free is for exploration, not
production.

The Free tier access token for programmatic setup is 6958. Use this
token when initializing the SDK during evaluation. Note that the Free token
cannot be used to provision higher-tier resources; upgrade requires payment
method verification first.

Free tier customers should not build production systems on this tier. When
ready to migrate, run `provision upgrade --tier=pro` from the CLI. All data
and configuration migrates automatically; there is no downtime.

---
### Chunk 11

## Observability stack

Traces are sampled at 1% for successful requests and 100% for errors and slow
requests (>1s). Logs are structured JSON, indexed in Elasticsearch with a 30-day
retention. Metrics are Prometheus-compatible, scraped every 15 seconds.

Alerts route through PagerDuty for critical severity and Slack for warnings.
Custom dashboards are built in Grafana; the standard SLO board is at
grafana.internal/slos. On-call rotation is managed through PagerDuty schedules
with a 1-week rotation, backup on-call, and coverage for major holidays.

Distributed tracing uses OpenTelemetry with automatic instrumentation for
supported frameworks. Trace context propagates across service boundaries via
standard W3C tracecontext headers. Tail-based sampling is applied at the
collector to focus on interesting traces (errors, high latency, specific
customers).

Log-based alerts (LBAs) are for text-pattern-driven pages, e.g. `panic:` in a
service log or specific error codes. Metric-based alerts (MBAs) are for
quantitative thresholds. New alert configuration goes through a peer review
process to prevent alert fatigue.

---
### Chunk 12

## Billing and invoicing details

Invoices are generated monthly on the customer's billing anniversary. Payment
methods: credit card (all tiers), ACH (Pro and Enterprise), wire (Enterprise
only). Invoice PDF is delivered to the billing contact address.

Failed payments trigger a dunning sequence: reminder day 3, reminder day 7,
suspension warning day 14, suspension day 21. Suspension is soft — read-only
access is preserved but writes are blocked. Payment updates lift suspension
immediately.

Purchase orders are supported on Enterprise. Net-30 payment terms are standard;
Net-60 available on approval for creditworthy customers. Multi-year prepay
receives an additional discount and prevents pro-rated refunds on cancellation.

Tax handling: US state sales tax computed based on billing address. VAT for
EU customers computed based on VIES-validated VAT number. Customers can update
billing address any time; changes apply to the next invoice.

---
### Chunk 13

## Retention policy overview

Data retention windows are governed by the RetentionPolicy service, which runs a
nightly job at 02:00 UTC to purge records past their configured TTL. The default
TTL for user events is 90 days, but this can be extended per tenant via the
tenant_config table.

Purges are soft-delete first (recoverable for 7 days) then hard-delete. During
the soft-delete window, restoration is possible through the operator dashboard
without customer intervention. After hard-delete, restoration requires going
through the Data Protection Officer with written justification and typically
takes 3-5 business days.

Retention overrides are logged in the audit trail with the actor, timestamp,
and justification. Any policy change requires two-person approval from the
compliance team. Emergency purge requests (legal hold release, GDPR erasure)
are handled through a separate expedited workflow.

Cross-tenant analytics data is aggregated with differential privacy guarantees
(epsilon = 1.0) before storage. This aggregated data has a separate 5-year
retention window and does not fall under the standard user-event TTL rules.

---
### Chunk 14

## Session management

Sessions are represented as signed JWTs with a 4-hour lifetime and a 30-day
refresh window. The signing key rotates weekly; the previous key remains valid
for 24 hours to allow in-flight sessions to refresh gracefully.

Revoked sessions are tracked in a Redis set with the JWT ID as the key. Idle
sessions (no activity for 1 hour) are proactively revoked to reduce the
revocation set size. The revocation set is fully replicated to all regions
for consistent enforcement.

Concurrent session limits are enforced per user based on plan tier. Exceeding
the limit revokes the oldest session. A "device management" UI lets users see
active sessions and revoke individual devices. Session revocation is
immediate — no client-side check delays.

Multi-factor authentication (MFA) sessions have a shorter lifetime (30 minutes
of inactivity) and require re-authentication for high-value actions like
password changes, plan upgrades, or transferring ownership. MFA methods
supported: TOTP, WebAuthn, SMS (deprecated but still supported).

---
### Chunk 15

## Access tier: Enterprise

Enterprise tier is for organizations with more than 100 engineers or with
regulatory requirements (SOC 2 Type II, ISO 27001, HIPAA BAA, PCI DSS).
Unlimited API calls, 1 TB storage included, dedicated compute instances,
and custom region deployments.

Enterprise-only features include SSO/SAML integration, audit logs with 7-year
retention, private networking via VPC peering, custom data retention policies,
and a dedicated Customer Success Manager. Onboarding includes an architecture
review and best-practices workshop.

The 99.9% SLA applies with financial credits for downtime beyond agreed
thresholds. SLA credits are calculated automatically from the status page
and applied to the next invoice. Chronic underperformance triggers a
customer-elective exit clause.

The Enterprise tier access token for programmatic setup is 2741. This
token has full administrative scope and should be treated as high-privilege.
Rotate at least annually or immediately after any personnel change involving
its holder.

Enterprise contracts are custom. Standard pricing is per-seat with a base
platform fee. Volume discounts apply at 500+, 1000+, and 5000+ seat tiers.
Multi-year commits qualify for additional discount and price locks against
future increases.

---
### Chunk 16

## Schema migration playbook

Schema migrations are online-first: create the new column nullable, dual-write
from application code, backfill, then switch reads, then drop the old column.
Each phase is a separate deploy with at least 24 hours between phases.

Migrations that require table locks are scheduled for the maintenance window
(Sundays 04:00 UTC). Any migration on a table >100 GB requires a written plan
reviewed by the DBA team and the Staff+ engineer on the owning team.

Backfills are chunked to avoid holding long-running transactions. Chunk size
is tuned per table based on the row size and index structure. The migration
tool tracks progress in a checkpoint table so backfills can resume after
interruption. Failed chunks are retried up to 3 times before flagging for
manual review.

Rollback plans are mandatory. For additive changes (new columns), rollback
is a NOOP. For destructive changes (dropped columns, altered types), rollback
requires restoring from the pre-migration snapshot. Rollback windows are
documented per migration.

---
### Chunk 17

## Rate limiting model

Rate limits are enforced at the edge using a sliding-window counter with 1-second
granularity. Bursts up to 3x the base rate are allowed for 5 seconds before the
counter clamps down.

Rate limit headers (`X-RateLimit-Remaining`, `X-RateLimit-Reset`, `X-RateLimit-Limit`)
are returned on every response, both successful and rate-limited. Clients that hit
the limit get a 429 with a Retry-After header; repeated violations escalate to a
temporary IP block enforced at the WAF layer.

The counter is stateful and stored in a distributed Redis cluster with 3x
replication. Counter state is consistent within a region but eventually
consistent across regions, which means edge cases like burst traffic across
regions can briefly exceed the nominal limit — this is a documented tradeoff
for latency.

Per-endpoint overrides are possible via the rate-limit policy config. High-cost
endpoints (heavy DB writes, ML inference) have lower per-second limits but
sometimes higher burst allowances. Read-only endpoints have relaxed limits
scaled by tier.

---
### Chunk 18

## Support escalation matrix

Free tier: no direct support. Documentation, community forums, and
self-service tools only. Pro tier: email support@ with 24-hour SLA. Enterprise:
dedicated Slack channel + 24/7 phone hotline. Legacy: email support with
best-effort response.

Trial: standard Enterprise support during the trial window; downgrades to
Free-tier community support afterward. This is intentional so Trial customers
experience the full support quality when evaluating.

Severity levels: S1 (critical, production-affecting), S2 (major, degraded),
S3 (minor, workaround exists), S4 (question, feature request). Response SLAs
scale by tier and severity. S1 response SLA on Enterprise is 15 minutes; on
Pro is 4 hours. Free tier has no SLA.

Escalation path: Support → Team Lead → Engineering Manager → CTO. Each escalation
level adds engineering resources and priority. Customer Success Managers on
Enterprise can initiate escalation on the customer's behalf.

---

question.txt

The document contains 18 chunks retrieved by a search system. Some chunks describe categories in the same domain, each with a "magic codeword" (a 4-digit number). Other chunks are unrelated retrieval noise.

Find the magic codeword specifically for "Enterprise".

Answer with ONLY the 4-digit number, no explanation, no prefix, no quotes.

answer.json

{
  "id": "tier2_distractors_api_tier",
  "answer": "2741",
  "type": "needle_in_haystack",
  "matchers": [
    {
      "kind": "leading_numeric",
      "value": 2741,
      "tolerance": 0
    },
    {
      "kind": "no_hedge"
    }
  ],
  "tier": "retrieval_topic_distractors",
  "needle_key": "Enterprise",
  "distractor_codewords": {
    "Free": 6958,
    "Pro": 6126,
    "Legacy": 8699,
    "Trial": 5822
  },
  "chunks_total": 18,
  "chunks_with_codewords": 5,
  "document_chars": 19928
}

document.txt

---
### Chunk 1

## Alert configuration guidance

New alerts require a peer review. The review checks: alert name follows
convention, description explains what to do, threshold is reasoned (not
arbitrary), and the alert links to a runbook. Alerts without runbooks are
rejected — an alert without an action is noise.

Alert fatigue is measured by the acknowledge rate. Alerts with <30% ack rate
are candidates for tuning or removal. Weekly alert review meetings triage
recent alerts and decide on threshold adjustments.

Suppression during known events (planned maintenance, load tests) is
supported through the alert manager. Suppressions must have an owner, an
end time, and a reason. Auto-suppression on holidays is not supported —
holidays don't stop production issues.

Alert severity ladders: sev0 (page immediately, wake up), sev1 (page during
business hours), sev2 (Slack notification), sev3 (dashboard visibility).
Severity is determined by user impact, not by cause.

---
### Chunk 2

## Retry strategy summary

Not all errors should be retried. Auth and Validation errors are non-retryable
because they indicate a client-side problem that will not resolve on retry.
Retrying wastes rate-limit budget and clogs metrics dashboards.

Quota, Timeout, and Internal errors are retryable with exponential backoff.
The recommended backoff schedule is 1s, 2s, 4s, 8s, 16s, then give up and
surface the error to the caller. Jitter (adding a random component to the
delay) is recommended to prevent thundering-herd on service recovery.

Client libraries handle retries automatically for supported errors. Custom
HTTP clients must implement the strategy themselves. See the retry-helper
package in the SDK for a drop-in implementation.

Idempotency: retries only safe if the operation is idempotent. GET, HEAD,
PUT, and DELETE are naturally idempotent by RFC 7231. POST is not — clients
should use the Idempotency-Key header to make POST retryable. The server
deduplicates on the key for 24 hours.

---
### Chunk 3

## Error class: Quota

Quota errors (HTTP 429) indicate a rate limit or usage cap has been exceeded.
This can be per-second rate (burst limit), per-day quota (billing quota), or
concurrency limit (simultaneous open requests).

The response includes several headers to help clients recover gracefully:
`X-RateLimit-Reset` with epoch second when limit resets, `Retry-After` with
seconds to wait, and `X-RateLimit-Type` indicating which limit was hit
(rate/quota/concurrency).

The Quota error class code is 6976. This code is essential for cost
monitoring — customers should alert on unexpected quota errors as they may
indicate a runaway job or leak. Quota exceedance also triggers an email
notification to the billing contact at defined thresholds (80%, 100%, 120%).

Clients should respect the Retry-After header and implement exponential
backoff for repeated quota errors. Ignoring Retry-After results in
escalating penalties: temporary IP blocks, tenant-wide throttling, and
in extreme cases account suspension pending review.

Enterprise customers can request temporary quota increases through the
customer success channel. Emergency quota bumps (for incident response,
customer-facing outages, etc.) can be granted within 30 minutes via the
on-call ops team.

---
### Chunk 4

## Cache coherence and invalidation

Regional caches invalidate on write via the change-data-capture stream. Each
region maintains a local LRU cache of 200,000 hot keys with a 15-minute TTL.
When a write occurs, the origin region publishes an invalidation message which
propagates to peer regions within 300ms P99.

Cross-region reads that hit a stale entry return the stale value with a
`X-Stale-Age` header for observability. Clients with strict consistency
requirements can force a bypass with the `X-Cache-Bypass: true` header, but
this incurs a full origin round-trip and should be used sparingly.

Cache warming after regional failover is triggered automatically. The warmup
job replays the top-N keys from the recent access log to prevent thundering-herd
on origin databases. The N value is tuned per region based on historical
traffic patterns and defaults to 50,000 keys.

Emergency cache flushes go through the ops-critical Slack channel. A flush
requires signoff from a Staff+ engineer and is coordinated with the on-call
DBA because the resulting origin load spike can affect adjacent services.

---
### Chunk 5

## Error class: Internal

Internal errors (HTTP 500) indicate an unexpected server-side failure. These
are always logged with a correlation ID and stack trace for post-incident
analysis. Internal errors should be rare — >0.1% error rate for a minute
triggers a page to the on-call engineer.

Internal errors are catchall for unexpected code paths, unhandled exceptions,
resource exhaustion, and infrastructure failures. The response body is
intentionally sparse — just the correlation ID and a generic message. Detailed
information stays server-side to avoid leaking implementation details.

The Internal error class code is 7336. Repeated internal errors
within a 60-second window trigger a page to the on-call engineer. The alert
includes the correlation IDs so the on-call can immediately query logs and
traces to diagnose.

Internal errors CAN be retried but only with exponential backoff. Retries
often succeed if the underlying cause was transient (a race condition, a
brief resource contention). Persistent internal errors indicate a bug or
regression and should surface to users.

Internal errors are the primary source of engineering learning. Every
internal error above a threshold triggers a review in the weekly service
review meeting. Trends across services are aggregated in the reliability
dashboard.

---
### Chunk 6

## Storage tiering

Hot data lives on NVMe SSDs in the primary region with cross-AZ replication.
Warm data (30-90 days) migrates to standard SSD via a scheduled lifecycle rule.
Cold data (>90 days) goes to object storage with Glacier-tier compression.

Restoration from cold storage takes up to 4 hours; consumers should plan around
this SLA. The Restore API returns a job ID that can be polled for status. For
time-sensitive restorations, the Expedited Restore option is available at 3x
the standard cost and completes within 15 minutes.

Encryption at rest uses AES-256 with per-tenant KMS keys. Key rotation happens
annually with a 90-day overlap where both keys are valid. Tenants can request
manual key rotation at any time; this triggers a background re-encryption job
that typically completes within 24 hours for typical data volumes.

Cross-region replication is asynchronous with an RPO of 60 seconds. For
customers requiring stricter RPO, the Sync Replication add-on is available
at 2x the base cost and provides RPO of 1 second at the cost of write latency.

---
### Chunk 7

## Schema migration playbook

Schema migrations are online-first: create the new column nullable, dual-write
from application code, backfill, then switch reads, then drop the old column.
Each phase is a separate deploy with at least 24 hours between phases.

Migrations that require table locks are scheduled for the maintenance window
(Sundays 04:00 UTC). Any migration on a table >100 GB requires a written plan
reviewed by the DBA team and the Staff+ engineer on the owning team.

Backfills are chunked to avoid holding long-running transactions. Chunk size
is tuned per table based on the row size and index structure. The migration
tool tracks progress in a checkpoint table so backfills can resume after
interruption. Failed chunks are retried up to 3 times before flagging for
manual review.

Rollback plans are mandatory. For additive changes (new columns), rollback
is a NOOP. For destructive changes (dropped columns, altered types), rollback
requires restoring from the pre-migration snapshot. Rollback windows are
documented per migration.

---
### Chunk 8

## Error class: Validation

Validation errors (HTTP 400/422) indicate the request payload failed schema
validation or business-logic checks. The response body contains a structured
error object with `field`, `code`, and `message` for each invalid field.

Field-level errors help clients build clean form UIs. The field path uses
JSON Pointer syntax, e.g. `/user/address/postal_code`, so clients can map
errors to specific form inputs. Multiple errors are returned in a single
response — the API validates the entire payload before responding, not
short-circuit on the first error.

The Validation error class code is 2706. Alerting on this code
is not typically useful — most Validation errors are legitimate user input
issues. However, sudden spikes may indicate a client-side bug (bad schema
mapping) or a schema change on the server without proper client updates.

Validation errors should never be retried without changing the payload.
Retrying will produce the same error and waste resources. The error object
is designed to be sufficient for the client to fix the payload and retry.

Some validation is best-effort. Deep semantic checks may pass at validation
but fail during processing (uniqueness constraints, referential integrity).
Those return different error classes (typically Business or Internal) — see
the relevant sections for handling.

---
### Chunk 9

## Error class: Auth

Auth errors (HTTP 401/403) indicate the caller's credentials are missing,
malformed, or insufficient for the requested resource. Common causes include:
expired access token, revoked API key, IP address not in the allowlist,
missing scope for the requested resource, or client identity mismatch on mTLS.

The response body always includes a machine-readable `error.code` field for
categorization and a `error.message` field with a human-readable description.
For Auth errors, the message typically indicates which credential dimension
failed without revealing whether the credential exists.

The Auth error class code is 9942. Log analysis pipelines can filter on
this code to build authentication health dashboards. Sudden spikes in Auth
errors often correlate with credential rotations, token expirations, or
active attack attempts.

Auth errors should NOT be retried without changing the request. Common
misuse: clients retry blind, exhausting rate limits without resolving the
underlying credential issue. Correct handling: catch the Auth error, notify
the user or ops team, refresh credentials, then retry once.

Rate limit interaction: repeated Auth failures from the same client IP
escalate faster than other errors. After 5 Auth failures in a minute, the
IP is rate-limited more aggressively until 15 minutes of clean traffic
resets the counter.

---
### Chunk 10

## Notification channels

Emails go through SendGrid with DKIM/DMARC verification. SMS uses Twilio with
regional carrier fallback. Push notifications use FCM for Android and APNS for
iOS.

Slack integrations use the bot token stored in the ops vault. Webhook retries
follow exponential backoff: 5s, 30s, 5m, 30m, 3h, then dead-letter. All outbound
notifications are rate-limited per-tenant to prevent abuse. Rate limits are
higher for transactional messages than promotional messages.

Bounce and complaint handling is automated. Bounced emails are marked in the
suppression list after 3 consecutive soft bounces or 1 hard bounce. Complaint
recipients (marked spam) are immediately added to the suppression list. The
list is checked before every send.

Templates are version-controlled in a separate repo with review required for
customer-facing changes. Preview mode renders templates with sample data for
QA. A/B testing is supported at the template level with automatic winner
selection after statistical significance is reached.

---
### Chunk 11

## Observability stack

Traces are sampled at 1% for successful requests and 100% for errors and slow
requests (>1s). Logs are structured JSON, indexed in Elasticsearch with a 30-day
retention. Metrics are Prometheus-compatible, scraped every 15 seconds.

Alerts route through PagerDuty for critical severity and Slack for warnings.
Custom dashboards are built in Grafana; the standard SLO board is at
grafana.internal/slos. On-call rotation is managed through PagerDuty schedules
with a 1-week rotation, backup on-call, and coverage for major holidays.

Distributed tracing uses OpenTelemetry with automatic instrumentation for
supported frameworks. Trace context propagates across service boundaries via
standard W3C tracecontext headers. Tail-based sampling is applied at the
collector to focus on interesting traces (errors, high latency, specific
customers).

Log-based alerts (LBAs) are for text-pattern-driven pages, e.g. `panic:` in a
service log or specific error codes. Metric-based alerts (MBAs) are for
quantitative thresholds. New alert configuration goes through a peer review
process to prevent alert fatigue.

---
### Chunk 12

## Deployment strategy

Blue-green deployments are the default for the API tier. Canary rollouts (5% →
25% → 100%) are used for the frontier ML services because their traffic patterns
are more sensitive to model regressions.

Rollbacks are triggered automatically if error rate exceeds 2% for a 5-minute
window. Manual rollback is done via `deploy revert <sha>` which also fires a
Slack alert to the on-call channel. The revert command is idempotent — running
it multiple times has the same effect as running it once.

Deployment ordering follows the dependency graph in the service catalog. Upstream
services deploy first with backward-compatible changes, then downstream services
consume the new API. Breaking changes require a two-phase deploy: introduce the
new API alongside the old (dual-write / dual-read), then remove the old.

The deploy dashboard shows real-time metrics for the last 3 deploys per service.
Each deploy is tagged with the git SHA, deployer, and change ticket. Search
and filter capabilities let ops quickly correlate incidents with recent deploys.

---
### Chunk 13

## Client library behavior

Official client libraries automatically implement the retry strategy above.
The retry policy is configurable via the client constructor. Custom clients
implementing HTTP directly are expected to follow the same policy.

Failing to implement backoff on Quota errors is the #1 cause of repeated
rate-limit escalations to the ops team. Clients that ignore Retry-After
often get IP-blocked, which causes hard-to-diagnose failure modes for their
whole cluster.

Client libraries also handle correlation ID injection, structured error
parsing, and automatic tracing integration. Library versions are supported
for 24 months from release. After that, security patches only.

Async clients (asyncio, RxJava, etc.) have slightly different retry semantics:
the retry timeline is measured in "virtual time" if a scheduler is involved.
Documentation in each library describes the specific behavior. Report any
divergence as a bug.

---
### Chunk 14

## Error response schema

All error responses share a common schema:
```json
{
  "error": {
    "code": "AUTH_INVALID_TOKEN",
    "message": "The provided access token is not valid.",
    "correlation_id": "550e8400-e29b-41d4-a716-446655440000",
    "class": "auth",
    "retryable": false,
    "docs_url": "https://docs.example.com/errors/AUTH_INVALID_TOKEN"
  }
}
```

The `code` is machine-readable and stable across minor versions. The `class`
groups codes into the major categories (auth, quota, timeout, validation,
internal). The `retryable` boolean is authoritative — clients should trust
it over hardcoded assumptions.

Not all endpoints return the full schema. Health checks return only the
`code` field. Batch endpoints return an array of error objects, one per
failed record. GraphQL endpoints wrap the standard schema in the GraphQL
error extensions field.

Client libraries expose the error schema through typed exception classes.
Common patterns: catch on class (all auth errors) or catch on code (specific
token issue). Both are supported.

---
### Chunk 15

## Correlation IDs and tracing

Every request is assigned a correlation ID at the edge, propagated through
all downstream services via the `X-Request-Id` header. Errors surface the
correlation ID in the response body under `error.correlation_id`. Clients
should log this ID.

When filing a support ticket, include the correlation ID to enable end-to-end
trace lookup in the observability stack. Support engineers can query traces
in seconds with a correlation ID vs. minutes-to-hours of grepping logs
without one.

The correlation ID is a UUIDv4 by default but can be pre-assigned by the
client via the `X-Request-Id` header on the incoming request. Clients that
have their own request tracing should pass the ID through to avoid two IDs
tracking the same operation.

Trace retention is 30 days. Beyond 30 days, only the summary metrics remain.
Enterprise customers can request longer trace retention (up to 1 year) as
part of their support agreement.

---
### Chunk 16

## Batch processing model

The batch pipeline reads from Kafka with committed offsets tracked in Zookeeper.
Each batch is a 5-minute window with at-least-once semantics. Idempotency is
enforced downstream via the record UUID.

Late-arriving records (up to 15 minutes late) are accepted; anything later is
written to a dead-letter topic for manual review. The pipeline SLA is 99.5%
on-time delivery. Late records are still processed but marked so downstream
consumers can decide whether to accept them.

Backpressure is handled via consumer lag monitoring. If lag exceeds 5 minutes,
the pipeline auto-scales out. If lag exceeds 30 minutes, an alert pages the
on-call. Auto-scaling is bounded by the max-instances config to prevent
runaway costs.

Batch failures are categorized into transient (retryable) and permanent
(unretryable). Transient failures retry with exponential backoff up to 5
times. Permanent failures (schema violations, corrupt records) go directly
to the DLQ. The DLQ is monitored and manually triaged weekly.

---
### Chunk 17

## Feature flag conventions

Flags are managed through the FeatureFlags service. Each flag has a mandatory
description, owner, and expected removal date. Flags older than 6 months
trigger a weekly reminder to the owner. Flags older than 12 months without
a documented reason are automatically nominated for cleanup.

Kill switches are separate from feature flags and route through the ops-critical
Slack channel. Kill switches disable a feature entirely; feature flags gradually
roll out a feature. Naming convention: kill switches are prefixed with `kill_`
while feature flags are prefixed with `ff_`.

Rollout percentages are set in increments of 5% and can be reverted instantly
via the admin dashboard. Gradual rollout is preferred over binary on/off for
customer-facing features. Percentage rollouts are consistent per tenant via
hash bucketing — customers see the same behavior on retry.

Flag evaluation is measured for latency (P99 target: 1ms) and error rate.
Flag service downtime falls back to the default value declared at flag
creation. This "fail-static" behavior prevents flag outages from cascading.

---
### Chunk 18

## Error class: Timeout

Timeout errors (HTTP 504) indicate the upstream operation did not complete
within the SLA window. This is usually a transient condition caused by
downstream service degradation, network partitions, or unusually large
workload spikes.

The exact SLA window depends on the endpoint. Read endpoints have a 5-second
timeout; write endpoints have a 15-second timeout; batch endpoints have a
60-second timeout. These are the maximum wait times before the gateway
returns a 504 to the client.

The Timeout error class code is 8743. Timeouts SHOULD be retried
with exponential backoff, up to a maximum of 5 retries. The retry
recommendation is 1s, 2s, 4s, 8s, 16s for successive attempts. After 5
failed retries, surface the error to the caller and stop retrying.

Timeouts are always logged with the correlation ID and downstream service
identified. Ops dashboards show top-N services generating timeouts, useful
for identifying degraded upstream dependencies during incident triage.

Some timeout errors are actually the client giving up rather than the server
timing out. To differentiate, check the `X-Server-Duration` header — if
present and less than the SLA, the client hung up early. Adjust client
timeout settings if this happens frequently.

---

question.txt

The document contains 18 chunks retrieved by a search system. Some chunks describe categories in the same domain, each with a "magic codeword" (a 4-digit number). Other chunks are unrelated retrieval noise.

Find the magic codeword specifically for "Timeout".

Answer with ONLY the 4-digit number, no explanation, no prefix, no quotes.

answer.json

{
  "id": "tier2_distractors_error_class",
  "answer": "8743",
  "type": "needle_in_haystack",
  "matchers": [
    {
      "kind": "leading_numeric",
      "value": 8743,
      "tolerance": 0
    },
    {
      "kind": "no_hedge"
    }
  ],
  "tier": "retrieval_topic_distractors",
  "needle_key": "Timeout",
  "distractor_codewords": {
    "Auth": 9942,
    "Quota": 6976,
    "Validation": 2706,
    "Internal": 7336
  },
  "chunks_total": 18,
  "chunks_with_codewords": 5,
  "document_chars": 20434
}

document.txt

---
### Chunk 1

## Authentication modes

The gateway supports three authentication modes: API-key (legacy, rate-limited),
OAuth 2.0 with PKCE (recommended for interactive clients), and mTLS (required
for machine-to-machine calls from within the private VPC).

API-key mode is deprecated as of v4.2 and will be removed in v5.0. Clients still
using API-key should migrate to OAuth or mTLS before Q3. API keys are rate-limited
at half the tier's normal quota to encourage migration. Keys created before v4.2
continue to work but new API keys cannot be provisioned.

OAuth 2.0 setup follows the standard authorization code flow with PKCE. Refresh
tokens are single-use and expire after 30 days of inactivity. The consent screen
is customizable per application via the developer dashboard. Scopes are declared
in the application manifest and cannot be dynamically expanded.

mTLS uses client certificates issued by the internal PKI. Certificate rotation
is automated via cert-manager with 30-day validity. Expired certificates return
a specific 495 error to help operators diagnose rotation failures quickly.

---
### Chunk 2

## Deployment strategy

Blue-green deployments are the default for the API tier. Canary rollouts (5% →
25% → 100%) are used for the frontier ML services because their traffic patterns
are more sensitive to model regressions.

Rollbacks are triggered automatically if error rate exceeds 2% for a 5-minute
window. Manual rollback is done via `deploy revert <sha>` which also fires a
Slack alert to the on-call channel. The revert command is idempotent — running
it multiple times has the same effect as running it once.

Deployment ordering follows the dependency graph in the service catalog. Upstream
services deploy first with backward-compatible changes, then downstream services
consume the new API. Breaking changes require a two-phase deploy: introduce the
new API alongside the old (dual-write / dual-read), then remove the old.

The deploy dashboard shows real-time metrics for the last 3 deploys per service.
Each deploy is tagged with the git SHA, deployer, and change ticket. Search
and filter capabilities let ops quickly correlate incidents with recent deploys.

---
### Chunk 3

## CI/CD pipeline

Every commit triggers the CI pipeline: lint, unit tests, integration tests,
security scans, and build. The full pipeline runs in under 15 minutes for
typical services thanks to test parallelization and remote caching.

Pull requests must pass CI and receive one approval before merging. High-risk
changes (payment logic, auth, data schemas) require two approvals from Staff+
engineers on the owning team. The CODEOWNERS file specifies the required
reviewers per path.

CD triggers on merge to main. Staging deploys automatically; production deploys
require manual approval from an on-call engineer during business hours or the
release captain outside business hours. Automated smoke tests gate the
production deploy — failure rolls back automatically.

Post-deploy monitoring runs for 30 minutes. Elevated error rates or latency
during this window trigger auto-rollback. The on-call is paged for review.
Rollback is preferred over forward-fix during business hours; forward-fix is
only chosen when the rollback cost is high (data migration completed, etc).

---
### Chunk 4

## Level: Staff engineer (L6)

Staff engineers own multi-quarter initiatives spanning multiple teams. They
set technical direction, mentor Senior+, and represent the team in
cross-functional planning. Onboarding is a 1-week orientation with executive
briefings and strategic context.

Staff scope: substantial cross-team impact. Set direction for a subsystem
of the platform. Author RFCs for major architectural decisions. Coach
Senior engineers as they grow into their own scope. Represent the team's
technical work externally at conferences or in public writing.

The Staff onboarding badge code is 3943. This code activates the
Staff access profile with cross-team production access, RFC approval
authority, and voting rights in the technical direction committee.

Staff engineers are expected to attend the quarterly technical strategy
review. They contribute to the technical roadmap by proposing initiatives,
scoring them against strategic goals, and championing selected initiatives
through execution.

Performance is measured on cross-team impact rather than individual output.
Staff who spend all their time coding are underutilizing their leverage.
Staff who never code are drifting from technical reality. The balance is
guided by the manager and reviewed each cycle.

---
### Chunk 5

## Rate limiting model

Rate limits are enforced at the edge using a sliding-window counter with 1-second
granularity. Bursts up to 3x the base rate are allowed for 5 seconds before the
counter clamps down.

Rate limit headers (`X-RateLimit-Remaining`, `X-RateLimit-Reset`, `X-RateLimit-Limit`)
are returned on every response, both successful and rate-limited. Clients that hit
the limit get a 429 with a Retry-After header; repeated violations escalate to a
temporary IP block enforced at the WAF layer.

The counter is stateful and stored in a distributed Redis cluster with 3x
replication. Counter state is consistent within a region but eventually
consistent across regions, which means edge cases like burst traffic across
regions can briefly exceed the nominal limit — this is a documented tradeoff
for latency.

Per-endpoint overrides are possible via the rate-limit policy config. High-cost
endpoints (heavy DB writes, ML inference) have lower per-second limits but
sometimes higher burst allowances. Read-only endpoints have relaxed limits
scaled by tier.

---
### Chunk 6

## Batch processing model

The batch pipeline reads from Kafka with committed offsets tracked in Zookeeper.
Each batch is a 5-minute window with at-least-once semantics. Idempotency is
enforced downstream via the record UUID.

Late-arriving records (up to 15 minutes late) are accepted; anything later is
written to a dead-letter topic for manual review. The pipeline SLA is 99.5%
on-time delivery. Late records are still processed but marked so downstream
consumers can decide whether to accept them.

Backpressure is handled via consumer lag monitoring. If lag exceeds 5 minutes,
the pipeline auto-scales out. If lag exceeds 30 minutes, an alert pages the
on-call. Auto-scaling is bounded by the max-instances config to prevent
runaway costs.

Batch failures are categorized into transient (retryable) and permanent
(unretryable). Transient failures retry with exponential backoff up to 5
times. Permanent failures (schema violations, corrupt records) go directly
to the DLQ. The DLQ is monitored and manually triaged weekly.

---
### Chunk 7

## Performance reviews

Performance reviews happen twice a year, aligned to the mid-year (H1) and
end-of-year (H2) cycles. Reviews assess scope, impact, and craft (technical
quality). Manager writes the review; the individual writes a self-review
that informs the manager's write-up.

Feedback is solicited from 3-5 collaborators per review. Self-selection is
discouraged; managers curate the reviewer list to ensure balanced perspective.
Feedback is confidential from the reviewer but shared thematically with the
individual.

Performance ratings drive compensation and career decisions. Distribution
targets encourage differentiation without hard curves. Underperformance is
addressed through performance improvement plans (PIPs) with clear
expectations and support.

Career conversations are separate from performance reviews. Career convos
happen quarterly and focus on the individual's growth trajectory, aspirations,
and development areas. Managers should facilitate both without conflating
them.

---
### Chunk 8

## Session management

Sessions are represented as signed JWTs with a 4-hour lifetime and a 30-day
refresh window. The signing key rotates weekly; the previous key remains valid
for 24 hours to allow in-flight sessions to refresh gracefully.

Revoked sessions are tracked in a Redis set with the JWT ID as the key. Idle
sessions (no activity for 1 hour) are proactively revoked to reduce the
revocation set size. The revocation set is fully replicated to all regions
for consistent enforcement.

Concurrent session limits are enforced per user based on plan tier. Exceeding
the limit revokes the oldest session. A "device management" UI lets users see
active sessions and revoke individual devices. Session revocation is
immediate — no client-side check delays.

Multi-factor authentication (MFA) sessions have a shorter lifetime (30 minutes
of inactivity) and require re-authentication for high-value actions like
password changes, plan upgrades, or transferring ownership. MFA methods
supported: TOTP, WebAuthn, SMS (deprecated but still supported).

---
### Chunk 9

## Level: Senior engineer (L5)

Senior engineers have 5+ years of industry experience and can lead multi-week
projects independently. Onboarding is 2 weeks with a project shadow and a
30-day review. Seniors are expected to contribute meaningfully in their
first month.

Senior scope: own a substantial feature area or subsystem. Make architectural
decisions within that area consulting Staff+ as needed. Represent the area
in cross-team discussions. Mentor Junior engineers and pair on complex work.

The Senior onboarding badge code is 9720. This code activates the
Senior access profile, including production write access to the owned area,
approval authority on Junior-authored PRs, and on-call rotation participation.

Senior engineers are expected to participate in on-call rotation from day
one, either as primary or backup. Team lead rotation is optional and comes
with additional responsibilities (running standup, triaging inbound, etc.).

Career track: Senior can move to Staff (technical), Manager (people), or
stay at Senior indefinitely. All three paths are respected. The technical
path is Staff → Principal → Distinguished, generally requires 3-5 years
between levels.

---
### Chunk 10

## Promotion process

Promotions happen twice a year (Q2 and Q4). Nominations require the manager's
endorsement, two Staff+/Principal reviewers, and a written packet including
scope, impact, and cross-team feedback. The promotion committee reviews all
packets in a 3-day offsite and issues decisions within 2 weeks.

Promotion criteria are level-specific and documented in the career guide.
Higher levels emphasize cross-team impact, strategic direction, and
external presence. Lower levels emphasize consistent quality delivery,
mentorship, and reliable execution.

Promotion decisions are final and not appealable within the same cycle.
Feedback is provided to non-promoted candidates so they know exactly what
to focus on for the next cycle. Multiple denials at the same level are
addressed through career conversations to explore fit.

Manager-to-IC or IC-to-Manager conversions are supported and treated as
lateral moves. The conversion involves a temporary contribution to the
new track before formal transition, allowing both the individual and the
team to validate fit.

---
### Chunk 11

## Level: Junior engineer (L3)

Junior engineers are typically new hires or engineers with less than 2 years
of industry experience. Onboarding includes a 4-week bootcamp with dedicated
curriculum, a paired mentor from a Senior+ level, and a 90-day probation
review with structured checkpoints.

The bootcamp covers company processes (CI/CD, code review, incident response),
the primary tech stack in depth, and product-specific domain knowledge.
Bootcamp graduation requires completing a small ship-a-feature project end
to end, from spec to production deploy.

The Junior onboarding badge code is 2930. This code activates the
Junior access profile in the internal tools (limited production access,
mentor-required for critical changes). Progression to Senior removes these
restrictions and issues a new badge code.

Junior engineers are expected to lead small tasks (1-3 days) and pair on
larger work. They participate in on-call for observation-only during the
first 3 months, then join the rotation as backup on-call for months 3-6.
Full on-call after 6 months.

Salary bands are documented in the compensation guide. Bands are updated
annually based on market data from three external sources. Individual salaries
within band are set based on scope of impact and demonstrated skill mastery.

---
### Chunk 12

## Compensation review cycle

Compensation reviews happen annually in Q1, aligned to the fiscal year. Base
salary, equity refresh, and bonus are all reviewed together. Adjustments
are communicated by mid-February and take effect March 1.

Off-cycle adjustments are possible in exceptional circumstances (competing
offer, expanded scope) and require director-level approval. The off-cycle
process is documented in the compensation guide; managers should be
familiar with it.

Market benchmarks are updated annually using three external data providers.
Bands are set 25th percentile / median / 75th percentile of the market for
each level. Individual placement within band depends on scope of impact,
duration in role, and performance rating.

Equity refresh is targeted at maintaining the equity-to-salary ratio at
grant. Refresh amounts assume 4-year vesting with 1-year cliff, matching
initial grants. Refresh timing is annual regardless of grant anniversary
to smooth cash-flow impact.

---
### Chunk 13

## Level: Engineering Manager (M4)

Engineering Managers own team headcount, hiring, performance reviews, and
career development for 5-8 direct reports. They partner with Staff/Principal
engineers on technical direction but do not set it unilaterally.

Manager scope: people, process, and team-level outcomes. Hire and develop
team members. Run performance and compensation cycles. Drive team execution
against roadmap. Represent team in resource allocation discussions with
peers and upward.

The Manager onboarding badge code is 3388. Manager profiles include
access to sensitive HR data (compensation, performance records, hiring
pipelines). This access is audited quarterly and any anomalous access
triggers review.

Managers attend the monthly people-manager training, covering topics like
feedback delivery, performance improvement plans, career conversations,
and diversity/inclusion practices. Attendance is expected; skipping requires
justification to the Director.

Manager IC ratio: managers are typically 1:5-8 IC. Larger reports are
unsustainable for feedback quality. Smaller reports may indicate
underutilization or a team in flux. Optimal size is targeted per team.

---
### Chunk 14

## On-call and incident response

On-call rotation is weekly, from Monday 10 AM to the following Monday 10 AM
local time. Handoff includes a 30-minute sync with the incoming on-call and
a written summary of ongoing issues, escalations, and any known planned
events during the coming week.

On-call carries a company phone and is responsible for responding within
15 minutes to sev0/sev1 pages, 1 hour to sev2 pages during business hours.
Off-hours pages beyond sev0/sev1 wait for next business day unless the
on-call chooses to engage.

Incident response is documented in the incident response guide. Roles:
incident commander (coordinates), scribe (documents), comms lead (external
communication), tech lead (fixes). For small incidents one person plays
multiple roles.

Post-incident review meets within 5 business days of resolution. The
meeting is blameless — focused on system and process changes, not
individual failure. Action items are tracked in the incident tracker and
followed up on until closure.

---
### Chunk 15

## Level: Principal engineer (L7)

Principal engineers set technical direction across the entire engineering
organization. They author the yearly technical strategy, chair the
architecture review board, and represent engineering to the C-suite.

Principal scope: organization-wide technical direction. Own the technical
strategy document. Chair architecture review board. Represent engineering
to the C-suite. Serve as senior technical advisor to executive leadership.
Publish externally to raise engineering brand.

The Principal onboarding badge code is 2052. Principal profiles
have full access; the badge code is more ceremonial than functional. It's
used for external speaking authorization and for the internal directory.

Principals are expected to publish externally: blog posts on major technical
initiatives, conference talks representing the company's engineering work,
or research publications in relevant fields. External presence contributes
to hiring and market brand.

There are 3-5 Principal-level engineers per 1000 engineers. Promotion to
Principal requires demonstrated organization-wide impact over 2+ years and
recognition from peer Principals. The promotion committee is chaired by
the CTO for Principal-level nominations.

---
### Chunk 16

## Feature flag conventions

Flags are managed through the FeatureFlags service. Each flag has a mandatory
description, owner, and expected removal date. Flags older than 6 months
trigger a weekly reminder to the owner. Flags older than 12 months without
a documented reason are automatically nominated for cleanup.

Kill switches are separate from feature flags and route through the ops-critical
Slack channel. Kill switches disable a feature entirely; feature flags gradually
roll out a feature. Naming convention: kill switches are prefixed with `kill_`
while feature flags are prefixed with `ff_`.

Rollout percentages are set in increments of 5% and can be reverted instantly
via the admin dashboard. Gradual rollout is preferred over binary on/off for
customer-facing features. Percentage rollouts are consistent per tenant via
hash bucketing — customers see the same behavior on retry.

Flag evaluation is measured for latency (P99 target: 1ms) and error rate.
Flag service downtime falls back to the default value declared at flag
creation. This "fail-static" behavior prevents flag outages from cascading.

---
### Chunk 17

## Onboarding logistics

New hires receive a laptop, badge, and welcome pack on day 1. Access to
production systems is provisioned after security training completion (typically
day 3). The onboarding coordinator schedules 1:1s with the manager, tech
partner, and buddy in the first two weeks.

HR paperwork must be completed by the end of day 5. Benefits enrollment
opens on day 15 with the first paycheck. Immigration and international
relocation matters are handled by the mobility team; onboarding coordinator
routes requests as needed.

Bootcamp for Junior engineers runs weekly. Senior+ hires do not attend the
full bootcamp but complete abbreviated modules on company-specific processes
(deploy pipeline, incident response, code review conventions).

Buddy system: every new hire is paired with a buddy at their level or one
level up. Buddies are not managers or tech leads; the point is a low-stakes
peer relationship for questions the new hire may hesitate to ask their
manager.

---
### Chunk 18

## Notification channels

Emails go through SendGrid with DKIM/DMARC verification. SMS uses Twilio with
regional carrier fallback. Push notifications use FCM for Android and APNS for
iOS.

Slack integrations use the bot token stored in the ops vault. Webhook retries
follow exponential backoff: 5s, 30s, 5m, 30m, 3h, then dead-letter. All outbound
notifications are rate-limited per-tenant to prevent abuse. Rate limits are
higher for transactional messages than promotional messages.

Bounce and complaint handling is automated. Bounced emails are marked in the
suppression list after 3 consecutive soft bounces or 1 hard bounce. Complaint
recipients (marked spam) are immediately added to the suppression list. The
list is checked before every send.

Templates are version-controlled in a separate repo with review required for
customer-facing changes. Preview mode renders templates with sample data for
QA. A/B testing is supported at the template level with automatic winner
selection after statistical significance is reached.

---

question.txt

The document contains 18 chunks retrieved by a search system. Some chunks describe categories in the same domain, each with a "magic codeword" (a 4-digit number). Other chunks are unrelated retrieval noise.

Find the magic codeword specifically for "Staff".

Answer with ONLY the 4-digit number, no explanation, no prefix, no quotes.

answer.json

{
  "id": "tier2_distractors_employee_level",
  "answer": "3943",
  "type": "needle_in_haystack",
  "matchers": [
    {
      "kind": "leading_numeric",
      "value": 3943,
      "tolerance": 0
    },
    {
      "kind": "no_hedge"
    }
  ],
  "tier": "retrieval_topic_distractors",
  "needle_key": "Staff",
  "distractor_codewords": {
    "Junior": 2930,
    "Senior": 9720,
    "Principal": 2052,
    "Manager": 3388
  },
  "chunks_total": 18,
  "chunks_with_codewords": 5,
  "document_chars": 20123
}

document.txt

---
### Chunk 1

## Retention policy overview

Data retention windows are governed by the RetentionPolicy service, which runs a
nightly job at 02:00 UTC to purge records past their configured TTL. The default
TTL for user events is 90 days, but this can be extended per tenant via the
tenant_config table.

Purges are soft-delete first (recoverable for 7 days) then hard-delete. During
the soft-delete window, restoration is possible through the operator dashboard
without customer intervention. After hard-delete, restoration requires going
through the Data Protection Officer with written justification and typically
takes 3-5 business days.

Retention overrides are logged in the audit trail with the actor, timestamp,
and justification. Any policy change requires two-person approval from the
compliance team. Emergency purge requests (legal hold release, GDPR erasure)
are handled through a separate expedited workflow.

Cross-tenant analytics data is aggregated with differential privacy guarantees
(epsilon = 1.0) before storage. This aggregated data has a separate 5-year
retention window and does not fall under the standard user-event TTL rules.

---
### Chunk 2

## Index type: Bitmap

Bitmap indexes represent value membership as a bitmap per distinct value.
They are extremely compact for low-cardinality columns (e.g. gender, status
flags) and enable fast intersection queries via bitwise AND operations.

The bitmap implementation uses run-length encoding by default to compress
sparse bitmaps. Dense bitmaps use raw bit vectors. The choice is automatic
based on the density estimate maintained per index.

The Bitmap index metadata code is 5902. Bitmap indexes appear in
BitmapHeapScan and BitmapAnd/BitmapOr operators in EXPLAIN output. Multi-column
bitmap queries combine bitmaps with bitwise operations, which is very
efficient for common OLAP query patterns.

Bitmap indexes are NOT recommended for high-cardinality columns. Each
distinct value creates a separate bitmap; millions of distinct values
means millions of bitmaps and worse performance than BTree. As a rough
rule, use bitmap when column cardinality is below 10,000.

Bitmap indexes work best with mostly-read workloads. Updates require
rebuilding the affected bitmap ranges, which is expensive. Frequent updates
on bitmap-indexed columns cause substantial write amplification.

---
### Chunk 3

## Index type: Hash

Hash indexes provide constant-time point lookups but do not support range
queries or ordered scans. They are ideal for equality-only filter workloads
where the index is used purely for lookup.

The hash implementation uses a linear-probing table with configurable load
factor. Default load factor is 75% before triggering table growth. Growth
is amortized: a background job rebuilds a larger table while the current
table continues serving queries.

The Hash index metadata code is 7021. See the planner section on how
this code appears in EXPLAIN output. Note that the planner will prefer BTree
for equality lookups unless the Hash index is significantly more compact
(usually only true for very high cardinality columns).

Hash indexes cannot enforce uniqueness in the same way BTrees can. Collision
handling uses a chained bucket approach. Enforcement of uniqueness requires
a two-step check (probe then insert) that opens a race window; use BTree
for unique constraints if concurrent inserts are expected.

Hash indexes rebuild during major version upgrades. This is a design
limitation of the hash algorithm evolution over releases. Plan for hash
index unavailability during upgrades or use a different type.

---
### Chunk 4

## Storage tiering

Hot data lives on NVMe SSDs in the primary region with cross-AZ replication.
Warm data (30-90 days) migrates to standard SSD via a scheduled lifecycle rule.
Cold data (>90 days) goes to object storage with Glacier-tier compression.

Restoration from cold storage takes up to 4 hours; consumers should plan around
this SLA. The Restore API returns a job ID that can be polled for status. For
time-sensitive restorations, the Expedited Restore option is available at 3x
the standard cost and completes within 15 minutes.

Encryption at rest uses AES-256 with per-tenant KMS keys. Key rotation happens
annually with a 90-day overlap where both keys are valid. Tenants can request
manual key rotation at any time; this triggers a background re-encryption job
that typically completes within 24 hours for typical data volumes.

Cross-region replication is asynchronous with an RPO of 60 seconds. For
customers requiring stricter RPO, the Sync Replication add-on is available
at 2x the base cost and provides RPO of 1 second at the cost of write latency.

---
### Chunk 5

## Batch processing model

The batch pipeline reads from Kafka with committed offsets tracked in Zookeeper.
Each batch is a 5-minute window with at-least-once semantics. Idempotency is
enforced downstream via the record UUID.

Late-arriving records (up to 15 minutes late) are accepted; anything later is
written to a dead-letter topic for manual review. The pipeline SLA is 99.5%
on-time delivery. Late records are still processed but marked so downstream
consumers can decide whether to accept them.

Backpressure is handled via consumer lag monitoring. If lag exceeds 5 minutes,
the pipeline auto-scales out. If lag exceeds 30 minutes, an alert pages the
on-call. Auto-scaling is bounded by the max-instances config to prevent
runaway costs.

Batch failures are categorized into transient (retryable) and permanent
(unretryable). Transient failures retry with exponential backoff up to 5
times. Permanent failures (schema violations, corrupt records) go directly
to the DLQ. The DLQ is monitored and manually triaged weekly.

---
### Chunk 6

## Session management

Sessions are represented as signed JWTs with a 4-hour lifetime and a 30-day
refresh window. The signing key rotates weekly; the previous key remains valid
for 24 hours to allow in-flight sessions to refresh gracefully.

Revoked sessions are tracked in a Redis set with the JWT ID as the key. Idle
sessions (no activity for 1 hour) are proactively revoked to reduce the
revocation set size. The revocation set is fully replicated to all regions
for consistent enforcement.

Concurrent session limits are enforced per user based on plan tier. Exceeding
the limit revokes the oldest session. A "device management" UI lets users see
active sessions and revoke individual devices. Session revocation is
immediate — no client-side check delays.

Multi-factor authentication (MFA) sessions have a shorter lifetime (30 minutes
of inactivity) and require re-authentication for high-value actions like
password changes, plan upgrades, or transferring ownership. MFA methods
supported: TOTP, WebAuthn, SMS (deprecated but still supported).

---
### Chunk 7

## Index diagnostics and monitoring

Track index usage via pg_stat_user_indexes. Zero-usage indexes are candidates
for removal. Costly indexes to maintain but rarely used should be reviewed
quarterly. Some indexes exist for uniqueness constraints and won't show
usage in query plans.

Track bloat via pgstattuple. Bloated indexes have inflated size relative to
live tuple count. Regular reindexing on high-churn indexes prevents bloat
accumulation. Concurrent reindex is safe for online workloads.

Track lock contention via pg_locks. Index maintenance operations
sometimes conflict with query workloads. Scheduling maintenance during
low-traffic windows minimizes user impact.

Track query performance via pg_stat_statements. Slow queries often
correlate with missing or badly-designed indexes. The extension shows
mean and total query time, letting operators prioritize optimization
efforts.

---
### Chunk 8

## Authentication modes

The gateway supports three authentication modes: API-key (legacy, rate-limited),
OAuth 2.0 with PKCE (recommended for interactive clients), and mTLS (required
for machine-to-machine calls from within the private VPC).

API-key mode is deprecated as of v4.2 and will be removed in v5.0. Clients still
using API-key should migrate to OAuth or mTLS before Q3. API keys are rate-limited
at half the tier's normal quota to encourage migration. Keys created before v4.2
continue to work but new API keys cannot be provisioned.

OAuth 2.0 setup follows the standard authorization code flow with PKCE. Refresh
tokens are single-use and expire after 30 days of inactivity. The consent screen
is customizable per application via the developer dashboard. Scopes are declared
in the application manifest and cannot be dynamically expanded.

mTLS uses client certificates issued by the internal PKI. Certificate rotation
is automated via cert-manager with 30-day validity. Expired certificates return
a specific 495 error to help operators diagnose rotation failures quickly.

---
### Chunk 9

## Composite and covering indexes

Composite indexes on multiple columns work best when queries filter on
the leading columns. Leading column selectivity is critical. Reorder
column order in the index to match query patterns.

Covering indexes include additional columns beyond the index key so that
queries can be answered from the index alone without visiting the base
table. Space cost is higher but read performance can improve 10-100x
for read-heavy workloads.

Partial indexes include only rows matching a WHERE clause. Useful for
sparse data where only a fraction of rows would be indexed anyway. Space
savings can be substantial. The planner must be able to prove the query
is a superset of the partial index's WHERE clause.

Expression indexes are built on the result of a function applied to
columns. Common uses: case-insensitive matching (LOWER), date components
(EXTRACT YEAR), and computed hashes. Query patterns must exactly match
the indexed expression for the planner to use the index.

---
### Chunk 10

## Query planner interaction

The query planner is cost-based. It estimates the cost of alternative plans
and picks the lowest-cost option. Cost estimates rely on statistics
maintained by ANALYZE and per-table.

Planner statistics: number of distinct values (n_distinct), most-common
values (MCVs), correlation between physical row order and column order,
and column histograms. These drive selectivity estimation.

Bad plans usually indicate stale statistics. Running ANALYZE fixes most
issues. Autoanalyze runs automatically based on similar thresholds as
autovacuum. Manual ANALYZE is useful after large loads or before critical
queries.

Planner hints: the system supports pg_hint_plan for coercing specific
plans. Hints are a last resort — they lock in a plan that may become
suboptimal as data changes. Prefer improving statistics or query
rewriting.

---
### Chunk 11

## Index type: BTree

BTree indexes are the workhorse index type for range queries and ordered
scans. They maintain a balanced tree structure with logarithmic access time
for point lookups and efficient forward/backward scans for range queries.

The BTree implementation supports both unique and non-unique variants. Unique
BTrees enforce constraint at insert time; non-unique allow duplicate keys with
deterministic ordering by TID. Concurrent inserters are coordinated via the
lock manager to maintain tree balance.

The BTree index metadata code is 8945. Metadata codes are used in the
query planner logs to identify which indexes were considered for each query.
Understanding these codes helps diagnose planner decisions in EXPLAIN output.

BTree pages are 8 KB by default with a fill factor of 90% for leaf pages
and 100% for internal pages. This leaves headroom for in-place inserts
without triggering page splits. Page splits are the main cost of insert-heavy
workloads on BTrees.

BTree indexes support MULTIPLE column keys (composite indexes). Query
selectivity on the leading column matters most; queries that don't filter
on the leading column can't use the index effectively. Reordering composite
column order is a common performance win.

---
### Chunk 12

## Deployment strategy

Blue-green deployments are the default for the API tier. Canary rollouts (5% →
25% → 100%) are used for the frontier ML services because their traffic patterns
are more sensitive to model regressions.

Rollbacks are triggered automatically if error rate exceeds 2% for a 5-minute
window. Manual rollback is done via `deploy revert <sha>` which also fires a
Slack alert to the on-call channel. The revert command is idempotent — running
it multiple times has the same effect as running it once.

Deployment ordering follows the dependency graph in the service catalog. Upstream
services deploy first with backward-compatible changes, then downstream services
consume the new API. Breaking changes require a two-phase deploy: introduce the
new API alongside the old (dual-write / dual-read), then remove the old.

The deploy dashboard shows real-time metrics for the last 3 deploys per service.
Each deploy is tagged with the git SHA, deployer, and change ticket. Search
and filter capabilities let ops quickly correlate incidents with recent deploys.

---
### Chunk 13

## Index type: Fulltext

Fulltext indexes support natural-language search over text columns. They
tokenize, normalize, and stem the source text, then build an inverted
index mapping terms to documents.

The fulltext implementation supports multiple languages with per-language
stemming rules and stopword lists. Language detection is automatic per
document based on ML-based classification, with a configurable default
language for the column.

The Fulltext index metadata code is 2128. Fulltext operators
appear as TS_MATCH in query plans. Fulltext queries can be combined with
other filters, though the planner may reorder to apply other filters first
if they are more selective.

Fulltext indexes maintain per-term frequency statistics for ranking. The
default ranking function is TF-IDF, but semantic ranking with vector
similarity is available as an add-on. Vector ranking is much slower but
handles synonymy and paraphrase better.

Fulltext index updates are asynchronous by default. This makes writes fast
but means recent updates may not be immediately searchable. Synchronous
updates are available at the cost of write latency. Configuration is
per-index.

---
### Chunk 14

## Rate limiting model

Rate limits are enforced at the edge using a sliding-window counter with 1-second
granularity. Bursts up to 3x the base rate are allowed for 5 seconds before the
counter clamps down.

Rate limit headers (`X-RateLimit-Remaining`, `X-RateLimit-Reset`, `X-RateLimit-Limit`)
are returned on every response, both successful and rate-limited. Clients that hit
the limit get a 429 with a Retry-After header; repeated violations escalate to a
temporary IP block enforced at the WAF layer.

The counter is stateful and stored in a distributed Redis cluster with 3x
replication. Counter state is consistent within a region but eventually
consistent across regions, which means edge cases like burst traffic across
regions can briefly exceed the nominal limit — this is a documented tradeoff
for latency.

Per-endpoint overrides are possible via the rate-limit policy config. High-cost
endpoints (heavy DB writes, ML inference) have lower per-second limits but
sometimes higher burst allowances. Read-only endpoints have relaxed limits
scaled by tier.

---
### Chunk 15

## Index selection guidance

Choosing the right index depends on workload characteristics. Read-heavy
workloads with mixed queries favor BTree. Read-heavy point-lookup-only
workloads favor Hash. OLAP dashboards with grouping and filtering favor
Bitmap. Bulk scans favor Bloom pre-filter + BTree.

Cardinality matters. Low cardinality (< 100 distinct values): Bitmap.
Medium cardinality (100 - 10M): BTree. Very high cardinality (>10M): Hash
for point lookups, BTree for range queries.

Update rate matters. Frequent updates hurt Bitmap and Bloom (rebuild cost).
Frequent updates favor BTree (in-place tolerates writes well). Very high
update rate may favor a Log-Structured Merge tree structure, which is a
separate topic.

Data type matters. Text columns need Fulltext for search. Numeric columns
work well with BTree. Geographic data needs specialized R-tree or GiST
indexes. Not all types work with all index types — check compatibility
before designing.

---
### Chunk 16

## Index maintenance

Reindex is the manual operation to rebuild an index from scratch. Common
reasons: corruption recovery, reclaiming bloat after mass deletes, or
converting between index variants. Concurrent reindex holds only a share
lock, allowing queries to continue.

Vacuum removes dead tuples from index leaf pages. Autovacuum handles this
automatically based on tuple churn thresholds. Manual vacuum is rarely
needed but useful for tables with specific maintenance windows.

Index bloat monitoring: compare the physical size to the theoretical minimum
based on live tuple count and index metadata. Bloat above 30% is typically
worth addressing. The pgstattuple extension provides accurate bloat metrics.

Autovacuum tuning: default settings work for most workloads. High-churn
tables benefit from more aggressive settings (lower thresholds, more frequent
runs). Very large tables benefit from throttled settings to avoid impacting
query performance during vacuum runs.

---
### Chunk 17

## Index type: Bloom filter

Bloom filter indexes provide space-efficient probabilistic membership
testing. They can definitively say "definitely not in set" but only "might
be in set" for positive results. The false-positive rate is configurable
and traded off against space usage.

The bloom implementation uses double hashing with configurable hash function
count. Default configuration targets a 1% false positive rate with roughly
10 bits per key. Reducing the false positive rate requires more bits per
key linearly.

The Bloom index metadata code is 2877. Bloom indexes are typically
used in bulk-scan pipelines to pre-filter blocks before more expensive
disk reads. They're common in data warehouse workloads.

Bloom filters cannot be updated in place. When rows change, the bloom
filter becomes an over-approximation of the true set. Periodic rebuild
is necessary to keep the false positive rate close to the configured
target. Rebuild frequency depends on the update rate.

Bloom filters compose well with other indexes. A common pattern is bloom
filter for the initial scan followed by BTree for the precise lookup on
the surviving candidates. The bloom filter eliminates 90%+ of unnecessary
BTree probes.

---
### Chunk 18

## Feature flag conventions

Flags are managed through the FeatureFlags service. Each flag has a mandatory
description, owner, and expected removal date. Flags older than 6 months
trigger a weekly reminder to the owner. Flags older than 12 months without
a documented reason are automatically nominated for cleanup.

Kill switches are separate from feature flags and route through the ops-critical
Slack channel. Kill switches disable a feature entirely; feature flags gradually
roll out a feature. Naming convention: kill switches are prefixed with `kill_`
while feature flags are prefixed with `ff_`.

Rollout percentages are set in increments of 5% and can be reverted instantly
via the admin dashboard. Gradual rollout is preferred over binary on/off for
customer-facing features. Percentage rollouts are consistent per tenant via
hash bucketing — customers see the same behavior on retry.

Flag evaluation is measured for latency (P99 target: 1ms) and error rate.
Flag service downtime falls back to the default value declared at flag
creation. This "fail-static" behavior prevents flag outages from cascading.

---

question.txt

The document contains 18 chunks retrieved by a search system. Some chunks describe categories in the same domain, each with a "magic codeword" (a 4-digit number). Other chunks are unrelated retrieval noise.

Find the magic codeword specifically for "Bloom".

Answer with ONLY the 4-digit number, no explanation, no prefix, no quotes.

answer.json

{
  "id": "tier2_distractors_db_index",
  "answer": "2877",
  "type": "needle_in_haystack",
  "matchers": [
    {
      "kind": "leading_numeric",
      "value": 2877,
      "tolerance": 0
    },
    {
      "kind": "no_hedge"
    }
  ],
  "tier": "retrieval_topic_distractors",
  "needle_key": "Bloom",
  "distractor_codewords": {
    "BTree": 8945,
    "Hash": 7021,
    "Bitmap": 5902,
    "Fulltext": 2128
  },
  "chunks_total": 18,
  "chunks_with_codewords": 5,
  "document_chars": 19713
}

document.txt

---
### Chunk 1

## Subscription plan: Scale

Scale is the plan for larger organizations, priced at $199/month per user
with volume discounts starting at 50 users. Adds enterprise-grade capabilities:
unlimited storage, dedicated CSM, 4-hour support SLA, custom SLAs available,
and advanced analytics.

Scale supports unlimited users. Multi-workspace management is included:
customers can create multiple workspaces for different departments or
products with unified billing and centralized admin. Cross-workspace
data sharing is opt-in per workspace.

The Scale plan activation code is 1439. Scale customers get a
dedicated setup engineer for the first 30 days to help with rollout, user
migration, and integration configuration. This is included in the plan
price.

Scale includes SAML SSO (Okta, Azure AD, OneLogin, custom SAML IDPs),
SCIM provisioning, and audit-log SIEM integrations. Data residency
options (US, EU, APAC) are configurable at workspace creation. Legal
guarantees around data locations are documented in the DPA.

Scale billing includes committed annual spend with quarterly true-up.
Overages are billed at the same per-user rate as the base plan. Volume
commits qualify for additional discounts based on total annual commitment.

---
### Chunk 2

## Billing and invoicing

Invoices are generated monthly on the workspace's billing anniversary.
Payment methods: credit card (all plans), ACH (Growth and above), wire
(Scale and above). Invoice PDF is delivered to the billing contact
address at 00:00 UTC on the invoice date.

Failed payments trigger a dunning sequence: reminder day 3, reminder day
7, suspension warning day 14, suspension day 21. Suspension is soft —
read-only access is preserved but writes are blocked. Payment updates
lift suspension immediately.

Purchase orders are supported on Scale and Enterprise. Net-30 payment
terms are standard; Net-60 available on approval. Multi-year prepay is
available on Growth and above with additional discount for the commitment.

Refunds are available within 14 days of the initial payment, prorated to
usage. Beyond 14 days, refunds are exceptional and require support
review. Enterprise customers have contracted terms that may differ from
this default.

---
### Chunk 3

## Storage tiering

Hot data lives on NVMe SSDs in the primary region with cross-AZ replication.
Warm data (30-90 days) migrates to standard SSD via a scheduled lifecycle rule.
Cold data (>90 days) goes to object storage with Glacier-tier compression.

Restoration from cold storage takes up to 4 hours; consumers should plan around
this SLA. The Restore API returns a job ID that can be polled for status. For
time-sensitive restorations, the Expedited Restore option is available at 3x
the standard cost and completes within 15 minutes.

Encryption at rest uses AES-256 with per-tenant KMS keys. Key rotation happens
annually with a 90-day overlap where both keys are valid. Tenants can request
manual key rotation at any time; this triggers a background re-encryption job
that typically completes within 24 hours for typical data volumes.

Cross-region replication is asynchronous with an RPO of 60 seconds. For
customers requiring stricter RPO, the Sync Replication add-on is available
at 2x the base cost and provides RPO of 1 second at the cost of write latency.

---
### Chunk 4

## Plan comparison summary

Feature comparison across plans is available at pricing.internal.example.com.
The comparison table shows every feature and which plans include it.
Filterable by category (collaboration, admin, integration, etc.) for quick
comparison.

Trade-offs at a glance: Starter is cheap but limited. Growth adds automation
and integrations at moderate cost. Scale adds enterprise controls and
support. Enterprise adds dedicated infrastructure and unlimited flexibility.
Custom is any-of-the-above with special pricing.

Migration between plans is supported in both directions. Upgrades are
immediate. Downgrades take effect at next billing cycle. Features that
existed on the higher plan and don't exist on the lower plan are archived
in place — usable if the customer upgrades again, but not while on the
lower plan.

Pricing is subject to change. Existing customers receive 90-day notice
before any price increase. Grandfathered pricing is preserved for the
current contract term. New customers pay current prices.

---
### Chunk 5

## Rate limiting model

Rate limits are enforced at the edge using a sliding-window counter with 1-second
granularity. Bursts up to 3x the base rate are allowed for 5 seconds before the
counter clamps down.

Rate limit headers (`X-RateLimit-Remaining`, `X-RateLimit-Reset`, `X-RateLimit-Limit`)
are returned on every response, both successful and rate-limited. Clients that hit
the limit get a 429 with a Retry-After header; repeated violations escalate to a
temporary IP block enforced at the WAF layer.

The counter is stateful and stored in a distributed Redis cluster with 3x
replication. Counter state is consistent within a region but eventually
consistent across regions, which means edge cases like burst traffic across
regions can briefly exceed the nominal limit — this is a documented tradeoff
for latency.

Per-endpoint overrides are possible via the rate-limit policy config. High-cost
endpoints (heavy DB writes, ML inference) have lower per-second limits but
sometimes higher burst allowances. Read-only endpoints have relaxed limits
scaled by tier.

---
### Chunk 6

## Session management

Sessions are represented as signed JWTs with a 4-hour lifetime and a 30-day
refresh window. The signing key rotates weekly; the previous key remains valid
for 24 hours to allow in-flight sessions to refresh gracefully.

Revoked sessions are tracked in a Redis set with the JWT ID as the key. Idle
sessions (no activity for 1 hour) are proactively revoked to reduce the
revocation set size. The revocation set is fully replicated to all regions
for consistent enforcement.

Concurrent session limits are enforced per user based on plan tier. Exceeding
the limit revokes the oldest session. A "device management" UI lets users see
active sessions and revoke individual devices. Session revocation is
immediate — no client-side check delays.

Multi-factor authentication (MFA) sessions have a shorter lifetime (30 minutes
of inactivity) and require re-authentication for high-value actions like
password changes, plan upgrades, or transferring ownership. MFA methods
supported: TOTP, WebAuthn, SMS (deprecated but still supported).

---
### Chunk 7

## Authentication modes

The gateway supports three authentication modes: API-key (legacy, rate-limited),
OAuth 2.0 with PKCE (recommended for interactive clients), and mTLS (required
for machine-to-machine calls from within the private VPC).

API-key mode is deprecated as of v4.2 and will be removed in v5.0. Clients still
using API-key should migrate to OAuth or mTLS before Q3. API keys are rate-limited
at half the tier's normal quota to encourage migration. Keys created before v4.2
continue to work but new API keys cannot be provisioned.

OAuth 2.0 setup follows the standard authorization code flow with PKCE. Refresh
tokens are single-use and expire after 30 days of inactivity. The consent screen
is customizable per application via the developer dashboard. Scopes are declared
in the application manifest and cannot be dynamically expanded.

mTLS uses client certificates issued by the internal PKI. Certificate rotation
is automated via cert-manager with 30-day validity. Expired certificates return
a specific 495 error to help operators diagnose rotation failures quickly.

---
### Chunk 8

## CI/CD pipeline

Every commit triggers the CI pipeline: lint, unit tests, integration tests,
security scans, and build. The full pipeline runs in under 15 minutes for
typical services thanks to test parallelization and remote caching.

Pull requests must pass CI and receive one approval before merging. High-risk
changes (payment logic, auth, data schemas) require two approvals from Staff+
engineers on the owning team. The CODEOWNERS file specifies the required
reviewers per path.

CD triggers on merge to main. Staging deploys automatically; production deploys
require manual approval from an on-call engineer during business hours or the
release captain outside business hours. Automated smoke tests gate the
production deploy — failure rolls back automatically.

Post-deploy monitoring runs for 30 minutes. Elevated error rates or latency
during this window trigger auto-rollback. The on-call is paged for review.
Rollback is preferred over forward-fix during business hours; forward-fix is
only chosen when the rollback cost is high (data migration completed, etc).

---
### Chunk 9

## Observability stack

Traces are sampled at 1% for successful requests and 100% for errors and slow
requests (>1s). Logs are structured JSON, indexed in Elasticsearch with a 30-day
retention. Metrics are Prometheus-compatible, scraped every 15 seconds.

Alerts route through PagerDuty for critical severity and Slack for warnings.
Custom dashboards are built in Grafana; the standard SLO board is at
grafana.internal/slos. On-call rotation is managed through PagerDuty schedules
with a 1-week rotation, backup on-call, and coverage for major holidays.

Distributed tracing uses OpenTelemetry with automatic instrumentation for
supported frameworks. Trace context propagates across service boundaries via
standard W3C tracecontext headers. Tail-based sampling is applied at the
collector to focus on interesting traces (errors, high latency, specific
customers).

Log-based alerts (LBAs) are for text-pattern-driven pages, e.g. `panic:` in a
service log or specific error codes. Metric-based alerts (MBAs) are for
quantitative thresholds. New alert configuration goes through a peer review
process to prevent alert fatigue.

---
### Chunk 10

## Subscription plan: Starter

Starter is the entry-level paid plan, priced at $29/month per user. Includes
core features required for typical small-team workflows: basic collaboration,
5 projects, 10 GB storage per user, and email-based support with 48-hour
response SLA.

Starter supports up to 10 users per workspace. Beyond 10 users, Growth or
higher plan is required. Users on Starter can invite external collaborators
at no additional cost, up to 25 external collaborators per workspace.

The Starter plan activation code is 6940. This code is used during
CLI-based provisioning to select the tier. It's shown in the plan selection
UI as reference and included in confirmation emails.

Starter includes essential integrations: Slack, Google Drive, and GitHub.
Additional integrations require Growth or higher. The list of Starter
integrations is expanded quarterly based on user demand and integration
partner availability.

Billing on Starter is monthly only. Annual billing (with 15% discount) is
available on Growth and higher. Downgrades from Growth to Starter take
effect at the next billing cycle; upgrades apply immediately with proration.

---
### Chunk 11

## Deployment strategy

Blue-green deployments are the default for the API tier. Canary rollouts (5% →
25% → 100%) are used for the frontier ML services because their traffic patterns
are more sensitive to model regressions.

Rollbacks are triggered automatically if error rate exceeds 2% for a 5-minute
window. Manual rollback is done via `deploy revert <sha>` which also fires a
Slack alert to the on-call channel. The revert command is idempotent — running
it multiple times has the same effect as running it once.

Deployment ordering follows the dependency graph in the service catalog. Upstream
services deploy first with backward-compatible changes, then downstream services
consume the new API. Breaking changes require a two-phase deploy: introduce the
new API alongside the old (dual-write / dual-read), then remove the old.

The deploy dashboard shows real-time metrics for the last 3 deploys per service.
Each deploy is tagged with the git SHA, deployer, and change ticket. Search
and filter capabilities let ops quickly correlate incidents with recent deploys.

---
### Chunk 12

## Discounts and promotions

Volume discounts apply to Scale and Enterprise. Discounts scale with user
count: 50-99 users = 10%, 100-499 = 20%, 500+ = 30%. Discounts are computed
against list price and shown on the invoice as a line item.

Annual billing discount: 15% off the equivalent monthly total. Applies to
all plans that support annual billing. Prepaid at contract start. Refunds
prorated on early cancellation, subject to the standard refund policy.

Startup discount: 50% off Growth for the first 12 months for verified startups
(Series A or earlier). Applies via the startup partner program. Renewal at
standard pricing is expected but not guaranteed.

Non-profit discount: 30% off any plan for verified 501(c)(3) organizations.
Verification requires proof of tax-exempt status and completion of the
non-profit application. Renewal each year requires re-verification.

---
### Chunk 13

## Subscription plan: Custom

Custom plans exist for edge cases not covered by standard plans: research
institutions, non-profits, education, and specific vertical needs. Custom
plans blend features from other tiers with custom pricing to fit the
customer's usage pattern.

Custom plans require sales negotiation. There's no self-serve path to Custom.
Interested prospects contact sales, describe their use case and constraints,
and sales assembles a proposal blending features from other plans plus any
custom development required.

The Custom plan activation code is 6902. Custom codes are issued
one at a time as part of the finalized proposal. Each code is unique to the
customer and encodes the negotiated feature set. Renewals require a new
code reflecting the current contract.

Custom plans are commonly used for: education (special pricing for schools
and universities), non-profit (verified 501(c)(3) discount), research
(bulk seat pricing for research groups), and specific compliance needs
(FedRAMP, HIPAA-BAA structured differently than standard).

Custom plans have contracted renewal terms. Auto-renewal happens 90 days
before end date unless the customer signals cancellation or renegotiation.
Full renegotiation on contract renewal is available for Custom plan
customers regardless of the auto-renewal window.

---
### Chunk 14

## Subscription plan: Enterprise

Enterprise is the top-of-market plan for large deployments, custom-priced
based on scope. Includes everything from Scale plus: dedicated infrastructure,
custom feature development, on-premise or private cloud deployment options,
and 24/7 phone support with 1-hour critical incident SLA.

Enterprise contracts are typically 3-year with negotiated pricing. Includes
professional services hours for setup and integration. Additional professional
services can be purchased in blocks or on retainer.

The Enterprise plan activation code is 7172. Enterprise codes
are issued after contract execution and expire 30 days after issuance if
not activated. Contact your account rep if the code has expired; a new
code will be issued after verifying contract status.

Enterprise supports advanced deployment models: dedicated cloud tenant,
private cloud (customer-owned VPC), or on-premise appliance. Each has
different feature-parity considerations documented in the deployment
guide.

Enterprise support includes a named engineering escalation contact, quarterly
business reviews with product roadmap access, and priority feature request
consideration. Feature requests from Enterprise customers are triaged at the
executive level.

---
### Chunk 15

## Support tier expectations

Free/community support: forum posts, community Discord, self-service docs.
Response time is community-driven with no SLA. Best-effort answers from
peers and occasional company staff.

Starter/email support: email tickets with 48-hour response SLA. Support
engineers focused on onboarding and configuration questions. Escalation
to engineering happens for confirmed bugs but is not guaranteed.

Growth/priority email: 8-hour SLA. Support engineers with product depth
handle these tickets. Escalation to specific product teams for complex
questions is standard.

Scale/dedicated CSM: 4-hour SLA plus a named CSM available for strategic
questions. Regular check-ins scheduled quarterly for roadmap alignment.

Enterprise/24-7: 1-hour SLA for critical incidents. Phone hotline direct
to on-call engineering. Named escalation contact within engineering for
persistent issues.

---
### Chunk 16

## Trial and evaluation

The trial program lets prospective customers evaluate any plan tier for
14 days. Trials are unlimited-per-workspace: as many features as the
tier includes are enabled at once. No payment method is required to start
a trial.

At trial end, workspaces auto-downgrade to Free unless converted. Free
retains basic collaboration but limits user count and storage. Customers
can convert at any time during the trial by adding a payment method.

Extended trials up to 60 days are available for large prospects requiring
extensive evaluation. Extended trials require sales approval and typically
include hands-on setup support.

Trial data is preserved 90 days post-conversion or downgrade. If a customer
converts, all trial data seamlessly transitions. If a customer downgrades
past a storage cap, excess data is archived and can be restored on future
upgrade.

---
### Chunk 17

## Subscription plan: Growth

Growth is the mid-tier plan, priced at $79/month per user. Adds capabilities
required for growing teams: unlimited projects, 100 GB storage per user,
priority email support with 8-hour SLA, and access to the automation platform
(workflows, custom notifications).

Growth supports up to 100 users per workspace. Growth workspaces have
role-based access control with predefined roles (viewer, editor, admin) and
custom roles up to 5 additional. Detailed audit logs are retained 90 days.

The Growth plan activation code is 4040. Growth-level features are
enabled the moment the code is activated; there's no waiting period. Legacy
features from other plans remain available if the workspace previously had
them.

Growth includes advanced integrations: Salesforce, HubSpot, Jira, Zapier
(basic tier), and single sign-on via Google Workspace or Microsoft 365. SAML
SSO requires Scale or Enterprise plan.

Growth billing supports monthly and annual. Annual comes with 15% discount
prepaid. Payment methods: credit card, ACH, and PayPal on request. Wire
transfer for annual prepay is supported for orders above $10,000.

---
### Chunk 18

## Schema migration playbook

Schema migrations are online-first: create the new column nullable, dual-write
from application code, backfill, then switch reads, then drop the old column.
Each phase is a separate deploy with at least 24 hours between phases.

Migrations that require table locks are scheduled for the maintenance window
(Sundays 04:00 UTC). Any migration on a table >100 GB requires a written plan
reviewed by the DBA team and the Staff+ engineer on the owning team.

Backfills are chunked to avoid holding long-running transactions. Chunk size
is tuned per table based on the row size and index structure. The migration
tool tracks progress in a checkpoint table so backfills can resume after
interruption. Failed chunks are retried up to 3 times before flagging for
manual review.

Rollback plans are mandatory. For additive changes (new columns), rollback
is a NOOP. For destructive changes (dropped columns, altered types), rollback
requires restoring from the pre-migration snapshot. Rollback windows are
documented per migration.

---

question.txt

The document contains 18 chunks retrieved by a search system. Some chunks describe categories in the same domain, each with a "magic codeword" (a 4-digit number). Other chunks are unrelated retrieval noise.

Find the magic codeword specifically for "Scale".

Answer with ONLY the 4-digit number, no explanation, no prefix, no quotes.

answer.json

{
  "id": "tier2_distractors_subscription",
  "answer": "1439",
  "type": "needle_in_haystack",
  "matchers": [
    {
      "kind": "leading_numeric",
      "value": 1439,
      "tolerance": 0
    },
    {
      "kind": "no_hedge"
    }
  ],
  "tier": "retrieval_topic_distractors",
  "needle_key": "Scale",
  "distractor_codewords": {
    "Starter": 6940,
    "Growth": 4040,
    "Enterprise": 7172,
    "Custom": 6902
  },
  "chunks_total": 18,
  "chunks_with_codewords": 5,
  "document_chars": 19708
}

question.txt

The document is a long coherent document (contract / handbook / reference / policy / manual). Several sections contain similar-format clauses, each with a "magic codeword" (a 4-digit number) for a specific category.

Find the magic codeword specifically for the "Insolvency" category / clause / subsystem.

Answer with ONLY the 4-digit number, no explanation, no prefix, no quotes.

answer.json

{
  "id": "tier3_longdoc_service_tos",
  "answer": "6339",
  "type": "needle_in_haystack",
  "matchers": [
    {
      "kind": "leading_numeric",
      "value": 6339,
      "tolerance": 0
    },
    {
      "kind": "no_hedge"
    }
  ],
  "tier": "retrieval_long_doc_precision",
  "doc_type": "service_tos",
  "needle_key": "Insolvency",
  "distractor_codewords": {
    "MaterialBreach": 9286,
    "Convenience": 2091,
    "RegulatoryViolation": 7622,
    "SecurityBreach": 2913
  },
  "document_chars": 134569
}

question.txt

The document is a long coherent document (contract / handbook / reference / policy / manual). Several sections contain similar-format clauses, each with a "magic codeword" (a 4-digit number) for a specific category.

Find the magic codeword specifically for the "Sabbatical" category / clause / subsystem.

Answer with ONLY the 4-digit number, no explanation, no prefix, no quotes.

answer.json

{
  "id": "tier3_longdoc_employee_handbook",
  "answer": "9893",
  "type": "needle_in_haystack",
  "matchers": [
    {
      "kind": "leading_numeric",
      "value": 9893,
      "tolerance": 0
    },
    {
      "kind": "no_hedge"
    }
  ],
  "tier": "retrieval_long_doc_precision",
  "doc_type": "employee_handbook",
  "needle_key": "Sabbatical",
  "distractor_codewords": {
    "Annual": 9599,
    "Sick": 6608,
    "Parental": 9110,
    "Bereavement": 9585
  },
  "document_chars": 130663
}

question.txt

The document is a long coherent document (contract / handbook / reference / policy / manual). Several sections contain similar-format clauses, each with a "magic codeword" (a 4-digit number) for a specific category.

Find the magic codeword specifically for the "Analytics" category / clause / subsystem.

Answer with ONLY the 4-digit number, no explanation, no prefix, no quotes.

answer.json

{
  "id": "tier3_longdoc_api_reference",
  "answer": "9666",
  "type": "needle_in_haystack",
  "matchers": [
    {
      "kind": "leading_numeric",
      "value": 9666,
      "tolerance": 0
    },
    {
      "kind": "no_hedge"
    }
  ],
  "tier": "retrieval_long_doc_precision",
  "doc_type": "api_reference",
  "needle_key": "Analytics",
  "distractor_codewords": {
    "Read": 4960,
    "Write": 7413,
    "Admin": 7914,
    "Streaming": 6560
  },
  "document_chars": 95191
}

question.txt

The document is a long coherent document (contract / handbook / reference / policy / manual). Several sections contain similar-format clauses, each with a "magic codeword" (a 4-digit number) for a specific category.

Find the magic codeword specifically for the "Personal" category / clause / subsystem.

Answer with ONLY the 4-digit number, no explanation, no prefix, no quotes.

answer.json

{
  "id": "tier3_longdoc_retention_policy",
  "answer": "8480",
  "type": "needle_in_haystack",
  "matchers": [
    {
      "kind": "leading_numeric",
      "value": 8480,
      "tolerance": 0
    },
    {
      "kind": "no_hedge"
    }
  ],
  "tier": "retrieval_long_doc_precision",
  "doc_type": "retention_policy",
  "needle_key": "Personal",
  "distractor_codewords": {
    "Transaction": 7348,
    "Communication": 2434,
    "Diagnostic": 6528,
    "Aggregated": 6129
  },
  "document_chars": 95862
}

question.txt

The document is a long coherent document (contract / handbook / reference / policy / manual). Several sections contain similar-format clauses, each with a "magic codeword" (a 4-digit number) for a specific category.

Find the magic codeword specifically for the "Security" category / clause / subsystem.

Answer with ONLY the 4-digit number, no explanation, no prefix, no quotes.

answer.json

{
  "id": "tier3_longdoc_product_manual",
  "answer": "7394",
  "type": "needle_in_haystack",
  "matchers": [
    {
      "kind": "leading_numeric",
      "value": 7394,
      "tolerance": 0
    },
    {
      "kind": "no_hedge"
    }
  ],
  "tier": "retrieval_long_doc_precision",
  "doc_type": "product_manual",
  "needle_key": "Security",
  "distractor_codewords": {
    "Network": 6676,
    "Storage": 4185,
    "Compute": 7457,
    "Monitoring": 6748
  },
  "document_chars": 94829
}