Skip to content

Multi-Agent Research Assistant with LangGraph: Planner, Workers, and Critic

A production-minded AWS blueprint for a LangGraph research assistant that delegates bounded research, checks evidence, and returns traceable answers.

Ekanatha Reddy Urivakili19 min3,648 words

1. Context and scale

A research assistant is not just a chat endpoint with a search tool. It must turn an open-ended question into bounded work, gather evidence from multiple sources, detect gaps or contradictions, and return an answer whose claims can be inspected. This blueprint uses LangGraph to coordinate a planner, parallel research workers, a critic, and a synthesizer. The workflow is asynchronous: the API accepts a request, places a job on a queue, and lets the user poll for progress and the final report.

The design targets a medium-sized product: a handful of services, one primary cloud, one or two engineering teams, and enough traffic to justify a queue and a multi-AZ database—but not a fleet of independently deployed agents. The graph is application logic inside a worker service, not a collection of autonomous services.

Assumptions — planning inputs, not forecasts:

DimensionAssumption
Users20,000 daily active users; within a 10,000–100,000 daily-user product
API traffic1 request/second average, 8 requests/second peak; research jobs are a subset
Research jobs1,500 jobs/day; up to 4 parallel worker tasks per job
Data volume500 GB of source documents and extracted text; growth is mostly append-only
Team5–7 engineers across product and platform responsibilities
Monthly budget$4,500–$14,000, including an assumption-based model and search allowance

Treat these values as capacity-planning inputs. Actual model, search, and storage costs depend on prompt size, model selection, retrieval volume, cache hit rate, and provider pricing. Put hard limits on those variables before optimizing infrastructure.

The application accepts a question, optionally scopes it to an internal corpus, and returns a cited report. The internal corpus is curated and indexed ahead of time; workers can also query an approved external search provider. This is a research assistant, not an autonomous browser with permission to act on arbitrary websites or make changes to business systems.

2. Requirements

Functional requirements describe user-visible behavior and boundaries. Non-functional requirements turn those boundaries into measurable operating goals.

Functional requirementDesign implication
Accept a question and return a job identifierAPI validates, persists, and enqueues; it does not hold a request open for the entire research run
Plan a bounded set of research tasksPlanner emits structured tasks with a maximum count and explicit query scope
Research tasks concurrentlyWorkers use bounded fan-out, per-provider timeouts, and a shared job deadline
Retrieve internal and approved external sourcesRetrieval adapters return normalized evidence with canonical URLs and timestamps
Critique evidence before synthesisCritic checks coverage, source quality, contradictions, and unsupported claims
Return citations and job progressPersist source metadata and job state; expose polling endpoints
Support user cancellation and retentionCheck cancellation between graph steps; apply tenant-aware retention policies
Non-functional requirementInitial target / constraint
Availability99.9% monthly availability for job submission and status APIs
LatencyAPI acknowledgement under 500 ms at p95; report completion is asynchronous and separately measured
ReliabilityAt-least-once queue delivery with idempotent job processing
SafetyNo arbitrary tool execution; worker tools are allowlisted, read-only, and time-bounded
QualityEvery factual answer claim should have one or more evidence references, or be identified as uncertain
IsolationTenant-scoped authorization and data access; no cross-tenant retrieval
Cost controlLimits on task count, search calls, tokens, retries, and total job duration
OperabilityTrace graph steps and provider calls without logging prompts, answers, or document contents by default

A critic can identify weak evidence, but it cannot prove that a claim is true. The system’s quality contract is therefore narrower: make evidence provenance visible, reject unsupported statements where possible, and return uncertainty when available sources do not support a conclusion.

3. High-level design

AWS is the chosen cloud. The application has two deployable workloads: an API and a queue-driven worker. Both are containerized on Amazon ECS with AWS Fargate. The API handles authentication, validation, job creation, and status reads. The worker runs LangGraph, calls Amazon Bedrock for language-model inference, retrieves internal evidence, and uses an approved external search adapter.

flowchart LR
    User["User"] --> Cognito["Amazon Cognito"]
    Cognito --> ALB["Application Load Balancer"]
    ALB --> API["ECS Fargate API"]
    API --> Queue["Amazon SQS"]
    API --> DB["Amazon Aurora PostgreSQL"]
    Queue --> Worker["ECS Fargate LangGraph worker"]
    Worker --> DB
    Worker --> Bedrock["Amazon Bedrock"]
    Worker --> Search["Approved search provider"]
    Worker --> Index["Amazon OpenSearch Service"]
    Index --> S3["Amazon S3 corpus"]
    Worker --> Cache["Amazon ElastiCache Redis"]
    Worker --> Telemetry["CloudWatch and OpenTelemetry"]

The API creates durable work; the worker owns graph execution and evidence gathering.

The index contains searchable representations of approved internal material, with the original objects in S3. Aurora PostgreSQL stores jobs, source metadata, graph checkpoints, and results. OpenSearch provides lexical and vector retrieval; do not introduce a separate vector database until retrieval measurements show a need. Redis is an optional short-lived cache for repeated retrieval or provider responses, not the source of truth.

A job progresses through explicit states such as queued, running, completed, failed, and cancelled. A queue message contains an opaque job identifier, not the full question or sensitive evidence. The worker loads authorized job state from the database, executes the graph, and writes durable progress and results. API clients poll the status endpoint; server-sent events can be added later if product requirements justify maintaining connections.

4. Cloud service mapping

The AWS choices favor managed services and a small operational surface. Equivalents are approximate: feature sets, networking, identity models, and regional availability differ, so a migration is not a name-for-name replacement.

ComponentAWS choiceAzure equivalentGCP equivalent
Containerized API and workerAmazon ECS on AWS FargateAzure Container AppsCloud Run
Ingress and TLSApplication Load Balancer, ACMApplication Gateway, Key Vault certificatesCloud Load Balancing, Certificate Manager
User identityAmazon CognitoMicrosoft Entra External IDIdentity Platform
Job queue and dead-letter queueAmazon SQSAzure Service Bus queuesPub/Sub
Transactional data and checkpointsAmazon Aurora PostgreSQL, Multi-AZAzure Database for PostgreSQLCloud SQL for PostgreSQL
Internal search indexAmazon OpenSearch ServiceAzure AI SearchVertex AI Search or managed Elasticsearch-compatible service
Source objectsAmazon S3Azure Blob StorageCloud Storage
Model inferenceAmazon BedrockAzure AI Foundry modelsVertex AI
Optional cacheAmazon ElastiCache for RedisAzure Managed RedisMemorystore
Secrets and encryption keysAWS Secrets Manager, AWS KMSKey VaultSecret Manager, Cloud KMS
Logs and service metricsAmazon CloudWatchAzure MonitorCloud Monitoring and Cloud Logging
Distributed tracingOpenTelemetry Collector and an OTLP backendAzure Monitor OpenTelemetryGoogle Cloud Trace

Prefer the managed PostgreSQL service that your team can operate well. LangGraph checkpoint persistence should use the same operational discipline as application data: backups, connection limits, schema migration controls, and restore tests. The AWS service choice does not remove those responsibilities.

5. Data design

Keep job lifecycle, graph execution, and evidence provenance separate enough to reason about, but avoid turning every graph node into a database table. The graph checkpoint is execution state; the application tables are the durable product contract. Do not expose checkpoint internals as an API response.

erDiagram
    TENANT ||--o{ RESEARCH_JOB : owns
    RESEARCH_JOB ||--o{ RESEARCH_RUN : has
    RESEARCH_RUN ||--o{ RESEARCH_FINDING : records
    RESEARCH_FINDING ||--o{ CITATION : cites
    RESEARCH_JOB ||--o{ JOB_EVENT : emits
    RESEARCH_JOB {
        uuid id PK
        uuid tenant_id FK
        text status
        text question
        timestamptz created_at
    }
    RESEARCH_RUN {
        uuid id PK
        uuid job_id FK
        int attempt
        text graph_version
        timestamptz started_at
    }
    RESEARCH_FINDING {
        uuid id PK
        uuid run_id FK
        text task_id
        text summary
    }
    CITATION {
        uuid id PK
        uuid finding_id FK
        text canonical_url
        text title
    }
    JOB_EVENT {
        bigserial id PK
        uuid job_id FK
        text event_type
        timestamptz created_at
    }

The application data model records ownership, attempts, findings, citations, and auditable lifecycle events.

An initial schema can use PostgreSQL JSONB for evolving report structures while keeping query-critical fields relational. Apply tenant filters in the data-access layer and reinforce them with database row-level security where the team can test and maintain the policies. Checkpoints are stored by the LangGraph Postgres saver; they are intentionally not duplicated in this schema.

CREATE TABLE tenant (
    id uuid PRIMARY KEY,
    name text NOT NULL,
    created_at timestamptz NOT NULL DEFAULT now()
);

CREATE TABLE research_job (
    id uuid PRIMARY KEY,
    tenant_id uuid NOT NULL REFERENCES tenant(id),
    status text NOT NULL CHECK (
        status IN ('queued', 'running', 'completed', 'failed', 'cancelled')
    ),
    question text NOT NULL,
    result jsonb,
    error_code text,
    created_at timestamptz NOT NULL DEFAULT now(),
    completed_at timestamptz
);
CREATE INDEX research_job_tenant_created
    ON research_job (tenant_id, created_at DESC);

CREATE TABLE research_run (
    id uuid PRIMARY KEY,
    job_id uuid NOT NULL REFERENCES research_job(id),
    attempt integer NOT NULL CHECK (attempt > 0),
    graph_version text NOT NULL,
    started_at timestamptz NOT NULL DEFAULT now(),
    finished_at timestamptz,
    UNIQUE (job_id, attempt)
);

CREATE TABLE research_finding (
    id uuid PRIMARY KEY,
    run_id uuid NOT NULL REFERENCES research_run(id),
    task_id text NOT NULL,
    summary text NOT NULL,
    created_at timestamptz NOT NULL DEFAULT now()
);

CREATE TABLE citation (
    id uuid PRIMARY KEY,
    finding_id uuid NOT NULL REFERENCES research_finding(id),
    canonical_url text NOT NULL,
    title text,
    published_at timestamptz,
    retrieved_at timestamptz NOT NULL DEFAULT now(),
    content_hash text
);

CREATE TABLE job_event (
    id bigserial PRIMARY KEY,
    job_id uuid NOT NULL REFERENCES research_job(id),
    event_type text NOT NULL,
    details jsonb NOT NULL DEFAULT '{}'::jsonb,
    created_at timestamptz NOT NULL DEFAULT now()
);

Partitioning is not an initial requirement at this scale. Start with indexed relational tables and a retention policy. If job_event grows enough to affect maintenance or query latency, partition it by creation time and archive old events. Keep document chunks and embeddings in the search index; keep source object versions and ingestion manifests in S3 so the index can be rebuilt.

6. Low-level design

Model the graph as a bounded state machine. Each node should have one responsibility and a typed input/output contract. The planner proposes no more than a small configured number of tasks; workers return evidence-backed findings; the critic decides whether coverage is sufficient; and the synthesizer produces the final response. A revision returns to planning only while the run budget permits.

classDiagram
    class ResearchState {
        +question: str
        +tasks: list
        +findings: list
        +critique: str
        +iteration: int
        +report: dict
    }
    class Planner {
        +create_tasks(state) dict
    }
    class Worker {
        +research(task) dict
    }
    class Critic {
        +review(state) dict
    }
    class Synthesizer {
        +compose(state) dict
    }
    class SearchPort {
        +search(query) list
    }
    class RetrievalPort {
        +retrieve(query, tenant_id) list
    }
    Planner --> ResearchState
    Worker --> SearchPort
    Worker --> RetrievalPort
    Critic --> ResearchState
    Synthesizer --> ResearchState

Graph nodes depend on explicit retrieval and search ports rather than owning provider-specific behavior.

A minimal graph illustrates the LangGraph state and execution model without requiring a model account. It is runnable with pip install langgraph; the deterministic nodes make it useful as a local orchestration smoke test, not as a research implementation.

from typing import TypedDict
from langgraph.graph import END, START, StateGraph

class State(TypedDict):
    question: str
    plan: str
    answer: str

def planner(state: State) -> dict:
    return {"plan": f"Find evidence relevant to: {state['question']}"}

def worker(state: State) -> dict:
    return {"answer": f"Research task completed: {state['plan']}"}

builder = StateGraph(State)
builder.add_node("planner", planner)
builder.add_node("worker", worker)
builder.add_edge(START, "planner")
builder.add_edge("planner", "worker")
builder.add_edge("worker", END)

app = builder.compile()
print(app.invoke({"question": "How does a queue improve resilience?"}))

Notice that graph state is explicit and the compiled graph can be tested without cloud services.

The realistic pattern below uses parallel Send work, an injected search adapter, a bounded critic loop, and Bedrock-backed chat models. Install langgraph, langchain-aws, and boto3; configure AWS credentials and BEDROCK_MODEL_ID. Supply a real, approved asynchronous search adapter in production. The placeholder adapter returns no web results, so this example runs the graph but does not perform external research.

import operator
import os
from typing import Annotated, TypedDict

from langchain_aws import ChatBedrockConverse
from langgraph.graph import END, START, StateGraph
from langgraph.types import Send

class State(TypedDict, total=False):
    question: str
    tasks: list[str]
    findings: Annotated[list[str], operator.add]
    feedback: str
    iteration: int
    report: str

model = ChatBedrockConverse(
    model=os.environ["BEDROCK_MODEL_ID"],
    temperature=0,
)

async def search(query: str) -> list[str]:
    # Replace with an authenticated, rate-limited search-provider adapter.
    return []

async def plan(state: State) -> dict:
    prompt = (
        "Create at most three short, distinct research queries. "
        "Return one query per line. Question: "
        f"{state['question']}\nFeedback: {state.get('feedback', 'none')}"
    )
    response = await model.ainvoke(prompt)
    tasks = [line.strip("- ").strip() for line in response.content.splitlines()
             if line.strip()][:3]
    return {"tasks": tasks, "iteration": state.get("iteration", 0) + 1}

def dispatch(state: State):
    return [Send("worker", {"question": state["question"], "task": task})
            for task in state.get("tasks", [])]

async def worker(state: State) -> dict:
    evidence = await search(state["task"])
    prompt = (
        "Summarize only what this evidence supports. Identify uncertainty. "
        f"Question: {state['question']}\nTask: {state['task']}\n"
        f"Evidence: {evidence}"
    )
    response = await model.ainvoke(prompt)
    return {"findings": [response.content]}

async def critic(state: State) -> dict:
    prompt = (
        "Assess whether findings answer the question with sufficient evidence. "
        "Reply PASS or REVISE followed by a short reason.\n"
        f"Question: {state['question']}\nFindings: {state.get('findings', [])}"
    )
    response = await model.ainvoke(prompt)
    return {"feedback": response.content}

def route_after_critic(state: State) -> str:
    feedback = state.get("feedback", "")
    if feedback.startswith("PASS") or state.get("iteration", 0) >= 2:
        return "synthesize"
    return "plan"

async def synthesize(state: State) -> dict:
    prompt = (
        "Answer the question using only the findings. Mark gaps and uncertainty. "
        f"Question: {state['question']}\nFindings: {state.get('findings', [])}"
    )
    response = await model.ainvoke(prompt)
    return {"report": response.content}

builder = StateGraph(State)
builder.add_node("plan", plan)
builder.add_node("worker", worker)
builder.add_node("critic", critic)
builder.add_node("synthesize", synthesize)
builder.add_edge(START, "plan")
builder.add_conditional_edges("plan", dispatch, ["worker"])
builder.add_edge("worker", "critic")
builder.add_conditional_edges(
    "critic", route_after_critic, {"plan": "plan", "synthesize": "synthesize"}
)
builder.add_edge("synthesize", END)

graph = builder.compile()

Notice that fan-out is explicit, the critic loop has a hard iteration limit, and provider access is behind an adapter.

In production, use structured model outputs and schema validation instead of parsing free-form lines or prefixes. Persist source identifiers alongside each finding; do not expect the model to reliably reproduce URLs from raw text. Add per-node deadlines, a maximum evidence payload, a maximum report size, and cancellation checks. Use a LangGraph Postgres checkpointer with a stable thread_id derived from the job ID, and pin graph versions on each run so a deployment cannot silently reinterpret an in-flight checkpoint.

7. Key flows

The submission path is deliberately short. It authenticates the caller, checks tenant-level limits, creates a durable job record, and enqueues an identifier. If the database write succeeds but queue publication fails, a transactional outbox or a reconciler must ensure that the job is eventually queued.

sequenceDiagram
    actor User
    participant API as ECS API
    participant DB as Aurora PostgreSQL
    participant SQS as Amazon SQS
    participant Worker as ECS worker
    User->>API: Submit question
    API->>DB: Authorize and create job
    API->>SQS: Enqueue job identifier
    API-->>User: Return job identifier
    SQS->>Worker: Deliver message
    Worker->>DB: Claim job and load state
    Worker->>Worker: Run LangGraph research
    Worker->>DB: Persist report and status

Job submission and graph execution are separate operations, so API latency is not model latency.

The second flow covers a single research run. Individual workers can call both internal retrieval and the approved search service, but each adapter must enforce its own timeout, rate limit, and result-size cap. Citation records should be persisted from retrieved evidence, not inferred from the final prose.

sequenceDiagram
    participant Worker as LangGraph worker
    participant Planner
    participant Search as Search adapter
    participant Index as OpenSearch
    participant Model as Amazon Bedrock
    participant Critic
    Worker->>Planner: Create bounded tasks
    Planner-->>Worker: Task list
    par Research task
        Worker->>Index: Retrieve tenant-scoped evidence
        Index-->>Worker: Internal passages
    and Research task
        Worker->>Search: Search approved sources
        Search-->>Worker: Normalized results
    end
    Worker->>Model: Summarize evidence
    Model-->>Worker: Findings with source references
    Worker->>Critic: Review coverage and support
    Critic-->>Worker: Pass or bounded revision
    Worker->>Model: Synthesize cited report
    Model-->>Worker: Final report

Parallel retrieval reduces serial dependency while bounded execution keeps work and cost predictable.

8. Build and release

Keep infrastructure-as-code in the application repository and review it like application code. This Terraform fragment defines a queue and dead-letter queue; a complete deployment also needs encryption, IAM policies, private networking, alarms, and service integrations. Queue visibility timeout must exceed the worker’s expected processing window, or the worker must extend visibility while it is active. The timeout is an explicit capacity assumption, not a universal setting.

resource "aws_sqs_queue" "research_dlq" {
  name                      = "research-jobs-dlq"
  message_retention_seconds = 1209600
  sqs_managed_sse_enabled   = true
}

resource "aws_sqs_queue" "research_jobs" {
  name                       = "research-jobs"
  visibility_timeout_seconds = 900
  message_retention_seconds  = 345600
  sqs_managed_sse_enabled    = true

  redrive_policy = jsonencode({
    deadLetterTargetArn = aws_sqs_queue.research_dlq.arn
    maxReceiveCount     = 4
  })
}

Notice that poison messages are isolated after a bounded number of deliveries rather than retried forever.

A staging environment should mirror production’s identity flow, database topology, queue semantics, and model access as closely as practical. The following GitHub Actions outline uses placeholders for the AWS role and ECS deployment details; configure short-lived credentials through GitHub OIDC rather than storing long-lived access keys.

name: build-test-deploy
on:
  pull_request:
  push:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install -r requirements-dev.txt
      - run: ruff check .
      - run: mypy app
      - run: pytest -q
      - run: python -m compileall app

  deploy-staging:
    if: github.event_name == 'push'
    needs: test
    runs-on: ubuntu-latest
    permissions:
      id-token: write
      contents: read
    steps:
      - uses: actions/checkout@v4
      - uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: ${{ secrets.STAGING_DEPLOY_ROLE_ARN }}
          aws-region: ${{ vars.AWS_REGION }}
      - run: terraform -chdir=infra init
      - run: terraform -chdir=infra apply -auto-approve
      - run: ./scripts/build-push-deploy.sh staging
      - run: ./scripts/smoke-test.sh staging

Notice that tests gate staging deployment and that deployment assumes an environment-scoped role.

The test strategy should make graph behavior testable without a live model call. Use fakes for search and model adapters in unit tests, then reserve limited provider-backed tests for staging.

Test layerWhat to verify
UnitTask limits, state transitions, citation validation, cancellation, retry classification, prompt and output-schema handling
IntegrationPostgreSQL migrations and checkpoint persistence; SQS redelivery; tenant-scoped retrieval; S3/index ingestion
ContractSearch and model adapter schemas, timeout behavior, error mapping, provider response changes
LoadAPI submission and polling at peak assumptions; concurrent jobs; database pool saturation; queue age; model-provider throttling
EvaluationCurated question set, citation coverage, evidence relevance, unsupported-claim checks, regressions by graph version

Do not use a single aggregate “answer quality” score as the release gate. Inspect representative outputs, track measurable properties such as citation coverage, and make release decisions against a versioned evaluation set. A model change, prompt change, retrieval change, and graph change can each alter behavior; record those versions on each run.

9. Observability

Instrument the API, queue consumer, graph nodes, database operations, retrieval, and provider calls. Use OpenTelemetry trace context across the API-to-queue boundary by placing trace propagation data in message attributes. Do not put prompts, retrieved text, user identifiers, or access tokens into span attributes. Logs should carry stable correlation identifiers and bounded categorical fields.

A structured log event might look like this:

{
  "timestamp": "2026-11-09T12:00:00Z",
  "level": "INFO",
  "service": "research-worker",
  "event": "graph_node_completed",
  "job_id": "opaque-job-id",
  "tenant_id_hash": "tenant-hash",
  "node": "critic",
  "graph_version": "2026-11-09.3",
  "duration_ms": 842,
  "outcome": "revise",
  "iteration": 1,
  "trace_id": "otel-trace-id"
}

Log metadata that supports diagnosis; keep sensitive content out of normal application logs.

For RED, monitor rate, errors, and duration for API requests, queue processing, retrieval requests, model calls, and graph runs. For USE, monitor utilization, saturation, and errors on Fargate tasks, database connections and storage, queue backlog, and OpenSearch capacity. Break down metrics by operation and outcome, not by high-cardinality values such as job ID or user ID.

This Python example creates a manual OpenTelemetry span around a graph node. Configure an OTLP exporter and resource attributes at process startup; propagate the active context through SQS using the OpenTelemetry propagator rather than starting an unrelated trace in the worker.

from opentelemetry import trace

tracer = trace.get_tracer("research-assistant.graph")

async def traced_node(node_name: str, operation):
    with tracer.start_as_current_span("langgraph.node") as span:
        span.set_attribute("graph.node", node_name)
        try:
            result = await operation()
            span.set_attribute("graph.outcome", "ok")
            return result
        except Exception as exc:
            span.set_attribute("graph.outcome", "error")
            span.record_exception(exc)
            raise

Notice that the span records node identity and failure without recording question or evidence contents.

Use these initial objectives as service-level targets and revisit them after measuring real traffic. Completion time is a product-level experience with a long tail; define it against a specific job class and exclude user cancellation.

SLIInitial SLOMeasurement
Submission availability99.9% monthlySuccessful authenticated submissions / valid submission attempts
Status API latency99% under 500 msServer-side latency for authorized status reads
Queue start delay95% under 60 secondsEnqueue timestamp to worker claim
Job completion90% within 5 minutesCompleted eligible jobs within target window
Citation integrity99% of returned citations resolve to stored evidenceValid citation references / returned references

Dashboard panels should include API RED, queue age and dead-letter count, worker concurrency and task restarts, graph duration by node, model and search errors/latency, token use by model and job class, database connections and query latency, retrieval hit rate, and SLO burn rates. Alert on user impact or imminent loss of capacity: submission error-rate burn, sustained queue age, dead-letter growth, worker inability to claim messages, database connection exhaustion, and provider throttling that pushes completion SLOs out of budget. Do not page on an isolated model timeout if retries and SLOs remain healthy.

10. Security and compliance

Treat retrieved documents and search results as untrusted input. Prompt injection can appear in ordinary web pages or internal documents; instructions found in evidence must never grant new tools, broaden access, or override system policy. Workers should have read-only capabilities and receive only the minimum tenant-scoped evidence needed for a task. A search result is evidence to assess, not code or instructions to execute.

Use Cognito or the organization’s identity provider for user authentication, then enforce tenant authorization in every API operation and retrieval query. Give API and worker tasks separate IAM roles. The API needs permission to write jobs and enqueue messages; the worker needs permission to consume jobs, read authorized data, call approved model endpoints, and write results. Avoid shared administrator credentials and use Secrets Manager for provider secrets that cannot use workload identity.

Protect data in transit and at rest with managed TLS and encryption controls. Establish retention periods for questions, results, events, checkpoints, and source content separately. Redact or hash tenant identifiers in broad operational logs, and define access controls for the traces that may contain operational metadata. Do not send sensitive customer data to an external search or model provider unless the organization’s data processing terms and classification policy permit it.

Define a source allowlist, block private and link-local network ranges in URL fetchers, validate redirects, and cap response sizes. These controls reduce SSRF and malicious-content risk but do not make arbitrary web fetching safe. Document the applicable privacy, contractual, and regulatory obligations before production; the use of managed cloud services is not, by itself, a compliance determination.

11. Cost

The following is an assumption-based monthly estimate, not a quoted AWS price. It assumes one primary region, moderate multi-AZ database sizing, 1,500 research jobs/day, bounded worker fan-out, cached repeated retrieval, and a modest search-provider allowance. It excludes taxes, enterprise support, and significant egress. Recalculate with the current regional price calculator and measured model-token usage before committing a budget.

Cost areaMonthly planning rangeMain cost driver
ECS Fargate API and workers$500–$1,800Always-on API capacity and worker runtime
Aurora PostgreSQL, backups, and storage$700–$2,000Multi-AZ capacity, I/O, retention, and checkpoint writes
OpenSearch and S3$500–$2,000Search capacity, index size, replicas, and document retention
Amazon Bedrock inference$1,500–$6,000Model choice, input/output tokens, critic iterations, and retries
External search provider$300–$1,500Search calls per job and provider plan
SQS, Redis, networking, and telemetry$500–$1,500NAT/data transfer, cache size, logs, metrics, and traces
Planning total$4,000–$14,800Highly sensitive to inference and search usage

The most effective cost controls are usually application-level: cap worker tasks, constrain evidence bytes, cache stable retrieval results, select a suitable model per graph step, and stop revision loops when additional evidence is unlikely to improve the answer. Set a per-job budget and fail with a useful status when it is exhausted. Measure actual tokens and calls per completed job before rightsizing compute.

12. Failure modes, resilience, and disaster recovery

Assume every external dependency can fail independently. Queue delivery is at least once, so the worker must make processing idempotent: claim a job using a transactional status update, persist attempt numbers, and avoid duplicating externally visible results. A duplicate delivery should observe a completed or actively claimed job and exit safely. Use a transactional outbox or reconciliation process to close the gap between job creation and queue publication.

FailureBehavior and mitigation
Search provider timeout or quota exhaustionRetry transient failures with jitter; continue with internal evidence when policy allows; report reduced coverage
Model throttling or outageApply bounded retries and concurrency limits; defer jobs or mark them retryable rather than looping
Worker crashAllow SQS redelivery; recover graph state from checkpoints; use job-level idempotency
Poison messageMove to DLQ after bounded receives; alarm and provide a controlled replay procedure
Database unavailableStop claiming new jobs; keep API degradation explicit; restore from managed backups and verify checkpoint consistency
Search index unavailableReturn a degraded result only if the product contract permits it; do not silently imply a corpus search occurred
Critic loop or oversized outputEnforce iteration, token, evidence, and wall-clock limits; terminate with an honest partial status

Use multi-AZ managed services for availability within a region, but do not confuse that with disaster recovery. Define an RPO and RTO that the product can support, enable point-in-time recovery for PostgreSQL, version important S3 objects, and document how to rebuild the search index from manifests and source objects. Test database restores and index rebuilds. A cross-region recovery design introduces replication, identity, routing, and data-residency complexity; add it only when an agreed business objective requires it.

13. Evolving to the next scale

Keep the graph, provider adapters, and API contracts modular while the deployment remains small. A worker service can scale horizontally on queue depth; the API can scale independently on request load. The next scaling step is usually not “one microservice per agent.” First determine which resource is constrained: database connections, search capacity, model quotas, worker memory, queue delay, or provider rate limits.

SignalEvolution
Queue delay rises while workers are saturatedIncrease worker capacity within provider and database concurrency budgets; separate interactive and batch queues if their SLOs differ
Search traffic dominates latency or costAdd measured retrieval caching, improve indexing and filters, or separate ingestion from query capacity
Checkpoints or event tables affect database operationsTune retention and indexing; isolate checkpoint storage only if measured contention justifies the operational cost
Independent ownership emerges across teamsSplit a service at a stable capability boundary, such as ingestion or search, with an explicit API and SLO
Regional recovery becomes a business requirementDesign and test a regional recovery path, including data replication and model/search availability

An evaluation service, model gateway, or dedicated ingestion service can be useful later, but each adds deployment and operational overhead. Keep the critic as a graph node while its lifecycle and scaling needs are coupled to the research run. Extract it only for a concrete reason such as separate model governance or independently measured workload demand.

14. Phased implementation plan

Deliver this system in phases that make failure visible early. Do not begin with broad web access, multi-region replication, or a large autonomous agent framework. Start with a narrow corpus and an explicit quality contract; then add bounded concurrency and production controls.

PhaseDeliverablesExit criteria
1. Evidence contractDefine source policy, citation format, question classes, and a small evaluation setProduct and engineering agree what counts as supported output
2. Local graphImplement planner, one worker, critic, and synthesizer with fakesUnit tests prove task and iteration bounds; results retain source references
3. AWS vertical sliceDeploy API, SQS, worker, Aurora, and staging model accessA job can be submitted, resumed safely, and polled to completion
4. Retrieval and safetyAdd tenant-scoped OpenSearch retrieval and approved search adapterSSRF, injection, access-control, timeout, and payload-limit tests pass
5. OperabilityAdd tracing, dashboards, SLOs, alarms, DLQ replay, and restore runbooksOn-call can diagnose a failed job without inspecting sensitive prompt content
6. Controlled launchLimit concurrency and traffic; review cost and quality weeklySLOs and evaluation gates hold under representative load

A successful launch is not the moment the graph produces a convincing answer. It is the point at which the team can explain why an answer was produced, identify the evidence behind it, stop an unsafe or over-budget run, and recover from partial failure without losing the user’s job. LangGraph provides the orchestration model; production quality comes from bounded tools, durable state, evidence contracts, and disciplined operations.


This article was generated with AI. #AIGenerated