1. Context and scale
A customer asks a support question in a web application. The assistant should find relevant, approved guidance, answer in plain language, and offer a human escalation when the knowledge base does not support a confident answer. It is a single agent: one model-driven interaction loop, a small allowlist of tools, and no autonomous delegation or background planning.
This blueprint uses AWS, Python 3.12, Amazon Bedrock’s Converse API, PostgreSQL 16 on Amazon RDS, and AWS Lambda. It is deliberately sized for a small product team. The model decides whether to call a tool; application code—not the model—executes the operation and enforces access boundaries. The knowledge base is relational data with PostgreSQL full-text search, not a vector platform or a second data service.
The assistant is not an authority for account-specific or safety-critical decisions. It can search approved support content and create an escalation. It cannot issue refunds, change account settings, or query arbitrary customer records. Those boundaries are part of the product design, not prompt wording.
Assumptions
| Dimension | Assumption |
|---|---|
| Users | Up to 800 daily active customers; authenticated web application |
| Requests | 0.2 average requests/second, short peaks up to 3 requests/second |
| Data volume | 5,000 support articles, under 2 GB of relational data including cases |
| Team size | 2 engineers, with part-time support/content ownership |
| Monthly budget | $300 target for infrastructure and model usage; this is a planning constraint, not a quote |
| Availability | Single AWS region; planned maintenance and database restore are acceptable |
| Conversation shape | One user turn, up to three tool calls, then a final answer; no durable agent memory |
| Data policy | Support content is approved for customer display; conversation text is not retained by default |
The request rate is intentionally modest. Model latency and database connection management matter more than horizontal compute scale. Lambda keeps idle compute costs low, while RDS provides familiar SQL operations and straightforward backup and restore. The system avoids a queue, cache, vector database, and separate orchestration service until a measured requirement justifies them.
2. Requirements
Functional requirements
| ID | Requirement | Acceptance signal |
|---|---|---|
| F1 | Authenticate the customer before starting a support interaction | Requests without a valid identity are rejected |
| F2 | Search approved support articles | Answers based on retrieved articles include article references |
| F3 | Ask clarifying questions when the request is ambiguous | The assistant does not fabricate missing context |
| F4 | Create an escalation on request or when evidence is insufficient | A case is associated with the authenticated customer |
| F5 | Return a useful answer when the model or database is unavailable | A clear, non-misleading fallback is shown |
| F6 | Allow support staff to publish, retire, and revise articles | Changes have an owner and publication timestamp |
Non-functional requirements
| Area | Requirement | Design response |
|---|---|---|
| Safety | The model must not execute arbitrary SQL or access customer data by guessing identifiers | Tools are an application-owned allowlist; customer identity comes from the verified request context |
| Latency | Most interactions should complete within a few seconds, excluding unusually slow model responses | One model loop, small search result sets, bounded tool calls, explicit timeouts |
| Reliability | A transient model or database failure should not create duplicate cases or false answers | Graceful fallback, idempotency key for case creation, bounded retries |
| Privacy | Minimize retention of customer text and avoid sensitive content in logs | Log identifiers and outcomes, not prompts or tool payloads |
| Maintainability | A two-person team can understand and deploy the whole system | One service codebase, one database, one region, infrastructure in Terraform |
| Knowledge quality | Answers must be traceable to current approved content | Article IDs and versions accompany retrieved results and citations |
A response is not “grounded” merely because the model was asked to use the knowledge base. The application must retrieve evidence, pass it into the model, and make the final response cite that evidence. If retrieval returns nothing useful, the assistant should say so or escalate rather than improvise.
3. High-level design
The public API uses API Gateway with a Cognito JWT authorizer. A Lambda function handles the authenticated request, performs a bounded Bedrock Converse interaction, and accesses PostgreSQL through RDS Proxy. The model receives tool descriptions and arguments; the Lambda validates and executes each tool. The model never receives database credentials, and the database never receives SQL generated by the model.
flowchart LR
Customer["Customer web app"] --> Auth["Amazon Cognito"]
Customer --> Api["Amazon API Gateway HTTP API"]
Auth --> Api
Api --> App["AWS Lambda support service"]
App --> Bedrock["Amazon Bedrock Converse"]
App --> Proxy["Amazon RDS Proxy"]
Proxy --> Db["Amazon RDS PostgreSQL"]
App --> Secret["AWS Secrets Manager"]
App --> Logs["Amazon CloudWatch"]
Staff["Support content editor"] --> DbThe diagram shows the single-region request path and the managed AWS services used by the application.
The API response is synchronous. The user gets either a grounded answer, a clarification request, or an escalation confirmation. Support staff publish articles through a small administrative workflow that uses the same database, but with a separate role and write permissions. Do not expose article editing through the customer-facing Lambda role.
The service has one trust boundary between the public request and application code, and another between application code and model output. Treat both the user’s text and the model’s tool arguments as untrusted. Verify the JWT, validate every argument, apply limits, and construct SQL in application code with bound parameters.
4. Cloud service mapping
AWS is the chosen cloud. Equivalent services are included to make the architecture portable conceptually, not to imply identical operating models.
| Capability | AWS choice | Azure equivalent | Google Cloud equivalent |
|---|---|---|---|
| Customer identity | Amazon Cognito User Pools | Microsoft Entra External ID | Identity Platform |
| HTTPS API and JWT validation | Amazon API Gateway HTTP API | Azure API Management | API Gateway |
| Application compute | AWS Lambda, Python 3.12 | Azure Functions | Cloud Run functions |
| Foundation model inference | Amazon Bedrock Converse API | Azure AI Foundry model inference | Vertex AI |
| Relational knowledge store | Amazon RDS for PostgreSQL | Azure Database for PostgreSQL | Cloud SQL for PostgreSQL |
| Managed database connections | Amazon RDS Proxy | Azure Database for PostgreSQL connection pooling options | Cloud SQL connector and pooling |
| Secret storage | AWS Secrets Manager | Azure Key Vault | Secret Manager |
| Logs and metrics | Amazon CloudWatch | Azure Monitor | Cloud Logging and Cloud Monitoring |
| Service identity and permissions | AWS IAM execution role | Managed identities and RBAC | Service accounts and IAM |
| Build and deployment | GitHub Actions with AWS OIDC | GitHub Actions with workload identity federation | GitHub Actions with Workload Identity Federation |
Keep the model name in configuration rather than code. Bedrock model availability, quotas, and request limits vary by region and account. Choose a model that supports the Converse tool-use contract in the deployment region, then test its actual behavior against the tool schema and answer-quality criteria before release.
5. Data design
The initial data model has two operational concerns: approved articles and customer escalations. Article content is searchable with PostgreSQL full-text search. A published flag and publication timestamp prevent drafts from appearing in customer answers. The case table stores only the minimum information needed to follow up; it does not store the full conversation transcript.
erDiagram
KB_ARTICLE {
bigint id PK
text slug UK
text title
text body
text status
text locale
integer version
timestamptz published_at
timestamptz updated_at
}
SUPPORT_CASE {
uuid id PK
text customer_subject
text summary
text status
text idempotency_key UK
timestamptz created_at
}The relational model keeps searchable knowledge and escalation records separate.
CREATE EXTENSION IF NOT EXISTS pgcrypto;
CREATE TABLE kb_article (
id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
slug text NOT NULL UNIQUE,
title text NOT NULL,
body text NOT NULL,
status text NOT NULL CHECK (status IN ('draft', 'published', 'retired')),
locale text NOT NULL DEFAULT 'en',
version integer NOT NULL DEFAULT 1 CHECK (version > 0),
published_at timestamptz,
updated_at timestamptz NOT NULL DEFAULT now(),
search_vector tsvector GENERATED ALWAYS AS (
setweight(to_tsvector('english', coalesce(title, '')), 'A') ||
setweight(to_tsvector('english', coalesce(body, '')), 'B')
) STORED,
CHECK ((status = 'published') = (published_at IS NOT NULL))
);
CREATE INDEX kb_article_search_idx
ON kb_article USING gin (search_vector);
CREATE INDEX kb_article_locale_status_idx
ON kb_article (locale, status, published_at DESC);
CREATE TABLE support_case (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
customer_subject text NOT NULL,
summary text NOT NULL CHECK (length(summary) <= 2000),
status text NOT NULL DEFAULT 'open'
CHECK (status IN ('open', 'in_progress', 'closed')),
idempotency_key text NOT NULL UNIQUE,
created_at timestamptz NOT NULL DEFAULT now()
);
A production migration should also establish database roles: the application’s customer-facing role can select published articles and insert cases, but cannot edit articles. Keep editorial writes separate. Add a trigger or migration-maintained process if article update timestamps must be automatic; do not silently rely on application clients to set them correctly.
The full-text index is a deliberate first retrieval method. It is explainable, inexpensive to operate, and appropriate for a modest curated corpus. Search results should return only a few compact passages, including the article ID, slug, title, and version. If relevance is poor, improve article structure, synonym handling, and ranking before introducing embeddings. Retrieval quality is a content and evaluation problem as much as a database problem.
6. Low-level design
The application has four responsibilities: authenticate and validate the request, run the agent loop, execute approved tools, and format a stable response. These responsibilities can live in one Lambda package with separate modules; at this scale, a distributed service boundary would add operational work without improving isolation.
classDiagram
class SupportHandler {
+handle(request) Response
}
class AgentService {
+answer(question, subject) Answer
-invoke_model(messages) Message
}
class ToolRegistry {
+execute(name, args, context) ToolResult
}
class KnowledgeRepository {
+search(query, locale) Article[]
}
class CaseRepository {
+create(subject, summary, key) Case
}
class BedrockClient {
+converse(messages, tools) Message
}
SupportHandler --> AgentService
AgentService --> ToolRegistry
AgentService --> BedrockClient
ToolRegistry --> KnowledgeRepository
ToolRegistry --> CaseRepositoryThe model client is separate from tool execution, so model output cannot directly become a database operation.
Minimal example: a closed tool allowlist
This runnable Python example demonstrates the core safety property: tool names map to application functions, and unknown names fail closed. Real tools also validate schemas and authorization.
def search_kb(query: str) -> dict:
if not query or len(query) > 500:
raise ValueError("query must contain 1 to 500 characters")
return {"articles": []}
TOOLS = {"search_kb": search_kb}
def dispatch(name: str, arguments: dict) -> dict:
tool = TOOLS.get(name)
if tool is None:
raise ValueError(f"tool not allowed: {name}")
return tool(**arguments)
if __name__ == "__main__":
print(dispatch("search_kb", {"query": "reset my password"}))
What to notice: the model may request a tool, but the application decides which functions exist and rejects everything else.
Realistic example: bounded Bedrock tool loop
The following example uses Bedrock Converse and psycopg to search published articles. Configure AWS_REGION, BEDROCK_MODEL_ID, and DATABASE_URL in the Lambda environment; use Secrets Manager and IAM-backed retrieval for production secrets. Install boto3 and psycopg[binary]. The database connection approach shown is intentionally compact; production Lambda code should reuse a bounded connection pool or use RDS Proxy.
import json
import os
import boto3
import psycopg
from psycopg.rows import dict_row
bedrock = boto3.client("bedrock-runtime", region_name=os.environ["AWS_REGION"])
model_id = os.environ["BEDROCK_MODEL_ID"]
database_url = os.environ["DATABASE_URL"]
tool_config = {
"tools": [{
"toolSpec": {
"name": "search_kb",
"description": "Find approved support articles for a customer question.",
"inputSchema": {
"json": {
"type": "object",
"properties": {
"query": {"type": "string", "minLength": 1, "maxLength": 500},
"locale": {"type": "string", "enum": ["en"]}
},
"required": ["query"],
"additionalProperties": False
}
}
}
}]
}
def search_kb(query: str, locale: str = "en") -> dict:
if not query.strip() or len(query) > 500 or locale != "en":
raise ValueError("invalid search arguments")
sql = """
SELECT id, slug, title, version,
ts_headline('english', body, plainto_tsquery('english', %s),
'MaxFragments=2,MaxWords=45,MinWords=15') AS excerpt
FROM kb_article
WHERE status = 'published'
AND locale = %s
AND search_vector @@ plainto_tsquery('english', %s)
ORDER BY ts_rank(search_vector, plainto_tsquery('english', %s)) DESC
LIMIT 4
"""
with psycopg.connect(database_url, row_factory=dict_row) as conn:
rows = conn.execute(sql, (query, locale, query, query)).fetchall()
return {"articles": [dict(row) for row in rows]}
def answer(question: str, customer_subject: str) -> str:
# customer_subject is verified by the API authorizer and is not model input.
del customer_subject
messages = [{"role": "user", "content": [{"text": question[:4000]}]}]
system = [{
"text": (
"You are a customer support assistant. Use search_kb for product "
"policy or how-to questions. Answer only from retrieved evidence, "
"cite article slugs, and say when evidence is insufficient. "
"Never claim to have changed an account or completed a refund."
)
}]
for _ in range(3):
response = bedrock.converse(
modelId=model_id,
system=system,
messages=messages,
toolConfig=tool_config,
inferenceConfig={"maxTokens": 700, "temperature": 0.2},
)
message = response["output"]["message"]
messages.append(message)
tool_uses = [
block["toolUse"]
for block in message["content"]
if "toolUse" in block
]
if not tool_uses:
return "".join(
block.get("text", "") for block in message["content"]
).strip()
results = []
for call in tool_uses:
if call["name"] != "search_kb":
result = {"error": "tool not allowed"}
status = "error"
else:
try:
result = search_kb(**call["input"])
status = "success"
except (TypeError, ValueError):
result = {"error": "invalid tool arguments"}
status = "error"
results.append({
"toolResult": {
"toolUseId": call["toolUseId"],
"content": [{"json": result}],
"status": status,
}
})
messages.append({"role": "user", "content": results})
return "I could not complete the search. Please try again or contact support."
What to notice: the loop has a hard call limit, SQL is fixed and parameterized, and the authenticated subject is not accepted from model-generated arguments.
For case creation, add a separate create_case tool with a strict summary length limit. The handler—not the model—injects the verified subject and an idempotency key derived from the request. Do not put customer_subject in the tool schema. Validate tool arguments again at dispatch time even if the model provider validates against the advertised schema.
7. Key flows
The first sequence shows the ordinary grounded answer path. Tool use is an internal application operation; the customer only sees the final response and citations.
sequenceDiagram
actor Customer
participant Api as API Gateway
participant App as Lambda
participant Model as Bedrock
participant Db as RDS PostgreSQL
Customer->>Api: Authenticated question
Api->>App: Request with verified subject
App->>Model: Question and search tool schema
Model-->>App: search_kb tool request
App->>Db: Parameterized full-text search
Db-->>App: Ranked article excerpts
App->>Model: Tool results with article metadata
Model-->>App: Answer with article citations
App-->>Customer: Answer and referencesA normal question uses one model tool request, a bounded SQL search, and a final model response.
A second flow handles a request that needs human follow-up. The model proposes a concise summary, but application code binds the case to the verified customer identity and applies idempotency. A repeated client retry returns the existing case rather than creating duplicates.
sequenceDiagram
actor Customer
participant Api as API Gateway
participant App as Lambda
participant Model as Bedrock
participant Db as RDS PostgreSQL
Customer->>Api: Request for help
Api->>App: Verified subject and request key
App->>Model: Question and allowed tools
Model-->>App: create_case tool request
App->>App: Validate summary and attach subject
App->>Db: Insert case with idempotency key
Db-->>App: Case identifier
App->>Model: Case creation result
Model-->>App: Confirmation text
App-->>Customer: Confirmation with referenceEscalation is an explicit, auditable operation; retries do not multiply support cases.
8. Build and release
Terraform should own the application’s AWS resources. Keep networking, IAM, database, API, and Lambda definitions in version control, and use separate state and configuration per environment. The excerpt below shows the database boundary and the core compute resource; the complete module also defines private subnets, security groups, RDS Proxy, API Gateway, Cognito authorization, log retention, and narrowly scoped IAM policies.
variable "db_password" {
type = string
sensitive = true
}
resource "aws_db_instance" "support" {
identifier = "support-prod"
engine = "postgres"
engine_version = "16"
instance_class = "db.t4g.small"
allocated_storage = 30
max_allocated_storage = 100
db_name = "support"
username = "support_admin"
password = var.db_password
manage_master_user_password = false
storage_encrypted = true
publicly_accessible = false
backup_retention_period = 7
deletion_protection = true
skip_final_snapshot = false
final_snapshot_identifier = "support-prod-final"
}
This is an illustrative excerpt, not a complete production module. In production, avoid passing a long-lived database password through Terraform variables because it can enter Terraform state. Prefer RDS-managed master credentials or a dedicated secret workflow, and give the application a separate least-privilege database role. Pin and review provider versions; apply schema changes as an explicit release step.
A CI workflow should run fast checks on every pull request and deploy only from a protected branch or approved environment. Use GitHub’s OIDC federation to assume a narrowly scoped AWS role rather than storing static access keys.
name: support-assistant
on:
pull_request:
push:
branches: [main]
permissions:
contents: read
id-token: write
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- run: pip install -r requirements-dev.txt
- run: ruff check .
- run: pytest -q
- run: terraform fmt -check -recursive
- run: terraform validate
working-directory: infra
deploy:
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
needs: test
runs-on: ubuntu-latest
environment: production
steps:
- uses: actions/checkout@v4
- uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ vars.AWS_DEPLOY_ROLE_ARN }}
aws-region: ${{ vars.AWS_REGION }}
- run: ./scripts/package-and-deploy.sh
Test strategy
| Test level | What to test | Suitable technique |
|---|---|---|
| Unit | Argument validation, allowlist behavior, response formatting, idempotency key handling | pytest, fake model client, repository fakes |
| Integration | Full-text search, published-only filtering, migrations, role permissions | PostgreSQL container or disposable test database |
| Contract | Bedrock tool schema and response parsing; API request/response shape | Recorded fixtures, schema checks, provider integration smoke test |
| Load | Concurrent Lambda invocations, database connection pressure, model throttling, tail latency | Small staged load test with a capped budget and realistic question mix |
| Evaluation | Whether answers are supported by retrieved articles and citations are valid | Curated question set reviewed by support staff; regression review on changes |
Do not make a live-model call part of every unit test. Model output is variable, and a flaky test suite teaches engineers to ignore failures. Use deterministic fixtures for control flow, then run a small, explicitly budgeted provider smoke test in a protected environment. Treat prompt, model, retrieval, and article changes as behavior changes that require review.
9. Observability
The main operational question is not simply whether Lambda returned HTTP 200. The team needs to know whether customers received a grounded answer, whether the database or model failed, how many tool calls were needed, and whether cost or latency is trending outside its intended envelope.
Structured logs
Emit one structured completion event per request. Use a request or trace ID, a pseudonymous customer identifier where needed, and counts or outcomes instead of user text. Do not log raw prompts, model responses, article bodies, access tokens, or database connection strings.
{
"event": "support_request_completed",
"request_id": "req_01J9EXAMPLE",
"trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
"customer_ref": "hmac:8bb2f1",
"outcome": "grounded_answer",
"tool_calls": 1,
"retrieved_article_ids": [104, 231],
"model_id": "configured-model-alias",
"duration_ms": 1840,
"http_status": 200
}
Metrics: RED and USE
| Lens | Metrics |
|---|---|
| RED — request rate | API requests per second; requests by outcome (grounded, clarification, escalation, fallback) |
| RED — errors | 4xx/5xx rate; model errors and throttles; database errors; tool validation failures |
| RED — duration | End-to-end latency and separate model, SQL, and application duration percentiles |
| USE — Lambda | Concurrency, throttles, duration, memory utilization where available |
| USE — database and proxy | Database CPU, connections, free storage, proxy connection borrow latency, connection saturation |
| USE — API/model dependency | API Gateway throttles and integration latency; Bedrock invocation errors and token usage where available |
Instrument the service with OpenTelemetry and export traces through an AWS-supported collector or the chosen OpenTelemetry-compatible path. Keep high-cardinality values such as user IDs and raw question text out of metric labels. This FastAPI-style instrumentation example illustrates the span boundaries; in Lambda, initialize instrumentation once per execution environment and propagate the incoming trace context from API Gateway.
from opentelemetry import trace
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
provider = TracerProvider(
resource=Resource.create({"service.name": "support-assistant"})
)
provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))
trace.set_tracer_provider(provider)
tracer = trace.get_tracer(__name__)
def traced_search(search_fn, query: str):
with tracer.start_as_current_span("kb.search") as span:
span.set_attribute("kb.query_length", len(query))
rows = search_fn(query)
span.set_attribute("kb.result_count", len(rows))
return rows
Create a trace for the inbound request and child spans for Bedrock invocation, tool validation, and SQL search. Record durations, status, tool name, and result count, but not raw content. For Lambda, ensure the exporter configuration and timeouts do not delay the response; asynchronous or platform-native telemetry export is preferable to a blocking export on every invocation.
SLI/SLOs
These are initial targets for a small service and should be adjusted after collecting real traffic and support feedback.
| SLI | Initial SLO | Measurement window |
|---|---|---|
| Successful API availability | 99.5% of authenticated requests receive a valid response or explicit fallback | 30 days |
| Grounded answer rate | At least 90% of sampled answerable questions cite one or more retrieved articles | Weekly reviewed sample |
| Request latency | 95% of requests complete within 8 seconds, excluding client disconnects | 30 days |
| Escalation integrity | 99.9% of accepted escalation requests create or return one case reference | 30 days |
Use three dashboards: request outcomes and latency; model/tool/database dependency health; and knowledge quality, including empty searches, citation coverage, and escalation frequency. A separate cost view should show model invocation and token usage alongside infrastructure spend.
Page an engineer for sustained API failure, repeated Lambda throttling that impacts requests, database connection exhaustion, failed database backups, or a model dependency outage that removes the assistant’s primary function. Send a notification rather than paging for isolated slow requests, an increase in low-confidence escalations, or a small shift in empty-search rate. Those are usually product or content issues first.
10. Security and compliance
The primary security control is to keep the assistant’s powers deliberately narrow. A system prompt is useful behavioral guidance, but it is not an authorization mechanism.
- Require a verified Cognito JWT at the API boundary. Derive the customer subject from validated claims, never from request JSON or model arguments.
- Give the Lambda role permission only to retrieve its secret, invoke the configured Bedrock model, write logs, and access the required network resources.
- Use private database subnets, encrypted storage, TLS connections, restricted security groups, and separate database roles for customer requests and editorial changes.
- Expose fixed tools such as
search_kbandcreate_case. Do not expose SQL text, a generic query tool, or broad account-management operations. - Validate tool arguments server-side, enforce query length and result limits, and cap tool calls per turn. Apply rate limits at the API boundary.
- Treat retrieved article text as untrusted input. An article can contain accidental or malicious instructions; the model should use it as evidence, not as a higher-priority instruction.
- Avoid storing conversation transcripts by default. Establish retention and deletion rules before adding conversation history or analytics.
- Redact sensitive values before logs, traces, support case summaries, and error reporting. Keep secrets out of source control and CI logs.
- Review model-provider data handling, regional availability, and organizational compliance requirements before sending customer content to inference.
If the assistant later handles account-specific questions, add an explicit authenticated lookup tool with narrowly scoped queries and authorization checks against the customer subject. Do not turn that requirement into a generic “look up customer” capability.
11. Cost
The following is an assumption-based monthly estimate, not a pricing quote. It assumes one region, low traffic, a small RDS instance, modest storage, and a managed model selected for the deployed region. Actual charges depend on region, model, token lengths, availability settings, data transfer, and current AWS prices.
| Cost area | Monthly planning range | Main driver |
|---|---|---|
| RDS PostgreSQL and storage | $45–$110 | Instance size, storage, backup retention, and any Multi-AZ choice |
| RDS Proxy | $15–$45 | Proxy capacity and idle connection behavior |
| Lambda and API Gateway | $5–$35 | Invocation count, duration, memory, and API request volume |
| Bedrock inference | $20–$220 | Model pricing, prompt and output tokens, and requests per day |
| CloudWatch logs and metrics | $5–$25 | Retention, log volume, custom metrics, and trace export |
| Cognito, Secrets Manager, network and miscellaneous | $10–$45 | Identity usage, secret count, and network configuration |
| Indicative total | $100–$480 | Planning range only; validate against current regional pricing |
The model line has the widest uncertainty, so instrument token counts and invocation outcomes from the first deployment. Keep retrieved passages short, constrain output length, and avoid redundant calls. Do not optimize for a lower-priced model by default: compare answer quality, escalation rate, and operational behavior on a representative evaluation set. For this small workload, the simplest cost controls are request throttling, a bounded tool loop, concise context, and alerts on spend.
12. Failure modes, resilience and disaster recovery
| Failure | Customer impact | Response |
|---|---|---|
| Bedrock timeout, throttle, or unavailable model | No generated answer | Apply a short timeout and bounded retry policy; return a helpful fallback and offer human support |
| RDS or RDS Proxy unavailable | No article retrieval or escalation write | Do not claim grounding or case creation; return a clear retry path and alert on sustained failure |
| Empty or weak search results | Incorrect or unhelpful answer risk | Require evidence for product-specific claims; ask for clarification or escalate |
| Malformed or unknown tool call | Potentially unsafe execution attempt | Reject it, record a redacted validation event, and do not execute a fallback tool |
| Lambda retry after a client timeout | Duplicate escalation risk | Use a unique idempotency key and return the existing case for a duplicate |
| Bad article publication | Incorrect guidance | Keep versioned content, support retirement, and maintain an editorial review workflow |
| Accidental secret or transcript logging | Privacy incident | Minimize data at source, restrict log access, set retention, and follow incident response procedures |
Back up the RDS database with automated backups and periodically test a restore into a non-production environment. For the assumed scale, a single-AZ database is a cost-conscious starting point, not a high-availability guarantee. A database failure may require restore time; define a recovery time objective and recovery point objective with the product owner and support team rather than implying instant recovery.
Keep the infrastructure definitions, application artifact, schema migrations, and article seed/import process reproducible. A regional outage is outside the initial design: disaster recovery means restoring into the same region from backups. If the business later requires regional continuity, define how identity, model access, database replication, DNS, and secret distribution behave before building a second region.
13. Evolving to the next scale
Scale should follow a measured constraint, not an architecture diagram’s ambition. Start with this design while traffic is below roughly 1,000 daily users and the article corpus remains manageable. The thresholds below are operational signals, not hard product limits.
| Signal | Likely next change | Keep stable |
|---|---|---|
| Lambda connection pressure or proxy borrow latency rises | Tune pool behavior, connection limits, and database sizing; consider a queue for non-interactive work | Tool contracts and parameterized SQL |
| Search relevance is poor despite editorial cleanup | Add semantic or hybrid retrieval and evaluate it against a labeled question set | Article lifecycle, citations, and publication controls |
| Support team needs reliable transcript review | Add explicit consent, retention, access controls, and a separate conversation store | Minimal request logs and least-privilege identity |
| Editorial workflow becomes a bottleneck | Add an authenticated admin interface and approval/version workflow | Separate customer and editorial database roles |
| Multiple product teams own different support domains | Split tool modules or deploy bounded services only where ownership or security requires it | One agent policy and centralized tool authorization |
| Availability requirements exceed single-AZ recovery | Adopt Multi-AZ database deployment and a tested failover runbook | Single-region operation until a regional objective exists |
Adding tools is a product and risk decision. Each new tool expands the agent’s effective authority and must have an owner, argument schema, authorization rule, side-effect policy, idempotency strategy, and evaluation coverage. Resist adding a generic browser, shell, SQL, or “execute action” tool merely because the model can call it.
14. Phased implementation plan
- Define the support boundary. Select the questions the assistant may answer, the cases it may escalate, the content it may cite, and the actions it must never claim to perform. Agree on privacy and retention requirements.
- Prepare the knowledge base. Import a small, reviewed set of articles with stable slugs, locale, status, owner, and version. Fix contradictory or obsolete content before measuring model quality.
- Build the retrieval path. Deploy PostgreSQL, apply migrations, create the full-text index, and verify that published-only searches return useful, bounded excerpts.
- Add the tool loop. Implement the fixed tool registry, server-side schema validation, call limit, citation formatting, timeout behavior, and safe fallback. Test with a fake model client first.
- Integrate identity and inference. Add API Gateway JWT validation, the Lambda execution role, Bedrock access, RDS Proxy, secret retrieval, rate limits, and request-size limits.
- Instrument before launch. Add structured logs, traces, latency and error metrics, token tracking, dashboards, and the initial paging rules. Confirm that telemetry does not contain customer text.
- Evaluate and release gradually. Run unit, integration, contract, and load tests; review a curated answer set with support staff; deploy to a small internal audience before enabling customer traffic.
- Operate the content loop. Review empty searches, escalations, citation coverage, and customer feedback weekly. Improve articles and retrieval before increasing model autonomy or adding infrastructure.
The success criterion is not that the model appears conversational. It is that a customer receives a useful answer grounded in approved knowledge—or an honest escalation—through a system the small team can understand, secure, and operate.