Skip to content

SYSTEM Cited by 1 source

SmartProminence AI Orchestrator

A multi-tenant, HIPAA-eligible agentic workflow framework built by MHK (a Hearst Health company) and running entirely on AWS. It replaces the pattern of building a separate AI system — each with its own compliance infrastructure, audit trails, and security controls — for every medical-management workflow (medical, pharmacy, grievance, appeals). New AI workflows are deployed through configuration (define prompts, specify input/output schemas, register the agent); the orchestrator handles HIPAA compliance, encryption, audit trails, scaling, and orchestration. (Source: sources/2026-09-30-aws-how-mhk-built-a-hipaa-eligible-agentic-ai-solution-on-amazon-bedrock)

Architecture

The system is architected around a controller-agent pattern — the LLM-workflow application of the classic orchestrator/worker (workflow-vs-activity) split. It is stateless, event-driven, and independently scalable.

  • Agent Orchestration Core — the "central nervous system." Spring Boot on AWS Fargate, exposing a REST API for job submission, workflow management, LLM proxying, and token management. It is the only component with direct database access; controllers and agents interact exclusively through its API, enforcing strict data-access boundaries. All model calls flow through its proxy to Amazon Bedrock (Claude).
  • Workflow Engine Controller — determines what happens and in what order. On a job, it loads a version-pinned workflow definition, resolves step dependencies into a DAG using Kahn's algorithm (see concepts/write-dependency-graph), pre-creates step executions in a WAITING state, uses Spring Expression Language (SpEL) conditionals to decide which steps to run vs skip, then dispatches agents depth by depth — steps at the same depth run in parallel; the controller polls for completion before advancing.
  • LLM processing agents — stateless workers that execute individual steps via a standardized pipeline: input binding (resolve expressions over prior step results) → optional vision processing of scanned documents → prompt assembly with enriched context → Bedrock (Claude) invocation → post-processing (field extraction, type coercion, structured output, hallucination/confidence validation). For intra-step parallelism they spawn a Java virtual thread per execution (e.g. extract every page of a multi-page document concurrently).

Messaging

Communication flows through Amazon SQS:

  • ControllerTaskMessage → Controller Invoke Queue (start a workflow)
  • AgentTaskMessage → Agent Invoke Queue (dispatch a step)

Each message carries a capability token scoped to only that operation's data. This gives stateless processing, independent horizontal agent scaling via ECS Fargate, fault isolation (a failed task doesn't block parallel steps; dead-letter queues capture failures — see patterns/dead-letter-queue), and decoupled deployment. The only blocking call in the pipeline is the Bedrock LLM invocation — everything else is asynchronous, so the system handles "hundreds of concurrent jobs without resource contention."

Dynamic agent registry

Adding an agent type is configuration-driven, not engineering-driven (concepts/self-service-infrastructure): a developer defines prompt templates, input/output schemas, and model selection, then registers the agent through the core's API. Terraform auto-provisions the supporting SQS queues, IAM roles, and ECS task definitions. The agent is immediately available for workflow steps with no new compliance certification, because it runs inside the already-certified orchestrator.

Security and multi-tenancy

Defense-in-depth across every layer (concepts/defense-in-depth, concepts/tenant-isolation):

  • Per-client encryption. Each client has its own AWS KMS key. S3 documents are double-encrypted (S3 SSE + client-specific KMS); the database adds row-level encryption on top of RDS storage-level encryption. A misrouted job cannot be decrypted by the receiving agent — it lacks that client's KMS key.
  • Capability-token model. Per-dispatch, KMS-signed, least-privilege tokens (concepts/least-privileged-access): a controller token can read workflow/job data, dispatch agents, create step executions; an agent token can only read its step's input, write its own result, call the LLM proxy, and upload artifacts. A compromised agent cannot reach other steps/workflows/clients.
  • Network isolation. Database subnets have no internet access; AWS service traffic (S3, SQS, KMS, Secrets Manager, CloudWatch, ECR) flows over VPC endpoints; TLS 1.3 in transit.
  • Compliance controls. LLM request/response bodies are not logged — only token counts + content hashes (CloudWatch, no PHI). Workflow configs stored immutably in S3; agents receive only the minimum context for their step (data minimization).

Responsible-AI controls

Application-layer validation inside each agent's pipeline (concepts/structured-output-reliability, concepts/llm-hallucination): post-process responses against expected schemas, cross-reference extracted data with source documents to detect hallucinations, reject below a confidence threshold, and verify outputs reference only the patient's own records / policy-specific criteria. All AI-assisted decisions are logged with full audit trails for compliance and reproducibility.

Conversational memory

The orchestrator maintains context across executions for the same case. Each execution returns a job ID that the upstream system associates with the case record; over a case's lifetime (intake, policy review, appeal — 3+ executions) each produces structured outputs that stay available in S3 (client-KMS-encrypted) for later executions, so a 30-page record analyzed once is reusable without re-running ingestion (concepts/human-in-the-loop review weeks later finds the history already organized).

Results (MHK-reported)

  • 90% reduction in manual review effort (intake: 5–10 min → under 1 min; director reviews: hours → ~30-second approve/deny).
  • 85% faster AI feature deployment (3–6 months → ~2 weeks).
  • Unified compliance: single orchestrator-level HIPAA + SOC 2 attestation that all agents inherit.

Caveats

Customer-architecture post (MHK on the AWS Architecture Blog), not an AWS-internal retrospective. No absolute throughput/latency/token-cost numbers; model given only as "Claude." MHK uses Bedrock exclusively for inference and built orchestration, retrieval, and validation itself for healthcare-specific control.

Seen in

Last updated · 766 distilled / 2,225 read