PATTERN Cited by 1 source

Cloud model factory with edge deployment¶

Definition¶

An end-to-end pattern where model customization (data generation, fine-tuning, evaluation) runs in the cloud using managed ML services, while inference runs entirely on edge hardware. A managed cloud-to-edge bridge handles model packaging, versioning, and lifecycle management without requiring persistent connectivity to target devices.

Three layers: 1. Cloud-side model factory — automated training data generation + repeatable, versioned model customization (FMOps pipeline) 2. Managed cloud-to-edge bridge — deployment orchestration handling model versioning and lifecycle 3. Self-contained edge inference stack — lightweight runtime + orchestration framework + local knowledge base operating independently of cloud availability

Mechanism¶

[Cloud]                              [Edge]
Bedrock (data gen)                   Ollama (inference runtime)
  → SageMaker Pipelines (FT)        Strands Agents (orchestration)
    → S3 (model artifact)           ChromaDB (local RAG)
      → IoT Greengrass (deploy) ──→ Quantized model (GGUF)
                                     ←── Feedback queue (async)

When to use¶

Environments with intermittent or unavailable connectivity (offshore, remote agriculture, transportation, defense)
Strict latency requirements incompatible with network round-trips
Data locality constraints (on-premise data must not leave the facility)
Industrial environments where unplanned downtime has catastrophic cost

Trade-offs¶

Model capability limited by hardware: Edge GPUs constrain model size; must use small language models (SLMs) or quantized variants.
Update latency: Model improvements only reach edge devices on next deployment cycle (not real-time).
Fleet management complexity: Each edge device is an independent failure domain requiring its own observability.
Dual-maintain: RAG knowledge base must be synchronized separately from model weights.

Seen in¶

sources/2026-07-22-aws-architecting-offline-first-generative-ai-applications-for-edge-deployments — AWS reference architecture using Bedrock + SageMaker Pipelines → IoT Greengrass → Ollama + Strands Agents + ChromaDB. Hybrid FT+RAG strategy balances model compactness with up-to-date knowledge retrieval.