Skip to content

AWS Tier 1

Read original ↗

Eclipse Dataspace Components on AWS: Architecture patterns in production

Summary

Part 2 of a series on running Eclipse Dataspace Components (EDC) connectors on AWS. This post presents a production-grade deployment architecture that treats each EDC connector as an isolated architecture cell — independent compute (ECS/Fargate), persistence (Aurora, Secrets Manager, Cognito), storage (S3), and networking (API Gateway + NLB via VPC links) — wrapped in defense-in-depth security and IaC automation via CDK. The architecture is validated against three AWS Well-Architected pillars (Operational Excellence, Security, Reliability) and positions managed services as the substrate for reducing undifferentiated operational burden.

Key Takeaways

  1. Cell-per-connector isolation: Each EDC connector instance forms a self-contained "architecture cell" — all cloud resources for one connector are isolated from other connector instances. Enterprises deploy separate connector instances per use case for failure isolation, data governance, and separation of shared/received data (Source: raw article, "Larger enterprises... deploy separate connector instances per use case").

  2. Control plane / data plane separation at connector level: The EDC connector itself splits into a control plane (contract negotiation, policy management) and a data plane (actual data transfer). These are deployed as independent ECS services with distinct responsibilities, security contexts, and scaling characteristics (Source: raw article, "Deploy the control plane and data plane as independent services").

  3. Managed services over self-managed infrastructure: Aurora replaces self-managed Postgres; Secrets Manager replaces Vault; Cognito replaces self-hosted OAuth; Fargate replaces EC2 fleet management. The rationale is explicit: "shifts undifferentiated heavy lifting to AWS and reduces operational burden. You gain high availability, built-in security best practices, compliance certifications, and automatic updates" (Source: raw article).

  4. Infrastructure-as-Code via CDK for templatized deployment: CDK allows "automated, templatized deployment and management of EDC connectors over time" — a single cdk deploy stands up a full connector cell. CDK Nag provides automated validation/security scanning pre-deployment (Source: raw article).

  5. Defense in depth via four independent layers: Network isolation (VPC private subnets) + security group segmentation + IAM least privilege + encryption at rest and in transit. Each layer is described as independently protective: "if one layer is bypassed, others continue to provide protection" (Source: raw article).

  6. Principle of least privilege applied per-component: IAM roles scoped to individual services (separate roles for control plane vs data plane). Security groups restrict to necessary ports/sources. Internal-only NLB behind API Gateway prevents unintended public exposure (Source: raw article).

  7. Encryption everywhere: Data encrypted at rest (Aurora, S3, Secrets Manager), in transit (TLS enforcement, HTTPS-only egress), and during processing (encrypted environment variables). Comprehensive regardless of data residency stage (Source: raw article).

  8. ECS circuit breakers for automatic rollback: "ECS circuit breakers can automatically roll back failed deployments, automated health checks remove unhealthy targets, and point-in-time recovery supports rapid database restoration" — minimizes MTTR without manual intervention (Source: raw article).

  9. Multi-AZ resilience by default: Cross-zone load balancing distributes across AZs, Aurora replicates across AZs, Fargate tasks scheduled in any AZ. Architecture tolerates full AZ failure without service disruption (Source: raw article).

  10. API Gateway + VPC link for private API exposure: API Gateway exposes selected EDC API resources (management, data plane, DSP) while the actual services remain in a private VPC. VPC links provide secure, private connectivity. SigV4 protects both S3 and API Gateway interaction surfaces (Source: raw article).

  11. MCP proxy support for agentic integration: API Gateway's Model Context Protocol proxy support allows EDC APIs to be consumed by authorized AI agents and MCP clients for autonomous data collection and sharing — forward-looking integration pattern (Source: raw article).

Architecture & Numbers

Dimension Value
Connector architecture Control plane + data plane as separate ECS services
Compute substrate Amazon ECS + AWS Fargate (serverless containers)
Container registry Amazon ECR
Relational state Amazon Aurora
Secrets/credentials AWS Secrets Manager
OAuth 2.0 Amazon Cognito
Object storage Amazon S3
API exposure Amazon API Gateway + VPC links
Internal LB Network Load Balancer (internal only)
Network isolation Amazon VPC (private subnets)
IaC tool AWS CDK + CDK Nag
Auth protocol AWS SigV4
Resilience model Multi-AZ, ECS circuit breakers, Aurora replication
Deployment unit One CDK stack = one full connector cell
Reference impl Dataspace Connector on AWS

Extracted Systems/Concepts/Patterns

Systems: Eclipse Dataspace Components (EDC), Amazon ECS, AWS Fargate, Amazon Aurora, AWS Secrets Manager, Amazon Cognito, Amazon S3, Amazon API Gateway, AWS NLB, Amazon VPC, AWS IAM, AWS CDK, Amazon ECR, Amazon CloudWatch, Amazon EventBridge.

Concepts: concepts/cell-based-architecture, concepts/control-plane-data-plane-separation, concepts/defense-in-depth, concepts/least-privilege, concepts/observability, concepts/multi-az-resilience, concepts/infrastructure-as-code, concepts/managed-services-over-self-managed.

Patterns: Architecture cell per connector instance, IaC-templatized deployment (single CDK command → full cell), ECS circuit-breaker rollback, VPC-link private API exposure, SigV4 API protection.

Caveats

  1. No operational numbers disclosed (QPS, latency, throughput, connector count in production).
  2. No specific sizing guidance (ECS task CPU/memory, Aurora instance class, S3 throughput).
  3. Well-Architected pillars of Performance Efficiency, Cost Optimization, and Sustainability deferred to Part 3.
  4. No multi-region architecture discussed — resilience is AZ-level only.
  5. No data-plane transfer performance numbers (throughput of connector-to-connector data movement).
  6. IDSA protocol details (DSP, IDS-CP) not elaborated — Part 1 prerequisite.
  7. Reference implementation maturity/production adoption not quantified beyond Prometheus-X mention.
  8. MCP proxy integration described as possible, not as validated production pattern.
  9. No comparison with alternative EDC hosting approaches (Kubernetes, VM-based, multi-cloud).
  10. Security group rules and IAM policy specifics not disclosed at granular level.

Source

Last updated · 585 distilled / 1,765 read