--- title: Scaling Personalized Marketing for Multi-Tenant Commerce Platforms source: Instacart Engineering source_slug: instacart url: https://tech.instacart.com/scaling-personalized-marketing-for-multi-tenant-commerce-platforms-816f0c6a046b?source=rss----587883b5d2ee---4 published: 2026-05-14 fetched: 2026-05-15T14:00:50+00:00 ingested: true --- # Scaling Personalized Marketing for Multi-Tenant Commerce Platforms ## TL;DR ## Background: Marketing Across Marketplace and Storefront Instacart operates across two distinct commerce experiences: * Instacart Marketplace, our first-party consumer marketplace * Storefront Pro, our white-label e-commerce platform for retailers For years, our marketing automation infrastructure was built primarily to support Marketplace use cases. That model worked well in a first-party environment, where the product experience, customer relationship, and brand were all centrally managed by Instacart. Storefront Pro introduced a very different set of requirements. As the platform scaled to more than 350 retailers, we needed to support hundreds of independent brands, each with its own brand identity, customer base, and marketing strategy. Retailers wanted the same level of personalization and lifecycle marketing sophistication that is available on the Instacart Marketplace, but in a way that preserved their own brand and operational independence. That raised a core architectural challenge. How do we deliver Marketplace-grade personalization and lifecycle marketing capabilities to hundreds of retailers without sacrificing tenant isolation, performance, or ease of use? To succeed, the platform needed to let retail marketers: * Launch onboarding, winback, and promotional campaigns * Customize branding and messaging * Target specific customer segments * Measure performance and iterate quickly At the same time, we could not simply extend a single-tenant marketing system to a multi-tenant environment without introducing serious risks, including: * Cross-retailer data leakage * API rate-limit bottlenecks * Manual operational overhead * Inconsistent brand experiences * Tight vendor coupling Solving those constraints required rethinking the architecture so that it could support self-service marketing at scale while preserving the isolation and reliability each retailer expects. ## Architecture Overview Our solution builds on a third-party marketing automation platform, while adding critical multi-tenancy and scaling capabilities that weren’t available out-of-the-box. ## Third Party API At Instacart, we leverage a third party vendor to facilitate marketing campaigns across a variety of channels such as Email and Push Notifications. Historically, this has been limited to marketing for the Instacart Marketplace. To extend this functionality for our white-label retailers, we expanded our relationship with the vendor, building on an existing foundation and familiarity with the platform. We provision a dedicated workspace for each retailer within the third-party provider. A workspace is an isolated account with its own customer data, templates, and configuration. This per-retailer workspace model guarantees data isolation and brand integrity across the platform. ### Managing Workspaces at Scale Traditionally, Instacart marketers leveraged the third-party UI to manually manage campaigns & workspaces. However, this does not scale to the hundreds of Storefront Pro Retailers on the Instacart Platform. Rather than exposing retailers to a separate system, we built tooling within Instacart’s existing retailer platform. This keeps marketing configuration in the same place retailers already manage their business and ensures a consistent, streamlined experience. Press enter or click to view image in full size ## Key Components To make the architecture more concrete, it helps to walk through the system from the retailer marketer’s point of view all the way to message delivery. Press enter or click to view image in full size ## 1\. Instacart Tooling for Retailers Retail marketers start in an Instacart-built React interface. From there, they can configure supported campaign types, customize template variables such as subject lines, images, offers, and disclaimers, choose their target audience, preview email and push content in real time, and launch campaigns. This layer gives retailers meaningful flexibility while still enforcing the guardrails needed to keep campaigns consistent, supported, and safe to operate at scale. ## 2\. Campaigns Engine Once a campaign is launched, the Campaigns Engine handles audience evaluation and personalization setup. It determines which customers match the target audience, assigns each customer to an experiment variant, generates any associated offers, and emits one event per customer to our streaming platform. Because this engine is already used across both Instacart Marketplace and Storefront, we can reuse proven targeting and experimentation logic instead of creating separate systems for each business. ## 3\. Stream Consumer & Batching Layer The Campaigns Engine produces events at the individual customer level, but processing each one independently would create unnecessary overhead downstream. To solve that, we added a stream processing layer that consumes those events, batches them into groups of up to 50, and forwards the batched requests to the CRM Service. This batching layer significantly improves throughput and helps reduce pressure on downstream APIs. ## 4\. Instacart CRM Service The CRM Service is responsible for preparing and sending the messages. It validates idempotency, routes each request to the correct retailer workspace, assembles personalized content, and integrates with the third-party provider. Just as importantly, this service isolates provider-specific behavior behind a clean abstraction layer. That gives us flexibility to change providers in the future, support multiple providers at once, and avoid tightly coupling our core platform to any one vendor. ## 5\. Third-Party Provider Workspaces At the final stage, messages are sent through isolated third-party workspaces for each retailer. This model ensures customer data stays separated, templates remain brand-specific, rate limits are managed independently, and each retailer’s brand identity is preserved throughout the delivery flow. From there, messages are delivered to end customers through channels such as email and push notification. ## Solving Scale Challenges Supporting multi-tenant marketing at scale meant solving for both volume and efficiency. Large retail campaigns can involve hundreds of thousands, and in some cases millions, of personalized messages. Each message may include retailer-specific branding, customer-level experimentation, and dynamically generated offers. That scale introduced an important systems challenge: how do we preserve deep personalization without creating bottlenecks in the delivery pipeline? ### Batching for Throughput and API Efficiency One of the core constraints came from the shape of our internal campaign pipeline. Instacart’s Campaigns Engine emits one event per user after audience evaluation and personalization setup. Left as-is, that would require the CRM Service to process users one at a time, creating unnecessary network overhead and placing avoidable pressure on downstream systems. Those costs mattered because our third-party provider imposes strict API constraints. For example, requests are rate-limited per retailer, and individual send APIs support batches of up to 50 users per call. Processing users individually would have made large-scale campaign delivery both slower and more expensive. To address that, we introduced a stream consumer that rebatches customer-level campaign events before handing them off to the CRM Service. Instead of processing one user per request, the system groups users into batches of up to 50 and sends them downstream together. This change significantly improved throughput, reduced API pressure, and aligned our internal processing model with the capabilities of the provider. Press enter or click to view image in full size ### Horizontal Scaling and Reliable Delivery Batching improved efficiency, but we also needed the system to remain reliable under sustained load. The CRM Service is implemented as a Rails engine backed by asynchronous Sidekiq workers distributed across multiple nodes. When the stream consumer forwards a batch, the CRM Service first validates idempotency to account for at-least-once delivery semantics in the streaming layer. It then enqueues asynchronous jobs to prepare personalized messages and send them through the appropriate retailer workspace. This model gives us two important advantages. First, it keeps the ingestion path lightweight by offloading the heavier personalization and delivery work to background jobs. Second, it allows us to scale horizontally by adding worker capacity as campaign volume increases. We also persist and send results in a dedicated datastore, which gives us a durable record of message delivery and helps support observability, retry handling, and operational debugging. ## Operational Considerations ## Domain Configuration As part of sending emails on behalf of our whitelabel retailers, we needed to consider the “from address” that users would see in their inbox. How we modelled this became important, as it also has an impact on the IP warming (see below). ## Get Brent Scheibelhut’s stories in your inbox Join Medium for free to get updates from this writer. Remember me for faster sign in In order to streamline this process, we acquired an instacart-agnostic domain (eg. example.com). For each retailer onboarding to the offering, we configure an email address of [retailerName@example.com](mailto:retailerName@example.com) which we can manage and automate internally, rather than relying on the retailer to make any DNS changes. ## IP Warming Common to the industry, launching email campaigns for new retailers requires gradually warming IP addresses to maintain sender reputation. We automated this process to the following: 1. Start with small daily send volumes (50–1,000 emails) 2. Gradually increase over 4–6 weeks 3. Monitor bounce rates, spam complaints, and deliverability metrics 4. Automatically adjust send volume based on metrics To use infrastructure efficiently, we share IPs across retailers where appropriate. The system monitors deliverability signals — bounce rates, spam complaints, and engagement metrics — and dynamically adjusts send volume or triggers capacity expansion when thresholds are reached. This keeps us from over-provisioning while ensuring we can scale smoothly as campaign volume grows. ## Observability Because the platform spans campaign configuration, event streaming, asynchronous processing, and third-party delivery, observability is critical to operating it reliably. We instrument the system at multiple levels. At the campaign level, we track send rate, delivery success, opens, and clicks. At the system level, we monitor API latency, error rates, queue depth, and worker utilization. At the business level, we capture metrics such as revenue attribution, cost per send, and campaign ROI. These signals feed into both Datadog and Snowflake. Datadog supports operational dashboards and alerting, while Snowflake powers downstream reporting and helps surface performance insights back to retailers. ## Resilience and Recovery We also built the platform to handle failures gracefully. In a distributed, event-driven system, retries and downstream instability are inevitable, so reliability mechanisms need to be built into the core flow rather than added later. To support that, the platform uses idempotent message identifiers to prevent duplicate sends during retries, throttling controls to protect shared resources under load, and campaign pause mechanisms that allow problematic sends to be halted quickly. We also isolate failures at the workspace level so issues affecting one retailer do not cascade across the broader system. Together, these operational safeguards help ensure the platform remains reliable even as campaign volume, retailer count, and delivery complexity continue to grow. ## Template Tooling for Retailers ## Content Creation at Scale Managing email and push notification templates across 100+ retail brands presents a unique scaling challenge. Each retailer needs brand-specific content while maintaining consistency in campaign structure and quality. Our solution leverages automation and standardization to make template management efficient and scalable. We maintain a standardized library of email and push notification templates written in Liquid, a templating language, within an internal retailer email management repository. These templates are organized by lifecycle stage and support multiple campaign types out of the box. Template deployment is fully automated through our CI/CD pipeline. When marketers merge template changes or onboard a new retailer, our deployment scripts automatically: 1. Parse a template metadata file to determine which retailers need updates 2. Interface with Instacart’s CRM Service to generate retailer-specific versions with the appropriate branding variables 3. Upload or update templates across all relevant retailer accounts via API Press enter or click to view image in full size This automation eliminates the manual overhead of managing templates across dozens of retailer accounts. What previously required hours of manual work — logging into each workspace, uploading templates, verifying configurations — now happens automatically in minutes. The pipeline also supports retailer-specific customizations. If a retailer only wants a subset of our template offerings or requires custom modifications, we simply update their configuration in the metadata file. The deployment system handles the rest, ensuring each retailer receives exactly the templates they need. ## Self-Service Template Customization While automation handles template distribution, retailers still need the ability to customize content for their brand voice and marketing strategy. We built a self-service template editor that balances flexibility with guardrails. Each standardized template is designed with configurable variables — subject lines, content copy, images, promotional disclaimers, etc. — and provides sensible default values for each. Retailers can customize these variables to match their brand voice and marketing strategy, with their values injected at send time by our campaigns engine. Instacart’s tooling for retailers provides an intuitive React-based interface where retail marketers can: * Customize each base template’s content and styling by overriding template variables * Configure campaign-specific offers and incentives * Customize branding elements pulled from the retailer’s internal asset library * Preview changes in real-time across both email and push notification formats The live preview capability is particularly valuable — retailers can see exactly how their customizations will appear to end users before launching campaigns. This reduces iteration cycles and builds confidence in the final output. Behind the scenes, when a campaign launches, our campaigns engine merges these retailer-specific configurations with the base template, effectively overriding default values while maintaining the proven template structure. This approach gives retailers creative control while ensuring technical consistency and deliverability across all campaigns. Press enter or click to view image in full size Email Template Customization Press enter or click to view image in full size Push Notification Template Customization ## Results and Impact The platform successfully launched and is now powering marketing campaigns for multiple retail brands: * Hundreds of thousands of personalized messages sent per campaign * 99.9% delivery success rate maintained across all retailers * Sub-minute template updates from editor to production * Reduction in operational costs through optimization * Zero cross-retailer data leakage incidents due to robust isolation Perhaps most importantly, retail partners can now self-service create and launch campaigns without engineering involvement, dramatically reducing time-to-market for new marketing initiatives. ## AI-Driven Optimization and What’s Next One of the biggest advantages of this architecture is that it gives us a strong foundation for future optimization. Because campaign configuration, audience evaluation, message generation, and delivery are already separated into clear layers, we can improve individual parts of the system without having to redesign the entire platform. ## Adaptive Campaign Optimization Today, retailers can launch highly personalized campaigns at scale. Over time, we see an opportunity to make those campaigns more adaptive as well. Rather than relying only on pre-launch configuration, we are exploring ways to use early campaign performance signals to improve outcomes while a campaign is still running. For example, the system could monitor initial engagement, compare variant performance, and adjust subject lines, offers, or creative treatments based on what is working best. This would introduce a tighter feedback loop into lifecycle marketing and help retailers optimize for outcomes such as engagement, conversion, or revenue. ## AI-Assisted Content Generation We also see an opportunity to reduce the manual effort required to create effective marketing content. We are prototyping tools that can help marketers generate subject lines aligned to a retailer’s brand voice, suggest stronger copy based on historical campaign performance, and surface likely improvements before a campaign is launched. Over time, these tools could also help predict open rates, click-through rates, or identify segments that may need a different message strategy. The goal is not to replace marketer judgment, but to help teams move faster and make better-informed decisions. ## Multi-Channel Intelligence Longer term, we believe the same platform can support more coordinated decision-making across channels. Because the architecture already abstracts provider integrations and relies on an event-driven pipeline, it positions us well to expand beyond single-channel execution. That could include cross-channel orchestration across email, push, and SMS, send-time optimization at the customer level, and smarter allocation of messaging volume across campaign types and objectives. As we continue investing in the platform, these capabilities can be layered onto the existing foundation without requiring a full re-architecture. ## Conclusion Building a scalable, multi-tenant marketing platform required solving complex challenges around data isolation, performance, usability, and operational efficiency. By thoughtfully architecting around vendor limitations and prioritizing self-service capabilities, we transformed marketing from a manual, engineering-heavy process into a robust, automated platform. The result is a system that enables personalized, brand-specific marketing at scale — driving engagement and conversions for multiple retail brands while maintaining complete data isolation and operational excellence. ## Acknowledgments This project involved contributions from teams across Growth Engineering, Infrastructure, Marketing and Product. Special thanks to the team members who drove technical design, implementation, and launch: Ryan Martin, Brent Scheibelhut, Shradha Menon, and many others who contributed along the way.