Skip to content

PATTERN Cited by 1 source

Dynamic content filtering in MCP pipeline

Pattern

Treat the MCP server tier not as a static tool-description endpoint, but as a data-pipeline layer where every MCP tool call passes through a declarative content-filter (Bloblang mapping, Starlark/Python extension, or SQL-like predicate) that enforces per-call, per-field, or per-row policy before either the request reaches the downstream tool or the response reaches the agent.

The payoff: enterprise security teams get HBase-cell-ACL-grade granularity on what non-deterministic LLM agents can see or cause, while engineers keep iteration speed via full programming-language code extensions.

Canonical statement on the wiki

Alex Gallego's 2025-04-03 founder-voice argument (Source: Gallego 2025-04-03):

"On the surface, wrapping a single Google Docs collection as a tool for agents to call may seem overly onerous. However, the world does not want to hand over full access to their kingdom to non-deterministic machines like these LLMs. While from a technical perspective MCP does offer centralized auditing, logging, tracing, security, API tokens, etc for the systems builder, the business wants ultra-fine grain control — HBase had cell access control list at the cell level for a reason. Control for MCP HTTP calls comes in the form of dynamic pipeline filtering."

"For Redpanda Connect specifically, it is the ability to leverage full programming languages via custom code extensions to give engineers the speed of iteration while letting the security team sleep at night, knowing they can enforce overriding global policies for ultra-fine-grain access to any of the ~300 connectors in a declarative fashion."

Why per-call filtering matters for agents

Static tool-level ACLs ("agent X may call tool Y") are insufficient when:

  • The same tool is safe for some rows/cells and unsafe for others. An agent may be allowed to query a customers table but not see rows belonging to EU residents without GDPR consent.
  • Content transformations are part of the policy. Redacting PII columns, masking credit-card numbers, truncating free-text fields, or filtering out internal-only rows is a response-shape concern.
  • Context-aware denial is required. The same tool call may be allowed in one agent session and denied in another based on who the calling user is, what the agent's goal is, what data has already been accessed this session, etc.

These policies can't be expressed at the tool-registration layer; they need a per-call runtime filter the proxy evaluates.

Mechanism at Redpanda

The Redpanda-specific realisation layered on patterns/mcp-as-centralized-integration-proxy:

  • Declarative pipeline: each MCP tool is a Redpanda Connect pipeline definition — an input (the MCP request body), zero or more processors, an output. Processors can include filters, mappers, branches, switches.
  • Bloblang mapping language: declarative per-field transformation / redaction / filtering.
  • Starlark code extensions: Python-subset embedded scripting for the cases where Bloblang isn't expressive enough — "effectively Python without imports, but more importantly, it is all Python so no need to learn a new configuration language."
  • Security team as global-policy author: "the security team [can] enforce overriding global policies for ultra-fine-grain access to any of the ~300 connectors in a declarative fashion."

Orthogonal shapes on the wiki

Caveats

  • Aspirational framing. The source post describes the shape as a future ("MCP future is full pipelines with dynamic content transformation and filtering"); the specific Redpanda Connect pipeline + Bloblang + Starlark combination is available today, but the cell-ACL-grade governance story is a direction of travel, not a shipped product-ready feature.
  • HBase cell-ACL analogy is aesthetic. No actual per-cell ACL primitive is implemented; the analogy signals the granularity aspiration, not a drop-in mechanism.
  • Policy-correctness under LLM prompt injection. A filter in the MCP pipeline helps on the response side; it does not protect against the LLM being manipulated by a prompt-injection payload to call a differently-shaped tool. Complementary pattern: concepts/ai-agent-guardrails.
  • Performance budget. Per-call filtering adds latency; for high-throughput tools, the filter itself may become the bottleneck.

Seen in

Last updated · 470 distilled / 1,213 read