Skip to content

SYSTEM Cited by 3 sources

FastAPI

Definition

FastAPI is a modern async Python web framework built on Starlette + Pydantic. It provides native support for Server-Sent Events (SSE) and async request handlers, making it a natural fit for streaming LLM token responses — the use case that motivated Yelp's migration from Pyramid to FastAPI for BAA.

Role at Yelp (2026-03-27)

"Streaming was the biggest win: By streaming the response from the LLM we have very low latency for the TTFB (Time to first Byte). This was facilitated by migrating from pyramid to a fastapi server that supports sse (Server side events)." (Source: sources/2026-03-27-yelp-building-biz-ask-anything-from-prototype-to-product)

Load-bearing properties:

  • SSE support — enables token-by-token streaming to the browser.
  • Async handlers — compatible with async LangChain invocation for parallel question analysis.
  • TTFT dominant — streaming flips the user-visible latency from "wait for full response" to "see first token". See concepts/tail-latency-at-scale.

Role at Expedia STAR (2026-04-28)

STAR's web tier is a FastAPI service (with Celery + Redis behind it for async task execution). STAR's V0 leaned on FastAPI's built-in async primitives — async/await handlers + FastAPI background tasks — then migrated the long-running analyses to Celery as traffic grew:

"This service is mostly I/O bound, but we still have synchronous operations. ... For this, we initially used certain features from FastAPI such as async/await and background tasks. As part of scaling up, we moved to Celery with Redis acting as the broker and result backend to store the state and results of tasks." (Source: sources/2026-04-28-expedia-expedias-service-telemetry-analyzer)

Canonical small-arc of the FastAPI background-tasks → Celery graduation: fine for sub-second fire-and-forget, graduate when you need durable task state, retries, and horizontal worker scaling. See systems/celery.

Caveats

  • Stub page. Deeper FastAPI architecture (Pydantic model validation, dependency injection, OpenAPI schema generation) is not walked here.

Seen in

Runs on Cloudflare Python Workers via workers.asgi (2026-09-21)

FastAPI is the flagship framework Cloudflare showcased for Python Workers GA. Because FastAPI speaks ASGI, a workers.asgi connector bridges the incoming native JS request into the ASGI structures FastAPI expects and pipes the response back — so the same FastAPI app runs unchanged, with the Workers platform (not uvicorn) acting as the web server. Entry point is Default = asgi.entrypoint(app). Django/Flask ride the same mechanism (WSGI via workers.wsgi). This makes FastAPI's async/streaming strengths available on an edge-serverless runtime backed by Pyodide.

(Source: sources/2026-09-21-cloudflare-python-workers-are-now-generally-available)

Last updated · 766 distilled / 2,225 read