Skip to content

SYSTEM Cited by 1 source

Datasette llm Python library

Definition

llm is Simon Willison's open-source Python library (and CLI) for talking to LLMs, with a conversation API that holds multi-turn state client-side, plugin-based multi-provider support (OpenAI, Anthropic, local models via Ollama, etc.), and CLI-first ergonomics. The wiki's canonical anchor is its use as the OpenAI API wrapper inside Zalando's component-migration toolkit, specifically the conversation API's role in truncation recovery.

Wiki anchor

From sources/2025-02-19-zalando-llm-powered-migration-of-ui-component-libraries:

"We developed a Python based migration tool using the llm library's conversation API."

Two load-bearing uses at Zalando:

  1. Multi-turn conversation state. The conversation API persists the model's in-flight response so that on the next turn the model continues from its previous state instead of starting over.

  2. continue-prompt truncation recovery. When an output exceeds the 4K-token-per-response limit, Zalando sends a literal "continue" as the next turn on the same conversation and concatenates the completion. "This allowed the LLM to pick up where it left off and complete the transformation. As per our tests, a simple 'continue' prompt proved more reliable than more complex prompts to continue the transformation." See concepts/continue-prompt-for-truncated-output for the generalised concept.

Role vs alternatives

  • vs openai Python SDK: the openai SDK is the lower-level, provider-specific client. llm sits above it, wrapping Chat Completions behind a uniform interface and adding the conversation abstraction + CLI + plugin system. Zalando chose llm for "its extensive support for working with LLMs and its rich ecosystem of libraries".
  • vs LangChain / LlamaIndex: those frameworks build agents, retrieval pipelines, and chains; llm is a focused "talk to models" layer. For Zalando's single-call-per-file migration pipeline, the simpler abstraction was the right fit.

Seen in

Last updated · 501 distilled / 1,218 read