Skip to content

Figma — How We Built AI-Powered Search in Figma

Summary

Figma built AI-powered search (shipped at Config 2024) combining visual search (query by screenshot / selected frame / sketch, i.e. reverse image search over designs) and semantic search (query by natural text against component names/descriptions/files even when terminology doesn't match). The feature originated in a June-2023 three-day AI hackathon whose most ambitious prototype was design autocomplete — suggest the next component as the designer works. Building autocomplete forced Figma to build the underlying AI search layer because retrieval quality determines autocomplete quality ("we can improve AI outputs with examples from search" — their framing of RAG). User research then redirected priorities: 75% of all objects added to the Figma canvas come from other files — designers riff on existing work — so better search is the user-value bottleneck, not autocomplete. The team identified three target use cases (frame lookup / frame variations / broad inspiration), started with creation-tied search (exact matches and close variants) before less-frequent ideation queries, and tapped reverse image search as the established AI research lineage to deliver value quickly. Surfaced in Figma's Actions panel (the team's consolidated home for AI features) with "peek" previews for its narrower width and CMD+Enter for full-screen result inspection. Four shipping principles named: AI for existing workflows, rapid iteration (continuous deploys to staging + internal beta), systematic quality checks (custom eval tools), cross-disciplinary teamwork.

Key takeaways

  1. Selective indexing via UI-shape heuristics cuts the search corpus. Indexing everything is too costly. Figma targets top-level frames that look like UI designs by combining common UI frame dimensions as a heuristic with exceptions for non-top-level frames that meet the right conditions (designers often organize work in sections or nested frames, breaking the naive "top-level frame only" rule). Canonical heuristic-driven indexing policy.
  2. Duplicate and file-copy collapsing further shrinks the corpus. Designers riff by duplicating frames and tweaking — many pages with similar designs. Figma indexes only one of each near-duplicate and skips unaltered file copies entirely. Complementary to heuristic shape-based filtering; same goal = reduce the searchable set without losing designer-useful results.
  3. Quality signals are an experimental axis, not a solved problem. Files contain polished designs alongside "Graveyard"-named archive pages. Figma is experimenting with explicit signals like "frame marked ready for development" to surface the former without dredging the latter. This is a candid "still open" engineering statement and is the honest place to cite on quality-signal sourcing for search.
  4. Edit-quiescence as an indexing trigger. Figma holds off on indexing designs until the file hasn't been edited for four hours. Two wins: (a) unfinished work stays out of search results; (b) reduced system load. Wall-clock-based freshness heuristic that trades a few-hour staleness window for both quality and load. Canonical edit-quiescence shape.
  5. Relevance across similarity tiers is the product bar. Designer queries span exact specifics ("project [codename] theme picker"), generic descriptive ("red website with green squiggly lines"), and broad categorical ("checkout screen"). Research finding: users prefer to start with something closer or more similar, even when ultimately seeking diverse results. The derived product rule: "if we couldn't prove we could find the needle in the haystack, designers wouldn't trust the feature for broader exploration." Canonical similarity-tier retrieval framing.
  6. Eval grading on Figma's own infinite canvas via the plugin API. Figma built a human-labeling tool inside Figma itself using the public plugin API: results displayed on an infinite canvas, with keyboard shortcuts for rapid correct / incorrect marking, and historical views showing whether the search model had improved between runs. The eval set was seeded from internal-designer interviews + file-browser usage analysis. Canonical visual eval-grading surface — the evaluation UI is itself the product's own data model.
  7. Workflows don't fit a linear model; offer results across the spectrum and let the user pick. Designers oscillate between exploration mode (brainstorm, breadth) and execution mode (find a specific design, speed). Rather than trying to detect the mode, the design shipped with a unified refinement interface (created-by filters, file/author/recency metadata) that works across both + input types. A "rabbit holing" deeper-dive interaction was explored and scrapped for simplicity.
  8. RAG-framed autocomplete as the downstream dependency. The rationale for investing in search before autocomplete: RAG — "if design autocomplete could find designs similar to what a designer was working on, it could better suggest the next component." Search quality caps autocomplete quality via the same chain relevance labels → ranker → RAG output that Dropbox Dash articulates (see concepts/relevance-labeling). The search foundation is an enabling condition for the future autocomplete ship.
  9. Surface constraints drove shipping decisions. The Actions panel (a single home for AI features) was the right launch surface for consistency but offered less space for search results than the file browser. Response: added peek (quick previews in the narrow surface) and CMD+Enter full- screen drill-down. Surface-scoped UI trade-offs baked into the product, not just research output.
  10. 75% of Figma-canvas objects come from other files — the foundational usage datum. Research-observed, product-defining statistic that reframed the feature from "nice-to-have navigation improvement" to "core workflow leverage point." Designers don't start from scratch; they riff, revisit, and pull-in existing work. The statistic legitimises the engineering investment.

Reported results / numbers

  • 75% of all objects added to the Figma canvas come from other files (the foundational user-research datum motivating the investment).
  • Hackathon: June 2023, 3 days, 20 completed projects including a working design-autocomplete prototype.
  • Edit-quiescence threshold: 4 hours of no-edits before indexing.
  • Shipping: first launched publicly at Config 2024.

No latency, throughput, corpus-size, embedding-model-family, index-size, NDCG / recall / MRR numbers, or cost numbers are disclosed in the article.

Caveats

  • Quality signals section is explicitly experimental. "We're still experimenting with different approaches. One direction we're testing is creating 'quality signals,' like whether a frame is marked ready for development." Don't cite this as a production-proven mechanism.
  • No architecture diagram, no embedding-model identity, no vector- store identity. The post frames problem/solution/research, not infra. Concrete systems-level questions (what embedding model powers visual search? what ANN index? how large is the corpus?) are unanswerable from this source.
  • Actions panel layout constraint is a UI concession, not a backend constraint. Peek + CMD+Enter are presentation-layer responses to Actions-panel width, orthogonal to retrieval quality.
  • "Reverse image search" cited as inspiration, not claimed as implementation. The post leans on established AI research lineage for credibility but does not disclose whether the visual-search implementation matches that lineage in technique (e.g. CLIP-style visual embeddings vs specialised model). Treat reverse image search as framing, not mechanism, for citation purposes.
  • Relationship to autocomplete is aspirational. Autocomplete was not shipped — it was the initial hackathon project that motivated the search investment, then deprioritised in favour of search. The source doesn't claim autocomplete is live; its future ship depends on the search foundation.

Introduces / extends

New wiki pages introduced by this source:

  • systems/figma-ai-search — Figma's AI search feature combining visual (query-by-image) and semantic (query-by-text-understanding) search over Figma frames; shipped at Config 2024.
  • patterns/selective-indexing-heuristics — reduce search-index corpus via heuristic filters on what qualifies as indexable (UI-frame dimensions + duplicate collapsing + file-copy skipping); Figma's canonical instance.
  • patterns/edit-quiescence-indexing — delay indexing until a wall-clock no-edit window elapses (Figma: 4h); trades bounded staleness for quality + load reduction.
  • patterns/visual-eval-grading-canvas — build the human-labeling UI inside the product itself using its plugin/extension API; Figma's infinite-canvas + keyboard-shortcut eval tool is the canonical instance.
  • concepts/similarity-tier-retrieval — the product bar that search must deliver high-quality results across similarity tiers (exact → near → diverse); users start from close and expand outward, so failing the needle-in-haystack case destroys trust in the broader- exploration case.

Extends:

Citation anchors

  • "75% of all objects added to the Figma canvas come from other files."
  • "we can improve AI outputs with examples from search" (RAG framing applied to design autocomplete).
  • "we knew we needed … a focused approach that would deliver value while setting us up for future features like autocomplete."
  • "we couldn't index and search everything — it would be too costly."
  • "One quick fix was to hold off on indexing designs until we knew that a designer was done tinkering; we'd only index once a file hadn't been edited for four hours."
  • "if we couldn't prove we could find the needle in the haystack, designers wouldn't trust the feature for broader exploration."
  • Four shipping principles: AI for existing workflows / Rapid iteration / Systematic quality checks / Cross-disciplinary teamwork.

Source

Last updated · 200 distilled / 1,178 read