Skip to content

DATABRICKS 2026-10-16 Tier tier-3

Read original ↗

Closing the Binary Blind Spot: Domain-Knowledge Injection for LLM Malware Triage in npm Packages

Summary

Databricks' security research team built a real-time scanner that monitors the npm registry and triages candidate malware through a static stage followed by an LLM verdict. Studying 11,413 scanned package-versions corroborated against OSV advisories, they found that bundled binaries — 3.8% of corroborated malware (432 package-versions) — are both the rarest and the hardest vector for the pipeline: binary-bearing packages fail to reach a decisive malicious verdict more than twice as often as source-only ones (14.4% vs 6.2%). A stripped, opaque binary wrapped in a facade of clean source, professional metadata, and a node-gyp-build install script skews budget LLMs toward a benign verdict. The post evaluates nine models over ten crafted native-addon payloads at varying obfuscation, names four systematic failure modes (rationalization, dismissal, hedging, confabulation), and closes most of them with capability profiles — pre-computed, weighted summaries of a malicious technique injected into the prompt (~1,800 tokens) before the model reasons. With profiles, budget-tier escalation-threshold failures fall from 50% to 4%, no model returns a benign verdict, and a cheap model matches the detection accuracy of a model costing ~3× more.

Key takeaways

  1. Binaries are the triage blind spot. Of 11,413 scanned/corroborated package-versions, 96.2% hide the payload in source or fetch it from a server; only 3.8% (432) bundle a binary — yet binary-bearing packages miss a decisive malicious verdict 14.4% vs 6.2% of the time versus source-only malware. An adversary can assume the target already has the JS runtime, so shipping a compiled artifact is costlier for them but disproportionately effective at evading triage (Source: this article).
  2. The gap is contextual, not isolated-reasoning capability. Every one of the 35 budget-tier detection failures leaned on ≥2 benign signals and 24 on four at once: the node-gyp-build install script called "standard" (35/35), native-addon .node packaging (35/35), metadata "consistent with" stated purpose (32/35), clean lib/index.js (21/35), and the binary's own imports recast as "normal" (14/35). No single artifact misleads the model; the clean wrapper does (Source: this article).
  3. Four systematic failure modes, not random error. Rationalization (~37%, six models): contextual cues override binary signals — Haiku rated an unobfuscated stealer MALICIOUS 0.98 but dropped to SUSPICIOUS once strings were XOR-encoded (obfuscation inverts the expected severity relationship). Dismissal (22%, four models, almost entirely open-weight): a benign verdict without engaging the binary at all — Qwen3-Next returned BENIGN 0.98 on 9 of 10 payloads and recommended allowlisting a reverse shell. Hedging (16%): never escalating past SUSPICIOUS — GPT-5.4-nano sat at SUSPICIOUS 0.55–0.62 on 8 of 10, which passes a block-at-malicious policy. Confabulation (~23%, six models, both baselines included): correct verdict, fabricated justification — Gemini invented "a decryption routine at address 0x401490" on payloads that contain no cipher (Source: this article).
  4. Capability profiles close most of the gap. A profile is a declarative playbook encoding one technique (credential harvesting, persistence, binary masquerade) as weighted conditions over the scanner's static evidence; when a suspicious/malicious/critical threshold is reached, the engine injects a ~1,800-token structured summary into the prompt before the model forms its verdict (and may override it). With profiles, 32 of 35 no-profile failures (91%) recover, no model returns benign, escalation-threshold failures fall 50% → 4% (3 of 70) and the malicious-bar failures 73% → 34% (Source: this article).
  5. A cheap model + profiles matches a 3×-costlier model. The two baselines (Sonnet 4.6, Kimi K3) detected every payload without profiles at roughly 3× the per-scan cost of the budget tier; with profiles the budget tier matches that accuracy at the lower price. The clearest single case: Qwen3-Next flipped from BENIGN 0.98 (allowlist) to MALICIOUS 0.98 (block) on byte-identical input after four profiles fired (Source: this article).
  6. Majority voting does not rescue this. On the obfuscated sysmon stealer v2, nine verdicts split 3 benign / 2 suspicious / 1 highly-suspicious / 3 malicious — the majority sits below the detection threshold, so the vote returns the wrong answer. The failure modes compound rather than cancel; this matches known limits of majority voting under correlated LLM errors (Source: this article).
  7. Test-set selection is a methodology trap. A test set of a few unobfuscated / low-ambiguity packages makes a budget model look sufficient while hiding its failure on the obfuscated credential stealer — and obfuscation recurs in real malware (the Binary String Obfuscation profile fired on 74 of 425, 17%, of OSV-confirmed binary packages). Testing only high-ambiguity payloads has the opposite effect, condemning a serviceable model (Source: this article).
  8. Production validation. Against 425 OSV-confirmed binary-bearing package-versions and a 2,449-version benign corpus (native binary, >1M monthly downloads, never in OSV), the scanner reaches precision 0.77, recall 0.81, F1 0.79 at the escalation threshold (TP 345, FN 80, FP 104, TN 2345). A binary capability profile drove 74% of detections. False positives are 4.25% of benign package-versions, concentrated in 6 of 287 distinct packages (2%) (Source: this article).
  9. Inference-time knowledge, not fine-tuning. Prior work instruction-tunes open models on expert data, baking knowledge into the weights — which can't reach closed hosted models and needs retraining as threats shift. Databricks supplies the expertise at inference as pre-computed static evidence in the prompt, so the same profiles improve both budget and frontier models and both open and proprietary ones (Source: this article).

Architecture / mechanism

The scanner pipeline (defense-in-depth, static-before-LLM). A real-time registry monitor runs a static stage first — YARA over the tarball, Semgrep plus the npm registry API over the JavaScript and metadata, and a binary-analysis module over the compiled artifact — then assembles a structured prompt. The LLM returns a verdict with reasoning and may request additional files over up to three rounds. The static tools are deterministic gates; the LLM is the judgment layer on top (this is a concepts/defense-in-depth layering and a patterns/deterministic-tool-vs-llm-judgment split).

Capability-profile injection. A capability profile is a declarative playbook that encodes one malicious technique as weighted conditions over the static evidence, with its own suspicious / malicious / critical thresholds (a scale separate from the model's verdict). Flow: (1) static stage extracts evidence; (2) each profile scores that evidence; (3) when a threshold is reached, the engine injects a ~1,800-token structured summary into the prompt before the model reasons; (4) the model forms its own verdict over the combined evidence and may override a firing profile. This is a concrete instance of inference-time domain-knowledge injection — the prompt is engineered to carry pre-computed expert evidence rather than relying on model weights. Named profiles include T001 Credential Harvesting, T002 Network Exfiltration, T003 Environment Reconnaissance, T004 Binary Masquerade, T006 Reverse Shell, T007 Dropper/Stager, and a Binary String Obfuscation profile.

Adversarial payloads. Five native-addon (.node) payload types mirroring collected malware: sysmon-native (credential stealer, HIGH ambiguity — fopen, connect, send, gethostname), buildchain-native (dropper/backdoor, LOW — execve, connect, chmod, socket, fwrite), netpulse-native (reverse shell, LOW — execve, dup2, connect, socket), hashworker-native (cryptominer stub, MEDIUM — pthread_create, connect), taskd-native (persistence, MEDIUM — system, fopen, fwrite, chmod). The sysmon stealer spans a six-step obfuscation ladder (v1 readable → v6 symbols stripped + strings XOR-encoded); single-byte XOR was chosen deliberately because, unlike RC4, it leaves no structural markers in disassembly. Each payload wraps the binary in a professional package.json, README, JS bindings, and binding.gyp.

Models and protocol. Nine models across seven providers, held at temperature 0 with identical system/analysis prompts and an 8192-token output cap; each payload scanned twice (without and with profiles) on byte-identical input. Budget tier: Claude Haiku 4.5, Gemini 3.5 Flash, Gemma 3 12B, GPT-5.4-nano, Qwen3-Next 80B, GLM 5.3 Flash, Llama 3.1 8B. Behavior baselines: Claude Sonnet 4.6, Kimi K3. Cost reference: Claude Opus 4.6.

Operational numbers

  • Corpus studied: 11,413 scanned/corroborated package-versions; 96.2% source/remote-fetch payloads vs 3.8% (432) bundled binaries.
  • Decisive-malicious miss rate: 14.4% binary-bearing vs 6.2% source-only.
  • Budget tier without profiles: 35/70 (50%) failures at escalation threshold (20 outright benign), 51/70 (73%) short of the malicious bar.
  • Budget tier with profiles: 4% (3/70) escalation-threshold failures, 34% at malicious bar; 91% (32/35) of no-profile failures recovered; 0 benign verdicts.
  • Failure-mode prevalence (90 scans, 9 models): rationalization ~37%, dismissal 22%, hedging 16%, confabulation ~23%.
  • Token cost of profiles (~1,800-token block): budget-tier input +9% to +34%; Opus input −32% (10.5K → 7.2K, it stops requesting extra files); Kimi output −13%. Baselines ran ~3× the budget per-scan cost unaided.
  • Production malicious set: 470 OSV-confirmed binary-bearing npm package-versions / 604 binaries — PE 273, ELF 215 (all stripped), Mach-O 113, 3 unidentified; by compiler 363 C/C++, 183 Go, 58 Rust (241 Go/Rust = 40%); 50 UPX-packed (all ELF). Evaluation subset: 425.
  • Detection at escalation threshold: P 0.77 / R 0.81 / F1 0.79 (TP 345, FN 80, FP 104, TN 2345); a binary capability profile drove 74% of detections.
  • Benign set: 2,449 package-versions / 287 distinct packages; 4.25% FP rate on package-versions, 2% on distinct packages; hard-miss rate 4.7% (20 of 425 cleared benign).

Caveats

  • Tier-3 vendor security-research post. Included because it describes a real production detection pipeline (static + LLM + capability-profile injection), an at-scale OSV-corroborated corpus, failure-mode analysis, and production precision/recall — well past the 20% architecture-content bar. The pipeline internals are sketched, not fully specified.
  • Verdicts reflect one serving epoch. Hosted models sit behind continuously updated endpoints, so absolute verdicts may drift; the authors rest their conclusions on the within-epoch delta between no-profile and with-profile conditions on byte-identical input, not on any single verdict.
  • Recall is scored against OSV package-level labels, which mark a whole package malicious without isolating the responsible artifact. Of the 20 cleared "misses," about half bundle a recognizable third-party tool unchanged (ripgrep, node-pty, NSIS plugins) where OSV attributes the malice to the install/JS layer; binary-level detection could be higher or lower than the headline F1.
  • A correct verdict is not a correct label. Budget models often flagged maliciousness while assigning the wrong technique category, so the stated category/evidence are triage hints, not attribution ground truth. Confabulation means logged justifications may be fabricated even when verdicts are right — poisoning any later analyst review or model fine-tuned on them.

Systems, concepts & patterns extracted

  • Systems named (tools/models, no page per Systems gate — listed in frontmatter, plain text here): YARA, Semgrep, OSV / OpenSSF Malicious Packages, npm, node-gyp-build, UPX, and the evaluated models (Claude Haiku 4.5 / Sonnet 4.6 / Opus 4.6, Gemini 3.5 Flash, Gemma 3 12B, GPT-5.4-nano, Qwen3-Next 80B, GLM 5.3 Flash, Llama 3.1 8B, Kimi K3). The scanner itself is an unnamed internal Databricks research system — described in prose, no page.
  • Concepts: concepts/llm-hallucination (confabulation is the named worked case), concepts/llm-as-judge (the LLM renders the verdict), concepts/context-engineering (capability profiles = inference-time domain-knowledge injection into the prompt), concepts/defense-in-depth (static YARA/Semgrep/binary-analysis gates before LLM triage), concepts/benchmark-methodology-bias (test-set selection hides or condemns a model), concepts/fail-open-vs-fail-closed (block-at-malicious policy lets hedged SUSPICIOUS verdicts through). supply-chain-attack recorded as a tag/ prose only — first source explicitly on it; a Lint-promotion candidate at ≥2 sources.
  • Patterns: patterns/cheap-approximator-with-expensive-fallback (budget model + injected profiles matches a frontier model's accuracy — the cost framing of cheap-does-the-bulk), patterns/deterministic-tool-vs-llm-judgment (static deterministic tools feed an LLM judgment layer), patterns/human-calibrated-llm-labeling (verdicts graded against OSV ground-truth labels). capability-profile and domain-knowledge-injection-at- inference recorded as tags/prose, not minted — article-specific implementation of context-engineering, Lint-promotion candidates at ≥2 sources.

Source

Last updated · 854 distilled / 2,448 read