Paper Shows Portable Skill Matches Complex RAG Using Claude Code

A paper posted to arXiv on October 10, 2026, by a team including Istok Bako, Bonan Min, Tiffany Shao, Claire Cheng and Yashar Mehdad, takes aim at a standard assumption in enterprise AI: that pulling the right facts out of a large corpus of standard operating procedures and policy documents requires a heavy pipeline of document chunking, indexing, learned re-rankers and query rewriters. The team reports a deployment case study where a much simpler construction did the same job for customer support resolution workflows.

That construction is what the paper calls a portable skill: a natural-language instruction, a small set of tools including a retrieval API and format-checking scripts, and a corpus enriched with an LLM-generated navigation layer the authors describe as a structured, “Wikipedia-style” index over the underlying documents. The paper reports it is straightforward to build and that it carried over across different agentic harnesses, including Claude Code, without the engineering investment a bespoke retrieval system usually demands.

On the specific task the team measured, generating the correct sequence of resolution steps for a customer support case, the skill reached 99.2 percent procedural equivalence against ground truth that had been approved by human specialists, and 83.3 percent evidence sufficiency. The authors are candid about where the approach loses: it ran roughly an order of magnitude slower per case than their existing production assistant, closer to ninety seconds against eight or nine seconds. They state plainly that they are not claiming a latency advantage, only an accuracy and maintenance one.

The paper scopes its own claim tightly: a mature, well curated document corpus, a workflow that can tolerate the extra seconds, and a human reviewing every case before it ships. Inside that scope, the authors say maintaining the system became a matter of editing text and data rather than retraining models, which is the practical gain they are arguing for.

The numbers

• 99.2 percent procedural equivalence to human-approved ground truth on the resolution workflow task.
• 83.3 percent evidence sufficiency, the paper’s measure of whether the retrieved material actually backed the generated steps.
• 1-2 engineer days reported to build the portable skill, against the multi-component pipelines it replaced.

For a team weighing whether to build another layer of rerankers and rewriters, the paper’s honesty about the speed penalty matters more than its accuracy numbers. A support desk with tight response targets gains nothing from a method that trades nine seconds for ninety, however strong its evidence sufficiency score, so the real decision the paper hands a reader is whether their workflow can afford to wait for a simpler system to be right.