Paper Finds Generative Recommenders Vulnerable to Low-Volume Poisoning

Generative recommenders have mostly been studied for how flexibly they model a user’s history, not for how easily that history can be tampered with. This preprint asks the second question directly: if an attacker can only touch a small slice of the interaction log and otherwise treats the recommender as a sealed box, how far can they push it toward recommending something it otherwise would not?

The team’s answer is further than expected. Working against both single-mode and multi-mode generative recommendation setups, they built a framework called AlignPoison that searches a recommender’s own outputs for short prefixes, the handful of interactions that come right before a trigger event, which the model already treats as common and strongly tied to that trigger. Planting a few of those prefixes was enough to make a chosen target item far more likely to appear once the trigger showed up again in a real user’s session, even though the attacker never saw the model’s parameters or training process.

The study frames the weakness as decoding dependent rather than data dependent in the usual sense. Older ranking systems can be poisoned too, but the paper’s authors argue that autoregressive decoding gives an attacker a sharper lever: a prefix that aligns with how the model predicts the next interaction does more damage per injected row than an attacker could get from simply flooding a target with generic fake clicks.

The numbers

Two figures anchor the claim. The attack worked with fewer than 1% synthetic interactions added to the pool, a share small enough to hide inside normal data churn at most platforms. Against competitive poisoning baselines, the crafted prefixes pushed attack success rate up by as much as 16.3 percentage points, a gap the authors attribute to matching the prefix to the model’s own induced transition pattern rather than to brute volume.

The paper does not name a deployed product or vendor, and the authors describe their tests as benchmark experiments rather than an attack carried out against a live service. That leaves an open question for anyone operating a generative recommender in production: whether existing anomaly detection built for ranking-style poisoning would catch a prefix this short and this well aligned with the model’s own decoding behavior, or whether it would need a different kind of check altogether.