Paper Tests Wildfire Tasks Across Six Core Language Models

Large language models are being tested for wildfire work, and a new paper called AI4Fire tries to measure how well they actually do. Posted to arXiv on October 7, 2026, the paper runs six core models on five wildfire tasks, zero-shot, each one tested twice: once bare, with only the task input, and once grounded, with one task-specific addition, such as a smoke-free reference frame from the same camera for smoke detection. A wider sweep adds 29 more models on top of the six core ones. The authors say their own search of fire-related work turned up 138 prior papers, and none paired this roster of models with this task coverage and matched bare-versus-grounded runs.

The paper reports three findings. First, grounding mattered most where the extra information effectively contained the answer: giving a read-only SQL tool to query a database lifted every core model’s database accuracy from as low as 16 percent up to at least 88 percent. That is less a reasoning improvement than a demonstration that the model could read a result once it was handed one.

Second, simple rules were hard to beat elsewhere. No core model did better than a naive forecast built on the current staffing figure, and two open-weight models mostly just copied the median of similar past fire-days instead, which the authors call a worse forecast than the simple rule.

Third, the paper flags hazards in public wildfire datasets themselves: a fire-danger column in one release separates the holdout set perfectly, which can make a model look far more capable than it is, and 67 aerial fire frames carry smoldering or fire-free labels that were read off a clipped thermal maximum rather than checked directly.

The numbers

The authors reviewed 138 existing works on fire-related tasks before building their own benchmark, then tested six core models before expanding to a sweep of 29 further models. Database accuracy, bare, ran as low as 16 percent and reached at least 88 percent once a read-only SQL tool was attached. Separately, 67 aerial fire frames were found to carry labels derived from a clipped thermal maximum rather than ground truth, a detail the authors flag as a hazard for anyone training or evaluating on that public data.

The authors say they are releasing prompts, responses, scores, code and the survey record alongside the paper, which lets other researchers rerun the same bare-versus-grounded comparison on new models as they appear.