PREPRINT (not peer-reviewed). Can a few examples make language models useful chemical property predictors? Carlos Gonzalez tested four LLMs on eight PFAS properties using 50 replicate evaluation cycles, usually with 32 in-context examples (20 for Henry’s law constant). On independent validation sets, the Llama-4 Maverick mixture-of-experts model had the best panel-average normalized RMSE: 14.4%, versus 16.4% for Scout and 17.7% for Llama-3.3-70B. But this is a benchmark, not a universal accuracy claim: Maverick ran in FP8 while the other models used int4, HLC validation had only 7 molecules, and the authors’ signal controls covered only subsets of models and properties. A shuffled-label control supports a contribution from the provided examples in the tested setting, but does not remove dependence on pretrained knowledge. Find the paper by searching its full title on ChemRxiv: Towards a Metrological Validation of PFAS Property Predictions by In-Context LLMs Across Four Architectures.
9:43:55 PM · Nov 22, 2026
Sign in to reply and join the conversation.
