Fine-tuning a 350M Model for Better Structured Outputs in 100 GRPO Steps

This guide is a fully public, inexpensive recipe for making a small model substantially better at structured-output compliance. We fine-tune LFM2.5-350M with Group Relative Policy Optimization (GRPO) using the TRL library and evaluate it on the IFStruct benchmark . The full run takes around 500 samples and 100 training steps, small enough for a free-tier Colab or Kaggle GPU, and is available on GitHub . The results show that even a light fine-tuning procedure improves performance from 22.6% to 29.7% on the IFStruct benchmark.

Structured output is one of the most common real-world tasks for LLMs, yet most benchmarks fold it into broader reasoning or extraction scores rather than measuring it on its own. Whether a model reliably returns valid, parseable output in the requested format and shape — schema compliance — is often what decides whether it can be wired into a downstream system at all.

Note that the training pipeline described here is not the one used to train the RL model described in the IFStruct blog . This notebook doesn't aim to recreate the IFStruct benchmark score, but to show how task-specific fine-tuning of smaller models can improve performance and match that of far larger models.

We use nvidia/Nemotron-RL-instruction_following-structured_outputs , which pairs each prompt with a target JSON Schema and an expected field count. We use about 500 samples for training.

We load LiquidAI/LFM2.5-350M and attach a LoRA adapter. Because LFM2.5 uses a hybrid attention/convolution architecture, we target the LFM-specific module names:

Finally, we merge the LoRA adapter back into the base weights and save it as a single self-contained checkpoint, ready to convert to GGUF for serving:

A short GRPO run with about 500 samples and 100 steps can lift a small 350M parameter model from 22.6% to 29.7% on IFStruct. The takeaway is that a cheap, task-specific reward signal can make a small model substantially more reliable about form , closing much of the gap to models several times its size.

We will need uv for the Python tooling and llama.cpp for serving. Following the Liquid AI llama.cpp deployment docs , install llama.cpp with Homebrew and verify that llama-server is available:

Before we begin, let's evaluate LFM2.5-350M on the IFStruct benchmark and see whether we can reproduce the reported score of 21.1% .

IFStruct is a benchmark for testing the validity of LLM outputs and schema adherence. The benchmark is open-source in Liquid4All/ifstruct , with the public benchmark dataset available on Hugging Face at LiquidAI/ifstruct-v1.0 .