Same Local Model, Opposite Settings: Configuring an LLM Per Task

A few days ago I was playing with a local model (Gemma 4) for various tasks, coding among them. I hit all kinds of hangs and glitches, and in general the model underperformed at first. I turned to my good friend Claude, we went over the model configuration together, and we found several settings that were causing all the trouble.

The big conclusion: choosing a suitable model (size, training base, quantization) is not enough. The configuration can sink you in just as many places. Task X needs configuration X, and task Y needs configuration Y, on the exact same model. This is the long version of how I configure the same model differently per job, and why coding and an "opportunity scout" need almost opposite settings.

This post is part of a local LLM series: the coding capability test, the LM Studio setup, and where local models actually earn their keep.

First: hygiene that applies to every task

When loading the model you can choose the context size. Naturally I dragged the slider all the way, because why not? Well, no.

Do not load context length to max. It chokes prefill, inflates the KV cache, and hands out hangs left and right. Most tasks are fine on 32K to 64K. Prove to yourself that you need more first.
Keep maxInputTokens below the context size, otherwise your prompt gets silently truncated and you will never know.
Cap the output. Without a ceiling, a loop can run forever.

What actually changes between tasks?

Mostly two axes: whether you are generating something (code, text) or extracting (classification, extraction), and the price of hallucination, meaning how expensive a confident invention is for you. These two dictate most of the parameters.

The sharpest example of the gap: a coding model versus an opportunity scout. The exact same Gemma 4, with nearly opposite settings:

Setting            Coding (generate)     Scout (extract + judge)
Temperature        ~1                    0.1 - 0.3
Structured output  Off                   On
Output cap         8K - 16K              256 - 512
Thinking           Off                   Off (On only for final judgment)
Grounding          Not relevant          Mandatory

Why is coding "open"?

Because it is generation. You want variety, the output is immediately verifiable (the code runs or it does not), and invention is cheap: you will see the bug. So you open the tap: high temperature, large output, no grounding. Bonus fact: on Gemma 4, lowering the temperature actually hurts code quality.

Why is the scout "closed"?

Because it is extract-and-judge against real data. The failure mode is the model inventing an opportunity that does not exist in the posts, and high temperature only amplifies that. So you tighten everything: low temperature, structured output, and grounding, where every opportunity must cite a source post.

And the scout is not one configuration at all. It is a funnel:

Filtering (high volume): classification. Low temperature, thinking off, fast and cheap
Extraction with grounding: low temperature, a schema with a mandatory source_evidence field
Judging the survivors (low volume): here you can afford thinking and medium temperature, but tied to evidence, not free-form

The real lever for the scout is grounding, not temperature. Block generation when there is no source, and always verify the citation actually supports the claim. Citation hallucination survives even when overall accuracy improves.

Do you need 20 configs? No: 3 or 4 archetypes

I also tried email classification, something similar to extract but without the heavy grounding, because a misclassified email is cheap. Again: the task decides.

But "a config per profile" does not mean hand-tuning 20 models. The profiles cluster into 3 or 4 archetypes: generate, extract, and staged judgment. Almost every new profile falls into one of them. Define a preset per archetype once and point at it.

One last point people miss: sometimes what needs to change is not the settings but the model itself. A high-volume filtering task runs better on a small fast model. A judgment task needs a bigger one. A different profile sometimes means a different model, not just different knobs. This mapping of tasks to models and configs is exactly the kind of design work I do in custom agent projects.

Bottom line

There is no single config. It derives from the task at least as much as from the model, and the fact that something does not work does not necessarily mean the model is the problem. In the end you have 3 or 4 archetypes, not 20 snowflakes.

FAQ

Should I set my local model context length to the maximum?

No. Maxing the context slider chokes prefill, inflates the KV cache, and produces hangs. Most tasks run well on 32K to 64K. Prove you need more before raising it, keep maxInputTokens below the context size to avoid silent truncation, and always cap output so a loop cannot run forever.

Why does temperature depend on the task?

Coding is generation: you want variety, the output is instantly verifiable because the code either runs or does not, and an invention is cheap to catch. So temperature stays high. Extraction against real data is the opposite: the failure mode is the model inventing something that is not in the source, so you run low temperature, structured output, and mandatory grounding.