
Stop Telling Neural Networks They Are Senior Developers 🛑
You probably all use one of the basic rules of prompt engineering: start your query with a phrase like "Imagine you are a Senior Python Developer with 100 years of experience at Google." Yes, I do that too.
Well, a fresh study from USC has been released, proving that such phrases make the model dumber at writing code.
Why is that? 🧐
Raw knowledge (facts, algorithms, logic) is embedded into the model during pretraining. But the ability to role-play, follow a format, and be safe is the result of instruction-tuning (SFT/RLHF).
When you write
You are an expert..., you shift the model into a mode of strict instruction following. Its internal weights lean towards maintaining this mask (alignment tasks). The model spends "computational resources" on sounding like an expert instead of thinking like an expert. Interference occurs: instruction-following patterns suppress the neural pathways responsible for extracting pure facts from pretraining.Numbers from the paper: on benchmarks like MMLU and coding/math tasks, the base model with a bare prompt consistently outperforms the model with a persona attached (68% vs 71.6% accuracy). The neural network literally gets dumber in logic while trying to generate confident, beautifully structured bullshit. This effect hits reasoning models like DeepSeek-R1 especially hard, breaking their chains of thought.
Does this mean personas are completely unnecessary? No.
The study clearly distinguishes tasks:
1️⃣ Where persona harms (Pretraining-dependent tasks):
Coding, math, extracting raw facts, logic puzzles.
How to do it: Give bare context and a clear task. No "you are an expert."
2️⃣ Where persona works (Alignment-dependent tasks):
Writing texts, formatting (collecting data into a JSON of a certain structure), tone of communication, and, oddly enough, safety (refusing to write exploits).
How to do it: Here
You are a strict critic or You are a technical writer actually improves the structure of the response.For those who want to dig deeper: the researchers even put together a crutch called PRISM — a LoRA adapter with a gate that dynamically enables the persona for formatting and disables it when it comes to hardcore coding and facts.
In short, now we know that if you need working code or a solution to a complex architectural problem — stop convincing the machine that it's a genius. Just write the specification properly.
Comments
0No comments yet.
Sign in to join the discussion.