CounselKit

Why Your AI Gives Generic Legal Output (And How to Fix It)

March 23, 2026 · CounselKit

You paste a lease agreement into ChatGPT and ask it to identify the key issues. It returns five paragraphs about how leases generally contain provisions regarding rent escalation, maintenance responsibilities, and termination clauses. It reads like a law school textbook entry, not a review of your specific document. The analysis could apply to literally any lease on earth.

This is the generic output problem, and it's the number one reason lawyers conclude that AI "doesn't work" for legal tasks. But the AI isn't failing — it's doing exactly what you asked it to do. You asked a general question, so it gave a general answer. The fix is understanding why this happens and structuring your prompts to prevent it.

The three reasons AI defaults to generic output

Reason 1: It doesn't know who you are. When you open a new ChatGPT or Claude conversation, the AI knows nothing about you. It doesn't know you're a lawyer. It doesn't know which side of a transaction you're on. It doesn't know your risk tolerance, your client's priorities, or what "standard" means in your practice area. Without this context, it produces output calibrated for the broadest possible audience — which means it's useful to no one in particular.

Reason 2: It doesn't know what "review" means to you. When a partner says "review this contract," you know from years of experience that they mean: identify the provisions that deviate from your firm's template, flag anything that creates unusual exposure for the client, and organize your findings by importance. When an AI hears "review this contract," it has no comparable frame of reference. It defaults to summarizing provisions — which is the safest, most generic thing it can do.

Reason 3: It's trained to avoid risk. Language models are optimized to produce responses that are helpful across the widest range of contexts. For legal analysis, this means hedging, generalizing, and avoiding specific claims that might be wrong. The AI would rather give you a vague summary that's technically accurate than a pointed analysis that might miss the mark. You have to explicitly override this instinct.

The fix: four elements that eliminate generic output

Every effective legal AI prompt contains four components. Miss any one of them and the output degrades toward generic. Include all four and the output becomes genuinely useful.

Element 1: Role — tell it who it is

Before asking any question, assign the AI a specific professional role. This isn't a gimmick — it fundamentally changes how the model weights its response.

Without role "Review this NDA."
With role "You are a senior associate at a mid-sized firm. I represent the disclosing party. Review this NDA from our perspective, flagging provisions that give the receiving party more latitude than market standard."

The role assignment does three things at once: it sets the expertise level (senior associate, not a general assistant), it establishes perspective (disclosing party), and it defines what "standard" means (market standard, not some abstract baseline). All of this context was previously missing, and the AI was silently making assumptions about each one.

Element 2: Context — brief it like a colleague

Think about how you'd brief a junior associate on a task. You wouldn't hand them a 40-page agreement and say "review this." You'd tell them the deal structure, who the parties are, what's been negotiated already, what's still open, and what specifically you need them to focus on.

The AI needs the same briefing. Load context before asking questions: the situation, the jurisdiction, the constraints, what's in scope and what isn't. The structured context loading approach takes about 60 seconds of typing and prevents rounds of generic back-and-forth.

Element 3: Constraints — tell it what NOT to do

This is the most counterintuitive element, and it's the one most people skip. Telling the AI what to avoid is often more effective than telling it what to do.

No constraints Output includes fabricated "industry standard" benchmarks, cites nonexistent cases, blends quotation with analysis so you can't tell which is which.
With constraints "Do not cite cases or statutes I haven't provided. Do not reference 'industry standard' or 'market practice' unless I've given you comparison documents. Distinguish between what the document says (quote it) and your analysis (label it)."

Constraints work because they close off the AI's escape routes. Without them, the model fills uncertainty gaps by inventing plausible-sounding content — fabricated citations, imagined market standards, confident characterizations of provisions it hasn't fully analyzed. Constraints force it to flag uncertainty instead of hiding it, which is exactly what you need for reliable legal analysis.

Element 4: Output format — specify the deliverable

The default AI output format is a wall of prose paragraphs. This is almost never what you need for legal work. Specify the exact structure you want and the AI will follow it.

Example format specification
Format your analysis as: EXECUTIVE SUMMARY (3 sentences maximum) KEY ISSUES (organized as a table) | Issue | Contract Language (quoted) | Risk Level | Recommended Action | MISSING PROVISIONS (list of provisions you'd expect to see but don't) OPEN QUESTIONS (factual questions I need to resolve before this analysis is complete)

This format specification does something subtle but important: the "Open Questions" section gives the AI a sanctioned place to put its uncertainty. Without it, uncertainty gets hidden inside confident-sounding prose. With it, the AI surfaces questions rather than making silent assumptions — which makes the entire output more trustworthy.

What this looks like in practice

Putting all four elements together, a complete prompt for contract review might be 200-300 words before you paste the document. That feels like a lot compared to "review this contract." But the math strongly favors the structured approach: 60 seconds writing a thorough prompt versus 20 minutes correcting generic output — or, worse, missing something because the AI glossed over it with a vague summary.

The structured prompt doesn't produce a finished work product. It produces a first-pass analysis that's organized the way you need it, grounded in the actual document language, and honest about its limitations. That's the right level of trust for AI in legal work: a capable tool that needs your judgment, not a replacement for it.

The reusability advantage

Once you build a structured prompt that works for a particular task — contract review, research memo, client communication — you don't write it from scratch every time. You save it and reuse it, filling in the brackets with the specifics of each new matter. The upfront investment of 10 minutes building a good prompt template pays off across dozens or hundreds of uses.

This is the real shift in how lawyers will use AI going forward. Not typing ad hoc questions into a chat box, but maintaining a library of structured, tested prompts for each recurring workflow. The lawyers who build those libraries first — or acquire them — will have a meaningful productivity advantage over those still asking vague questions and getting vague answers.


CounselKit is a library of 24 structured prompts for legal workflows — contract review, research, analysis, and drafting. Each prompt includes all four elements: role, context, constraints, and output format, with anti-hallucination guardrails built in. Works with ChatGPT, Claude, and any AI tool.

Get CounselKit — $49