Verified Patch Generation: Why AI-Suggested Solidity Fixes Fail to Compile (and How to Fix It)

Ask any modern AI to fix a reentrancy bug in your withdraw() function and it will give you a confident, polished patch. Paste that patch into your codebase, run forge build, and roughly one third of the time you will get a wall of errors:

Error: Source "@openzeppelin/contracts/utils/cryptography/MessageHashUtils.sol" not found
Error: Wrong argument count for function call: 0 arguments given but expected 1.
Error: Identifier not found or not unique. Counters.Counter

The patch is structurally sensible. The problem is that the LLM made it for a different version of OpenZeppelin than the one your project actually uses. To the model, "OpenZeppelin Ownable" is a single concept; to the compiler, OZ v4 and v5 are two incompatible products that happen to share a name.

This post walks through why LLMs make these mistakes, why off-the-shelf "AI patch" features in security tools rarely catch them, and the two-layer architecture ContractScan now uses to deliver patches that actually compile against your codebase.

---

The Failure Mode: Same Concept, Two Incompatible Stacks

Consider this seemingly trivial fix for a signature-verification bug:

// AI suggests:
import {ECDSA} from "@openzeppelin/contracts/utils/cryptography/ECDSA.sol";

function _hashAndRecover(bytes32 hash, bytes memory sig) internal pure returns (address) {
    return ECDSA.recover(ECDSA.toEthSignedMessageHash(hash), sig);
}

This compiles cleanly against OpenZeppelin Contracts v4. It does not compile against v5. In v5, toEthSignedMessageHash was moved out of ECDSA into a dedicated MessageHashUtils library:

// What v5 actually wants:
import {ECDSA} from "@openzeppelin/contracts/utils/cryptography/ECDSA.sol";
import {MessageHashUtils} from "@openzeppelin/contracts/utils/cryptography/MessageHashUtils.sol";

function _hashAndRecover(bytes32 hash, bytes memory sig) internal pure returns (address) {
    return ECDSA.recover(MessageHashUtils.toEthSignedMessageHash(hash), sig);
}

If your project is on OZ v5 and the AI hands you the v4 version of the patch, solc will reject ECDSA.toEthSignedMessageHash as a missing identifier. That is not a malicious bug or a clever attack — it is an LLM looking at a training corpus that contains both APIs and picking the wrong one.

The same trap appears across at least seven other places in the v4→v5 migration:

What changed	v4	v5
Owner constructor	constructor() Ownable() {}	constructor() Ownable(initialOwner) {}
Counter helper	using Counters for Counters.Counter	removed — use a plain uint256
ReentrancyGuard path	security/ReentrancyGuard.sol	utils/ReentrancyGuard.sol
EIP-712 base	draft-EIP712.sol	EIP712.sol
Permit interface	draft-IERC20Permit.sol	IERC20Permit.sol
Access control	AccessControl.sol	AccessControlDefaultAdminRules.sol (recommended)
Errors	string requires	custom errors throughout

Any patch that touches one of these areas can flip from "perfect" to "won't compile" depending on which major version the user is on.

---

Why Off-the-Shelf AI Patches Rarely Catch This

There are three common ways security tools try to use AI for patches today, and each leaves the version-mismatch hole open.

1. Pure prompt with the buggy snippet

The simplest approach: feed the buggy function to a model and ask "how do I fix this?" The model has no context about which OZ version, which Solidity version, or which other libraries are in scope. It generates code in whatever style it last saw most often in its training data — usually the most popular version at the time of training, which is often not the version the user has installed.

2. Static "Recommended Pattern" library

Some scanners ship a fixed library of remediation snippets ("for reentrancy, add nonReentrant"). These are version-agnostic by design but trivial — they cannot capture the actual structural rewrite the user needs, and they often reference imports the user does not have.

3. Prompt with version pinned, but no verification

The next step up: pass the user's pragma and detected libraries to the model and ask it to use those versions. This helps a lot, but the model can still hallucinate. It might still generate MessageHashUtils for a v4 codebase if the prompt is ambiguous, or invent an API that does not exist in any version. Without an actual compile step, those failures ship to the user.

---

The Two-Layer Architecture

ContractScan now applies two layers to every patch the AI suggests. The first improves the input to the model; the second proves the output is correct.

Layer 1 — Detect-and-Match

Before generating a report, the pipeline runs a fast static pass over the user's source to extract:

• The exact pragma solidity version
• All import paths
• Which OZ major version is in use (signaled by paths like MessageHashUtils.sol for v5 or Counters.sol for v4, with a Solidity-version fallback)
• Whether Solady, Solmate, Chainlink, Aave, or Uniswap are imported

The detected stack is rendered as a hard instruction block at the top of the LLM prompt:

## Detected Stack — Match patches to THIS exactly

- Solidity: 0.8.20
- OpenZeppelin Contracts v5 — use MessageHashUtils.toEthSignedMessageHash,
  Ownable(initialOwner) constructor, AccessControlDefaultAdminRules,
  custom errors. DO NOT use _msgSender() inheritance, Counters,
  draft-EIP712, or security/ReentrancyGuard.sol (these are removed/renamed in v5).
- Chainlink oracles — use AggregatorV3Interface.latestRoundData();
  check updatedAt and answeredInRound for staleness.

Patch rule: every before/after example must compile against the user's
detected dependencies above. Do not invent imports they don't have.

This alone removes most of the obvious mismatches. The LLM is no longer guessing which OZ generation you are on.

Layer 2 — Verify-Compile

Detection alone is not enough. LLMs still hallucinate occasionally, and the cost of a non-compiling patch shipped to a developer is high (lost trust, wasted time). So every ```solidity block that comes back from the model goes through a real compile step:

1. The block is classified as contract-level, function-level, or statements based on its leading keywords.
2. It is wrapped into a minimal compilation unit using the user's pragma and imports.
3. The wrapped unit is fed to the resolved solc binary (the same one used in the main scan), with --allow-paths pointing at the vendored node_modules so that @openzeppelin/, @chainlink/, and friends resolve correctly.
4. The result is annotated directly into the report:

> ✅ Patch verified — compiles cleanly against your Solidity version
>    and imported dependencies.

or, when verification fails:

> ⚠️ Patch needs review — did not compile against your stack:
>    `Error: Identifier not found or not unique. Counters.Counter`.
>    Adjust manually before applying.

The compile step is fail-graceful: if solc is unavailable, the vendored libraries are missing, or any internal exception fires, the report passes through unannotated rather than blocking the scan.

---

Three Example Outcomes

To illustrate, three patches from a recent scan against an OZ v5 lending vault:

Patch A — passes verification

function withdraw(address to, uint256 amount) external nonReentrant {
    uint256 bal = address(this).balance;
    require(amount <= bal);
    balances[msg.sender] -= amount;
    (bool s,) = to.call{value: amount}("");
    require(s);
}

✅ Patch verified — compiles cleanly against your Solidity version and imported dependencies.

Patch B — v4 import on v5 stack (model hallucination)

import "@openzeppelin/contracts/utils/Counters.sol";

contract Bad {
    using Counters for Counters.Counter;
    Counters.Counter private _id;
    function next() external returns (uint256) {
        _id.increment();
        return _id.current();
    }
}

⚠️ Patch needs review — did not compile against your stack: Error: Identifier not found or not unique. Counters. Adjust manually before applying.

Patch C — invented function name

function fix() external returns (uint256) {
    return SomeUndefinedLib.zzzNotARealFn();
}

⚠️ Patch needs review — did not compile against your stack: Error: Undeclared identifier "SomeUndefinedLib". Adjust manually before applying.

In all three cases, the developer sees the real compiler verdict alongside the patch — not just the model's confidence.

---

Why This Matters Beyond One Tool

The pattern generalizes. Any AI feature that emits code into someone else's codebase has to either:

1. Constrain the input (Detect-and-Match), so the model cannot stray outside the user's stack, and
2. Verify the output (Verify-Compile), so hallucinations are caught before the user copies them into a PR.

Either step alone is insufficient. Constraining alone trusts the model not to drift. Verifying alone wastes a generation budget on output that was always going to fail. Together they turn LLMs from creative assistants into deterministic patch authors.

For developer tooling, the implications go further. Linters, refactor agents, dependency-upgrade bots — anything that emits code — should be doing the same two-step. If your AI tool gives you a patch and there is no green check next to it telling you "this compiles in your repo," treat it like any other unverified suggestion: review every line.

---

What ContractScan Does With This

Every paid scan now goes through Detect-and-Match in the prompt and Verify-Compile in post-processing. The verification step adds 1–3 seconds per scan in exchange for catching the dependency-mismatch class of failures entirely.

The longer-term roadmap is to extend this past compile into:

• Test generation — Foundry test cases that prove the original is exploitable and the patched version is not.
• PR-ready diffs — unified git apply-able patches with auto-generated commit messages.
• Severity-tiered patches — minimal one-line fixes versus full refactors, so you can pick the change scope you want.

Run a scan at ContractScan to see verified patches against your own contract, including the AI-suggested fixes annotated with real solc results.

---

Related: OpenZeppelin v4 vs v5 Migration: 7 API Changes That Break Your Contracts — the version-pair this post keeps referencing, with full before/after for each change.

Related: Smart Contract Security Tools Comparison — where AI patch generation fits in the broader scanner landscape.

Important Notes

This post is for informational and educational purposes only. It does not constitute financial, legal, or investment advice. Compile-verification reduces but does not eliminate the need for manual review of AI-generated code. Always run a professional audit before deploying smart contracts to production.