Can AI repair tears and scratches? Feasibility numbers

Yes — and this post records the measurements that took it from roadmap hope to a shipping feature: what the pipeline used to do to damaged scans, the proof-of-concept numbers that justified building the stage, and the end-to-end results of the version that now ships behind the “Repair scratches & tears” checkbox.

What the pipeline did to a torn photo before

Restoration had no inpainting stage — nothing that could remove a region and synthesize a replacement. Verified on a heavily cracked portrait: inside face regions, damage vanished incidentally (the face model re-synthesizes the whole crop) and then reappeared abruptly at the blend boundary — a crack that stops at the cheek, which can look worse than no repair. Everywhere else, damage was enhanced: the sharpening pass crisped tear edges as if they were detail, and the colouriser tinted white crack lines as scene content.

Proof of concept

Test subject: a 510×817 black-and-white portrait with a web of emulsion cracks. Hardware: a deliberately weak CPU-only sandbox, so every timing is a worst case. Two stages: a scratch-detection network (from Microsoft's open-source “Bringing Old Photos Back to Life” project^[2]) produces a damage mask; the LaMa inpainting model^[1] then fills only the masked regions in 512 px tiles, both on ONNX Runtime's CPU provider^[3].

The damage mask covered 18.4% of pixels on this heavily damaged photo. Quality outcome: the crack web was removed across face, clothing, and background, with no damaging false positives — the lace tablecloth, the classic trap for scratch detectors, survived.

What shipped, measured end to end

The production stage runs first in the restore order, so cracks are never colourised or sharpened before removal. Detection runs at short-edge 256 with a 2 px mask dilation; inpainting is patch-wise at native resolution, capped at 12 patches per photo with the largest damage regions winning. Two invariants are enforced rather than hoped for: only masked pixels are ever written — every undamaged pixel in the output is byte-identical to the input — and a coverage gate skips the stage entirely when the detected area is under 0.5% (nothing worth repairing) or over 35% (more likely misdetected texture than damage).

A later production revision added a second, sharper discriminator to the auto gate: shape. Coverage alone cannot tell a hairline tear from detector noise, but their geometry differs by an order of magnitude — a tear is long and thin, noise is blobs. For each connected component of the mask, with bounding box b_w×b_h and pixel area A, we score

Calibration points from the bench: a clean marketing graphic scores a maximum elongation of 2.4; the crack-web scan scores 29.8. The threshold of 6 sits well clear of both, and each production run logs its coverage and elongation so the gate keeps being tuned on real traffic rather than on two photos.

Two controls: with the checkbox off, the stage never ran and timings matched the previous pipeline exactly. On a near-clean photo, the detector flagged only the genuinely damaged 0.81% of pixels — leftover corner cracks and dust — with zero false positives on faces, lace, or texture.

Why it is opt-in rather than default

The known failure mode of automatic damage repair is false positives: a detector that flags fence wires, hair strands, or lace as scratches will erase real content, and an inpainter is very good at erasing things. Two control photos are evidence, not a benchmark. The stage therefore ships as an explicit checkbox, and each run logs its damage-coverage figure so the thresholds can be tuned on real traffic before any promotion to default. We also rejected the shortcut of sending photos to an external generative image API: this tool processes photos transiently on our own infrastructure, and a full generative re-render drifts identity — the goal is to repair the damage, not repaint your relative.