Hard-Negative Mining for Lookalike Products

Published 2026-08-23 by Faraz Rahimi

The model is not confused in general. It is confused about two items that share a silhouette. Those pairs should drive the next training loop.

Hard-Negative Mining for Lookalike Products

Tags: Machine Learning, Computer Vision, SKU, Feedback, unlisted

Most warehouse visual errors are not random. They are twins: same black charger brick, same brown carton, same generic bottle with a label you cannot read from three feet away.

If you train on random batches forever, you will keep getting 92% accuracy and the same 8% of SKUs will keep eating the day.

Let operators label the twins

The "not this item" button should store query image, wrong SKU, and right SKU when they pick the correct card. That triplet is a hard negative you would never have found sitting at a laptop.

I review those pairs weekly. If two SKUs collide more than a handful of times, they get extra floor photos, and I add a simple rule: if the matcher is torn between them, force a human pick and do not auto-advance.

Do not overfit the twins in isolation

It is tempting to fine-tune only on the failures. Then you break the easy SKUs. I keep a mix: confirmed dwells as positives, twin rejections as hard negatives, and a slice of easy barcodes as an anchor so the model does not forget the obvious.

Original post: https://farazrahimi.com/posts/hard-negative-mining-for-lookalike-products