A/B testing a link preview

Testing two previews against each other is technically simple and statistically awkward: you can’t run both variants on one URL, click-through rates on shared links are noisy, and the audiences for two posts are never quite the same. It’s worth doing at volume, with a clear single variable. Below that volume, consistency beats testing.

That’s an unglamorous answer, and it saves people from concluding things their data doesn’t support.

Why it’s harder than page testing

On a website you split incoming traffic and both variants face the same audience simultaneously. Link previews don’t work that way.

One URL has one set of tags. Testing means either two URLs, or the same URL shared twice at different times — and each introduces a confound.

The platform decides who sees it. Reach isn’t controlled by you. Two posts from the same account reach different numbers of different people, algorithmically, and that variation dwarfs the effect of a different image.

Timing dominates. The same post at 9am Tuesday and 9pm Saturday performs differently for reasons that have nothing to do with your card.

Volumes are small. Detecting a modest difference in click-through rate needs more impressions than a typical post gets. With a few hundred impressions per variant, almost any observed difference is noise.

Caching interferes. Changing tags on a shared URL doesn’t change already-rendered cards, so a “before and after” test often isn’t testing what you think.

When it’s worth doing

High volume. Paid promotion, a large mailing list, a high-traffic account. Enough impressions that a real difference can separate from noise.

One variable. Two images, same title. Or two titles, same image. Changing both tells you nothing about which mattered.

Repeated over time. One test is an anecdote. The same variant winning across several attempts on different days is a signal.

A big expected difference. Testing whether a card with an image beats one without is worth doing; testing two similar images differing in shade is not.

How to run one honestly

Two URLs, both canonical to one page. Different og: tags per URL, both with rel="canonical" pointing at the same address, so search engines consolidate. Share one variant in one place and the other in a comparable place.

Or the same content, twice, spaced out, with variant order alternated across several rounds so time of day doesn’t systematically favour one.

Or use utm_content to distinguish variants of the same destination, which keeps attribution clean in analytics.

Whichever you pick: decide beforehand what result would change your behaviour. “The image variant gets a third more clicks across three rounds” is a decision rule. “Let’s see which does better” produces a number and a story to explain it.

Read the result carefully

Click-through rate, not clicks. Raw clicks reflect reach more than card quality.

Compare across the same platform. Card layouts and audiences differ enough that cross-platform comparison is meaningless.

Watch for a lopsided reach split. If one variant got three times the impressions, the platform made a decision that swamps yours.

Small differences aren’t differences. Without a lot of data, a 10% gap is noise. Don’t rebuild your template on it.

The cheaper alternative

Most of the available gain isn’t in optimisation, it’s in fixing broken and boring cards. Before testing anything:

Make sure every page has a real card. A page with no image, or a card built from your homepage’s tags, is a much bigger loss than a suboptimal image.

Front-load titles so they survive truncation.

Make descriptions add information rather than restating the title.

Make images legible at card size. Large text, high contrast, real margins.

Be consistent so your links become recognisable. This has a compounding effect that no single test will match, and it’s the opposite of what heavy testing produces — testing pushes toward whatever won last week, which erodes recognisability.

Those five cost one afternoon of template work and apply to every future page. A test tells you about one card.

What to test if you’re going to

In rough order of expected effect:

1. Image or no image — the largest difference, and usually not really a test, just a fix.
2. Card type — the large-image layout versus the small thumbnail. A one-tag change with a visible effect.
3. Title framing — question versus statement, specific versus general. The most interesting genuine test.
4. Image style — text-on-colour versus photographic. Worth one test to settle a house style, then stop.
5. Description phrasing — smallest effect, and often not shown at all in compact layouts.

Anything below that list is below the noise floor of most people’s traffic. Fix the fundamentals, pick a consistent style, and spend the attention you saved on the page the link points at.