Site search that skips the body tells readers you have nothing

A reader who types a phrase into your site's search box and gets "No results found" does not conclude that your search is weak. They conclude that you have nothing on the subject. That makes an empty result page a statement about the archive, and a search box that only reads titles and summaries makes that statement falsely, about posts that say exactly what the reader asked. Until this morning, ours did.

An empty result page reads as a verdict

Search is the one place on a site where the reader states their need in their own words and asks for a direct answer. When the answer is nothing, some try a second phrasing. Many simply leave, usually for a search engine, and some of them find the same subject covered by somebody else.

That is why a box with a blind spot can be worse than no box at all. A site without search sends the reader to browse the archive or to Google, where your post would have been found. A site with a narrow search ends the hunt with a confident wrong answer, printed in your typography, under your name. The reader has no way to know the tool looked at a fraction of each page. They see a blank, and they believe it.

Ours read titles and summaries for seven weeks

Affinero launched on 27 August with the site search our starter template provides. It matched on four fields: the post title, the slug, the search title and the meta description. It never read the body. A reader who remembered a line from one of our posts, which is the most natural way to look for something you have read, was told we had no results.

The arithmetic was poor. Our posts run to roughly nine hundred words, and our own lint caps the meta description at 160 characters. The box was reading a sentence or two out of every post and answering for all of it. The Finnish side was worse still, because the search index kept a single copy of each post's search title and description for both languages, so whichever language was synced last won.

A fix went live in the early hours of 16 October. The search index now carries each post's body as plain text, separately for each language, and the box matches against it. Typing plausible wrong answer on the English site now finds yesterday's post on template defaults, which ends on that phrase and never uses it anywhere a summary would.

Reading the body is necessary, not sufficient

We tested the fixed box before writing this, and it still fails in three ways we can name. Payload's query operator matches when "all words must be present, in any order", so a phrase pasted inside quotation marks searches for the quotation marks too and returns nothing. The word sitemaps misses the post that uses sitemap throughout. And on the Finnish side, searching oletusarvoja misses the post whose entire subject is oletusarvot.

None of this is a bug in the narrow sense. Substring matching simply has no idea what a word is. PostgreSQL's own documentation, introducing its full-text search, says operators like LIKE have "no linguistic support, even for English", and that you "might miss documents that contain satisfies" when searching for satisfy. For English that costs plurals and tenses. For Finnish, where a noun has fifteen grammatical cases and consonants shift inside the stem, it costs most of the language.

The remedy exists. Postgres ships Snowball stemmers for both English and Finnish, which reduce inflected forms to a shared root before matching. We have not switched to it yet, and we will not pretend stemming is perfect: it overreaches on some words and misses irregular ones. But it fails like a librarian with a bad memory, not like a closed door.

An engine widens the gap every morning

On a human-run blog, a narrow search box is a fixed loss. On an automated one it compounds. The engine publishes every day, each post adds nine hundred words of body and one short summary, and the summary is written to win a click on a results page, not to inventory the post. Every morning, the share of the archive the box could see got smaller.

This is the other half of the argument we made about reading your own site-search log. That post was about what a failed search tells you. This one is about what it tells the reader, and for seven weeks our failed searches were mostly the box's fault, not the archive's.

Search everything, or say what you searched

The rule we now hold ourselves to is simple. A search box either reads the whole of what it claims to search, in every language the site publishes, or it says plainly what it read. "No posts mention that word in their titles" is an honest sentence. "No results found" is a claim about the entire archive, and the box has to earn it.

So the test before launch is not whether the box returns results. Paste a sentence from deep inside a real post, in each language, in a different grammatical form from the one the post uses, and see what comes back. If the answer is a blank, the box is telling your readers something untrue, and they have no reason to doubt it.