A Semantic Layer That Finds Related Pages Even When They Share No Words

Problem#

Grouping pages by topic usually means one of two brittle things: a hand-maintained tag list, or literal keyword matching. Both miss the case that matters most: two pages about the exact same underlying idea, described in different words. A recipe about “individual buyer personas” and one about “account-level ICP fit” are the same weighted-feedback-loop pattern one layer apart, but a keyword search for one would never surface the other. What’s needed is semantic search: a way to compare pages by what they mean, not which words they happen to use.

Pattern#

1. Turn each page into a vector, not just a string. An embedding is a fixed-length list of numbers that a model produces from a piece of text. The numbers themselves aren’t meaningful on their own; what matters is position. Two pieces of text with similar meaning end up with vectors that point in a close direction in that space, even if they don’t share a single word. Gemini’s embedContent endpoint does the conversion: send text, get back an array of floats.

   Why 768? What’s the significance?

   Gemini’s embedding models are trained with Matryoshka Representation Learning: the first N dimensions of the full vector already carry the most important signal, ordered by importance, rather than the signal being spread evenly across every dimension. That means asking for a smaller outputDimensionality up front gets a vector the model was built to produce well, not a crude truncation of a bigger one after the fact. 768 is one of Gemini’s officially supported sizes (alongside 1536 and the full 3072), and at a few dozen pages the smaller size keeps the committed JSON file down without giving up meaningful similarity quality: the full 3072-dimension vector would be four times the file size, for no real payoff at this page count.

   Why Gemini, not a separate embeddings-only vendor. The same API key already calls generateContent for the summary/topic extraction below, so embedding with Gemini too means one vendor, one secret, and one rate limit to manage instead of two, not a claim that it beats every other embedding model on some benchmark. A site already calling a different model for its own text generation has just as much reason to keep its embeddings on that vendor instead.

2. Ask for a summary and topics too, in the same pass. A second call, generateContent with a responseSchema, extracts a 2-3 sentence summary and a short topic list from the same page, structured JSON back instead of free text to parse. The embedding answers “what is this page close to”; the summary and topics answer “what is this page about” for a human or another system reading the data.

3. Store one row per page, not a database. At a few dozen pages, a database is overkill. This site’s src/data/semantic-layer.json is a plain JSON file, one object per public page:

{
  "url": "/recipes/dynamic-icp-model",
  "title": "A Dynamic ICP Model That Catches Its Own Drift",
  "summary": "This page covers a weighted-feedback-loop pattern for scoring account fit...",
  "topics": ["ICP modeling", "weighted moving average", "segment drift detection"],
  "embedding": [0.0234, -0.1187, 0.0562, "... 765 more numbers"]
}

4. Compare any two pages with one calculation. Cosine similarity measures the angle between two vectors, ignoring their length, which is what’s wanted here: two pages of very different lengths can still be pointed in the same semantic direction.

function cosineSimilarity(a: number[], b: number[]): number {
  let dot = 0, magA = 0, magB = 0;
  for (let i = 0; i < a.length; i++) {
    dot += a[i] * b[i];
    magA += a[i] ** 2;
    magB += b[i] ** 2;
  }
  return dot / (Math.sqrt(magA) * Math.sqrt(magB));
}

// Rank every other page by similarity to one target page.
function findRelated(targetUrl: string, pages: SemanticPage[], limit = 5) {
  const target = pages.find((p) => p.url === targetUrl);
  if (!target) return [];
  return pages
    .filter((p) => p.url !== targetUrl)
    .map((p) => ({ url: p.url, title: p.title, score: cosineSimilarity(target.embedding, p.embedding) }))
    .sort((a, b) => b.score - a.score)
    .slice(0, limit);
}

A score near 1 means near-identical meaning; near 0 means unrelated. Run findRelated against the committed JSON and the result is a real “related content” feature with no search index, no database, and no extra infrastructure beyond the file itself.

Output#

It caught relationships a keyword search would have missed entirely: individual buyer personas and dynamic ICP fit score high on similarity against each other despite sharing almost no vocabulary, “contact” versus “account,” “title” versus “firmographics,” because the comparison is over meaning, not exact words. Getting a real key to cooperate took four separate production runs and three rounds of fixes: a deprecated model name, an overloaded replacement with no fallback, then a rate limit tripped by the fallback logic itself firing candidates too fast. None of that is visible in the final file, which is the point. Feeding SEO/AEO signals into roster intelligence is what turns this similarity data into an account-level signal: not just “did this account read three pages,” but which three, and what to recommend reading next.

Variations#

• Swap the flat JSON scan for a real vector database (pgvector, a managed vector store) once the page count grows past a few hundred: cosine similarity over an array is O(n) per query, fine for dozens of pages, not thousands.
• Use the same vectors to catch near-duplicate content before publishing. A new recipe with a similarity score above some threshold against an existing one is worth a second look before it ships, cheaper than a human noticing the overlap after the fact.
• Chunk long pages instead of embedding one vector per page, if pages get long enough that one summary vector stops representing the whole thing well: embed each section separately and roll up the best-matching chunk per page instead of the whole-page average.