From c185256034d3567b74218a45f61e442ad9faf2c9 Mon Sep 17 00:00:00 2001 From: jbranchaud Date: Sun, 27 Sep 2026 11:38:39 -0500 Subject: [PATCH] Add Translate To Canonical Tags With Small Model as an LLM TIL --- README.md | 3 +- ...late-to-canonical-tags-with-small-model.md | 81 +++++++++++++++++++ 2 files changed, 83 insertions(+), 1 deletion(-) create mode 100644 llm/translate-to-canonical-tags-with-small-model.md diff --git a/README.md b/README.md index 0007ef8..4b33238 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ working across different projects via [VisualMode](https://www.visualmode.dev/). For a steady stream of TILs, [sign up for my newsletter](https://visualmode.kit.com/newsletter). -_1882 TILs and counting..._ +_1883 TILs and counting..._ See some of the other learning resources I work on: @@ -744,6 +744,7 @@ If you've learned something here, support my efforts writing daily TILs by - [Generate Image Alt Text Across Claude Models](llm/generate-image-alt-text-across-claude-models.md) - [Include A File With Message To `ant`](llm/include-a-file-with-message-to-ant.md) - [Send cURL To Claude Text Completion API](llm/send-curl-to-claude-text-completion-api.md) +- [Translate To Canonical Tags With Small Model](llm/translate-to-canonical-tags-with-small-model.md) - [Use The llm CLI With Claude Models](llm/use-the-llm-cli-with-claude-models.md) ### Mac diff --git a/llm/translate-to-canonical-tags-with-small-model.md b/llm/translate-to-canonical-tags-with-small-model.md new file mode 100644 index 0000000..a521ba7 --- /dev/null +++ b/llm/translate-to-canonical-tags-with-small-model.md @@ -0,0 +1,81 @@ +# Translate To Canonical Tags With Small Model + +In [Don't Classify, Hallucinate](https://softwaredoug.com/blog/2026/08/10/hypothetical-classifications), +Doug Turnbull discusses a technique for hallucinating classifications for a +product and then using MiniLM to translate those classifications to actual +product classifications in your system / vocabulary. + +The second step in the process he described seemed most interesting to me, so I +wanted to experiment with it. + +First, here is my classification system -- that is, these are all the categories +of [TIL posts](https://github.com/jbranchaud/til) that I write. + +```bash +# from my TIL repo +❯ git ls-files '*/*.md' | cut -d/ -f1 | sort -u | paste -sd\| - | sed 's/.*/"&"/' +"ack|ansible|astro|aws|bash|brew|chrome|claude-code|clojure|css|cursor|deno|devops|docker|drizzle|elixir|gatsby|git|github|github-actions|go|groq|heroku|html|http|inngest|internet|java|javascript|jj|jq|kitty|linux|llm|mac|math|mise|mongodb|mysql|neovim|netlify|next-auth|nextjs|phoenix|planetscale|pnpm|postgres|prisma|python|rails|react|react_native|react-testing-library|reason|remix|rspec|ruby|sed|shell|sqlite|streaming|tailwind|taskfile|tmux|typescript|unix|vercel|vim|vscode|webpack|workflow|xstate|yaml|zed|zod|zsh" +``` + +I'll grab that output and use it in the following set of Python REPL steps. Here +I need to import +[`sentence_transformers`](https://sbert.net/docs/quickstart.html#sentence-transformer) +which is "applicable for a wide range of tasks, such as semantic textual +similarity, semantic search, clustering, classification, paraphrase mining, and +more." + +```python +# I initiate this REPL with: +# ❯ uv run --with sentence_transformers python +>>> from sentence_transformers import SentenceTransformer +>>> model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2") +>>> raw_tags = "ack|ansible|astro|aws|bash|brew|chrome|claude-code|clojure|css|cursor|deno|devops|docker|drizzle|elixir|gatsby|git|github|github-actions|go|groq|heroku|html|http|inngest|internet|java|javascript|jj|jq|kitty|linux|llm|mac|math|mise|mongodb|mysql|neovim|netlify|next-auth|nextjs|phoenix|planetscale|pnpm|postgres|prisma|python|rails|react|react_native|react-testing-library|reason|remix|rspec|ruby|sed|shell|sqlite|streaming|tailwind|taskfile|tmux|typescript|unix|vercel|vim|vscode|webpack|workflow|xstate|yaml|zed|zod|zsh" +>>> tags = raw_tags.split("|") +>>> tag_vecs = model.encode(tags, normalize_embeddings=True) +>>> def resolve(fake_tags, threshold=0.5): +... fake_vecs = model.encode(fake_tags, normalize_embeddings=True) +... scores = fake_vecs @ tag_vecs.T +... best, best_score = scores.argmax(1), scores.max(1) +... return {tags[i] for i, s in zip(best, best_score) if s >= threshold +... +``` + +With that all set up, I take a set of tags for a post like this one that +you're reading and see if I can get them to resolve to one or more tags directly +from the canonical tag vocabulary. + +```python +>>> resolve(["Python 3.13", "uv", "ML / LLM / Classification"]) +{'python', 'llm'} +``` + +A set of tags with varying structure and specificity can be translated into the +relevant canonical tags that I started with. + +I can also experiment a little with the threshold. Where is the line for +"Version Control" being associated to "git"? + +```python +>>> resolve(["Version Control"]) +set() +>>> resolve(["Version Control"], 0.2) +{'git'} +>>> resolve(["Version Control"], 0.3) +{'git'} +>>> resolve(["Version Control"], 0.4) +{'git'} +>>> resolve(["Version Control"], 0.5) +set() +>>> resolve(["Version Control"], 0.45) +set() +>>> resolve(["Version Control"], 0.42) +{'git'} +``` + +A quick note on what is happening in the `resolve` function above. We have the +normalized embedding vectors for the _fake tags_ and the set of canonical tags. +We then compute the dot product of the fake tags embeddings with the +_transposed_ canonical tags. This gives us the cosine similarly between all of +them which is an indication of how similar each fake tag vector is to each +canonical tag vector. The ones above the `0.5` threshold make the cut and that +set is returned. We may match one tag, a couple tags, or even no tags.