A korrentour readingWhat is a korrent?
Vector embeddings will not solve search: a decades-old term-frequency algorithm still beats most of them at ranking.
Drawn from what Aravind Srinivas said
A korrentour readingWhat is a korrent?
Drawn from what Aravind Srinivas said
Word for word, with the source under each one. They did not write this page.
Co-founder and CEO of Perplexity, an AI answer engine
There is an algorithm called BM25 precisely for this, which is a more sophisticated version of TF-IDF. TF-IDF is term frequency times inverse document frequency, a very old-school information retrieval system that just works actually really well even today. And BM25 is a more sophisticated version of that, that is still beating most embeddings on ranking.
Added to korrents 19 Jun 2024 · How quotes work · Something wrong? Tell us
Our reading — they may agree, disagree or merely touch the same thing. Closest first: a shared subject counts for most, then how near the wording is.