korrents

korrents · piece

Predictive Human Preference: From Model Ranking to Model Routing

Chip Huyen · 28 Feb 2024 · huyenchip.com ↗

3 korrents from this piece

Chip Huyen did not write this page.

Every claim below was made in this piece, quoted word for word and numbered in the order the piece makes them, so you can read it there rather than take our word for it. The sentence above each quote is our reading of the claim, not their wording. Each quote was checked against a stored copy of the page at build time; where the two differ, the quote is the fact.

  1. Our reading

    Routing prompts to different AI models can improve response quality while lowering cost and latency.

    Their wordsModel routing has the potential to increase response quality while reducing costs and latency.

  2. Our reading

    Weaker AI models match stronger ones on simple prompts, but strong models are increasingly preferred as prompts get more challenging.

    Their wordsOne pattern is that for simple prompts, weak models can do (nearly) as well as strong models. For more challenging prompts, however, users are much more likely to prefer stronger models.

  3. Our reading

    As more AI models are developed with varying capabilities and costs, routing between them carries real economic value.

    Their wordsWith more and more models being developed, each with different capabilities and a cost structure, model routing has clear economic values.