What François Chollet thinks about benchmarks

@francois-chollet · 52 positions · 2 changes of mind · On ppll ↗

Creator of the Keras deep-learning library and the ARC-AGI benchmark.

François Chollet did not write this page.

We collected these quotes from things they published elsewhere, and every quote links to where it was said. They have no account here and have not endorsed this site. Quotes are word for word; the short line under each one is our own restatement, not their wording. Their own site. Is this you? Claim it or ask us to remove it. Or tell us what is wrong here.

Filter benchmarks · Newest first

Topic

All 52benchmarks 1AI alignment 8measuring intelligence 8LLMs 6world models 6AGI 5scaling laws 5AI sentience 3consciousness 3

17 more topics

OpenAI 3ARC-AGI 2AI slop 1behavioural science 1coding agents 1curiosity 1design 1developer tools 1documentation 1education 1energy 1government 1open source 1recursive self-improvement 1reinforcement learning 1robotics 1social media 1

Sort

NewestOldest

Only François Chollet on benchmarks. See everyone on benchmarks

1 dated position, 2019, in their own words. Our reading of what François Chollet has said — not written or endorsed by them.

1 position so far — this page is not yet offered to search engines.

1. 5 Nov 2019

   Our readingThe Abstraction and Reasoning Corpus can measure human-like general fluid intelligence and enable fair comparisons between AI systems and humans.

   ARC-AGImeasuring intelligence

   Their wordsWe argue that ARC can be used to measure a human-like form of general fluid intelligence and that it enables fair general intelligence comparisons between AI systems and humans.

   ↗On the Measure of Intelligencearxiv.org All 5 from this paper