korrents

ARC-AGI

A benchmark created by Francois Chollet that tests whether a system can solve novel reasoning tasks it was not trained on.

arcprize.org ↗

FilterEveryone, all time3 from 2 people
Group All Technologists 1Scientists 2
Who Everyone FCFrançois Chollet 2 CFChelsea Finn 1
When All time Past 1 yearPast 3 yearsPast 5 years
  1. CF

    Chelsea Finn quoted

    Assistant professor of computer science at Stanford

    Our readingCombining an avocado and a chair into one image is evidence a model conceptually understands both, not that it memorised pictures of them.

    Their wordswhen you can basically bridge the concept of an avocado and a chair and show that you kind of can combine those two, it means that the model has at least some kind of conceptual understanding of what an avocado is and what a chair is to the point that it can combine them into something that exhibits both concepts at the same time.

    ↗Chelsea Finn: This is the State of the Art in Roboticsyoutube.com 10th of 22 in this recording

  2. 20 months earlier
  3. FC

    François Chollet quoted

    Creator of the Keras deep-learning library and the ARC-AGI benchmark

    Our readingPassing ARC-AGI does not amount to achieving AGI: o3 still fails on some very easy tasks, indicating fundamental differences from human intelligence.

    Their wordsPassing ARC-AGI does not equate to achieving AGI, and, as a matter of fact, I don't think o3 is AGI yet. o3 still fails on some very easy tasks, indicating fundamental differences with human intelligence.

    ↗OpenAI o3 Breakthrough High Score on ARC-AGI-Pubarcprize.org

  4. 5 years earlier
  5. FC

    François Chollet quoted

    Our readingThe Abstraction and Reasoning Corpus can measure human-like general fluid intelligence and enable fair comparisons between AI systems and humans.

    measuring intelligence

    Their wordsWe argue that ARC can be used to measure a human-like form of general fluid intelligence and that it enables fair general intelligence comparisons between AI systems and humans.

    ↗On the Measure of Intelligencearxiv.org All 5 from this paper