AI Program Beats Humans On College Acceptance Test
Humanoid robot Pepper is on display at SoftBank Mobile shop in Tokyo. Koji Sasahara/AP File Photo
Japanese researchers have created a computer program capable of limited humanistic understanding.
Artificial intelligence—the kind where robots understand a wide number of human concepts—has been a military focus for decades, and scientists in Japan may have just made a college-ready AI.
For the first time, a Japanese artificial intelligence program has scored above average on a national standardized college entry exam, performing well enough to have an 80 percent likelihood of getting into 33 national universities, according to the National Institute of Informatics, or NII.
The AI program scored 511 points out of 950 on the National Center Test for University Admissions (similar to the SATs), with a particularly strong performance in math and history, and well above the national average of 416. It was a first for the program, which performed less well in 2013 and 2014, according to the Wall Street Journal. (Physics proved harder for the program, which still showed difficulty in comprehending questions.) NII announced the breakthrough on Saturday.
The goal of the 4-year-old Todai Robot Project is to create by 2021 an artificial intelligence that can win admission to the incredibly prestigious University of Tokyo, and thereby display a uniquely humanistic understanding of language and concepts. And yet: While getting into any college seems like a remarkable achievement in terms of AI comprehension, it’s not actually as grand as mastering most day-to-day human interaction, according to the researchers.
“What makes the University of Tokyo entrance exam harder [than other intelligence challenges] is that the rules are less clearly defined than they are for shogi [a Japanese strategy game similar to chess] or a quiz show. From the perspective of using knowledge and data to answer questions, the university entrance exam requires a more human-like approach to information processing. However, it does not rely as much on common sense as an elementary school exam or everyday life, so it’s a reasonable target for the next step in artificial intelligence research,” Yusuke Miyao, one of the project’s sub-directors, said in a 2013 interview on the group’s website.
The U.S. Defense Department remains keenly interested in artificial intelligence as a force multiplier. One notable success in the history of military-funded AI research is Siri, the AI assistant that has been standard on most iPhones since 2011. Siri was a spinoff of a project called Cognitive Assistant that Learns and Organizes, or CALO, funded by the Defense Advanced Research Projects Agency as part of an 2003-08 endeavor called the Personalized Assistant that Learns, or PAL.
The current military push for greater autonomy in drones can be considered one aspect of AI research. Last November, Undersecretary of Defense Frank Kendall issued a memo to the Defense Science Board, calling for a new intellectual study of artificial intelligence for defense, to “identify the science, engineering, and policy problems that must be solved to permit greater operational use of autonomy across all war-fighting domains … Emphasis will be given to exploration of the bounds-both technological and social-that limit the use of autonomy across a wide range of military operations. The study will ask questions such as: What activities cannot today be performed autonomously? When is human intervention required? What limits the use of autonomy? How might we overcome those limits and expand the use of autonomy in the near term as well as over the next 2 decades?”
The Japanese program’s test performance brings the world one step closer to an AI that can take a humanistic approach to different challenges and problem sets. That, in turn, could be a boost to the military’s human-machine teaming efforts. It also means that humans will be able to leave more to the machines.