Software Edits Images Via Human 'Mind Control'
Westend61/Getty Images
The development has numerous potential applications.
New software can perform computerized image editing with electric signals from human brains as the only input.
“We can make a computer edit images entirely based on thoughts generated by human subjects. The computer has absolutely no prior information about which features it is supposed to edit or how. Nobody has ever done this before,” says Tuukka Ruotsalo, an associate professor in the computer science department at the University of Copenhagen.
In the new study, 30 participants were equipped with hoods containing electrodes which map electrical brain signals (electro-encephalography or EEG). All participants were given the same 200 different facial images to look at. Also, they were given a series of tasks such as looking for female faces, looking for older people, looking for blond hair, etc.
The participants would not perform any actions, just look briefly at the images—0.5 second for each image. Based on their brain activity, the machine first maps the given preference and then edits the images accordingly. So, if the task was to look for older people, the computer would modify the portraits of the younger persons, making them look older. And if the task was to look for a given hair color, everybody would get that color.
“Notably, the computer had no knowledge of face recognition and would have no idea about gender, hair color, or any other relevant features. Still, it only edited the feature in question, leaving other facial features unchanged,” says Keith Davis, a PhD student at the University of Helsinki.
Some may argue that plenty of software capable of manipulating facial features already exist. That would be missing the point, Davis says.
“All the existing software has been previously trained with labeled input. So, if you want an app which can make people look older, you feed it thousands of portraits and tell the computer which ones are young, and which are old,” he says. “Here, the brain activity of the subjects was the only input. This is an entirely new paradigm in artificial intelligence—using the human brain directly as the source of input.”
One possible application for the new work could be in medicine.
“Doctors already use artificial intelligence in interpretation of scanning images. However, mistakes do happen. After all, the doctors are only assisted by the images but will take the decisions themselves. Maybe certain features in the images are more often misinterpreted than others. Such patterns might be discovered through an application of our research,” says Ruotsalo.
Another application could be assistance to certain groups of disabled people, for instance allowing a paralyzed person to operate his or her computer.
“That would be fantastic,” says Ruotsalo. “However, that is not the focus of our research. We have a broad scope, looking to improve machine learning in general. The range of possible applications will be wide. For instance, 10 or 20 years from now we may not need to use a mouse or type commands to operate our computer. Maybe we can just use mind control!”
There are some reasons for caution, however, according to Ruotsalo.
“Collecting individual brain signals does involve ethical issues. Whoever acquires this knowledge could potentially obtain deep insight into a persons’ preferences. We already see some trends. People buy ‘smart’ watches and similar devices able to record heart rate etc., but are we sure that data are not generated which give private corporations knowledge which we wouldn’t want to share?”
“I see this as an important aspect of academic work. Our research shows what is possible, but we shouldn’t do things just because they can be done. This is an area which in my view needs to be regulated by guidelines and public policies. If these are not adapted, private companies will just go ahead.”
The researchers shared a paper on their work at the CVPR 2022 (Computer Vision and Pattern Recognition).