Wednesday, January 10, 2018

"researchers have created a wonderful attack on image recognition systems that uses specially printed stickers that are so interesting to the AI that it completely fails to see anything else"

They accomplished it by training an adversary system to create small circles full of features that distract the target system, trying out many configurations of colors, shapes and sizes and seeing which causes the image recognizer to pay attention. Specific curves that the AI has learned to watch for, combinations of color that indicate something other than background and so on.

Eventually out comes a psychedelic swirl like those shown here.

Put it next to another object the system knows, like a banana, and it will immediately forget the banana and think the picture is “of” the swirl.