Researchers Zhi Wang, Chaoge Liu and Xiang Cui published a paper last Monday demonstrating a new technique for slipping malware past automated detection tools, in this case by hiding it in a neural network.
The three embedded 36.9 MiB of malware in a 178 MiB AlexNet model without significantly changing the function of the model itself. The malware-embedded model classified images with nearly identical accuracy, within 1% of the malware-free model. (This is possible because the number of layers and total number of neurons in a convolutive neural network are pre-trained — meaning, just like in human brains, many of the neurons in a trained model end up being largely or completely inactive. )
Just as importantly, by squirting the malware into the model, it was split up to avoid detection by standard antivirus engines. VirusTotal, a service that “inspects items with more than 70 antivirus scanners and URL/domain blocking services, in addition to a host of tools to extract signals from the studied content,” raised no suspicions about the malware’s built-in model.
The researchers’ technique chooses the best layer to work with in an already trained model and then embeds the malware in that layer. In an existing trained model, e.g. a widely available image classifier, there can be an undesirably large impact on accuracy because there are not enough or mostly dormant neurons.
If the accuracy of a model built into malware is insufficient, the attacker can choose to start with an untrained model, add many extra neurons, and then train the model on the same dataset that the original model used. This should yield a model with a larger size but equivalent accuracy, plus the approach offers more room to hide nasty things in it.
The good news is that we’re really just talking about steganography: the new technique is a way of hiding malware, not executing it. In order for the malware to actually run, it must be extracted from the poisoned model by another malicious program and then reassembled into its working form. The bad news is that neural network models are significantly larger than typical photographic images, giving attackers the opportunity to hide much more illegal data in them without detection.
Cybersecurity researcher Dr. Lukasz Olejnik told Motherboard he didn’t think the new technique had much to offer an attacker. “Today it wouldn’t be easy to detect it by antivirus software, but this is only because no one is looking.” But the technique represents yet another way to potentially smuggle data past digital sentries and into a potentially less secure internal network.