Designing Deep Convolutional Neural Networks using a Genetic Algorithm for Image-based Malware Classification

In recent years, deep Convolutional Neural Networks (CNNs) have shown great potential in malware classification. CNNs, which are originally designed for image processing, identify malware binaries visualised as images. Despite offering promising performance, these human-designed networks are very large requiring more resources to train and deploy them. Evolutionary algorithms have been successfully used in designing deep neural networks automatically for different application domains. In this work, we use a Genetic Algorithm (GA) to optimise the CNN topology and hyperparameters for image-based malware classification. Computational experiments with two different malware datasets, Malimg and Microsoft Malware, show that the GA-evolved networks are very competitive to the networks designed by experts in classifying malware, yet they are also considerably smaller in size comparison.