Intrusion Detection in MANET Using Classification Algorithms: The Effects of Cost and Model Selection

Intrusion detection is frequently used as a second line of defense in Mobile Ad-hoc Networks (MANETs). In this paper we examine how to properly use classification methods in intrusion detection for MANETs. In order to do so we evaluate five supervised classification algorithms for intrusion detection on a number of metrics. We measure their performance on a dataset, described in this paper, which includes varied traffic conditions and mobility patterns for multiple attacks. One of our goals is to investigate how classification performance depends on the problem cost matrix. Consequently, we examine how the use of uniform versus weighted cost matrices affects classifier performance. A second goal is to examine techniques for tuning classifiers when unknown attack subtypes are expected during testing. Frequently, when classifiers are tuned using cross-validation, data from the same types of attacks are available in all folds. This differs from real-world employment where unknown types of attacks may be present. Consequently, we develop a sequential cross-validation procedure so that not all types of attacks will necessarily be present across all folds, in the hope that this would make the tuning of classifiers more robust. Our results indicate that weighted cost matrices can be used effectively with most statistical classifiers and that sequential cross-validation can have a small, but significant effect for certain types of classifiers.