Distributed anomaly detection, using cooperative learners and association rule analysis
Titel:
Distributed anomaly detection, using cooperative learners and association rule analysis
Auteur:
Deshmeh, G. Rahmati, M.
Verschenen in:
Intelligent data analysis
Paginering:
Jaargang 12 (2008) nr. 4 pagina's 339-357
Jaar:
2008-08-29
Inhoud:
Anomaly detection is an important branch of the classification problem which has attracted much attention during the previous years. This, as well as the growing need for distributed data mining techniques, and concerns for privacy and security issues of gathering all distributed data in a central location, emphasizes the importance of the distributed anomaly detection problem, which has thus far received little attention. In this paper, we address the problem of detecting anomalies in horizontally distributed data, where only a limited ratio of the instances at each remote site are allowed to be shared, and no single entity is allowed to observe the whole dataset, neither at once nor incrementally. In our proposed method, local predictors are trained and association rules are extracted, using the difference between predicted and actual values on a context dataset. These association rules are used to represent normal and anomalous behaviors, while a final set of learners use these representations to detect anomalies. The contributions of our work are: 1) distributed anomaly detection, where (a) both data and process are distributed, (b) only a limited form of sharing is allowed and (c) no single entity is allowed to observe the whole data, in anyway, 2) solving the problem in cases where concept drifts might occur, 3) providing a solution which is able to handle potential dishonesty from participating entities, and 4) using association rules for anomaly detection, while maintaining the speed requirement in anomaly detection which is necessary in various applications. We have conducted a set of experiments, comparing our proposed method to other typical anomaly detection methods (oversampling, undersampling, SMOTE), which indicate the superiority of the proposed method, while preserving the privacy of participating datasets by avoiding the communication of all local samples to other local datasets.
Tijdschrift beschrijving