Improving information-theoretic competitive learning by accentuated information maximization
Titel:
Improving information-theoretic competitive learning by accentuated information maximization
Auteur:
Kamimura, Ryotaro
Verschenen in:
International journal of general systems
Paginering:
Jaargang 34 (2005) nr. 3 pagina's 219-233
Jaar:
2005-06
Inhoud:
In this paper, we propose a new computational method for information-theoretic competitive learning. We have so far developed information-theoretic methods for competitive learning in which competitive processes can be simulated by maximizing mutual information between input patterns and competitive units. Though the methods have shown good performance, networks have had difficulty in increasing information content, and learning is very slow to attain reasonably high information. To overcome the shortcoming, we introduce the rth power of competitive unit activations used to accentuate actual competitive unit activations. Because of this accentuation, we call the new computational method “accentuated information maximization”. In this method, intermediate values are pushed toward extreme activation values, and we have a high possibility to maximize information content. We applied our method to a vowel-consonant classification problem in which connection weights obtained by our methods were similar to those obtained by standard competitive learning. The second experiment was to discover some features in a dipole problem. In this problem, we showed that as the parameter r increased, less clear representations could be obtained. For the third experiment of economic data analysis, much clearer representations were obtained by our method, compared with those obtained by the standard competitive learning method.
Tijdschrift beschrijving