Artificial intelligence

Abstract: This paper deals with the problem of information representation into a form that allows to make associations, measure similarity and integrate new information with respect to previously stored. Several simple models for encoding information into sparse distributed representation are explored. These models based on the idea that information about stimuli is stored in the population, not an individual neuron, thus each neuron learns many partial features. Results show formation of a sparse representation of image data with high overlap for similar images. Each cell develops multiple receptive fields that together create a population receptive field. It was possible due to incorporation of dendritic tree into standard neuron model. Also, models were tested on a classification of handwritten digits from MNIST dataset. Results from unsupervised representation show poor accuracy compared to the state-of-the-art supervised methods, however, due to the presence of interesting properties further development of an idea should be continued.

Keywords: dendritic computation, sparse representation, sparse coding, unsupervised learning.

References:

1. Minsky M.L. The Emotional Machine : Commonsense Thinking , Artificial Intelligence , and the Futureof the Human Mind. 2006.
2. Flynn M.J., Kanerva P., Bhadkamkar N. “Sparse distributed memory: Principles of operation,” Science (80)., 1989.
3. Kanerva P. “Hyperdimensional computing: An introduction to computing in distributed representationwith high-dimensional random vectors,” Cognit. Comput., vol. 1, no. 2, pp. 139–159, 2009.
4. Rachkovskij D.A., Kussul E.M., Baidyk T.N. “Building a world model with structure-sensitive sparsebinary distributed representations,” Biol. Inspired Cogn. Archit., vol. 3, pp. 64–86, 2013.
5. Krizhevsky A., Sutskever I., Hinton G.E. “ImageNet Classification with Deep Convolutional NeuralNetworks,” in ImageNet Classification with Deep Convolutional Neural Networks, 2012.
6. Field D.J., Olshausen B. “Emergence of simple-cell receptive field properties by learning a sparse codefor natural images.” 1996.
7. Bell A.J., Sejnowski T.J. “The ‘independent components’ of natural scenes are edge filters,” VisionRes., vol. 37, no. 23, pp. 3327–3338, 1997.
8. Lee H., Battle A., Raina R., Ng A.Y. “Efficient Sparse coding algorithms,” Adv. neural informationProcess. Syst., vol. 19, no. 2, pp. 801–808, 2006.
9. Olshausen B.A., Field D.J. “Sparse coding of sensory inputs,” Curr. Opin. Neurobiol., vol. 14, no. 4,pp. 481–487, 2004.
10. Arora S., Ge R., Ma T., Moitra A. “Simple, Efficient, and Neural Algorithms for Sparse Coding,” 2015.
11. Földiák P. “Forming sparse representations by local anti-Hebbian learning,” Biol. Cybern., vol. 64, no.2, pp. 165–170, 1990.
12. Cui Y., Ahmad S., Hawkins J. “The HTM Spatial Pooler: a neocortical algorithm for online sparsedistributed coding,” bioRxiv, 2016.
13. Kastellakis G., Cai D.J., Mednick S.C., Silva A.J., Poirazi P. “Synaptic clustering within dendrites: Anemerging theory of memory formation,” Prog. Neurobiol., vol. 126, pp. 19–35, 2015.
14. Mel B.W., Schiller J., Poirazi P. “Synaptic plasticity in dendrites: complications and coping strategies,”Curr. Opin. Neurobiol., vol. 43, pp. 177–186, 2017.
15. Tran-Van-Minh A., Cazé R.D., Abrahamsson T., Cathala L., Gutkin B.S., DiGregorio D.A.“Contribution of sublinear and supralinear dendritic integration to neuronal computations,” Front. Cell.Neurosci., vol. 9, no. March, pp. 1–15, 2015.
16. Sjöström P.J., Rancz A., Roth A., Häusser M., Sjostrom P.J., Rancz E.A., Roth A., Hausser M., S. PJ,R. EA, R. A, H. M, Sjostrom P.J., Rancz E.A., Roth A., Hausser M. “Dendritic Excitability andSynaptic Plasticity,” Physiol. Rev., vol. 88, no. 2, pp. 769–840, 2008.
17. London M., Häusser M. “Dendritic Computation,” Annu. Rev. Neurosci., vol. 28, no. 1, pp. 503–532, 2005.
18. Hawkins J., Ahmad S. “Why Neurons Have Thousands of Synapses, a Theory of Sequence Memory inNeocortex,” Front. Neural Circuits, vol. 10, no. March, pp. 1–20, 2016.
19. Govindarajan A., Kelleher R.J., Tonegawa S. “A clustered plasticity model of long-term memoryengrams.,” Nat. Rev. Neurosci., vol. 7, no. July, pp. 575–583, 2006.
20. Eurich C.W.C., Schwegler H., Woesler R. “Coarse coding: applications to the visual system ofsalamanders,” Biol. Cybern., vol. 47, pp. 41–47, 1997.
21. Hinton G.E., McClelland J.L., Rumelhart D.E. “Distributed representations,” Parallel Distrib. Process.,pp. 77–109, 1986.