Please use this identifier to cite or link to this item: http://ena.lp.edu.ua:8080/handle/ntb/55842
Title: Схемотехнічна реалізація моделі розпаралеленої штучної нейронної мережі нечіткої теорії адаптивного резонансу
Other Titles: Hardware implementation of parallelized fuzzy adaptive resonance theory neural network
Authors: Тимощук, П.
Шатний, С.
Tymoshchuk, P.
Shatnyi, S.
Affiliation: Національний університет “Львівська політехніка”
Lviv Polytechnic National University
Bibliographic description (Ukraine): Тимощук П. Схемотехнічна реалізація моделі розпаралеленої штучної нейронної мережі нечіткої теорії адаптивного резонансу / П. Тимощук, С. Шатний // Computer Design Systems. Theory and Practice. — Lviv : Lviv Politechnic Publishing House, 2019. — Vol 1. — No 1. — P. 1–11.
Bibliographic description (International): Tymoshchuk P. Hardware implementation of parallelized fuzzy adaptive resonance theory neural network / P. Tymoshchuk, S. Shatnyi // Computer Design Systems. Theory and Practice. — Lviv : Lviv Politechnic Publishing House, 2019. — Vol 1. — No 1. — P. 1–11.
Is part of: Computer Design Systems. Theory and Practice, 1 (1), 2019
Issue: 1
Issue Date: 28-Feb-2019
Publisher: Видавництво Львівської політехніки
Lviv Politechnic Publishing House
Place of the edition/event: Львів
Lviv
DOI: doi.org/10.23939/cds2019.01.001
Keywords: функціональна блок-схема
нечітка теорія адаптивного резонансу
нейронна мережа
вибір категорії
переможець-забирає-все
рівняння стану з розривною правою частиною
Functional block-diagram
fuzzy Adaptive Resonance Theory
neural network
category choice
winner-take-all
state equation with a discontinuous right-hand side
Number of pages: 11
Page range: 1-11
Start page: 1
End page: 11
Abstract: У статті описана і змодельована схемотехнічна реалізація розпаралеленої штучної нейронної мережі нечіткої теорії адаптивного резонансу. У мережі реалізовані паралельний вибір категорії та резонансу. Нейронні схеми типу “winner-take-all” неперервного та дискретного часу забезпечують ідентифікацію найбільших з М-входів. Схеми неперервного часу описані рівняннями стану з розривною правою частиною. Дискретний аналог описано різницевим рівнянням. Відповідні функціональні блок-діаграми схем містять М жорсткообмежувальних нейронів прямого зв’язку та один нейрон зворотного зв’язку, який використовують для обчислення динамічного зсуву входів. Схеми поєднують у собі такі переваги, як довільна скінченна роздільна здатність входів, висока швидкість збіжності операції “winner-take-all”, низька обчислювальна складність і складність апаратної реалізації та незалежність від початкових умов. Схеми також використовують для знаходження елементів вхідного вектора з мінімальними/максимальними значеннями для його нормування у діапазоні [0,1].
A hardware implementation design of parallelized fuzzy Adaptive Resonance Theory neural network is described and simulated. Parallel category choice and resonance are implemented in the network. Continuous-time and discrete-time winner-take-all neural circuits identifying the largest ofM inputs are used as the winner-take-all units. The continuous-time circuit is described by a state equation with a discontinuous right-hand side. The discrete-time counterpart is governed by a difference equation. Corresponding functional block-diagrams of the circuits include M feed-forward hardlimiting neurons and one feedback neuron, which is used to compute the dynamic shift of inputs. The circuits combine arbitrary finite resolution of inputs, high convergence speed to the winner-take-all operation, low computational and hardware implementation complexity, and independence of initial conditions. The circuits are also used for finding elements of input vector with minimal/maximal values to normalize them in the range [0,1].
URI: http://ena.lp.edu.ua:8080/handle/ntb/55842
Copyright owner: © Національний університет „Львівська політехніка“, 2019
© Тимощук П., Шатний С., 2019
References (Ukraine): 1. Carpenter G. A., Grossberg S., and Rosen D. B., “Fuzzy ART: Fast Stable Learning and Categorization of Analog Patterns by an Adaptive Resonance System”, Neural Networks, vol. 4, no. 6, pp. 759–771, Jun. 1991.
2. Grossberg S. and Levine D. S., “Neural dynamics of attentionally modulated Pavlovian conditioning: blocking, inter-stimulus interval, and secondary reinforcement”, Applied Optics, vol. 26, no.23, pp. 5015–5030, Dec. 1987.
3. Wunsch II D. C., “ART properties of interest in engineering applications”, in Proc. Int. Joint Conf. Neural Networks, 2009, pp. 3380–3383.
4. Grossberg S., “Adaptive Resonance Theory: How a brain learns to consciously attend, learn, and recognize a changing world”, Neural Networks, vol. 37, pp. 1–47, Jan. 2013.
5. Tan A.-H., Carpenter G. A., and Grossberg S., “Intelligence through interaction: Towards a unified theory for learning”, in Proc. 4th Int. Symp. Neural Networks, LNCS 4491, 2007, pp. 1094–1103.
6. Martínez-Zarzuela M., Pernas F., Díez Higuera J., and Antón-Rodríguez M., “Fuzzy ART neural network parallel computing on the GPU”, in Proc. 9th Int. Work-Conf. Art. Neural Networks, LNCS 4507, 2007, pp. 463–470.
7. Martnez-Zarzuela M., Pernas F., Pablos A. de, Rodrguez M., Higuera J., Giralda D., and Ortega D., “Adaptive Resonance Theory fuzzy networks parallel computation using CUDA”, in Proc. 10th Int. Work-Conf. Art. Neural Networks, LNCS 5517, 2009, pp. 149–156.
8. Ho C. S., Liou J. J., Georgiopoulos M., Heileman G. L., and Christodoulou C., “Analogue circuit design and implementation of an adaptive resonance theory (ART) neural network architecture”, Int. J. Electronics, vol. 76, no 2, pp. 271–291, Apr. 1994.
9. Tsay S. W. and Newcomb R. W., “VLSI implementation of ART1 memories”, IEEE Trans. Neural Networks, vol. 2, no. 2, pp. 214–221, March 1991.
10. Wunsch D. C., Caude U. T. P., Capps C. D., Marks R. J., and Falk R. A., “An optoelectronic implementation of the adaptive resonance fuzzy neural network”, IEEE Trans. Neural Networks, vol. 4, no. 4, pp. 673–684, July 1993.
11. Wunsch II D. C., Morris D. J., McGann R. K., and Caudell T. P., “Photorefractive Adaptive Resonance Neural Network”, Applied Optics, vol. 32, no. 8, pp. 1399–1407, Mar. 1993.
12. Serrano-Gotarredona T. and Linares-Bamanco B., “A modified ARTI algorithm more suitable for VLSI implementations”, Neural Networks, vol. 9, no. 6, pp. 1025–1043, Aug. 1996.
13. Serrano-Gotarredona T. and Linares-Bamanco B., “A real-time clustering microchip neural engine”, IEEE Trans. VLSI Systems, vol. 4 no. 2, pp. 195–209, June 1996.
14. Carpenter G. A. and Grossberg S., “A massively parallel architecture for a self-organizing neural pattern recognition machine”, Computer Vision, Graphics, and Image Processing, vol. 37, no. 1, pp. 54–115, Jan. 1987.
15. Versace M., Kozma R. T., and Wunsch D. C., “Adaptive resonance theory design in mixed memristive-fuzzy hardware”, in Advances in Neuromorphic Memristor Science and Applications, R. Kozma, R. Pino, and G. Pazienza, Eds. Netherlands: Springer, 2012, pp. 133–153.
16. Xu R. and Wunsch II D., “Survey of clustering algorithms”, IEEE Trans. Neural Networks, vol. 13, no 3, pp. 645–678, May 2005.
17. Kalouptisidis N., Signal Processing Systems. Theory and Design. New York: Wiley, 1997.
18. Tank D. W. and Hopfield J. J., “Simple neural optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit”, IEEE Trans. Circuits Syst., vol. 33, no. 5, pp. 533–541, May 1986.
19. Xia Y. and Wang J., “A one-layer recurrent neural network for support vector machine learning”, IEEE Trans. Systems, Man and Cybernetics – Part B: Cybernetics, vol. 34, no. 2, pp. 1261–1269, 2004.
20. Carpenter G. A., Grossberg S., and Reynolds J. H., “ARTMAP: supervised real-time learning and classification of nonstationary data by a self-organizing neural network”, Neural Networks, vol. 4, no 5, pp. 565–588, Feb. 1991.
21. Liu L., Huang L., Lai M., and Ma C., “Projective ART with buffers for the high dimensional space clustering and application to discover stock associations”, Neurocomputing, vol. 72, nos. 4-6, pp. 1283–1295, Jan. 2009.
22. Kim S. and Wunsch II D., “A GPU based parallel hierarchical fuzzy ART clustering”, in Proc. Int. Joint Conf. Neural Networks, 2011, pp. 2778–2782.
23. Tymoshchuk P. V., “A dynamic K-winners take all analog neural circuit”, in Proc. IVth Int. Conf. “Perspective technologies and methods in MEMS design”, Lviv-Polyana, Ukraine, May 21–24, 2008, pp. 13–18.
24. Tymoshchuk P. V., “A discrete-time dynamic K-winners-take-all neural circuit”, Neurocomputing, vol. 72, nos. 13–15, pp. 3191–3202, Aug. 2009.
25. Cai X., Prokhorov D. and Wunsch D., “Training winner-take-all simultaneous recurrent neural networks”, IEEE Trans. Neural Networks, vol. 18, no 3, pp. 674–684, May 2007.
26. Carpenter G. A., Grossberg S., Markuzon N., Reynolds J. H., and Rosen D. B., “Fuzzy ARTMAP: a neural network architecture for incremental supervised learning of analog multidimensional maps”, IEEE Trans. Neural Networks, vol. 3, no. 5, pp. 698–713, Sept. 1992.
27. Lopes M. L. M., Minussi C. R., and Lotufo A. D. P., “Electric load forecasting using a fuzzy ART and ARTMAP neural network”, Applied Soft Computing, vol. 5, no. 2, pp. 235–244, Jan. 2005.
28. Meng L., Tan A.-H., and Xu D., “Semi-supervised heterogeneous fusion for multimedia data coclustering”, IEEE Trans. Knowledge Data Engineering, vol. 26, no. 9, pp. 2293–2306, 2014.
29. Carpenter G. A., Grossberg S., and Rosen D. B., “A neural network realization of fuzzy ART”, Technical Report CAS/CNS-91-021. Boston, MA: Boston University, 1991.
30. Serrano-Gotarredona T., Linares-Barranco B., and Andreou A. G., Adaptive Resonance Theory Microchips: Circuit Design Techniques. Norwell, MA: Kluwer, 1998.
31. Xu R., Xu J., and Wunsch II D. C., “Using Default ARTMAP for cancer classification with MicroRNA expression signatures”, in Proc. Int. Joint Conf. Neural Networks, 2009, pp. 3398–3404.
32. Xu R., Xu J., and Wunsch II D. C. “MicroRNA expression profile based cancer classification using Default ARTMAP”, Neural Networks, vol. 22, pp. 774–780, June 2009.
33. Meng L., Tan A.-H., and Wunsch D. C., “Vigilance adaptation in adaptive resonance theory”, in Proc. Int. Joint Conf. Neural Networks, 2013, pp. 1–7.
34. Tymoshchuk P. V., “A simplified continuous-time model of analogue K-winners-take-all neural circuit”, in Proc. XI Int. Conf. “The Experience of Designing and Application of CAD Systems in Microelectronics”, Polyana- Svalyava, Ukraine, February 23–25, 2011, pp. 121–125.
35. Cichocki A. and Unbehauen R., Neural networks for optimization and signal processing, Baffins Lane, Chichester: John Wiley & Sons, 1993.
References (International): 1. Carpenter G. A., Grossberg S., and Rosen D. B., "Fuzzy ART: Fast Stable Learning and Categorization of Analog Patterns by an Adaptive Resonance System", Neural Networks, vol. 4, no. 6, pp. 759–771, Jun. 1991.
2. Grossberg S. and Levine D. S., "Neural dynamics of attentionally modulated Pavlovian conditioning: blocking, inter-stimulus interval, and secondary reinforcement", Applied Optics, vol. 26, no.23, pp. 5015–5030, Dec. 1987.
3. Wunsch II D. C., "ART properties of interest in engineering applications", in Proc. Int. Joint Conf. Neural Networks, 2009, pp. 3380–3383.
4. Grossberg S., "Adaptive Resonance Theory: How a brain learns to consciously attend, learn, and recognize a changing world", Neural Networks, vol. 37, pp. 1–47, Jan. 2013.
5. Tan A.-H., Carpenter G. A., and Grossberg S., "Intelligence through interaction: Towards a unified theory for learning", in Proc. 4th Int. Symp. Neural Networks, LNCS 4491, 2007, pp. 1094–1103.
6. Martínez-Zarzuela M., Pernas F., Díez Higuera J., and Antón-Rodríguez M., "Fuzzy ART neural network parallel computing on the GPU", in Proc. 9th Int. Work-Conf. Art. Neural Networks, LNCS 4507, 2007, pp. 463–470.
7. Martnez-Zarzuela M., Pernas F., Pablos A. de, Rodrguez M., Higuera J., Giralda D., and Ortega D., "Adaptive Resonance Theory fuzzy networks parallel computation using CUDA", in Proc. 10th Int. Work-Conf. Art. Neural Networks, LNCS 5517, 2009, pp. 149–156.
8. Ho C. S., Liou J. J., Georgiopoulos M., Heileman G. L., and Christodoulou C., "Analogue circuit design and implementation of an adaptive resonance theory (ART) neural network architecture", Int. J. Electronics, vol. 76, no 2, pp. 271–291, Apr. 1994.
9. Tsay S. W. and Newcomb R. W., "VLSI implementation of ART1 memories", IEEE Trans. Neural Networks, vol. 2, no. 2, pp. 214–221, March 1991.
10. Wunsch D. C., Caude U. T. P., Capps C. D., Marks R. J., and Falk R. A., "An optoelectronic implementation of the adaptive resonance fuzzy neural network", IEEE Trans. Neural Networks, vol. 4, no. 4, pp. 673–684, July 1993.
11. Wunsch II D. C., Morris D. J., McGann R. K., and Caudell T. P., "Photorefractive Adaptive Resonance Neural Network", Applied Optics, vol. 32, no. 8, pp. 1399–1407, Mar. 1993.
12. Serrano-Gotarredona T. and Linares-Bamanco B., "A modified ARTI algorithm more suitable for VLSI implementations", Neural Networks, vol. 9, no. 6, pp. 1025–1043, Aug. 1996.
13. Serrano-Gotarredona T. and Linares-Bamanco B., "A real-time clustering microchip neural engine", IEEE Trans. VLSI Systems, vol. 4 no. 2, pp. 195–209, June 1996.
14. Carpenter G. A. and Grossberg S., "A massively parallel architecture for a self-organizing neural pattern recognition machine", Computer Vision, Graphics, and Image Processing, vol. 37, no. 1, pp. 54–115, Jan. 1987.
15. Versace M., Kozma R. T., and Wunsch D. C., "Adaptive resonance theory design in mixed memristive-fuzzy hardware", in Advances in Neuromorphic Memristor Science and Applications, R. Kozma, R. Pino, and G. Pazienza, Eds. Netherlands: Springer, 2012, pp. 133–153.
16. Xu R. and Wunsch II D., "Survey of clustering algorithms", IEEE Trans. Neural Networks, vol. 13, no 3, pp. 645–678, May 2005.
17. Kalouptisidis N., Signal Processing Systems. Theory and Design. New York: Wiley, 1997.
18. Tank D. W. and Hopfield J. J., "Simple neural optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit", IEEE Trans. Circuits Syst., vol. 33, no. 5, pp. 533–541, May 1986.
19. Xia Y. and Wang J., "A one-layer recurrent neural network for support vector machine learning", IEEE Trans. Systems, Man and Cybernetics – Part B: Cybernetics, vol. 34, no. 2, pp. 1261–1269, 2004.
20. Carpenter G. A., Grossberg S., and Reynolds J. H., "ARTMAP: supervised real-time learning and classification of nonstationary data by a self-organizing neural network", Neural Networks, vol. 4, no 5, pp. 565–588, Feb. 1991.
21. Liu L., Huang L., Lai M., and Ma C., "Projective ART with buffers for the high dimensional space clustering and application to discover stock associations", Neurocomputing, vol. 72, nos. 4-6, pp. 1283–1295, Jan. 2009.
22. Kim S. and Wunsch II D., "A GPU based parallel hierarchical fuzzy ART clustering", in Proc. Int. Joint Conf. Neural Networks, 2011, pp. 2778–2782.
23. Tymoshchuk P. V., "A dynamic K-winners take all analog neural circuit", in Proc. IVth Int. Conf. "Perspective technologies and methods in MEMS design", Lviv-Polyana, Ukraine, May 21–24, 2008, pp. 13–18.
24. Tymoshchuk P. V., "A discrete-time dynamic K-winners-take-all neural circuit", Neurocomputing, vol. 72, nos. 13–15, pp. 3191–3202, Aug. 2009.
25. Cai X., Prokhorov D. and Wunsch D., "Training winner-take-all simultaneous recurrent neural networks", IEEE Trans. Neural Networks, vol. 18, no 3, pp. 674–684, May 2007.
26. Carpenter G. A., Grossberg S., Markuzon N., Reynolds J. H., and Rosen D. B., "Fuzzy ARTMAP: a neural network architecture for incremental supervised learning of analog multidimensional maps", IEEE Trans. Neural Networks, vol. 3, no. 5, pp. 698–713, Sept. 1992.
27. Lopes M. L. M., Minussi C. R., and Lotufo A. D. P., "Electric load forecasting using a fuzzy ART and ARTMAP neural network", Applied Soft Computing, vol. 5, no. 2, pp. 235–244, Jan. 2005.
28. Meng L., Tan A.-H., and Xu D., "Semi-supervised heterogeneous fusion for multimedia data coclustering", IEEE Trans. Knowledge Data Engineering, vol. 26, no. 9, pp. 2293–2306, 2014.
29. Carpenter G. A., Grossberg S., and Rosen D. B., "A neural network realization of fuzzy ART", Technical Report CAS/CNS-91-021. Boston, MA: Boston University, 1991.
30. Serrano-Gotarredona T., Linares-Barranco B., and Andreou A. G., Adaptive Resonance Theory Microchips: Circuit Design Techniques. Norwell, MA: Kluwer, 1998.
31. Xu R., Xu J., and Wunsch II D. C., "Using Default ARTMAP for cancer classification with MicroRNA expression signatures", in Proc. Int. Joint Conf. Neural Networks, 2009, pp. 3398–3404.
32. Xu R., Xu J., and Wunsch II D. C. "MicroRNA expression profile based cancer classification using Default ARTMAP", Neural Networks, vol. 22, pp. 774–780, June 2009.
33. Meng L., Tan A.-H., and Wunsch D. C., "Vigilance adaptation in adaptive resonance theory", in Proc. Int. Joint Conf. Neural Networks, 2013, pp. 1–7.
34. Tymoshchuk P. V., "A simplified continuous-time model of analogue K-winners-take-all neural circuit", in Proc. XI Int. Conf. "The Experience of Designing and Application of CAD Systems in Microelectronics", Polyana- Svalyava, Ukraine, February 23–25, 2011, pp. 121–125.
35. Cichocki A. and Unbehauen R., Neural networks for optimization and signal processing, Baffins Lane, Chichester: John Wiley & Sons, 1993.
Content type: Article
Appears in Collections:Комп'ютерні системи проектування теорія і практика. – 2019. – Том 1, № 1



Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.