Toward nanomagnetic implementation of energy-based machine learning
https://doi.org/10.17586/2220-8054-2023-14-6-613-625
Abstract
Some approaches to machine learning (ML) such as Boltzmann machines (BM) can be reformulated as energy based models, which are famous for being trained by minimization of free energy. In the standard contrastive divergence (CD) learning the model parameters optimization is driven by competition of relaxation forces appearing in the target system and the model one. It is tempting to implement a physical device having natural relaxation dynamics matching minimization of the loss function of the ML model. In the article, we propose a general approach for the design of such devices. We systematically reduce the BM, the restricted BM and BM for classification problems to energy based models. For each model we describe a device capable of learning model parameters by relaxation. We compare simulated dynamics of the models using CD, Monte-Carlo method and Langevin dynamics. Benchmarks of the proposed devices on generation and classification of hand-written digits from MNIST dataset are provided.
Supporting agencies: The work is supported by Russian Science Foundation grant 22-22-00565: https://rscf.ru/en/project/22-22-00565/
About the Author
I. S. Lobanov
Faculty of Physics, ITMO University
Russian Federation
Russian Federation
Igor S. Lobanov
Lomonosova Str. 9, Saint Petersburg, 191002
References
1. Min B., Ross H., Sulem E., et al. Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey. ACM Computing Surveys, 2023, 56(2), P. 1–40.
2. Chen Y., Nazhamaiti M., Xu H., et al. All-analog photoelectronic chip for high-speed vision tasks. Nature, 2023, 623(7985), P. 48–57.
3. Grollier J., Querlioz D., Camsari K.Y., et al. Neuromorphic spintronics. Nature Electronics, 2020, 3(7), P. 360–370.
4. Li S., Kang W., Zhang X., et al. Magnetic skyrmions for unconventional computing. Materials Horizons, 2021, 8(3), P. 854–868.
5. Yokouchi T., Sugimoto S., Rana B., et al. Pattern recognition with neuromorphic computing using magnetic fieldinduced dynamics of skyrmions. Science Advances, 2022, 8(39), P. eabq5652.
6. Gu K., Guan Y., Hazra B.K., et al. Three-dimensional racetrack memory devices designed from free- standing magnetic heterostructures. Nature Nanotechnology, 2022, 17(10), P. 1065–1071.
7. Blasing R., Khan A.A., Filippou P.C., et al. Magnetic Racetrack Memory: From Physics to the Cusp of ApplicationsWithin a Decade. Proceedings of the IEEE, 2020, 108(8), P. 1303–1321.
8. Marullo C., Agliari E. Boltzmann Machines as Generalized Hopfield Networks: A Review of Recent Results and Outlooks. Entropy, 2020, 23(1), 34 pp.
9. Khater A., Abou Ghantous M. Magnetic properties of 2D nano-islands I: Ising spin model. Journal of Magnetism and Magnetic Materials, 2011, 323(22), P. 2717–2726.
10. Yang S., Shin J., Kim T., et al. Integrated neuromorphic computing networks by artificial spin synapses and spin neurons. NPG Asia Materials, 2021, 13(1), P. 1–10.
11. Dixit V., Selvarajan R., Alam M.A., et al. Training Restricted Boltzmann MachinesWith a D-Wave Quantum Annealer. Frontiers in Physics, 2021, 9, 589626, 10 pp.
12. Lecun Y., Chopra S., Hadsell R., et al. A tutorial on energy-based learning. In G. Bakir, T. Hofman, B. Scholkopt, A. Smola, B. Taskar, editors, Predicting structured data. MIT Press. 2006.
13. Dayan P., Hinton G.E., Neal R.M., et al. The Helmholtz Machine. 1999. Neural Comput., 1995, 7(5), P. 889–904.
14. Teh Y.W., Welling M., Osindero S., et al. Energy-Based Models for Sparse Overcomplete Representa- tions. Journal of Machine Learning Research, 2003, 4, P. 1235–1260.
15. Du Y., Mordatch I. Implicit Generation and Modeling with Energy Based Models. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc.
16. Lobanov I. Spin Boltzmann machine. Nanosystems: Physics, Chemistry, Mathematics, 2022, 13(6), P. 593–607.
17. Kiraly B., Knol E.J., Van Weerdenburg W.M.J., et al. An atomic Boltzmann machine capable of self- adaption. Nature Nanotechnology, 2021, 16(4), P. 414–420.
18. Aksenova E., Dobrun L., Kovshik A., et al. Magnetic Field-Induced Macroscopic Alignment of Liquid- Crystalline Lanthanide Complexes. Crystals, 2019, 9(10), P. 499.
19. Yair O., Michaeli T. Contrastive Divergence Learning is a Time Reversal Adversarial Game. URL/arXiv: https://doi.org/10.48550/arXiv.2012.03295.
20. Colombo S., Pedrozo-Peafiel E., Adiyatullin A.F., et al. Time-reversal-based quantum metrology with many-body entangled states. Nature Physics, 2022, 18(8), P. 925–930.
21. Purcell E.M., Pound R.V. A Nuclear Spin System at Negative Temperature. Physical Review, 1951, 81(2), P. 279–280.
22. Hakonen P., Lounasmaa O.V. Negative Absolute Temperatures: “Hot” Spins in Spontaneous Magnetic Order. Science, 1994, 265(5180), P. 1821– 1825.
23. Medley P., Weld D.M., Miyake H., et al. Spin Gradient Demagnetization Cooling of Ultracold Atoms. Physical Review Letters, 2011, 106(19), 195301, 4 pp.
24. Ramsey N.F. Thermodynamics and Statistical Mechanics at Negative Absolute Temperatures. Physical Review, 1956, 103(1), P. 20–28.
25. Baldovin M., Iubini S., Livi R., et al. Statistical mechanics of systems with negative temperature. Physics Reports, 2021, 923, P. 1–50.
26. Deng L. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 2012, 29(6), P. 141–142.
27. Hoffmann M., Blgel S. Systematic derivation of realistic spin models for beyond-Heisenberg solids. Physical Review B, 2020, 101(2), P. 024418.
28. Yoshioka N., Akagi Y., Katsura H. Transforming generalized Ising models into Boltzmann machines. Physical Review E, 2019, 99(3), P. 032113.
Review
For citations:
Lobanov I.S. Toward nanomagnetic implementation of energy-based machine learning. Nanosystems: Physics, Chemistry, Mathematics. 2023;14(6):613-625. https://doi.org/10.17586/2220-8054-2023-14-6-613-625
Views:
9
Email this article 