Spin Boltzmann machine
https://doi.org/10.17586/2220-8054-2022-13-6-593-607
Abstract
Boltzmann machine (BM) is a recurrent network, which has a wide range of applications in machine learning (ML) including dimensionality reduction, feature learning and classification. Standard BM is described by the Ising model and can be implemented as a spin ice based device. Such hardware implementation is faster and more energy efficient than a simulation on digital computers. At the moment, a hardware BM is a single purpose device designed on digital computers for a specific task. In the paper we propose a generalized BM capable of fitting parameters by demonstration of training examples, which is done completely inside the spintronic device. Our generalization is based on the Heisenberg model, which is more accurate than the Ising model for spin ice. We show that for some systems minimization of Kullback-Leibler divergence during training of BM is equivalent to minimization of free energy with respect to the biases of the units, hence training of the ML model can be done by energy dissipation. We include the biases as degrees of freedom of the device, whose dynamics is described by the same Landau-Lifschitz-Gilbert equation as for spins representing units of BM. The demonstration of samples from the training set is done by fixing inputs and outputs according to ground truth. The training samples are remembered by the machine becoming minima on the energy landscape implementing a kind of long-term potentiation. The performance of the proposed machine is compared with a single layer perceptron artificial neural network and with a Bernoulli restricted BM on a binary classification problem.
References
1. Alzubaidi L., Zhang J., Humaidi A.J. et al. Review of deep learning: concepts, CNN architectures, challenges, applications, future directions. J. Big Data, 2021, 8(53).
2. Karniadakis G.E., Kevrekidis I.G., Lu L. et al. Physics-informed machine learning. Nat Rev Phys, 2021, 3, P. 422-440.
3. Hebb D.O. The Organization of Behavior. New York: Wiley & Sons.: 1949, 365 p.
4. Marvin M., Papert S.A. Perceptrons: An Introduction to Computational Geometry. The MIT Press.: 1969, 292 p.
5. Reuther A., Michaleas P., Jones M. AI and ML Accelerator Survey and Trends. 2022 IEEE High Performance Extreme Computing Conference (HPEC).
6. Siegelmann H.T. Neural Networks and Analog Computation: Beyond the Turing Limit. Springer: 1999.
7. Sun Z., Pedretti G., Ambrosi E. et. al. Solving matrix equations in one step with cross-point resistive arrays. PNAS, 2019, 116(10), P. 4123.
8. Hughes T.W., Williamson A.D., Minkov M., Fan Sh. Wave physics as an analog recurrent neural network. Science Advances, 2019, 5(12).
9. Grollier J., Querlioz D., Stiles M. D. Spintronic Nanodevices for Bioinspired Computing. Proc IEEE Inst Electr Electron Eng., 2016, 104(10), P. 2024-2039.
10. Torrejon J., Riou M., Araujo F. et al. Neuromorphic computing with nanoscale spintronic oscillators. Nature, 2017, 547, P. 428-431.
11. Stephan A.W., Lou Q., Niemier M.T., Hu X.S., Koester S.J. Nonvolatile Spintronic Memory Cells for Neural Networks. IEEE Journal on Exploratory Solid-State Computational Devices and Circuits, 2019, 5(2), P. 67-73.
12. Ostwal V., Zand R., De Mara R. A Novel Compound Synapse Using Probabilistic Spin-Orbit-Torque Switching for MTJ-Based Deep Neural Networks. IEEE Journal on Exploratory Solid-State Computational Devices and Circuits, 2019, 5(2), P. 182-187.
13. Bla¨sing R., Khan A.A., Filippou P.Ch. Magnetic Racetrack Memory: From Physics to the Cusp of Applications Within a Decade Proceedings of the IEEE, 2020, 108(8), P. 1303-1321.
14. Sai L., Wang K., Xichao Zh. Magnetic skyrmions for unconventional computing. Materials Horizons, 2021, 8(3), P. 854-868.
15. Vakili H., Xu J-W., Zhou W. Skyrmionics-Computing and memory technologies based on topological excitations in magnets. Journal of Applied Physics, 2021, 130, 070908.
16. Song K. M., Jeong J-S., Pan B, et. al. Skyrmion-based artificial synapses for neuromorphic computing. Nature Electronics, 2020., 3, P. 148-155.
17. Chakravarty A., Mentinka J.H., Davies C.S., Yamada K.T., Kimel A.V., Rasinga Th. Supervised learning of an opto-magnetic neural network with ultrashort laser pulses. Appl. Phys. Lett., 2019, 114, 192407.
18. Pinna D., Bourianoff G., Everschor-Sitte K. Reservoir Computing with Random Skyrmion Textures. Phys. Rev. Applied, 2020, 14, 054020.
19. Hopfield J.J. Neural networks and physical systems with emergent collective computational abilities. PNAS, 1982, 79(8), P. 2554-2558.
20. Hopfield J.J. Neurons With Graded Response Have Collective Computational Properties Like Those of Two-State Neurons. Proceedings of the National Academy of Sciences, 1984, 81(10), P. 3088-3092.
21. Hopfield J.J. Searching for memories, Sudoku, implicit check bits, and the iterative use of not-always-correct rapid neural computation Neural Comput, 2008, 20(5), P. 1119-64.
22. Sherrington D., Kirkpatrick S. Solvable Model of a Spin-Glass. Physical Review Letters, 1975, 35(26), P. 1792.
23. Hinton G., Sejnowsk T.J. Analyzing Cooperative Computation. 5th Annual Congress of the Cognitive Science Society. Rochester, New York, 1983.
24. MacKay D. Information Theory, Inference, and Learning Algorithms. University of Cambridge, 2003.
25. Neelakanta P., De Groff D.F. Neural Network Modeling: Statistical Mechanics and Cybernetic Perspectives. CRC Press, Boca Raton.: 1994. 256 p.
26. Freeman W.J., Vitiello G. Dissipation and spontaneous symmetry breaking in brain dynamics. J. Phys. A: Math. Theor, 2008, 41(30), P. 304042.
27. Gori M., Maggini M., Rossi A. Neural network training as a dissipative process Neural Networks, 2016, 81, P. 72-80.
28. Mo¨ller M., Mo¨llenhoff T., Cremers D. Controlling Neural Networks via Energy Dissipation.International Conference on Computer Vision (ICCV), 2019, P. 3255-3264.
29. Fukami S., Ohno H. Perspective: Spintronic synapse for artificial neural network featured. Journal of Applied Physics, 2018, 124, P. 151904.
30. Saeedi M., Markov I.L. Synthesis and optimization of reversible circuits - a survey. ACM Computing Surveys, 2013, 45(2), P. 1-34.
31. Nielsen M.A., Chuang I.L. Quantum Computation and Quantum Information. Cambridge University Press. 2011.
32. Ventra M.D., Traversa F.L. Perspective: Memcomputing: Leveraging memory and physics to compute efficiently. Journal of Applied Physics, 2018, 123, P. 180901.
33. Camsari K.Y., Faria R., Sutton B.M., Datta S. Stochastic p-Bits for Invertible Logic. Phys. Rev. X, 2017, 7, P. 031014.
34. Borders W.A., Pervaiz A.Z., Fukami S., Camsari K.Y., Ohno H., Datta S.Integer factorization using stochastic magnetic tunnel junctions. Nature, 2019, 573, P. 390-393.
35. Bearden S.R.B., Pei Y.R., Di Ventra M. Efficient solution of Boolean satisfiability problems with digital memcomputing. Sci Rep, 2020, 10, P. 19741.
36. Gypens P., Waeyenberge B.V., Di Ventra M., Leliaert J., Pinna D. Nanomagnetic Self-Organizing Logic Gates. Phys. Rev. Applied, 2021, 16(2), P. 024055.
37. Balynskiy M., Chiang H., Gutierrez D., Kozhevnikov A., Filimonov Yu., Khitun A. Reversible magnetic logic gates based on spin wave interference Journal of Applied Physics, 2018, 123, P. 144501.
38. Chauwin M., Hu X., Garcia-Sanchez F., Betrabet N., Paler A., Moutafis C., Friedman J.S. Skyrmion Logic System for Large-Scale Reversible Computation Phys. Rev. Applied, 2019, 12(6), P. 064053.
39. Amit D.J., Gutfreund H., Sompolinsky H. Spin-glass models of neural networks. Phys. Rev. A, 1985, 32, P. 1007.
40. Bramwell S.T., Harris M.J. The history of spin ice. Journal of Physics: Condensed Matter, 2020, 32(37), P. 374010.
41. Farhan A., Derlet P., Kleibert A., Balan A., Chopdekar R. V., Wyss M., Anghinolfi L., Nolting F., Heyderman L.J. Exploring hyper-cubic energy landscapes in thermally active finite artificial spin-ice systems. Nature Phys, 2013, 9, P. 375-382.
42. Kaffash M.T., Lendinez S., Jungfleisch M.B. Nanomagnonics with artificial spin ice. Physics Letters A, 2021, 402, P. 127364.
43. Jensen J.H., Folven E., Tufte G.Computation in artificial spin ice. Proceedings of the ALIFE 2018: Tokyo, Japan. July 23-27, 2018. (P. 15-22).
44. Gypens P., Leliaert J., Van Waeyenberge B. Balanced Magnetic Logic Gates in a Kagome Spin Ice. Phys. Rev. Applied, 2018, 9, P. 034004.
45. Arava H., Leo N., Schildknecht D., Cui J., Vijayakumar J., Derlet P. M., Kleibert A., Heyderman L.J. Engineering Relaxation Pathways in Building Blocks of Artificial Spin Ice for Computation. Phys. Rev. Applied, 2019, 11, P. 054086.
46. Banas L., Brzezniak Z., Neklyudov M., Prohl A. Stochastic Ferromagnetism: Analysis and Numerics. De Gruyter. 2014.
47. Hoffmann M., Blu¨gel S. Systematic derivation of realistic spin models for beyond-Heisenberg solids. Phys. Rev. B, 2020, 101, P. 024418.
48. Bramwell S.T., Gingras M.J.P. Spin Ice State in Frustrated Magnetic Pyrochlore Materials. Science, 2001, 294(5546), P. 1495-1501.
49. Klinglera S., Pirrob P., Bra¨cher T., Leven B., Hillebrands B., Chumak A.V. Spin-wave logic devices based on isotropic forward volume magneto-static waves. Appl. Phys. Lett, 2015, 106, P. 212406.
Review
For citations:
Lobanov I.S. Spin Boltzmann machine. Nanosystems: Physics, Chemistry, Mathematics. 2022;13(6):593-607. https://doi.org/10.17586/2220-8054-2022-13-6-593-607
Views:
1
Email this article 