This book presents and discusses innovative ideas in the design, modelling, implementation, and optimization of hardware platforms for neural networks. The rapid growth of server, desktop, and embedded applications based on deep learning has brought about a renaissance in interest in neural networks, with applications including image and speech processing, data analytics, robotics, healthcare monitoring, and IoT solutions. Efficient implementation of neural networks to support complex deep learning-based applications is a complex challenge for embedded and mobile computing platforms with limited computational/storage resources and a tight power budget. Even for cloud-scale systems it is critical to select the right hardware configuration based on the neural network complexity and system constraints in order to increase power- and performance-efficiency. Hardware Architectures for Deep Learning provides an overview of this new field, from principles to applications, for researchers, postgraduate students and engineers who work on learning-based services and hardware platforms.
The book provides an overview of the basic concepts and recent advances in some of the key deep learning hardware research areas. The book is structured into five parts. Part I of the book contains three chapters to introduce the fundamental concepts of neural networks. This part first provides a background on the neural network concepts and models and then reviews in more detail some design issues of two important neural network classes, i.e., recurrent neural network (RNN) and feedforward models. Part II has two chapters devoted to low-precision data representation for neural networks. The chapters review and introduce some state-of-the-art proposals to simplify neural network execution by using stochastic and binary data representations. The chapters show that due to their inherent error-tolerance property, neural networks experience negligible or sublinear accuracy degradation when these approximate data representations are employed. Other topics include deep learning and model sparsity; convolutional neural networks for embedded systems; deep learning on analog accelerators; and inverter-based memristive neuromorphic circuit for ultra-low-power IoT smart applications.