VLSI Artificial Neural Networks Engineering, 1994

Engineers have long been fascinated by how efficient and how fast biological neural networks are capable of performing such complex tasks as recognition. Such networks are capable of recognizing input data from any of the five senses with the necessary accuracy and speed to allow living creatures to survive. Machines which perform such complex tasks as recognition, with similar ac­ curacy and speed, were difficult to implement until the technological advances of VLSI circuits and systems in the late 1980's. Since then, the field of VLSI Artificial Neural Networks (ANNs) have witnessed an exponential growth and a new engineering discipline was born. Today, many engineering curriculums have included a course or more on the subject at the graduate or senior under­ graduate levels. Since the pioneering book by Carver Mead; "Analog VLSI and Neural Sys­ tems", Addison-Wesley, 1989; there were a number of excellent text and ref­ erence books on the subject, each dealing with one or two topics. This book attempts to present an integrated approach of a single research team to VLSI ANNs Engineering.

1 An Overview.- 1.1 Introduction.- 1.2 Biological Neural Networks.- 1.3 Artificial Neural Networks (ANNs).- 1.4 Artificial Neural Network Algorithms.- 1.5 Supervised Neural Networks.- 1.6 Unsupervised Neural Networks.- 1.7 Neural Network Architectures and Implementations.- 1.8 Book Overview.- 2 A Sampled-Data CMOS VLSI Implementation of a Multi-Character ANN Recognition System.- 2.1 Introduction.- 2.2 ANN Implementation Techniques.- 2.3 Developed CMOS Circuits for ANNs.- 2.4 The Prototype MLP ANN Model Architecture.- 2.5 MLP ANN Model Simulations.- 2.6 ANN Circuit Simulations.- 2.7 The Developed VLSI Architectures.- 2.8 The Developed Two-Character ANN Recognizer.- 2.9 The Proposed Multi-Character ANN Recognition System.- 2.10 Conclusions.- 3 A Design Automation Environment for Mixed Analog/Digital ANNs.- 3.1 Introduction.- 3.2 Mixed Analog/Digital ANN Hardware.- 3.3 Overview of the Design Automation Environment.- 3.4 Data Flow Graph.- 3.5 The Analyzer.- 3.6 The Design Library.- 3.7 The Synthesizer.- 3.8 Design Examples.- 3.9 Conclusions.- 4 A Compact VLSI Implementation of Neural Networks.- 4.1 Introduction.- 4.2 The Building Blocks.- 4.3 The Circuit Implementation Example.- 4.4 Expanding the Network with Multiple Chips.- 4.5 Conclusions.- 5 An All-Digital VLSI ANN.- 5.1 Introduction.- 5.2 Neocognitron Neural Network Model.- 5.3 Digi-Neocognitron (DNC): A Digital Neural Network Model for VLSI.- 5.4 Character Recognition Example.- 5.5 Advantages for VLSI Implementation.- 5.6 Conclusions.- 6 A Neural Predictive Hidden Markov Model Architecture for Speech and Speaker Recognition.- 6.1 Introduction.- 6.2 Automatic Speech Recognition Methodologies.- 6.3 An ANN Architecture for Predictive HMMs.- 6.4 Discriminative Training of the Neural Predictive HMM.- 6.5 Speaker Recognition Using the Neural Predictive HMM.- 6.6 Conclusions.- 7 Minimum Complexity Neural Networks for Classification.- 7.1 Introduction.- 7.2 Adaptive Probabilistic Neural Networks: APNN and ANNC.- 7.3 Bayesian PDF Model Selection.- 7.4 Maximum Likelihood and Maximum Mutual Information Train-ing.- 7.5 Experimental Results.- 7.6 The Adaptive Feature Extraction Nearest Neighbor Classifier “AFNN”.- 7.7 Conclusions.- 8 A Parallel ANN Architecture for Fuzzy Clustering.- 8.1 Introduction.- 8.2 Fuzzy Clustering and Neural Networks.- 8.3 Fuzzy Competitive Learning Algorithm.- 8.4 Mapping Algorithm Onto Architecture.- 8.5 Fuzzy Clustering Neural Network (FCNN) Architecture: Processing Cells.- 8.6 Comparison With The Fuzzy C-Mean (FCM) Algorithm.- 8.7 Conclusions.- 9 A Pipelined Ann Architecture for Speech Recognition.- 9.1 Introduction.- 9.2 Definition and Notation.- 9.3 PNN Architecture: Processing Stages.- 9.4 Case Studies.- 9.5 Performance Analysis.- 9.6 Conclusions.

This book offers a unique engineering approach to the design of VLSI Artificial Neural Networks (ANNs). The design of analog, digital and mixed analog/digital VLSI ANNs are represented. A design methodology and a CAD environment are presented to highlight the tradeoff design factors. System applications of ANNs to automatic speech recognition and pattern recognition are included.An overview. A sampled-data CMOS VLSI implementation of a multi-character ANN recognition system. A design automation environment for mixed analog/digital ANNs. A compact VLSI implementation of neural networks. An all-digital VLSI ANN. A neural predictive Hidden Markov model architecture for speech and speaker recognition. Minimum complexity neural networks for classification. A parallel ANN architecture for fuzzy