Advanced Dynamic-System Simulation (2^nd Ed.) Model Replication and Monte Carlo Studies

PREFACE xiii

CHAPTER 1 DYNAMIC-SYSTEM MODELS AND SIMULATION 1

SIMULATION IS EXPERIMENTATION WITH MODELS 1

1-1 Simulation and Computer Programs 1

1-2 Dynamic-System Models 2

1-3 Experiment Protocols Define Simulation Studies 3

1-4 Simulation Software 4

1-5 Fast Simulation Program for Interactive Modeling 5

ANATOMY OF A SIMULATION RUN 8

1-6 Dynamic-System Time Histories Are Sampled Periodically 8

1-7 Numerical Integration 10

1-8 Sampling Times and Integration Steps 11

1-9 Sorting Defined-Variable Assignments 12

SIMPLE APPLICATION PROGRAMS 12

1-10 Oscillators and Computer Displays 12

1-11 Space-Vehicle Orbit Simulation with Variable-Step Integration 15

1-12 Population-Dynamics Model 17

1-13 Splicing Multiple Simulation Runs: Billiard-Ball Simulation 17

INRODUCTION TO CONTROL-SYSTEM SIMULATION 21

1-14 Electrical Servomechanism with Motor-Field Delay and Saturation 21

1-15 Control-System Frequency Response 23

1-16 Simulation of a Simple Guided Missile 24

STOP AND LOOK 28

1-17 Simulation in the Real World: A Word of Caution 28

References 29

CHAPTER 2 MODELS WITH DIFFERENCE EQUATIONS, LIMITERS, AND SWITCHES 31

SAMPLED-DATA SYSTEMS AND DIFFERENCE EQUATIONS 31

2-1 Sampled-Data Difference-Equation Systems 31

2-2 Solving Systems of First-Order Difference Equations 32

2-3 Models Combining Differential Equations and Sampled-Data Operations 35

2-4 Simple Example 35

2-5 Initializing and Resetting Sampled-Data Variables 35

TWO MIXED CONTINUOUS/SAMPLED-DATA SYSTEMS 37

2-6 Guided Torpedo with Digital Control 37

2-7 Simulation of a Plant with a Digital PID Controller 37

DYNAMIC-SYSTEM MODELS WITH LIMITERS AND SWITCHES 40

2-8 Limiters, Switches, and Comparators 40

2-9 Integration of Switch and Limiter Outputs, Event Prediction, and Display Problems 43

2-10 Using Sampled-Data Assignments 44

2-11 Using the step Operator and Heuristic Integration-Step Control 44

2-12 Example: Simulation of a Bang-Bang Servomechanism 45

2-13 Limiters, Absolute Values, and Maximum/Minimum Selection 46

2-14 Output-Limited Integration 47

2-15 Modeling Signal Quantization 48

EFFICIENT DEVICE MODELS USING RECURSIVE ASSIGNMENTS 48

2-16 Recursive Switching and Limiter Operations 48

2-17 Track/Hold Simulation 49

2-18 Maximum-Value and Minimum-Value Holding 50

2-19 Simple Backlash and Hysteresis Models 51

2-20 Comparator with Hysteresis (Schmitt Trigger) 52

2-21 Signal Generators and Signal Modulation 53

References 55

CHAPTER 3 FAST VECTOR–MATRIX OPERATIONS AND SUBMODELS 57

ARRAYS, VECTORS, AND MATRICES 57

3-1 Arrays and Subscripted Variables 57

3-2 Vector and Matrices in Experiment Protocols 58

3-3 Time-History Arrays 58

VECTORS AND MODEL REPLICATION 59

3-4 Vector Operations in DYNAMIC Program Segments: The Vectorizing Compiler 59

3-5 Matrix–Vector Products in Vector Expressions 61

3-6 Index-Shift Operation 63

3-7 Sorting Vector and Subscripted-Variable Assignments 64

3-8 Replication of Dynamic-System Models 64

MORE VECTOR OPERATIONS 65

3-9 Sums, DOT Products, and Vector Norms 65

3-10 Maximum/Minimum Selection and Masking 66

VECTOR EQUIVALENCE DECLARATIONS SIMPLIFY MODELS 67

3-11 Subvectors 67

3-12 Matrix–Vector Equivalence 67

MATRIX OPERATIONS IN DYNAMIC-SYSTEM MODELS 67

3-13 Simple Matrix Assignments 67

3-14 Two-Dimensional Model Replication 68

VECTORS IN PHYSICS AND CONTROL-SYSTEM PROBLEMS 69

3-15 Vectors in Physics Problems 69

3-16 Vector Model of a Nuclear Reactor 69

3-17 Linear Transformations and Rotation Matrices 70

3-18 State-Equation Models of Linear Control Systems 72

USER-DEFINED FUNCTIONS AND SUBMODELS 72

3-19 Introduction 72

3-20 User-Defined Functions 72

3-21 Submodel Declaration and Invocation 73

3-22 Dealing with Sampled-Data Assignments, Limiters, and Switches 75

References 75

CHAPTER 4 EFFICIENT PARAMETER-INFLUENCE STUDIES AND STATISTICS COMPUTATION 77

MODEL REPLICATION SIMPLIFIES PARAMETER-INFLUENCE STUDIES 77

4-1 Exploring the Effects of Parameter Changes 77

4-2 Repeated Simulation Runs Versus Model Replication 78

4-3 Programming Parameter-Influence Studies 80

STATISTICS 84

4-4 Random Data and Statistics 84

4-5 Sample Averages and Statistical Relative Frequencies 85

COMPUTING STATISTICS BY VECTOR AVERAGING 85

4-6 Fast Computation of Sample Averages 85

4-7 Fast Probability Estimation 86

4-8 Fast Probability-Density Estimation 86

4-9 Sample-Range Estimation 90

REPLICATED AVERAGES GENERATE SAMPLING DISTRIBUTIONS 91

4-10 Computing Statistics by Time Averaging 91

4-11 Sample Replication and Sampling-Distribution Statistics 91

RANDOM-PROCESS SIMULATION 95

4-12 Random Processes and Monte Carlo Simulation 95

4-13 Modeling Random Parameters and Random Initial Values 97

4-14 Sampled-Data Random Processes 97

4-15 “Continuous” Random Processes 98

4-16 Problems with Simulated Noise 100

SIMPLE MONTE CARLO EXPERIMENTS 100

4-17 Introduction 100

4-18 Gambling Returns 100

4-19 Vectorized Monte Carlo Study of a Continuous Random Walk 102

References 106

CHAPTER 5 MONTE CARLO SIMULATION OF REAL DYNAMIC SYSTEMS 109

INTRODUCTION 109

5-1 Survey 109

REPEATED-RUN MONTE CARLO SIMULATION 109

5-2 End-of-Run Statistics for Repeated Simulation Runs 109

5-3 Example: Effects of Gun-Elevation Errors on a 1776 Cannnonball Trajectory 110

5-4 Sequential Monte Carlo Simulation 113

VECTORIZED MONTE CARLO SIMULATION 113

5-5 Vectorized Monte Carlo Simulation of the 1776 Cannon Shot 113

5-6 Combined Vectorized and Repeated-Run Monte Carlo Simulation 115

5-7 Interactive Monte Carlo Simulation: Computing Runtime Histories of Statistics with DYNAMIC-Segment DOT Operations 115

5-8 Example: Torpedo Trajectory Dispersion 117

SIMULATION OF NOISY CONTROL SYSTEMS 119

5-9 Monte Carlo Simulation of a Nonlinear Servomechanism: A Noise-Input Test 119

5-10 Monte Carlo Study of Control-System Errors Caused by Noise 121

ADDITIONAL TOPICS 123

5-11 Monte Carlo Optimization 123

5-12 Convenient Heuristic Method for Testing Pseudorandom Noise 123

5-13 Alternative to Monte Carlo Simulation 123

References 125

CHAPTER 6 VECTOR MODELS OF NEURAL NETWORKS 127

ARTIFICIAL NEURAL NETWORKS 127

6-1 Introduction 127

6-2 Artificial Neural Networks 127

6-3 Static Neural Networks: Training, Validation, and Applications 128

6-4 Dynamic Neural Networks 129

SIMPLE VECTOR ASSIGNMENTS MODEL NEURON LAYERS 130

6-5 Neuron-Layer Declarations and Neuron Operations 130

6-6 Neuron-Layer Concatenation Simplifies Bias Inputs 130

6-7 Normalizing and Contrast-Enhancing Layers 131

6-8 Multilayer Networks 132

6-9 Exercising a Neural-Network Model 132

SUPERVISED TRAINING FOR REGRESSION 134

6-10 Mean-Square Regression 134

6-11 Backpropagation Networks 137

MORE NEURAL-NETWORK MODELS 140

6-12 Functional-Link Networks 140

6-13 Radial-Basis-Function Networks 142

6-14 Neural-Network Submodels 145

PATTERN CLASSIFICATION 146

6-15 Introduction 146

6-16 Classifier Input from Files 147

6-17 Classifier Networks 147

6-18 Examples 149

PATTERN SIMPLIFICATION 155

6-19 Pattern Centering 155

6-20 Feature Reduction 156

NETWORK-TRAINING PROBLEMS 157

6-21 Learning-Rate Adjustment 157

6-22 Overfitting and Generalization 157

6-23 Beyond Simple Gradient Descent 159

UNSUPERVISED COMPETITIVE-LAYER CLASSIFIERS 159

6-24 Template-Pattern Matching and the CLEARN Operation 159

6-25 Learning with Conscience 163

6-26 Competitive-Learning Experiments 164

6-27 Simplified Adaptive-Resonance Emulation 165

SUPERVISED COMPETITIVE LEARNING 167

6-28 The LVQ Algorithm for Two-Way Classification 167

6-29 Counterpropagation Networks 167

EXAMPLES OF CLEARN CLASSIFIERS 168

6-30 Recognition of Known Patterns 168

6-31 Learning Unknown Patterns 173

References 174

CHAPTER 7 DYNAMIC NEURAL NETWORKS 177

INTRODUCTION 177

7-1 Dynamic Versus Static Neural Networks 177

7-2 Applications of Dynamic Neural Networks 177

7-3 Simulations Combining Neural Networks and Differential-Equation Models 178

NEURAL NETWORKS WITH DELAY-LINE INPUT 178

7-4 Introduction 178

7-5 The Delay-Line Model 180

7-6 Delay-Line-Input Networks 180

7-7 Using Gamma Delay Lines 182

STATIC NEURAL NETWORKS USED AS DYNAMIC NETWORKS 183

7-8 Introduction 183

7-9 Simple Backpropagation Networks 184

RECURRENT NEURAL NETWORKS 185

7-10 Layer-Feedback Networks 185

7-11 Simplified Recurrent-Network Models Combine Context and Input Layers 185

7-12 Neural Networks with Feedback Delay Lines 187

7-13 Teacher Forcing 189

PREDICTOR NETWORKS 189

7-14 Off-Line Predictor Training 189

7-15 Online Trainng for True Online Prediction 192

7-16 Chaotic Time Series for Prediction Experiments 192

7-17 Gallery of Predictor Networks 193

OTHER APPLICATIONS OF DYNAMIC NETWORKS 199

7-18 Temporal-Pattern Recognition: Regression and Classification 199

7-19 Model Matching 201

MISCELLANEOUS TOPICS 204

7-20 Biological-Network Software 204

References 204

CHAPTER 8 MORE APPLICATIONS OF VECTOR MODELS 207

VECTORIZED SIMULATION WITH LOGARITHMIC PLOTS 207

8-1 The EUROSIM No. 1 Benchmark Problem 207

8-2 Vectorized Simulation with Logarithmic Plots 207

MODELING FUZZY-LOGIC FUNCTION GENERATORS 209

8-3 Rule Tables Specify Heuristic Functions 209

8-4 Fuzzy-Set Logic 210

8-5 Fuzzy-Set Rule Tables and Function Generators 214

8-6 Simplified Function Generation with Fuzzy Basis Functions 214

8-7 Vector Models of Fuzzy-Set Partitions 215

8-8 Vector Models for Multidimensional Fuzzy-Set Partitions 216

8-9 Example: Fuzzy-Logic Control of a Servomechanism 217

PARTIAL DIFFERENTIAL EQUATIONS 221

8-10 Method of Lines 221

8-11 Vectorized Method of Lines 221

8-12 Heat-Conduction Equation in Cylindrical Coordinates 225

8-13 Generalizations 225

8-14 Simple Heat-Exchanger Model 227

FOURIER ANALYSIS AND LINEAR-SYSTEM DYNAMICS 229

8-15 Introduction 229

8-16 Function-Table Lookup and Interpolation 230

8-17 Fast-Fourier-Transform Operations 230

8-18 Impulse and Freqency Response of a Linear Servomechanism 231

8-19 Compact Vector Models of Linear Dynamic Systems 232

REPLICATION OF AGROECOLOGICAL MODELS ON MAP GRIDS 237

8-20 Geographical Information System 237

8-21 Modeling the Evolution of Landscape Features 239

8-22 Matrix Operations on a Map Grid 239

References 242

APPENDIX: ADDITIONAL REFERENCE MATERIAL 245

A-1 Example of a Radial-Basis-Function Network 245

A-2 Fuzzy-Basis-Function Network 245

References 248

USING THE BOOK CD 251

INDEX 253

GRANINO A. KORN, PhD, is Professor of Electrical and Computer Engineering at the University of Arizona and a partner with G.A. and T.M. Korn Industrial Consultants, a company that designs systems for interactive simulation of dynamic systems and neural networks. He is the author of fifteen books, a Fellow of the IEEE, and the recipient of several awards for his work on computer simulation.