Comprehensive guide on learning automata, introducing two variants to accelerate convergence and computational update speed
Learning Automata and Their Applications to Intelligent Systems provides a comprehensive guide on learning automata from the perspective of principles, algorithms, improvement directions, and applications. The text introduces two variants to accelerate the convergence speed and computational update speed, respectively; these two examples demonstrate how to design new learning automata for a specific field from the aspect of algorithm design to give full play to the advantage of learning automata.
As noisy optimization problems exist widely in various intelligent systems, this book elaborates on how to employ learning automata to solve noisy optimization problems from the perspective of algorithm design and application.
The existing and most representative applications of learning automata include classification, clustering, game, knapsack, network, optimization, ranking, and scheduling. They are well-discussed. Future research directions to promote an intelligent system are suggested.
Written by two highly qualified academics with significant experience in the field,Learning Automata and Their Applications to Intelligent Systems covers such topics as:Mathematical analysis of the behavior of learning automata, along with suitable learning algorithmsTwo application-oriented learning automata: one to discover and track spatiotemporal event patterns, and the other to solve stochastic searching on a lineDemonstrations of two pioneering variants of Optimal Computing Budge Allocation (OCBA) methods and how to combine learning automata with ordinal optimizationHow to achieve significantly faster convergence and higher accuracy than classical pursuit schemes via lower computational complexity of updating the state probability
A timely text in a rapidly developing field,Learning Automata and Their Applications to Intelligent Systems is an essential resource for researchers in machine learning, engineering, operation, and management. The book is also highly suitable for graduate level courses on machine learning, soft computing, reinforcement learning and stochastic optimization.
About the Authors ix
Preface xi
Acknowledgments xiii
A Guide to Reading this Book xv
Organization of the Book xvii
1 Introduction 1
1.1 Ranking and Selection in Noisy Optimization 2
1.2 Learning Automata and Ordinal Optimization 5
1.3 Exercises 7
2 Learning Automata 9
2.1 Environment and Automaton 9
2.1.1 Environment 9
2.1.2 Automaton 10
2.1.3 Deterministic and Stochastic Automata 11
2.1.4 Measured Norms 15
2.2 Fixed Structure Learning Automata 16
2.2.1 Tsetlin Learning Automaton 16
2.2.2 Krinsky Learning Automaton 18
2.2.3 Krylov Learning Automaton 19
2.2.4 IJA Learning Automaton 20
2.3 Variable Structure Learning Automata 21
2.3.1 Estimator-Free Learning Automaton 22
2.3.2 Deterministic Estimator Learning Automaton 24
2.3.3 Stochastic Estimator Learning Automaton 26
2.4 Summary 27
2.5 Exercises 28
3 Fast Learning Automata 31
3.1 Last-position Elimination-based Learning Automata 31
3.1.1 Background and Motivation 32
3.1.2 Principles and Algorithm Design 35
3.1.3 Difference Analysis 37
3.1.4 Simulation Studies 40
3.1.5 Summary 45
3.2 Fast Discretized Pursuit Learning Automata 46
3.2.1 Background and Motivation 46
3.2.2 Algorithm Design of Fast Discretized Pursuit LAs 48
3.2.3 Optimality Analysis 54
3.2.4 Simulation Studies 59
3.2.5 Summary 63
3.3 Exercises 63
4 Application-Oriented Learning Automata 67
4.1 Discovering and Tracking Spatiotemporal Event Patterns 67
4.1.1 Background and Motivation 69
4.1.2 Spatiotemporal Pattern Learning Automata 70
4.1.3 Adaptive Tunable Spatiotemporal Pattern Learning Automata 73
4.1.4 Optimality Analysis 76
4.1.5 Simulation Studies 83
4.1.6 Summary 89
4.2 Stochastic Searching on the Line 89
4.2.1 Background and Motivation 89
4.2.2 Symmetrical Hierarchical Stochastic Searching on the Line 95
4.2.3 Simulation Studies 99
4.2.4 Summary 104
4.3 Fast Adaptive Search on the Line in Dual Environments 104
4.3.1 Background and Motivation 109
4.3.2 Symmetrized ASS with Buffer 111
4.3.3 Simulation Studies 114
4.3.4 Summary 118
4.4 Exercises 118
5 Ordinal Optimization 123
5.1 Optimal Computing-Budget Allocation 123
5.2 Optimal Computing-Budget Allocation for Selection of Best and Worst Designs 125
5.2.1 Background and Motivation 125
5.2.2 Approximate Optimal Simulation Budget Allocation 126
5.2.3 Simulation Studies 138
5.2.4 Summary 150
5.3 Optimal Computing-Budget Allocation for Subset Ranking 151
5.3.1 Background and Motivation 151
5.3.2 Approximate Optimal Simulation Budget Allocation 153
5.3.3 Simulation Studies 159
5.3.4 Summary 167
5.4 Exercises 167
6 Incorporation of Ordinal Optimization into Learning Automata 175
6.1 Background and Motivation 175
6.2 Learning Automata with Optimal Computing Budget Allocation 178
6.3 Proof of Optimality 182
6.4 Simulation Studies 187
6.5 Summary 193
6.6 Exercises 193
7 Noisy Optimization Applications 199
7.1 Background and Motivation 200
7.2 Particle Swarm Optimization 202
7.2.1 Parameters Configurations 203
7.2.2 Topology Structures 203
7.2.3 Hybrid PSO 203
7.2.4 Multiswarm Techniques 204
7.3 Resampling for Noisy Optimization Problems 204
7.4 PSO-Based LA and OCBA 205
7.5 Simulations Studies 209
7.6 Summary 223
7.7 Exercises 224
8 Applications and Future Research Directions of Learning Automata 231
8.1 Summary of Existing Applications 231
8.1.1 Classification 231
8.1.2 Clustering 233
8.1.3 Games 233
8.1.4 Knapsack Problems 234
8.1.5 Decision Problems in Networks 235
8.1.6 Optimization 236
8.1.7 LA Parallelization and Design Ranking 238
8.1.8 Scheduling 240
8.2 Future Research Directions 241
8.3 Exercises 243
References 243
Index 249
