Multiple Gain Adaptations for Improved Neural Networks Training
- Resource Type
- Conference
- Authors
- Challagundla, Jeshwanth; Tyagi, Kanishka; Chugh, Tushar; Manry, Michael
- Source
- 2024 IEEE 14th Annual Computing and Communication Workshop and Conference (CCWC) Computing and Communication Workshop and Conference (CCWC), 2024 IEEE 14th Annual. :0414-0420 Jan, 2024
- Subject
- Communication, Networking and Broadcast Technologies
Components, Circuits, Devices and Systems
Computing and Processing
Power, Energy and Industry Applications
Robotics and Control Systems
Signal Processing and Analysis
Training
Heuristic algorithms
Conferences
Neural networks
Nonhomogeneous media
Newton method
Multilayer Perceptron
Back Propagation
Target Weight Refinement
Adaptive multiple gain adaptations
Orthogonal Least Squares
- Language
A two stage algorithm developed called Multiple Gain Adaptations for Improved Networks (MGAIN) is presented. MAGAIN alternatively finds output weights and uses several gain factors to update the input weights in a Multi-Layer Perceptron. The gain factors are computed using Newtons method. Our method dynamically adjusts the quantity of gain factors calculated to maximize the reduction in loss with each epoch. The results demonstrate that our approach outperforms existing second order algorithms across the majority of diverse datasets.