Comparision of Solutions of Numerical Gradient Descent Method and Continous Time Gradient Descent Dynamics and Lyapunov Stability

Yagmur N., ALAGÖZ B. B.

27th Signal Processing and Communications Applications Conference (SIU), Sivas, Türkiye, 24 - 26 Nisan 2019 identifier identifier

  • Yayın Türü: Bildiri / Tam Metin Bildiri
  • Cilt numarası:
  • Doi Numarası: 10.1109/siu.2019.8806396
  • Basıldığı Şehir: Sivas
  • Basıldığı Ülke: Türkiye
  • İnönü Üniversitesi Adresli: Evet

Özet

Gradient descent dynamics is an optimization techniques that is widely used in machine learning applications. This technique updates model parameter in the direction of descending of learning error. In this study, Lyapunov stability of continuous time gradient descent dynamics is investigated and robust stability condition, which is needed for implementation of gradient descent dynamics in intelligent control system applications, is evaluated. In a illustrative example, for a De Jong's function type error function, solutions of continuous gradient descent dynamics and Euler method based numerical solutions are compared and stability concerns is discussed.