Stochastic optimization methods are widely used in machine learning and related fields, such as system identification and signal processing. In this talk, three such approaches are overviewed. (1) The first one studies pseudo-contraction based recursive algorithms, fundamental for reinforcement learning (RL), in case the underlying update operator is inhomogeneous. Such methods are motivated by learning in time-varying Markov decision processes (MDPs). A general relaxed convergence theorem is presented and demonstrated on RL methods. (2) The second part addresses the problem of estimating autoregressive systems based on binary quantized measurements, without assuming the knowledge of the noise distribution (which is typical for "textbook" solutions). A strongly consistent algorithm is suggested, assuming the threshold of the quantizer can be controlled. (3) Finally, in the third part, the effect of momentum acceleration on the least mean square (LMS) adaptive filter is analyzed, under the assumption of stationary, ergodic and mixing signals. The trade-off between the rate of convergence and the covariance of the asymptotic distribution is explored.
Joint work with: (part 1) László Monostori, SZTAKI; (part 2) Erik Weyer, University of Melbourne; (part 3) László Gerencsér, SZTAKI, and Sotirios Sabanis, University of Edinburgh
