https://blog.bj-yan.top/series/convergence-analysis/ Recent content in Convergence Analysis on Bingjie's Blog https://blog.bj-yan.top/favicon.png https://blog.bj-yan.top/favicon.png Hugo -- gohugo.io zh-CN https://blog.bj-yan.top/p/convergence-analysis-in-deep-learning-part-5/ Tue, 08 Aug 2023 15:04:00 +0800 https://blog.bj-yan.top/p/convergence-analysis-in-deep-learning-part-5/ 从 FedAvg 的本地更新推导全参与模型下的非凸驻点界与强凸误差界，区分梯度噪声、客户端漂移和采样误差，并讨论部分参与与异质性修正。 https://blog.bj-yan.top/p/convergence-analysis-in-deep-learning-part-4/ Mon, 07 Aug 2023 15:03:00 +0800 https://blog.bj-yan.top/p/convergence-analysis-in-deep-learning-part-4/ 推导同步、异步及 K-Async SGD 的强凸收敛界，明确独立采样、条件无偏、梯度滞后与新鲜概率假设。 https://blog.bj-yan.top/p/convergence-analysis-in-deep-learning-part-3/ Tue, 25 Jul 2023 15:03:00 +0800 https://blog.bj-yan.top/p/convergence-analysis-in-deep-learning-part-3/ 在条件无偏和方差有界假设下推导 mini-batch SGD 的强凸误差界与非凸驻点界，说明步长和噪声的影响。 https://blog.bj-yan.top/p/convergence-analysis-in-deep-learning-part-2/ Mon, 24 Jul 2023 15:02:00 +0800 https://blog.bj-yan.top/p/convergence-analysis-in-deep-learning-part-2/ 推导光滑与非光滑、凸与非凸条件下的一阶方法收敛界，明确步长、输出点及迭代复杂度。 https://blog.bj-yan.top/p/convergence-analysis-in-deep-learning-part-1/ Sun, 23 Jul 2023 15:01:00 +0800 https://blog.bj-yan.top/p/convergence-analysis-in-deep-learning-part-1/ 统一凸性、强凸性、光滑性与随机梯度的定义和假设，并推导收敛性分析常用的不等式。