https://blog.bj-yan.top/en/tags/convergence-analysis/ Recent content in Convergence-Analysis on Bingjie's Blog https://blog.bj-yan.top/favicon.png https://blog.bj-yan.top/favicon.png Hugo -- gohugo.io en-US Tue, 08 Aug 2023 15:04:00 +0800 https://blog.bj-yan.top/en/p/blog-convergence-analysis-in-deep-learning-part-5/ Tue, 08 Aug 2023 15:04:00 +0800 https://blog.bj-yan.top/en/p/blog-convergence-analysis-in-deep-learning-part-5/ Deriving non-convex stationary point bounds and strongly convex error bounds under full participation from FedAvg's local updates, distinguishing gradient noise, client drift, and sampling error, while discussing partial participation and heterogeneity corrections. https://blog.bj-yan.top/en/p/blog-convergence-analysis-in-deep-learning-part-4/ Mon, 07 Aug 2023 15:03:00 +0800 https://blog.bj-yan.top/en/p/blog-convergence-analysis-in-deep-learning-part-4/ Deriving strong convexity convergence bounds for synchronous, asynchronous, and K-Async SGD, clarifying assumptions of independent sampling, conditional unbiasedness, gradient staleness, and freshness probability. https://blog.bj-yan.top/en/p/blog-convergence-analysis-in-deep-learning-part-3/ Tue, 25 Jul 2023 15:03:00 +0800 https://blog.bj-yan.top/en/p/blog-convergence-analysis-in-deep-learning-part-3/ Deriving strong convex error bounds and non-convex stationary point bounds for mini-batch SGD under assumptions of conditionally unbiased gradients and bounded variance, explaining the impact of step size and noise. https://blog.bj-yan.top/en/p/blog-convergence-analysis-in-deep-learning-part-2/ Mon, 24 Jul 2023 15:02:00 +0800 https://blog.bj-yan.top/en/p/blog-convergence-analysis-in-deep-learning-part-2/ Derive first-order method convergence bounds under smooth/non-smooth and convex/non-convex conditions, clarifying step sizes, output points, and iteration complexity. https://blog.bj-yan.top/en/p/blog-convergence-analysis-in-deep-learning-part-1/ Sun, 23 Jul 2023 15:01:00 +0800 https://blog.bj-yan.top/en/p/blog-convergence-analysis-in-deep-learning-part-1/ Unify definitions and assumptions regarding convexity, strong convexity, smoothness, and stochastic gradients, and derive inequalities commonly used in convergence analysis.