https://blog.bj-yan.top/en/tags/distributed-machine-learning/ Recent content in Distributed-Machine-Learning on Bingjie's Blog https://blog.bj-yan.top/favicon.png https://blog.bj-yan.top/favicon.png Hugo -- gohugo.io en-US Wed, 09 Aug 2023 00:54:19 +0800 https://blog.bj-yan.top/en/p/blog-efficiency-analysis-of-distributed-machine-learning/ Wed, 09 Aug 2023 00:54:19 +0800 https://blog.bj-yan.top/en/p/blog-efficiency-analysis-of-distributed-machine-learning/ Analyzing the stochastic runtime of synchronous, asynchronous, and K-worker SGD, and discussing the time to reach target accuracy and parallel speedup limits in conjunction with convergence bounds. https://blog.bj-yan.top/en/p/blog-convergence-analysis-in-deep-learning-part-5/ Tue, 08 Aug 2023 15:04:00 +0800 https://blog.bj-yan.top/en/p/blog-convergence-analysis-in-deep-learning-part-5/ Deriving non-convex stationary point bounds and strongly convex error bounds under full participation from FedAvg's local updates, distinguishing gradient noise, client drift, and sampling error, while discussing partial participation and heterogeneity corrections. https://blog.bj-yan.top/en/p/blog-convergence-analysis-in-deep-learning-part-4/ Mon, 07 Aug 2023 15:03:00 +0800 https://blog.bj-yan.top/en/p/blog-convergence-analysis-in-deep-learning-part-4/ Deriving strong convexity convergence bounds for synchronous, asynchronous, and K-Async SGD, clarifying assumptions of independent sampling, conditional unbiasedness, gradient staleness, and freshness probability. https://blog.bj-yan.top/en/p/blog-convergence-analysis-in-deep-learning-part-3/ Tue, 25 Jul 2023 15:03:00 +0800 https://blog.bj-yan.top/en/p/blog-convergence-analysis-in-deep-learning-part-3/ Deriving strong convex error bounds and non-convex stationary point bounds for mini-batch SGD under assumptions of conditionally unbiased gradients and bounded variance, explaining the impact of step size and noise.