Level Method convergence for SGD optimizers.pptx

Model convergecne for SGD

1. Level method convergence by Raghu Pasupathy

2. The Level Method Choose Denote Define:  

3. The Level Method

4. Level Method Notice:  

5. Level Method 0. Choose accuracy 0, and coefficient . 1. th iteration. (a) Compute and (b) If then stop. (c) set .  

6. Complexity Result Suppose after steps,  

7. Lemma: Suppose is bounded, and diam, If for some , then (Proof ?) Lemma: For the sequence generated by the level method, (Proof ?)