At some point in every optimist's life, we are going to be asked the age-old question: "How does your method compare with Adam?" or, even more annoyingly, "Does this analysis help understand Adam?" I mean, Adam is a nice method that works very well for a very unique set of optimization problems, and not, like, … Continue reading We need to talk about Adam
Tag: paper reviews
Convergence proofs IV: My journey with automatic proof generators
Post by Yifan Sun In comparison to past posts, this post is really about some (relatively) recent work by a bunch of people ([1],[2],[3] to name a few). I therefore tried to spend a lot less time detailing how these things work, as these authors have posted their own excellent blog posts and presentations; however, … Continue reading Convergence proofs IV: My journey with automatic proof generators
Convergence proofs II: (simple) estimate sequences
Post by Yifan Sun tl;dr: Estimate sequences are a very interesting way of isolating the problem of finding rates over a wide family of unconstrained problems, and can also be used to generate accelerated methods. However it's unclear whether they can easily be used on non-NAG-like methods, at least when restricted to quadratic functions. The … Continue reading Convergence proofs II: (simple) estimate sequences