What is a p-value? It's what we use to say "my experiment isn't full of crap". Specifically, if I want to prove a drug is effective, then I take my data, compute a p-value, and if it's below some acceptable value (say, 0.05) I get to say that my drug was effective. Similarly, I can … Continue reading This post is kind of (p)-hacky.
I’m almost sure this converges (to a random variable)
It's always nice to have friends who encourage you to revisit old concepts that you had seen before as a student, and at that time had just concluded "yeah, there's no way I can ever understand that", but now are forced encouraged to confront it a second time. So today's post is going to be … Continue reading I’m almost sure this converges (to a random variable)
What does it mean to be subgaussian?
For many of us who have visited the steps of a concentration inequality from time to time, we are probably familiar with this term "subgaussian". We know it means something like "variance", in that if the variance of a random variable is 0, then it is also subgaussian with constant 0. We also know it … Continue reading What does it mean to be subgaussian?
Spectral graph theory: deriving effective resistance
There are some themes that you hear about here and there for years. You know it's cool, you know it's probably profound, but you've never really taken it seriously. Then, one day, you decide to take a deeper look at it, and you find yourself sucked in a black hole of mathematical revelations. To me, … Continue reading Spectral graph theory: deriving effective resistance
Tangoing with nonsymmetric contraction matrices: Understanding Gelfand’s formula and its implications on linear convergence rates
I have finally reached the point in my life when I have the misfortune of meeting not one, but two nonsymmetric "contraction" matrices, and have had to try to understand why it is that, even after all the hard work of proving that a matrix's largest eigenvalue is $\lambda_{\max}\leq \rho <1$, it does not mean that … Continue reading Tangoing with nonsymmetric contraction matrices: Understanding Gelfand’s formula and its implications on linear convergence rates
High dimensional mean value theorem.
Here's a new "overthinking it" issue for you. For a while now, I've been relying on the high-dimensional version of a Taylor series approximation result to help me flip variables and gradients. That is, for 1-D functions with continuous Hessians everywhere, the mean value theorem says that there always exists a $z$ where $x\leq z … Continue reading High dimensional mean value theorem.
Projected gradient descent, revisited
It turns out that some of my previous statements on PGD may not have been, well, correct. At first I thought it was just a few typos, but as I studied my notes further, I realized there may be some fundamental flaws. So this blog post revisits some ideas and hopefully irons out some past … Continue reading Projected gradient descent, revisited
Video time! Discretization methods on quadratic and logistic regression
Hi all, this is another "tiny project" that I've been wanting to do for a while. Lately, I've been somewhat obsessed with method flows and discretization methods. While we can sit here and write integrals until our faces turn blue, I think it's time to just simulate some stuff, and just see what happens. In … Continue reading Video time! Discretization methods on quadratic and logistic regression
Some proofs of continuous time things
In trying to understand how continuous time interpretations can help optimization research, I find myself doing a lot of "recreating the wheel", but in a nontrivial way; I am getting these proofs that I'm sure are standard knowledge, but it's not like we learned them back when we were kids. So I'm going to try … Continue reading Some proofs of continuous time things
A fun convex set fact
This is actually not a new thing at all, but a thing I played with a couple years ago, and I ended up using in a numerical optimization lecture to try to impress some undergrads. They were not impressed. Since then I've tuned it a bit and just kind of take it for granted now, … Continue reading A fun convex set fact