Jun 21, 2016

For a mathy academic survey of gradient descent methods (from this month, June 2016), I recommend: https://arxiv.org/abs/1606.04838

(It was submitted by someone else a few days ago, but saw no traction.)