Jul 14, 2017

There has been some work related to what you are describing:

Learning to learn by gradient descent by gradient descent https://arxiv.org/abs/1606.04474

Related (but less so), there are also some papers about learning neural network architectures:

Designing Neural Network Architectures using Reinforcement Learning https://arxiv.org/abs/1611.02167

Neural Architecture Search with Reinforcement Learning https://arxiv.org/abs/1611.01578

Jan 02, 2017

Other equally exciting papers that relates to learning to learn in DL.

"Neural Architecture Search with Reinforcement Learning"

https://arxiv.org/abs/1611.01578

"RL^2: Fast Reinforcement Learning via Slow Reinforcement Learning"

https://arxiv.org/abs/1611.02779

"Designing Neural Network Architectures using Reinforcement Learning"

https://arxiv.org/abs/1611.02167