https://vaipatel.com/tags/lagrange-multipliers/Recent content in Lagrange Multipliers on Vaibhav Patelhttps://vaipatel.com/https://vaipatel.com/Hugo -- gohugo.ioen-usTue, 04 Feb 2020 07:18:43 +0000https://vaipatel.com/posts/deriving-the-adjoint-equation-for-neural-odes-using-lagrange-multipliers/Tue, 04 Feb 2020 07:18:43 +0000https://vaipatel.com/posts/deriving-the-adjoint-equation-for-neural-odes-using-lagrange-multipliers/A Neural ODE ​1​ expresses its output as the solution to a dynamical system whose evolution function is a learnable neural network. In other words, a Neural ODE models the transformation from input to output as a learnable ODE. Since our model is a learnable ODE, we use an ODE solver to evolve the input to an output in the forward pass and calculate a loss. For the backward pass, we would like to simply store the function evaluations of the ODE solver and then backprop through them to calculate the loss gradient.