https://vaipatel.com/tags/machine-learning/Recent content in Machine Learning on Vaibhav Patelhttps://vaipatel.com/https://vaipatel.com/Hugo -- gohugo.ioen-usFri, 21 Feb 2020 10:09:59 +0000https://vaipatel.com/posts/how-is-the-vector-jacobian-product-invoked-in-neural-odes/Fri, 21 Feb 2020 10:09:59 +0000https://vaipatel.com/posts/how-is-the-vector-jacobian-product-invoked-in-neural-odes/This post just tries to explicate the claim in Deriving the Adjoint Equation for Neural ODEs Using Lagrange Multipliers that the vector-Jacobian product $\lambda^\intercal \frac{\partial f}{\partial z}$ can be calculated efficiently without explicitly constructing the Jacobian $\frac{\partial f}{\partial z}$. The claim is made in the Solving PL, PG, PM with Good Lagrange Multiplier section. This post is inspired by a question asked about this topic in the comments post there.https://vaipatel.com/posts/deriving-the-adjoint-equation-for-neural-odes-using-lagrange-multipliers/Tue, 04 Feb 2020 07:18:43 +0000https://vaipatel.com/posts/deriving-the-adjoint-equation-for-neural-odes-using-lagrange-multipliers/A Neural ODE ​1​ expresses its output as the solution to a dynamical system whose evolution function is a learnable neural network. In other words, a Neural ODE models the transformation from input to output as a learnable ODE. Since our model is a learnable ODE, we use an ODE solver to evolve the input to an output in the forward pass and calculate a loss. For the backward pass, we would like to simply store the function evaluations of the ODE solver and then backprop through them to calculate the loss gradient.