MathActivation Saturation and Jacobian Attenuation in Neural Ordinary Differential Equations
Neural Ordinary Differential Equations (Neural ODEs) offer a principled way to model continuous dynamical systems with neural networks [1]. The hidden state evolves according to where is a multilayer perceptron with trained weights . Once training is finished, is fixed and will not change. The network no longer adapts—it defines a static vector field on […]Neural Ordinary Differential Equations (Neural ODEs) offer a principled way to model continuous dynamical systems with neural networks [1]. The hidden state evolves according to where is a multilayer perceptron with trained weights . Once training is finished, is fixed and will not change. The network no longer adapts—it defines a static vector field on […]
