I'm building an LSTM deep neural network for sequence-to-one regression and need to implement a custom loss function for training. The issue is that my loss function depends on the input data X that I feed to the network. I'm running out of ideas how to properly hand the input data to the output layer. For custom regression output layers there seems to be no way to use the NN input data X as an input to the loss function. Also, I tried adding the input data to the target data array T, but then the size of T doesn't match the expected format compared to the NN output. Does anyone know how I can properly implement this?