The beginning of your question is a bit unclear. You mention you want to take the derivative of H with respect to M, but you then go on to say you want to find dM/dH, which is actually the derivative of M with respect to H. For the purposes of answering your question, I will assume you mean to find dM/dH.
You say you have some equation that relates M and H. You will need to solve for M in this equation. Then you will have a function M(H). You can then find dM/dH by simply taking the derivative of this equation with respect to H. If you were trying to find dH/dM, just do the same process, but solve for H instead of M.
While the simulation is time-based, there is no requirement that you actually use the time measurements for anything. You can just create new vectors to represent the values that H and M are dependent on inside the "variables" or "parameters" section of the custom component.
Below I've linked a few documentation pages on the sections of Simscape components that may be relevant to you:
https://www.mathworks.com/help/physmod/simscape/lang/variables.html
https://www.mathworks.com/help/physmod/simscape/lang/parameters.html
Good luck with writing your custom component,
Rollin
