You can never remove all of the noise for the reasons you mention. You can eliminate low-frequency baseline wander and any d-c offset, as well as frequencies that are not part of the EKG spectrum (including atrial and ventricular tachyarrhythmias and other abnormal rhythms) by using a bandpass filter with a passband of about 2-100 Hz. For that, your EKG signal has to be sampled (or resampled if less than or equal to 200 Hz) to a sampling frequency >200 Hz, usually chosen to be 256 Hz (because many EKGs are sampled at this rate, so you don’t have to re-design the filters for different EKG records).
The mains frequency interference is usually eliminated with a notch filter. See the documentation for Remove the 60 Hz Hum from a Signal to see one way of eliminating it.
The filter type (Butterworth, Chebyshev, etc.) is your choice, depending on your requirements. I’ve used these and a number of others, in hardware and software. My typical filter design procedure is described in How to design a lowpass filter for ocean wave data in Matlab?.
