Heart disease is the leading cause of mortality worldwide, and it is of utmost importance that clinicians and researchers understand the dynamics of the heart. As an electrical measure of the heart’s activity, the electrocardiogram, or ECG, is the gold standard for recording the cardiac state, whether monitoring the structure of the traces that make up the ECG or indicating key metrics such as heart rate variability. Long-term monitoring of ECG is often required to identify cardiovascular issues but proves impractical; therefore, patients will remotely collect their data. However, ECG signals can become contaminated with various noise sources during data collection. This paper proposes a custom loss function capable of denoising electrode motion artefact in ECG data to a higher standard than other, more common loss functions. We implement our custom loss function with a convolutional neural network to return high-quality ECG, suitable for calculating the aforementioned key metrics from a previously unobtainable state. The proposed model improves ECG signals overall signal-to-noise ratio and preserves the R waves structure. The model outperforms a standard mean squared error loss function with an improvement of 0.5 dB in terms of signal to noise ratio and improves the heart rate estimation by 25%.