Hilbert–Huang transform (HHT), continuous wavelet transform (CWT) and discrete wavelet transform (DWT) are well-known signal processing methods that are widely utilized for feature extraction and fault detection by protection systems in smart grids. In this paper, we assess the performances of these methods encountering challenging situations in distribution networks, i.e. high impedance arcing fault (HIF) and current transformer (CT) saturation. Low fault current amplitude in HIF case causes the overcurrent protection, which is the predominant protection method in distribution grids, to fail. Furthermore, some faults may lead to CT saturation, which may result in delayed operation of the relay. To overcome the mentioned problems, researchers employ signal processing approaches such as HHT, DWT or CWT for feature extraction from voltage and current waveforms and import the features to artificial intelligence-based algorithms to detect and discriminate the problems from other normal conditions in power networks. In this regard, HHT, CWT, and DWT are compared under different fault conditions, such as HIF and CT saturation, as well as sudden load increasing, capacitor bank switching, and inrush current of distribution transformers as normal conditions. As a result, simulation studies demonstrate that CWT and DWT are more appropriate for applications of CT saturation and HIF detection in protection of power networks.