Single particle cryo-electron microscopy (cryo-EM) is a technique that can provide detailed structural information on the architecture of biological macromolecules with resolution ranging from atomic details to quaternary arrangement (Nogales and Scheres, 2015). It particularly excels at characterizing large biological assemblies or heterogeneous samples, which are extremely challenging targets for other high-resolution structural techniques such as X-ray crystallography or nuclear magnetic resonance (NMR) spectroscopy. Since the introduction of direct electron detection (DED) cameras roughly a decade ago, cryo-EM has undergone rapid development both on the hardware and software fronts (Chua et al., 2022). Improved automation and throughput during data collection, coupled with efficient and user-friendly data processing software, has enabled wide adoption of this technique in many areas of biological research.

Despite the achievements so far, many aspects of cryo-EM can still benefit from further improvements. One such area concerns efficient recognition and exclusion of non-ideal cryo-EM micrographs, given the increasing number and size of cryo-EM datasets. Currently, a beam-image shift scheme (Cheng et al., 2018) is commonly used during cryo-EM data collection using software such as SerialEM (Mastronarde, 2005), EPU, and Leginon (Cheng et al., 2021), which greatly increases data collection throughput. Coupled with newer detectors with shorter exposure times, 300 or more cryo-EM movies can be obtained per hour with little compromise on achievable resolution (Fréchin et al., 2023, Peck et al., 2022). Effort is also being made to automate microscope operation through deep learning methods (Bouvette et al., 2022, Cheng et al., 2023, Fan et al., 2024), which would further accelerate the data collection process. Such advances, combined with the need for a large amount of data for low-concentration or highly heterogeneous samples, have led to increasingly large dataset sizes, typically ranging from a few thousand to tens of thousands of movies. Inevitably, a portion of the collected data will be non-ideal for processing and will thus need to be excluded. A common method for performing filtering is based on contrast transfer function (CTF) fitting, in which the user defines a threshold for acceptable CTF fit resolution or defocus value for a given micrograph. This method is very efficient at excluding micrographs with drift issues or beam aberrations, but it is not ideal for filtering out many other issues such as crystalline ice or off-target support film images. To achieve high filtering accuracy, manual inspection of micrographs is often needed, which is slow and requires expert knowledge to perform. Such an approach becomes less practical as the dataset size increases.

A convolutional neural network (CNN) is a machine learning algorithm that is specialized for processing matrix-like data, such as images. CNNs have been successfully applied to a wide range of computer vision tasks such as image recognition, classification, and segmentation (Alzubaidi et al., 2021). CNNs have also proven useful in many cryo-EM processing steps such as particle picking (Bepler et al., 2019, Tegunov and Cramer, 2019, Wagner et al., 2019), 2D class selection (Kimanius et al., 2021, Li et al., 2020), and micrograph segmentation (Sanchez-Garcia et al., 2020). Because cryo-EM micrograph filtering is an image classification task at its essence, a CNN-based approach is highly attractive. Cianfrocco and co-workers were the first to demonstrate such an approach (Li et al., 2020). By training a CNN based on a ResNet34 model on a labeled dataset of “good” and “bad” micrographs, they were able to show that CNN-based micrograph filtering greatly improves prediction accuracy (∼93 %) over traditional CTF-based filtering (∼78 %), in particular by dramatically reducing the false negative rate (“good” micrographs predicted as “bad”). The improvement over CTF-based filtering is especially significant for the classification of tilted images as tilting inherently leads to lower CTF resolution. A more recent version of their tool, MicAssess 1.0 (Li and Cianfrocco, 2021), implements a hierarchical process to further subclassify the “good” and “bad” classes with ∼75 and 80 % subclass prediction accuracy, respectively. The success of the CNN-based approach to micrograph filtering raises new questions. For example, can we further improve the accuracy of CNN-based filtering given limited training data? Additionally, can we better encode the reasoning behind micrograph filtering into CNN-based methods?

In this work, we examined various strategies to improve CNN-based filtering. First, we tested whether using a CNN pretrained on the ImageNet dataset (Deng et al., 2009), which does not resemble cryo-EM micrographs, as a starting point for training can achieve better micrograph filtering accuracy than training a CNN on cryo-EM data from scratch. This method, known as fine-tuning or transfer learning, has been shown to greatly improve attainable accuracy of CNN models when only limited training data is available (Kolesnikov et al., 2019, Yadav and Jadhav, 2019). Second, we examined whether the direct inclusion of Fourier space information as part of the CNN input could result in a better prediction accuracy, since many issues such as crystalline ice or sample drift can be better spotted in the power spectrum than in the real-space image. Finally, we aimed to expand the versatility of CNN-based filtering in terms of the type of samples and detectors that it can be applied to and provide integration with common processing software packages including RELION (Kimanius et al., 2021) and cryoSPARC (Punjani et al., 2017). We named the resulting tool Miffi, which stands for cryo-EM micrograph filtering utilizing Fourier space information (Fig. 1). Miffi is open-source and freely available for public use (https://github.com/ando-lab/miffi). Importantly, we find that fine-tuning provides significant improvement over training from scratch and that inclusion of power spectra as a second input channel suppresses the false positive rate (“bad” micrographs predicted as “good”), largely through improved detection of micrographs with crystalline ice. While we provide Miffi for public use, our results also indicate that CNNs pretrained on the ImageNet dataset provide a useful starting point for any user interested in training a model on custom datasets.