TY - JOUR
AU - Nakamae,, Koji
AB - Abstract We try to improve the limit of the phase estimation of the interference fringe at low electron dose levels in electron holography by a noise reduction method. In this paper, we focus on unsupervised approaches to apply it to electron beam-sensitive and unknown samples and describe an overview of denoising methods used widely in image processing, such as wiener filter, total variation denoising, nonlocal mean filters and wavelet thresholding. We compare the wavelet hidden Markov model (WHMM) denoising that we have studied so far with the other conventional noise reduction methods. We evaluate the denoise performance of each method using the peak signal-to-noise ratio between noise-free and the target holograms (noisy or denoised holograms) and the root mean-square error (RMSE) between the true phase of the fringe and the measured phase by the discrete Fourier transform phase estimator. We show the denoised holograms for simulation and experimental data by using each noise reduction method and then discuss evaluation indexes obtained from these denoised holograms. From experimental results, it can be seen that the WHMM denoising can reduce the RMSE of fringe phase to about 1/4.5 for noisy simulation holograms and it has stable and good performance for noise reduction of observed holograms with various image qualities. electron holography, wavelet transform, hidden Markov model, denoising, phase estimation 1 Introduction Electron holography, which was invented by Gabor [1] and observed for the first time by Haine and Mulvey [2], is used to observe the nanometer-scale electromagnetic field distribution of electronic and magnetic materials [3–6]. A coherent electron wave emitted from the electron source propagates to a sample. Half of the wave passes through the sample (object wave), and the other half passes through a vacuum area (reference wave). The object and reference waves are superimposed by using a biprism filament under the object plane and then an interference fringe pattern is formed. The electromagnetic field is visualized by detecting the phase shifts of the fringe pattern. It is necessary to reduce noise in electron hologram to measure a minute phase change accurately. Walkup and Goodman [7] have reported limitations of fringe–parameter estimation for photoemissive detectors. Fundamental limitations of estimating phases of interference fringes at low light levels are determined by the finite number of photo events registered in the measurement. By modeling the receiver as a one-dimensional spatial array of photon-counting detectors, results are obtained that permit specification of the minimum number of photo events required for estimation of fringe parameters to a given accuracy. The fringe phase is estimated by calculating the arctangent of the ratio of the imaginary and real parts of the discrete Fourier-transform (DFT) components (referred to as the DFT phase estimator). As shown in later Fig. 7, the root mean-square error (RMSE) of the fringe phase, which is the standard deviation of the error between the measured phase and the true phase of the fringe, depends on the parameter |$\gamma (m_0)$|⁠, where |$m_0$| is an integer number of period of fringe patterns, and |$\gamma (m_0)$| is a signal-to-noise ratio (SNR) of the |$m_0$|th DFT component (see Performance evaluation index). For example, it is seen from this graph that |$\gamma (m_0)$| of about 10 is required to obtain a phase error of 0.1 radians. For |$\gamma (m_0)=3$|⁠, the phase error becomes about 0.3 radians. Noise in an electron hologram is due to single-electron events. The shot noise in the electron hologram can be considered as Poisson-distributed with a standard deviation given by |$\sigma ^2=N_e$|⁠, where |$N_e$| is the average number of detected electrons per pixel [8]. Lichte et al. [9] have introduced the SNR of the DFT phase estimator defined in light-optics to electron holography. Ruijter [10] proposed that RMSE of the fringe phase is |$\sqrt{7}$| times larger if the spatial frequency and the direction of the interference fringe are unknown. Harscher [11] verified that experimental results using a slow-scan CCD camera were close to the above definition. Furthermore, Voelkl [12, 13] defined the low bound of the phase noise parameter, which is the standard deviation measured from an arbitrarily selected area in the phase image. The expression includes the effect of the reconstruction aperture and the reference hologram on the Fourier based phase reconstruction, and the binning factor and conversion factor that converts the pixel values effectively into electron counts. Chang [14] used the phase error considering the detective quantum efficiency (DQE) and the modulation transfer function (MTF) of a direct detection camera. These proposals are extensions of the definition of Walkup and Goodman by adding a constant multiplication according to the detector characteristics and observation conditions. The RMSE of the fringe phase is inversely proportional to the square root of the total electron dose in the measured area and the fringe visibility. To obtain a hologram with a good SNR, it is possible to increase the beam current and the exposure time. These approaches may, however, be difficult in terms of sample damage. It is considered to be effective to improve the phase estimation accuracy of interference fringes at low electron dose levels by using signal and image processing such as noise reduction methods. Roels et al. [15] have reported the image restoration (denoising and deconvolution) techniques that are currently used in electron microscopy imaging. These techniques are mainly distinguished between local linear and nonlinear filters, nonlocal self-similarity filters, manifold representation-based, and probabilistic (Bayesian) methods as follows. Local linear filters Local linear filters are computationally efficient. Two of the most commonly used are smoothing filters and the Wiener filter (WF). The smoothing filters, such as mean filters, Gaussian filters and spline interpolation [16], restore a pixel value by linearly combining pixel values in the neighborhood. These filters are generally unable to resolve local structures well enough. The WF estimates a pre-degraded image that minimizes the mean squared error between the observed image and the simulation image degraded by assumed blur and noise [17]. Local nonlinear filters Nonlinearities are added to avoid over-smoothing of image edges. The non-linear method allows the shrinking of the amplitude of the transform to separate signals or remove noise. The medial filter (MF), the bilateral filter (BF) and the wavelet thresholding (WT) are representative methods. The MF estimates the median of a local neighborhood of the target pixel as a noise-free signal [17]. The BF is an edge-preserving spatial filter that has two parameters about the intensity and spatial distance [18]. The BF replaces the intensity of each pixel with a weighted average of a local neighborhood to less blur along edges. The WT is often used as wavelet-based denoising. Wavelets have proved remarkably successful for estimating signals in additive white Gaussian noise [19, 20]. The compression property indicates that the wavelet transform typically compacts signals into just a few coefficients of large magnitude. Because the wavelet transform is orthogonal, it leaves white noise evenly distributed across many coefficients of small magnitude. Therefore, the WT effectively removes noise without degrading the signal by setting small wavelet coefficients to zero (see Fig. 1(a) with 2 wavelet scales). Barsanti and Gilmore [21] have reported the WT performs better than the Fourier-based thresholding for a variety of signals corrupted with additive Gaussian noise. When noise depends on the signal level, such as Poisson noise, variance stabilization (VST) is usually applied as pre-processing (see Variance Stabilization Variance stabilization). Nonlocal self-similarity-based filters Self-similarity is a statistical property that an object is exactly or approximately similar to a part of itself. Buades et al. [22] have proposed a nonlocal means filter (NLM) with the self-similarity prior. The NLM suppresses noise and other distortions by searching for similar patches at different locations in a given image and then by weighted averaging all pixels. The weights are determined by how similar these patches are to the target patch. Manifold representation-based restoration For more specialized and restricted targets, a noise reduction method that aggressively utilizes the characteristics of signal and noise can be considered. These methods use a transformation that converts the image to a (preferably lower-dimensional) manifold where noise can be isolated more efficiently. The transformation is mainly derived by supervised machine learning for a given training dataset, such as dictionary learning [23] and convolutional neural networks [24]. Because the supervised method needs a pair of degraded (input) and degradation-free (output) images, it is difficult to apply them when images without degradation can not be obtained (for example, it is necessary to generate labeled images by high precision simulation). Bayesian image restoration Image restoration is probabilistically performed based on a Bayesian framework. Bayesian inference computes the posterior probability from a prior probability and a likelihood function derived from a statistical model of the observation (from a ground truth image to an observed image). The degradation-free image is estimated from the observed image by using maximum a posteriori (MAP) algorithm. Sorzano et al. [25] have reported application results for electron microscope images. Various models exist depending on the target image formation and noise characteristics. Here we discuss denoising methods based on total variation (TV) prior, one of the most used priors. The TV prior assumes that images should consist of flat regions delineated by a relatively small amount of edges [26]. Therefore, TV denoising (TVD) can reduce oscillations and discontinuities. Zhanjiang et al. [27] have introduced the primal-dual method to improve denoise stability and high efficiency. In this paper, we try to improve the limits of phase estimation at low electron dose levels in electron holography. Here we focus on an unsupervised based noise reduction considering an application to unknown samples vulnerable to damage by the electron beam. M. Crouse et al. [28] have reported wavelet-based statistical signal processing using hidden Markov models where attractive properties of the wavelet transform are utilized. We have applied the wavelet-based hidden Markov model (WHMM) to the noise reduction of an electron hologram [29–31]. The WHMM denoising is a method combines local nonlinear filters and Bayesian image restoration. This paper gives an overview of the WHMM denoising and also shows comparison results with other methods. The outline of this work is organized as follows. Methods introduces the WHMM denoising. Result is devoted to simulation and experimental results. Finally, the conclusion is drawn in Concluding Remarks. Fig. 1. Open in new tabDownload slide Overview of wavelet-based denoising for 2 wavelet scales: (a) Wavelet-based thresholding (WT) and (b) Wavelet hidden Markov model (WHMM) denoising. The original image is divided into 4 subband images by the wavele transform. The noises included in 3 detail components (HL, LH, HH) are suppressed by the noise reduction method. The denoised image is finally obtained by the inverse wavelet transform. Fig. 1. Open in new tabDownload slide Overview of wavelet-based denoising for 2 wavelet scales: (a) Wavelet-based thresholding (WT) and (b) Wavelet hidden Markov model (WHMM) denoising. The original image is divided into 4 subband images by the wavele transform. The noises included in 3 detail components (HL, LH, HH) are suppressed by the noise reduction method. The denoised image is finally obtained by the inverse wavelet transform. Fig. 2 Open in new tabDownload slide Preliminary simulation results for parameter optimization of WHMM denoising: PSNR of denoised simulation holograms and RMSE of the fringe phase calculated from denoised simulation holograms: (a) Daubechies 8 as a function of |$\kappa$|⁠, (b) seven wavelets with |$\kappa =1.5$|⁠. Fig. 2 Open in new tabDownload slide Preliminary simulation results for parameter optimization of WHMM denoising: PSNR of denoised simulation holograms and RMSE of the fringe phase calculated from denoised simulation holograms: (a) Daubechies 8 as a function of |$\kappa$|⁠, (b) seven wavelets with |$\kappa =1.5$|⁠. 2 Methods Wavelet-based denoising methods, such as WT and WHMM, have proved successful for estimating signals in additive white Gaussian noise. The phase error is, however, determined by mainly the Poisson noise in the electron hologram. Weighted sum |$aX+bY$| of two random variables |$X$| and |$Y$| with independent Gaussian (normal) distribution is also normally distributed, where |$a$| and |$b$| are arbitrary real numbers. Since the property does not generally hold for the Poisson distribution, the distribution of wavelet coefficients for electron hologram cannot be expressed with a simple formula. This makes it difficult to select an appropriate threshold of denoising and results in signal-dependent noise reduction. To avoid this, variance stabilization (VST) is often used as a pre-processing of the noise reduction. VST and WHMM will be described in the following. Fig. 3. Open in new tabDownload slide Application results of noise reduction methods for simulated holograms with |$N=256$|⁠, |$m_0=10$|⁠, |$V_s=1.0$|⁠, |$x_s=1$|⁠, |$x_b=0$|⁠, |$\phi =0$|⁠. Fig. 3. Open in new tabDownload slide Application results of noise reduction methods for simulated holograms with |$N=256$|⁠, |$m_0=10$|⁠, |$V_s=1.0$|⁠, |$x_s=1$|⁠, |$x_b=0$|⁠, |$\phi =0$|⁠. 2.1 Variance stabilization The rationale behind applying a variance stabilization is to remove the signal dependence of noise variance. One of the most popular VST is the Anscombe transformation defined by the following equation, $$\begin{equation}f(z) = 2\sqrt{z+3/8}.\end{equation}$$(1) Applying eq. (1) to Poisson distributed data gives a signal whose noise is asymptotically additive standard normal |$N(0, 1)$|⁠. Then the noise is removed from |$f(z)$| using the wavelet-based methods, WT and WHMM. Finally, the denoised image is obtained by applying an inverse VST to the processed image. To minimize the bias error of the inverse transformation, we use optimal inversion proposed by M|$\ddot{a}$|kitalo [32]. 2.2 Wavelet hidden Markov model denoising The wavelet transform divides an original image into 4 subband images; approximation (LL), horizontal (HL), vertical (LH), diagonal (HH) detail components (wavelet coefficients), as shown in Fig. 1. The approximation image is given by low-pass filtering and downscaling. The three detail images are given by high-pass filtering with different bands and downscaling. The following wavelet transform is repeatedly applied to the approximation image at the previous wavelet scale. The wavelet transform has several attractive properties that make it natural for signal and image processing: locality, multiresolution, compression, clustering and persistence [28]. Clustering means that if a particular wavelet coefficient is large/small, then adjacent coefficients are very likely to also be large/small. Persistence means that large/small values of wavelet coefficients tend to propagate across scales. Persistence suggests that wavelet coefficients can have strong dependencies across scale, whereas clustering and locality suggest that coefficients can have strong dependencies within scale. The conventional threshold-based denoising, as shown in Fig. 1 (a), usually ignores possible dependencies between signal wavelet coefficients, and hence, these methods do not exploit key clustering and persistence properties. Markovian dependencies between the hidden state variables are introduced to characterize the key dependencies between the wavelet coefficients. These dependencies are described by a probabilistic graph or tree as shown in Fig. 1(b) that are referred to as hidden Markov models (HMM’s). Fig. 4. Open in new tabDownload slide Comparisons of horizontal line profiles at the center in the vertical direction of the holograms in Fig. 3. Fig. 4. Open in new tabDownload slide Comparisons of horizontal line profiles at the center in the vertical direction of the holograms in Fig. 3. Fig. 5. Open in new tabDownload slide Application results of noise reduction methods for simulated holograms with |$N=256$|⁠, |$m_0=10$|⁠, |$V_s=0.4$|⁠, |$x_s=1$|⁠, |$x_b=0$|⁠, |$\phi =0$|⁠. Fig. 5. Open in new tabDownload slide Application results of noise reduction methods for simulated holograms with |$N=256$|⁠, |$m_0=10$|⁠, |$V_s=0.4$|⁠, |$x_s=1$|⁠, |$x_b=0$|⁠, |$\phi =0$|⁠. Fig. 6. Open in new tabDownload slide Comparisons of horizontal line profiles at the center in the vertical direction of the holograms in Fig. 5. Fig. 6. Open in new tabDownload slide Comparisons of horizontal line profiles at the center in the vertical direction of the holograms in Fig. 5. Simple Markovian structures are considered on the wavelet tree (a pixel on the LL component at a scale and the corresponding 4 pixels on LL, HL, LH and HH components at the lower scale) to account for residual dependencies that remain between the wavelet coefficients. Markov models in the tree are considered to account for the properties of the wavelet transform: clustering and persistence. These structures reflect Markov dependencies between the states (hidden states) of the wavelet coefficients rather than the values of the wavelet coefficients themselves. By fitting an HMM to the noisy signal wavelet coefficients and then subtracting the added variance due to noise, the signal wavelet model can be estimated. Using a 2-state Gaussian mixture model for each wavelet coefficient, the parameters for the hidden Markov tree (HMT) model are 1) $p_{S_0}(m)$⁠, the probability mass function (pmf) for the root node $S_0$⁠; 2) $\epsilon ^{mr}_{i,\rho (i)}$⁠, the conditional probability that $S_i$ is in state $m$ given $S_{\rho (i)}$ is in state $r$⁠; 3) $\sigma ^2_{i,m}$⁠, the variance of the wavelet coefficient $w_i$ given $S_i$ is in state $m$⁠. (the mean of the wavelet coefficient is assumed to be 0 at all nodes). These parameters can be grouped into a model parameter vector |$\boldsymbol{\theta }$|⁠. There are two steps associated with the wavelet-domain HMM’s where the image size is assumed as |$N\times N$|⁠: a) Training: Given one or more sets of observed wavelet coefficients (⁠$\mathbf{w}=\{w_0,w_1,\cdots ,w_{N^2-1}\}$⁠), determine the wavelet-domain HMT parameters $\boldsymbol{\theta }$ that best characterize the wavelet coefficients. The Baum-Welch algorithm, one of the expectation-maximization (EM) algorithm, is used to estimate HMT parameters. b) State Estimation: Given a fixed wavelet-domain HMT with parameters, determine the most likely sequence of hidden states for an observed set of wavelet coefficients. Once we have trained the HMT in step (a), state probabilities are obtained in step (b). The denoised signal wavelet coefficients (⁠|$\mathbf{w}^{\prime}=\{w_0^{\prime},w_1^{\prime},\cdots ,w_{N^2-1}^{\prime}\}$|⁠) are predicted by an empirical Bayesian estimation procedure as follows: $$\begin{eqnarray}w_i^{\prime}=\sum_{m=1}^M p(S_i=m|\mathbf{w},\boldsymbol{\theta}) \frac{\sigma^2_{i,m}}{\kappa \sigma^2_n+\sigma^2_{i,m}} w_i.\end{eqnarray}$$(2) Here, |$p(S_i=m|\mathbf{w},\boldsymbol{\theta })$| is the hidden state probabilities given |$\mathbf{w}$| and the model |$\boldsymbol{\theta }$|⁠. |$\sigma ^2_n$| (the variance of wavelet coefficients in |$HH^1$| in Fig. 1(b)) is an estimation value of additive noise power. |$\kappa$| is an adjustment parameter for noise power. The final denoised signal is computed as the inverse wavelet transform of these estimates of the signal wavelet coefficients. Note that only the wavelet coefficients are processed. The original scaling coefficients are used in the inverse transform. The overview of the WHMM denoising is shown in Fig. 1(b) with 2 wavelet scales. 3 Results This section gives some simulation and experimental results for electron holography using various noise reduction methods; WF, MF, BF, WT (includes VST and inverse VST), NLM, TVD and WHMM (includes VST and inverse VST). 3.1 Performance evaluation index We evaluate noise reduction methods by using two quantitative indices. One is the peak signal-to-noise ratio (PSNR) between noise-free (ground-truth) and the target images used commonly in image processing. The other is the root mean-square error (RMSE) of the DFT phase estimator, |$\langle \Delta \phi \rangle$|⁠. PSNR of the hologram PSNR is defined by $$\begin{equation}{PSNR}=10 log_{10} MAX_{I}^2/MSE,\end{equation}$$(3) where |$MAX_{I}$| is the maximum possible value of the hologram. We will use |$MAX_{I}=1$| in the following experiments. |$MSE$| is the mean-squared error for a noise-free simulation hologram |$s(x,y)$| and the target hologram |$I(x,y)$| (noisy or denoised holograms), as defined in the following equation: $$\begin{eqnarray}\mbox{MSE}=\frac{1}{N^2}\sum_{x=0}^{N-1}\sum_{y=0}^{N-1} \{s(x,y)-I(x,y)\}^2,\end{eqnarray}$$(4) where |$N\times N$| is the size of a 2D hologram. The more the denoised hologram closed to its noise-free simulation hologram, the more the value of PSNR increases. RMSE of the fringe phase The electron hologram with interference fringes oriented parallel to the |$y$|-axis can be written as $$\begin{eqnarray}N_e(x,y)=x_s[1+V_s \cos(2\pi x m_0 /N +\phi)]+x_b, \nonumber\\ x=0, 1, \cdots, N-1, \,\,y=0, 1, \cdots, N-1,\end{eqnarray}$$(5) where the parameters |$x_s$| and |$x_b$| are the spatially averaged mean signal and background counts per detector. |$V_s$| is the fringe visibility. |$m_0$| and |$\phi$| are the spatial period and phase of the fringe, respectively. Walkup and Goodman [7] approximated the RMSE of the fringe phase as the root mean square (RMS) SNR of phase fluctuation of the DFT component with index |$m_0$|⁠. The noise contained in a hologram results in an error |$\Delta \phi$| between the measured and the true phases of the fringe pattern. |$\gamma (m_0)$| is an RMS SNR of the DFT phase estimator derived from the assumption of a one-dimensional spatial array of photon-counting detectors. |$\gamma (m_0)$| is given by the following equations: $$\begin{equation}\bar{R}(m_0)=\frac{x_s V_s}{2} \cos\phi,\end{equation}$$(6) $$\begin{equation}\bar{I}(m_0)=\frac{x_s V_s}{2} \sin\phi,\end{equation}$$(7) $$\begin{equation}\sigma^2(m_0)=\sigma^2[R(m_0)]=\sigma^2[I(m_0)]=\frac{(x_s+x_b)}{2N},\end{equation}$$(8) $$\begin{equation}\gamma(m_0)= \left[\frac{R^2(m_0)+I^2(m_0)}{\sigma^2(m_0)}\right]^{\frac{1}{2}} =V_s\left[\frac{N x_s}{2}\right]^{\frac{1}{2}} \left[1+\frac{x_b}{x_s}\right]^{-\frac{1}{2}},\end{equation}$$(9) where |$\bar{R}(m_0)$| and |$\bar{I}(m_0)$| are real and imaginary parts of the DFT component with index |$m_0$|⁠, respectively. The variance of |$R(m_0)$| and |$I(m_0)$| are theoretically both |$\sigma ^2(m_0)$|⁠. The RMSE of the fringe phase |$\langle \Delta \phi \rangle$| is approximated by $$\begin{equation}\langle\Delta\phi\rangle\cong[\gamma(m_0)]^{-1}= \sqrt{\frac{2(1+x_b/x_s)}{V_s^2 N x_s}}.\end{equation}$$(10) Each noise reduction method is applied to a 2D hologram. Then |$\langle \Delta \phi \rangle$| is calculated on one horizontal line (⁠|$N$|⁠) at the center in the vertical direction of the denoised hologram. The higher denoise performance, the smaller |$\langle \Delta \phi \rangle$| will be than the approximation value. Table 1 Quantitative comparison of denoising performance by PSNR [dB] and RMSE [rad] for simulation holograms . Figure 3 . Figure 5 . . PSNR . RMSE . PSNR . RMSE . (b) Noisy holo. -2.14E-3 0.0935 2.86E-5 0.2213 (c) BF 3.56 0.0520 4.35 0.0955 (d) TVD -1.10 0.0810 0.698 0.2033 (e) WT 13.3 0.0509 15.71 0.1417 (f) MF 6.97 0.0719 6.93 0.1590 (g) WF 7.09 0.0496 9.60 0.1091 (h) NLM 7.21 0.0359 12.7 0.0968 (i) WHMM 19.0 0.0189 20.686 0.0499 . Figure 3 . Figure 5 . . PSNR . RMSE . PSNR . RMSE . (b) Noisy holo. -2.14E-3 0.0935 2.86E-5 0.2213 (c) BF 3.56 0.0520 4.35 0.0955 (d) TVD -1.10 0.0810 0.698 0.2033 (e) WT 13.3 0.0509 15.71 0.1417 (f) MF 6.97 0.0719 6.93 0.1590 (g) WF 7.09 0.0496 9.60 0.1091 (h) NLM 7.21 0.0359 12.7 0.0968 (i) WHMM 19.0 0.0189 20.686 0.0499 Open in new tab Table 1 Quantitative comparison of denoising performance by PSNR [dB] and RMSE [rad] for simulation holograms . Figure 3 . Figure 5 . . PSNR . RMSE . PSNR . RMSE . (b) Noisy holo. -2.14E-3 0.0935 2.86E-5 0.2213 (c) BF 3.56 0.0520 4.35 0.0955 (d) TVD -1.10 0.0810 0.698 0.2033 (e) WT 13.3 0.0509 15.71 0.1417 (f) MF 6.97 0.0719 6.93 0.1590 (g) WF 7.09 0.0496 9.60 0.1091 (h) NLM 7.21 0.0359 12.7 0.0968 (i) WHMM 19.0 0.0189 20.686 0.0499 . Figure 3 . Figure 5 . . PSNR . RMSE . PSNR . RMSE . (b) Noisy holo. -2.14E-3 0.0935 2.86E-5 0.2213 (c) BF 3.56 0.0520 4.35 0.0955 (d) TVD -1.10 0.0810 0.698 0.2033 (e) WT 13.3 0.0509 15.71 0.1417 (f) MF 6.97 0.0719 6.93 0.1590 (g) WF 7.09 0.0496 9.60 0.1091 (h) NLM 7.21 0.0359 12.7 0.0968 (i) WHMM 19.0 0.0189 20.686 0.0499 Open in new tab Fig. 7. Open in new tabDownload slide Dependence of |$\gamma (m_0)$| on PSNR of the hologram by changing the average number of electrons. Fig. 7. Open in new tabDownload slide Dependence of |$\gamma (m_0)$| on PSNR of the hologram by changing the average number of electrons. Fig. 8. Open in new tabDownload slide Dependence of |$\gamma (m_0)$| on RMSE of the fringe phase by changing the average number of electrons. Fig. 8. Open in new tabDownload slide Dependence of |$\gamma (m_0)$| on RMSE of the fringe phase by changing the average number of electrons. 3.2 Experimental environment All simulation experiments were performed on MATLAB 2018b including Wavelet and Image Processing toolboxes in the Windows 10 (64bit) environment with an Intel Core i7-7700HQ CPU@2.80GHz and 64GB of RAM. Fig. 9. Open in new tabDownload slide An example of the observed electron hologram in a vacuum. The hologram was recorded using a direct electron detection camera in electron counting mode. Fig. 9. Open in new tabDownload slide An example of the observed electron hologram in a vacuum. The hologram was recorded using a direct electron detection camera in electron counting mode. Fig. 10. Open in new tabDownload slide Application results of noise reduction methods for an observed hologram with exposure time 1.0 [sec]. Fig. 10. Open in new tabDownload slide Application results of noise reduction methods for an observed hologram with exposure time 1.0 [sec]. Each noise reduction method has different parameters. In this experiment, the following conditions were used. WF:$[5\times 5]$ neighborhood size was used to estimate the local image mean and standard deviation. Additive noise power was obtained by the mean of the local variance. MF: Each output pixel was given by the median value in the $[5\times 5]$ neighborhood around the corresponding pixel. BF: Twice the variance (equal to average value) of Poisson noise was given as the amount of smoothing. The standard deviation of the spatial Gaussian smoothing filter was 1. WT:wthrmngr and wdencmp functions are used to determine level-dependent thresholds and to denoise, respectively. Thresholds were determined based on the Birgé–Massart strategy (thresholding method = penalhi in MATLAB) with the sparsity parameter $\alpha =2.5$⁠. The hard thresholding was applied with different thresholds for each orientation (horizontal, vertical, and diagonal) and each wavelet level. 4 wavelet scales and Daubechies 8 were used. The wavelet type and the threshold determination method were selected based on past comparative experiments [29]. NLM: The degree of smoothing was determined by the standard deviation of noise in the input image. The search and comparison window sizes were $s=21$ and $c=5$⁠, respectively. The NLM searches for similar neighborhoods within the $[s\times s]$ region surrounding the target pixel and computes similarity weights using the $[c\times c]$ neighborhood. TVD: We used the TV-L1 model optimized with a primal-dual algorithm [27]. The regularization parameter and the number of iteration were 0.01 and 100, respectively. WHMM: Two hidden states (one group with large wavelet coefficients and the other group with small wavelet coefficients) are assumed. Like WT, 4 wavelet scales and Daubechies 8 were selected. The adjustment parameter of noise power was $\kappa =1.5$⁠. These values were adjusted through the preliminary simulation results shown in Fig. 2, where seven kinds of wavelets were used to compare denoise performance; haar, biorsplines (bior)2.6, bior6.8, symlets (sym)8, coiflets (coif)4, discrete meyer (dmey), Daubechies (db)8. Figure 3(b) was used in the preliminary simulation results. 3.3 Simulation results 3.3.1 Simulation of the electron hologram We simulated a 2D electron hologram with the shot noise using the following equation: $$\begin{equation}J(x,y)=poisson\_random(N_e(x,y)),\end{equation}$$(11) where |$poisson\_random(a)$| is a random number generated from Poisson-distribution with |$\sigma ^2=a$|⁠. Figures 3 (a) and 5(a) show the noise-free simulation holograms |$N_e(x,y)$| calculated by equations (5) with |$V_s=1.0$| and |$V_s=0.4$|⁠, respectively. Other hologram parameters were common, |$N=256$|⁠, |$m_0=10$|⁠, |$x_s=1$|⁠, |$x_b=0$|⁠, |$\phi =0$|⁠. In this case, the maximum value of |$N_e(x,y)$| were 2.0 and 1.4, respectively. Figures 3(b) and 5(b) were simulation holograms with Poisson noise |$J(x,y)$|⁠. |$\gamma (m_0)$| of these holograms were 11.31 and 4.53 from equation (9), respectively. Also, the mean-square phase errors of both noisy holograms were calculated as 0.0884 and 0.221 from equations (9) and (10), respectively. 3.3.2 Denoised holograms We applied noisy reduction methods (BF, TVD, WT, MF, WF, NLM, TVD and WHMM) to noisy simulation holograms with different |$V_s$|⁠. Figures 3 and 5(c–i) show the denoised holograms by using each method. Figures 4 and 6 also show horizontal line profiles at the center in the vertical direction of each hologram in Figs 3 and 5, respectively. Results for the WHMM denoising have fringe patterns close to that in the noise-free simulation hologram, which shows that it has a higher noise suppression effect while suppressing the occurrence of artifacts as compared with the other noise reduction methods. Table 1 shows also PSNR of noisy and denoised holograms and RMSE of the fringe phase estimated from each simulation hologram shown in Figs 3 and 5. It means that improving the image quality of the hologram leads to phase error reduction. The WHMM denoising showed the best performance in the simulation experiments. The RMSE can be reduced to about 1/4.5 compared to the original noisy hologram. 3.3.3 Dependence of $\gamma (m_0)$ on PSNR and RMSE We obtained the dependence of the SNR of phase fluctuation based on the DFT phase estimator, |$\gamma (m_0)$|⁠, on the PSNR of the hologram and the RMSE of the fringe phase by changing the average number of electrons (correspond to 0.5 to 15 in |$\gamma (m_0)$|⁠). Figure 7 shows the relationship between |$\gamma (m_0)$| and the PSNR of the hologram. It can be seen that the PSNR is improved by each noise reduction at any |$\gamma (m_0)$| except TVD. When using Bayesian image restoration, it is necessary to consider prior information more suitable for electron holograms than the total variation prior. Figure 8 shows the relationship between |$\gamma (m_0)$| and the RMSE of the fringe phase. The RMSE for the noisy simulation hologram was plotted with orange circle symbols. A blue line is the approximation curve calculated by equation (10). This is a reproduction of the results of Walkup and Goodman [7]. The RMSEs for the denoised hologram by each noise reduction method were plotted with different symbols and lines. From this result, it can be seen that the WHMM denoising has stable and good performance for noise reduction of holograms with various image qualities and it can reduce the RMSE from 1/5 to 1/3. Fig. 11. Open in new tabDownload slide Comparisons of horizontal line profiles at the center in the vertical direction of the holograms in Fig. 10. Fig. 11. Open in new tabDownload slide Comparisons of horizontal line profiles at the center in the vertical direction of the holograms in Fig. 10. Fig. 12. Open in new tabDownload slide Comparisons of the RMSE of the fringe phase for observed holograms with different exposure times 0.5, 1.0 and 2.0 [sec]. Fig. 12. Open in new tabDownload slide Comparisons of the RMSE of the fringe phase for observed holograms with different exposure times 0.5, 1.0 and 2.0 [sec]. 3.4 Experimental results We took electron holograms in vacuum with different exposure times; 0.5, 1.0, 2.0 and 40 (2.0 sec |$\times$| 20 shots) [sec]. The experiment was performed with a HITACHI ultra-high-voltage (1.2 MV) holography electron microscope [33, 34]. Holograms were recorded using a direct electron detection camera (K2 Summit, Gatan Inc.) in electron counting mode with the dark and gain correction. Figure 9 shows an observed electron hologram with exposure time 40 [sec]. We extracted the region of interest by image rotation and trimming. Figure 10(a and b) show extracted regions from holograms with exposure time 40 and 1.0 [sec], respectively. Figure 10(c–i) show denoised holograms obtained by applying each noise reduction method to Fig. 10(b) as well as for the simulation hologram. Figure 11 also shows horizontal line profiles at the center in the vertical direction of each hologram in Fig. 10 after position alignment. It was confirmed that the fringe period of the hologram observed with a high electron dose was close to the fringe period of the denoised hologram by the WHMM denoising. Next, we obtained a phase of each horizontal line using the DFT phase estimator and then calculated the standard deviation of the fringe phase as the RMSE. Figure 12 shows the RMSEs of the fringe phase estimated from the holograms of every exposure time and noise reduction method. From these results, it is clear that the WHMM denoising has a noise reduction effect superior to the other methods even for observed holograms. 4 Concluding remarks We have tried to improve the limit of the phase estimation of the interference fringe at low electron dose levels in electron holography by using noise reduction methods. Here we focused on unsupervised based noise reductions used widely in image processing so that it can be applied to unknown samples vulnerable to damage by the electron beam. As noise reduction methods, BF, TVD, WT, MF, WF, NLM and WHMM denoising were described. VST was introduced to avoid the signal-dependent noise reduction in wavelet-based denoising methods. We evaluated the effects of noise reduction by using PSNR between noise-free (ground-truth) and the target images used commonly in image processing, and the RMSE between the true phase of the fringe and the measured phase by the DFT phase estimator. From simulation and experimental results, it is found that the WHMM denoising has stable good performance as noise reduction both for the signal intensity of holograms and the phase of the DFT phase estimator. The WHMM denoising can reduce the mean-square phase error to about 1/4.5 than that original noisy simulation hologram. Acknowledgments We would like to gratefully and sincerely thank Yasukazu Murakami of Kyushu University, Hiroyuki Shinada, Toshiaki Tanigaki, Tetsuya Akashi and Yoshio Takahashi of Hitachi Ltd. for their immense help and support. Funding Core Research for Evolutional Science and Technology (CREST), Japan Science and Technology Agency (JST) (Grant No. JPMJCR1664). References 1 Gabor D ( 1948 ) A new microscopic principle . Nature 161 : 777 – 778 . Google Scholar Crossref Search ADS PubMed WorldCat 2 Haine M E and Mulvey T ( 1952 ) The formation of the diffraction image with electrons in the Gabor diffraction microscope . J. Opt. Soc. Am. 42 ( 10 ): 763 – 773 . Google Scholar Crossref Search ADS WorldCat 3 Tonomura A ( 1987 ) Applications of electron holography . Rev. Mod. Phys. 59 ( 3 ): 639 . Google Scholar Crossref Search ADS WorldCat 4 Cowley J M ( 1992 ) Twenty forms of electron holography . Ultramicroscopy 41 ( 4 ): 335 – 348 . Google Scholar Crossref Search ADS WorldCat 5 Lichte H and Lehmann M ( 2007 ) Electron holography – basics and applications . Rep. Prog. Phys. 71 ( 1 ): 016102 . Google Scholar Crossref Search ADS WorldCat 6 Tanigaki T , Harada K, Murakami Y, Niitsu K, Akashi T, Takahashi Y, Sugawara A, and Shindo D ( 2016 ) New trend in electron holography . J. Phys. D Appl. Phys. 49 : 244001 . Google Scholar Crossref Search ADS WorldCat 7 Walkup J F , and Goodman J W ( 1973 ) Limitations of fringe-parameter estimation at low light levels . J. Opt. Soc. Am. 63 : 399 – 407 . Google Scholar Crossref Search ADS WorldCat 8 Volkl E , Allard L F, and Joy D C ( 1999 ) Introduction to electron holography ( Springer Science & Business Media ). Google Scholar Crossref Search ADS Google Scholar Google Preview WorldCat COPAC 9 Lichte H , Herrmann K H, and Lenz F ( 1987 ) Electron noise in off-axis image plane holography . Optik 77 ( 3 ): 135 – 140 . OpenURL Placeholder Text WorldCat 10 De Ruijter W J and Weiss J K ( 1993 ) Detection limits in quantitative off-axis electron holography . Ultramicroscopy 50 ( 3 ): 269 – 283 . Google Scholar Crossref Search ADS WorldCat 11 Harscher A and Lichte H ( 1996 ) Experimental study of amplitude and phase detection limits in electron holography . Ultramicroscopy 64 ( 1–4 ): 57 – 66 . Google Scholar Crossref Search ADS WorldCat 12 Voelkl E ( 2010 ) Noise in off-axis type holograms including reconstruction and CCD camera parameters . Ultramicroscopy 110 ( 3 ): 199 – 210 . Google Scholar Crossref Search ADS PubMed WorldCat 13 Voelkl E and Tang D ( 2010 ) Approaching routine 2|$\pi$|/1000 phase resolution for off-axis type holography. Ultramicroscopy 110 ( 5 ): 447 – 459 . Crossref Search ADS 14 Chang S L , Dwyer C, Barthel J, Boothroyd C B, and Dunin-Borkowski R E ( 2016 ) Performance of a direct detection camera for off-axis electron holography . Ultramicroscopy 161 : 90 – 97 . Google Scholar Crossref Search ADS PubMed WorldCat 15 Roels J , Aelterman J, Luong H, Lippens S, Pižurica A, Saeys Y, and Philips W ( 2018 ) An overview of state-of-the-art image restoration in electron microscopy . J. Microsc. 271 : 239 – 254 . Google Scholar Crossref Search ADS PubMed WorldCat 16 Karczewicz M and Gabbouj M ( 1998 ) Robust B-spline image modeling with application to image processing . IEEE Trans. Image Process. 7 ( 6 ): 912 – 917 . Google Scholar Crossref Search ADS PubMed WorldCat 17 Lim Jae S ( 1990 ) Two-Dimensional Signal and Image Processing ( Prentice Hall ). Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC 18 Tomasi C , and Manduchi R ( 1998 ) Bilateral filtering for gray and color images . In Proc. 1998 IEEE Int. Conf. Comput. Vision, pp 836 – 846 . OpenURL Placeholder Text WorldCat 19 Chipman H , Kolaczyk E, and McCulloch R ( 1997 ) Adaptive Bayesian wavelet shrinkage . J. Amer. Stat. Assoc. 92 ( 440 ): 1413– 1421 Google Scholar Crossref Search ADS WorldCat 20 Donoho D and Johnstone I ( 1995 ) Adapting to unknown smoothness via wavelet shrinkage . J. Amer. Stat. Assoc. 90 ( 432 ): 1200 – 1224 . Google Scholar Crossref Search ADS WorldCat 21 Barsanti R J , and Gilmore J ( 2011 ) Comparing noise removal in the wavelet and Fourier domains. In IEEE 43rd Southeastern Symposium on System Theory, pp 163 – 167 . OpenURL Placeholder Text WorldCat 22 Buades A , Coll B, and Morel J-M ( 2005 ) A Non-Local Algorithm for Image Denoising. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. 2 : 60 – 65 . OpenURL Placeholder Text WorldCat 23 Anada S , Nomura Y, Hirayama T, and Yamamoto K ( 2019 ) Sparse coding and dictionary learning for electron hologram denoising . Ultramicroscopy 206 : 112818 . Google Scholar Crossref Search ADS PubMed WorldCat 24 Haan K , Ballard Z S, Rivenson Y, Wu Y, and Ozcan A ( 2019 ) Resolution enhancement in scanning electron microscopy using deep learning . Sci. Rep. 9 : 12050 . Google Scholar Crossref Search ADS PubMed WorldCat 25 Sorzano C O S , Ortiz E, López M and Rodrigo J ( 2006 ) Improved Bayesian image denoising based on wavelets with applications to electron microscopy . Pattern Recognit. 39 : 1205 – 1213 . 26 Rudin L I , Osher S, and Fatemi E ( 1992 ) Nonlinear total variation based noise removal algorithms . Physica D 60 ( 1–4 ): 259 – 268 . Google Scholar Crossref Search ADS WorldCat 27 Zhanjiang Z , Shi B, and Sun Y ( 2016 ) Primal-dual method to smoothing TV-based model for image denoising . J. Algorithms Comput. Technol. 10 ( 4 ): 235 – 243 . Google Scholar Crossref Search ADS WorldCat 28 Crouse M S , Nowak R D, and Baraniuk R G ( 1998 ) Wavelet-based statistical signal processing using hidden Markov models . IEEE Trans. Signal Process. 46 ( 4 ): 886 – 902 . Google Scholar Crossref Search ADS WorldCat 29 Midoh Y , and Nakamae K ( 2017 ) Denoising of noisy electron hologram by using a wavelet-based hidden Markov model . Proc. 37th NANO Testing Symposium (NANOTS) , pp 123 – 128 . OpenURL Placeholder Text WorldCat 30 Midoh Y , and Nakamae K ( 2018 ) Effect of variance stabilization on denoising of low SNR fringe patterns using wavelet hidden Markov models . Proc. 38th NANO Testing Symposium (NANOTS) , pp 84 – 87 . OpenURL Placeholder Text WorldCat 31 Midoh Y and Nakamae K (2018) Noise reduction in electron hologram by using dual-tree complex wavelet transform and hidden Markov tree models. Proc. the 19 International Microscopy Congress (IMC19), pp 1062. 32 Makitalo M and Foi A ( 2011 ) Optimal Inversion of the Anscombe Transformation in Low-Count Poisson Image Denoising . IEEE Trans. Image Process. 20 ( 1 ): 99 – 109 . Google Scholar Crossref Search ADS PubMed WorldCat 33 Shinada H ( 2015 ) Development of Holography Electron Microscope with Atomic Resolution . Hitachi Rev. 64 ( 8 ): 533 – 546 . OpenURL Placeholder Text WorldCat 34 Tanigaki T , Akashi T, Takahashi Y, Kawasaki T, and Shinada H ( 2016 ) Quest for ultimate resolution using coherent electron waves: An aberration-corrected high-voltage electron microscope . Adv. Imaging Electron Phys. 198 : 69 – 125 . Google Scholar Crossref Search ADS WorldCat © The authors 2020. Published by Oxford University Press on behalf of the Institute of Mathematics and its Applications. All rights reserved. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model)
TI - Accuracy improvement of phase estimation in electron holography using noise reduction methods
JF - Microscopy
DO - 10.1093/jmicro/dfz115
DA - 2020-04-08
UR - https://www.deepdyve.com/lp/oxford-university-press/accuracy-improvement-of-phase-estimation-in-electron-holography-using-kXIGbC7vrK
SP - 123
VL - 69
IS - 2
DP - DeepDyve
ER -