2.2 Multiscale Digital Rock Images Fusion by CycleGAN
Another challenge in fusing multiscale digital rock images is that the images are acquired at different locations of the rock sample, which means that the training samples are unpaired. For this reason, we adopt a CycleGAN, which is an unsupervised method designed for image cross-domain transfer (Zhu et al., 2017). Specifically, in this study, we aim to transfer the micro-CT images from the low-resolution domainLR to the high-resolution domain HR by integrating the information from SEM data.
CycleGAN consists of four networks: (1) a generator \(G_{L2H}\) mapping the images from domain LR to domain HR, (2) a generator \(G_{H2L}\) mapping the images from domain HR to domain LR, (3) a discriminator \(D_{H}\) aiming to encourage\(G_{L2H}\) to transfer LR into outputs indistinguishable from domain HR and (4) a similar discriminator \(D_{L}\) for\(G_{H2L}\). The major difference of CycleGAN from traditional GANs is that it includes two generators to constrain each other. It ensures that\(G_{L2H}\) outputs the high-resolution image that is conditional to the input low-resolution image and vice versa for \(G_{H2L}\). To achieve this goal, apart from the adversarial loss (Eq. 1), we need to add a regularization term to the loss function of the generators, namely the cycle-consistent loss given by
\(\mathcal{L}_{\text{CYC}}\left(G_{L2H},\ G_{H2L}\right)=E_{x\sim p_{\text{LR}}\left(x\right)}\left[\left\|G_{H2L}\left(G_{L2H}\left(x\right)\right)\ -x\right\|_{1}\right]+E_{y\sim p_{\text{HR}}(y)}\left[\left\|G_{L2H}\left(G_{H2L}\left(y\right)\right)\ -y\right\|_{1}\right]\ \ (2)\).
The first term in Eq. 2 aims that for each image \(x\) from domainLR, the image transfer cycle can transform \(x\) back to the original input, i.e., \(G_{H2L}\left(G_{L2H}\left(x\right)\right)\ \approx x\). Similarly, for each image \(y\) from domain HR, the second term in Eq. 2 ensures backward cycle consistency:\(G_{L2H}\left(G_{H2L}\left(y\right)\right)\ \approx y\). A detailed network architecture of CycleGAN can be found in the Supporting Information Figure S3.