2.2 Multiscale Digital Rock Images Fusion by CycleGAN
Another challenge in fusing multiscale digital rock images is that the
images are acquired at different locations of the rock sample, which
means that the training samples are unpaired. For this reason, we adopt
a CycleGAN, which is an unsupervised method designed for image
cross-domain transfer (Zhu et al., 2017). Specifically, in this study,
we aim to transfer the micro-CT images from the low-resolution domainLR to the high-resolution domain HR by integrating
the information from SEM data.
CycleGAN consists of four networks: (1) a generator \(G_{L2H}\) mapping
the images from domain LR to domain HR, (2) a
generator \(G_{H2L}\) mapping the images from domain HR to
domain LR, (3) a discriminator \(D_{H}\) aiming to encourage\(G_{L2H}\) to transfer LR into outputs indistinguishable
from domain HR and (4) a similar discriminator \(D_{L}\) for\(G_{H2L}\). The major difference of CycleGAN from traditional GANs is
that it includes two generators to constrain each other. It ensures that\(G_{L2H}\) outputs the high-resolution image that is conditional to the
input low-resolution image and vice versa for \(G_{H2L}\). To achieve
this goal, apart from the adversarial loss (Eq. 1), we need to add a
regularization term to the loss function of the generators, namely
the
cycle-consistent loss given by
\(\mathcal{L}_{\text{CYC}}\left(G_{L2H},\ G_{H2L}\right)=E_{x\sim p_{\text{LR}}\left(x\right)}\left[\left\|G_{H2L}\left(G_{L2H}\left(x\right)\right)\ -x\right\|_{1}\right]+E_{y\sim p_{\text{HR}}(y)}\left[\left\|G_{L2H}\left(G_{H2L}\left(y\right)\right)\ -y\right\|_{1}\right]\ \ (2)\).
The first term in Eq. 2 aims that for each image \(x\) from domainLR, the image transfer cycle can transform \(x\) back to the
original input, i.e.,
\(G_{H2L}\left(G_{L2H}\left(x\right)\right)\ \approx x\).
Similarly, for each image \(y\) from domain HR, the second
term in Eq. 2 ensures backward cycle consistency:\(G_{L2H}\left(G_{H2L}\left(y\right)\right)\ \approx y\). A
detailed network architecture of CycleGAN can be found in the Supporting
Information Figure S3.