CycleISP: Real Image Restoration via Improved Data Synthesis
Motivation:
- CNN在AWGN 的数据集上表现很好，但是在real noise 中表现差，其原因是没有考虑到ISP将noise转变了，不在符合假设的高斯噪声。
- Noise at the RAW sensor space is signal-dependent, after demosaicking, it becomes spatio-chromatically correlated; and after passing through the rest of the pipeline, its probability distribution not necessarily remains Gaussian. This implies that the camera ISP heavily transforms the sensor noise, and therefore more sophisticated models that take into account the influence of imaging pipeline are needed to synthesize realistic noise than uniform AWGN model.
The CNNs achieve impressive results on these synthetic datasets, they do not perform well when applied on real camera images, as reported in recent benchmark datasets.
This is mainly because the AWGN is not adequate for modeling the real camera noise which is signal-dependent and heavily transformed by the camera imaging pipeline
On synthetic datasets, existing deep learning-based denoising models yield impressive results, but they exhibit poor generalization to real camera data as compared to conventional methods [8, 15]. This trend is also demonstrated in recent benchmarks [1, 44].
Method
作者提出使用两个对称NN来模拟ISP的处理过程，一个是RGB2RAW， 一个是RAW2RGB。RGB2RAW的网络目的是为了去构建原图的raw图像，在得到了raw图像后，为了生成图像去训练，还需要将raw转回RGB，并且提供可以加噪声的RGB。
这样让NN train 在这个生成的数据集(spatio-aware ISP like noise)而不是AWGN(spatio-none aware independent noise)上面。就可以在real noise中达到较好的效果。
