Comparative Analysis on Deep Learning based Pan-Sharpening of Very High-Resolution Satellite Images
Yıl: 2021 Cilt: 8 Sayı: 2 Sayfa Aralığı: 150 - 165 Metin Dili: İngilizce DOI: 10.30897/ijegeo.834760 İndeks Tarihi: 02-10-2021
Comparative Analysis on Deep Learning based Pan-Sharpening of Very High-Resolution Satellite Images
Öz: Pan-sharpening is a fundamental task of remote sensing, aiming to produce a synthetic image having high spatial and spectralresolution of original panchromatic and multispectral images. In recent years, as in other tasks of the remote sensing field, deeplearning based approaches have been developed for this task. In this research, a detailed comparative analysis was conducted toevaluate the performance and visual quality of pan-sharpening results from traditional algorithms and deep learning-based models.For this purpose, the deep learning based methods that are CNN based pan-sharpening (PNN), Multiscale and multi-depthconvolutional neural networks (MSDCNN) and Pan-sharpened Generative Adversarial Networks (PSGAN) and traditional methodsthat are Brovey, PCA, HIS, Indusion and PRACS were applied. Analysis was performed on regions with different land covercharacteristics to evaluate the stability of the methods. In addition, effects of the filter size, spectral indices, activation and lossfunctions on the pan-sharpening were investigated. For the accuracy assessment, commonly used with-reference and withoutreference quality metrics were computed in addition to visual quality evaluations. According to results, the deep learning-basedmethods provided promising results in both the reduced resolution and full resolution experiments, while PRACS methodoutperformed other traditional algorithms in most of the experimental configurations.
Anahtar Kelime: Belge Türü: Makale Makale Türü: Araştırma Makalesi Erişim Türü: Erişime Açık
- Aiazzi, B., Alparone, L., Baronti, S., Garzelli, A. (2002). Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis. IEEE Transactions on Geoscience and Remote Sensing. doi.org/ 10.1109/TGRS.2002.803623.
- Aiazzi, B., Alparone, L., Baronti, S., Garzelli, A., Selva, M. (2003). An MTF-based spectral distortion minimizing model for pan-sharpening of very high resolution multispectral images of urban areas. 2nd GRSS/ISPRS Joint Workshop on Remote Sensing and Data Fusion over Urban Areas, URBAN 2003. doi.org/10.1109/DFUA.2003.1219964
- AIRBUS. (2020). Pleiades Products. https://www.intelligence-airbusds.com/optical-andradar-data/#pleiades
- Chavez Jr., P. S., Yaw Kwarteng, A. (1989). Extracting spectral contrast in Landsat Thematic Mapper image data using selective principal component analysis. Photogrammetric Engineering and Remote Sensing.
- Cheng, M., Wang, C., Li, J. (2014). Sparse representation based pansharpening using trained dictionary. IEEE Geoscience and Remote Sensing Letters. doi.org/10.1109/LGRS.2013.2256875
- Choi, J., Yu, K., Kim, Y. (2011). A new adaptive component-substitution-based satellite image fusion by using partial replacement. IEEE Transactions on Geoscience and Remote Sensing. doi.org/10.1109/TGRS.2010.2051674
- Christian Ledig Ferenc Huszar, L. T. (2017). PhotoRealistic Single Image Super-Resolution Using a Generative Adversarial Network. 19.
- Çeli̇ k, O., Gazi̇ oğlu, C. (2020). Coastline Difference Measurement (CDM) Method. International Journal of Environment and Geoinformatics, 7(1), 1-5.doi. 10.30897/ijegeo.706792.
- DigitalGlobe. (2020). Tools and Resources. http://www.digitalglobe.com/resources.
- Dong, C., Loy, C. C., He, K., Tang, X. (2016). Image Super-Resolution Using Deep Convolutional Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence. doi.org/10.1109/TPAMI.2015.2439281
- Erdem, F., Avdan, U. (2020). Comparison of Different U-Net Models for Building Extraction from HighResolution Aerial Imagery. International Journal of Environment and Geoinformatics, 7(3), 221-227. doi.10.30897/ijegeo.684951
- Gao, B. (1996). NDWI—A Normalized Difference Water Index for Remote Sensing of Vegetation Liquid Water From Space. Remote Sensing of Environment, 266(April), 257–266. doi.org/10.1016/S0034-4257(96)00067-3
- Gillespie, A. R., Kahle, A. B., Walker, R. E. (1987). Color enhancement of highly correlated images. II. Channel ratio and “chromaticity” transformation techniques. Remote Sensing of Environment. https://doi.org/10.1016/0034-4257(87)90088-5
- Goodfellow, I. J., and Pouget-Abadie, J. (2014). Generative Adversarial Nets. Veterinary Immunology and Immunopathology, 155(4), 270–275. https://doi.org/10.1016/j.vetimm.2013.08.005
- He, K., Zhang, X., Ren, S., Sun, J. (2015). Deep Residual Learning for Image Recognition. doi.org/10.1109/CVPR.2016.90
- Kang, X., Li, S., Benediktsson, J. A. (2014). Pansharpening with Matting Model. IEEE Transactions on Geoscience and Remote Sensing. doi.org/10.1109/TGRS.2013.2286827
- Khan, M. M., Chanussot, J., Condat, L., Montanvert, A. (2008). Indusion: Fusion of multispectral and panchromatic images using the induction scaling technique. IEEE Geoscience and Remote Sensing Letters, 5(1), 98–102.doi.org/10.1109/LGRS.2007. 909934
- Laben, C. Brower, B. . (2000). Process for enhancing the spatial resolution of multispectral imagery using pansharpening. United States Patent 6. https://doi.org/10.1074/JBC.274.42.30033.(51)
- Liu, J. G. (2000). Smoothing Filter-based Intensity Modulation: A spectral preserve image fusion technique for improving spatial details. International Journal of Remote Sensing. doi.org/10.1080/014311600750037499
- Liu, J. G. (2010). Smoothing Filter-based Intensity Modulation : A spectral preserve image fusion technique for improving spatial details. 1161. doi.org/10.1080/014311600750037499
- Liu, X., Wang, Y., and Liu, Q. (2018a). PSGAN: A Generative Adversarial Network for Remote Sensing Image Pan-Sharpening. 1–5. doi.org/10.1109/ICIP.2018.8451049
- Liu, X., Wang, Y., Liu, Q. (2018b). Remote Sensing Image Fusion Based on Two-Stream Fusion Network. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 10704 LNCS, 428–439. https://doi.org/10.1007/978- 3-319-73603-7_35
- Masi, G., Cozzolino, D., Verdoliva, L., Scarpa, G. (2016). Pansharpening by convolutional neural networks. Remote Sensing. doi.org/10.3390/rs8070594
- Nencini, F., Garzelli, A., Baronti, S., Alparone, L. (2007). Remote sensing image fusion using the curvelet transform. Information Fusion. doi.org/10.1016/j.inffus.2006.02.001
- Nouri, H., Beecham, S., Anderson, S., and Nagler, P. (2013). High spatial resolution WorldView-2 imagery for mapping NDVI and its relationship to temporal urban landscape evapotranspiration factors. Remote Sensing, 6(1), 580–602. doi.org/10.3390/rs6010580
- Open Remote Sensing. (2016). A Critical Comparison among Pansharpening Algorithms. http://openremotesensing.net/
- Ozcelik, F., Alganci, U., Sertel, E., Unal, G. (2020). Rethinking CNN-Based Pansharpening: Guided Colorization of Panchromatic Images via GANs. IEEE Transactions on Geoscience and Remote Sensing, 1–16. doi.org/10.1109/tgrs.2020.3010441
- Ozturk, O., Saritürk, B., Seker, DZ. (2020). Comparison of Fully Convolutional Networks (FCN) and U-Net for Road Segmentation from High Resolution Imageries. International Journal of Environment and Geoinformatics, 7(3), 272-279.doi.10.30897/ ijegeo.737993
- Palsson, F., Ulfarsson, M. O., Sveinsson, J. R. (2020). Model-Based Reduced-Rank Pansharpening. IEEE Geoscience and Remote Sensing Letters. doi.org/10.1109/LGRS.2019.2926681
- Panchal, S., Thakker, R. (2015). Implementation and Comparative Quantitative Assessment of Different Multispectral Image Pansharpening Approaches. Signal and Image Processing : An International Journal, 6(5), 35–48. doi.org/10. 5121/sipij.2015.6503
- Pohl, C., Van Genderen, J. L. (1998). Review article Multisensor image fusion in remote sensing: Concepts, methods and applications. In International Journal of Remote Sensing (Vol. 19, Issue 5). doi.org/10.1080/014311698215748
- Rouse, J.W.; Haas, R.H.; Schell, J.A.; Deering, D.W.; Harlan., J. C. (1974). Monitoring the vernal advancement and retrogradation (green wave effect) of natural vegetation. 1–8.
- Scarpa, G., Vitale, S., Cozzolino, D. (2018). TargetAdaptive CNN-Based Pansharpening. IEEE Transactions on Geoscience and Remote Sensing. doi.org/10.1109/TGRS.2018.2817393
- Shettigara, V. K. (1992). A generalized component substitution technique for spatial enhancement of multispectral images using a higher resolution data set. Photogrammetric Engineering and Remote Sensing. doi.org/10.1038/050073a0
- Tensorflow. (2015). https://www.tensorflow.org/
- Tonbul, H., Kavzoglu, T. (2020). A Spectral Band Based Comparison of Unsupervised Segmentation Evaluation Methods for Image Segmentation Parameter Optimization. International Journal of Environment and Geoinformatics, 7(2), 132-139. doi.10.30897/ijegeo.641216
- Tu, T.-M., Su, S.-C., Shyu, H.-C., Huang, P. S. (2001). A new look at IHS-like image fusion methods. Information Fusion. doi.org/10.1016/S1566- 2535(01)00036-7
- Vivone, G., Alparone, L., Chanussot, J., Dalla Mura, M., Garzelli, A., Licciardi, G. A., Restaino, R., Wald, L. (2015). A critical comparison among pansharpening algorithms. IEEE Transactions on Geoscience and Remote Sensing. doi.org/10.1109/TGRS.2014. 2361734
- Wald, L., Ranchin, T., Mangolini, M. (1997). Fusion of satellite images of different spatial resolutions : Assessing the quality of resulting images. Photogrammetric Engineering and Remote Sensing.
- Wei, Y., Yuan, Q. (2017). Deep residual learning for remote sensed imagery pansharpening. RSIP 2017 - International Workshop on Remote Sensing with Intelligent Processing, Proceedings. doi.org/10.1109/RSIP.2017.7958794
- Yang, X., Jian, L., Yan, B., Liu, K., Zhang, L., Liu, Y. (2018). A sparse representation based pansharpening method. Future Generation Computer Systems, 88(June), 385–399. doi.org/10.1016/j.future.2018. 04.096
- Yin, H. (2015). Sparse representation based pansharpening with details injection model. Signal Processing. doi.org/10.1016/j.sigpro.2014.12.017
- Yuan, Q., Wei, Y., Meng, X., Shen, H., Zhang, L. (2018). A Multiscale and Multidepth Convolutional Neural Network for Remote Sensing Imagery PanSharpening. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. doi.org/10.1109/JSTARS.2018.2794888
- Yuhas, R., Goetz, A. F. H., Boardman, J. W. (1992). Descrimination among semi-arid landscape endmembers using the Spectral Angle Mapper (SAM) algorithm. Summaries of the Third Annual JPL Airborne Geoscience Workshop, JPL Publ. 92– 14, Vol. 1.
- Zhong, S., Zhang, Y., Chen, Y., Wu, D. (2017). Combining component substitution and multiresolution analysis: A novel generalized BDSD Pansharpening algorithm. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. doi.org/10.1109/JSTARS.2017.2697445
- Zhou, W., Bovik, a C. (2002). A universal image quality index. Signal Processing Letters, IEEE. doi.org/10.1109/97.995823
- Zhu, X. X., Bamler, R. (2013). A sparse image fusion algorithm with application to pan-sharpening. IEEE Transactions on Geoscience and Remote Sensing. https://doi.org/10.1109/TGRS.2012.2213604
| APA | WANG p, Alganci U, Sertel E (2021). Comparative Analysis on Deep Learning based Pan-Sharpening of Very High-Resolution Satellite Images. , 150 - 165. 10.30897/ijegeo.834760 |
| Chicago | WANG peijuan,Alganci Ugur,Sertel Elif Comparative Analysis on Deep Learning based Pan-Sharpening of Very High-Resolution Satellite Images. (2021): 150 - 165. 10.30897/ijegeo.834760 |
| MLA | WANG peijuan,Alganci Ugur,Sertel Elif Comparative Analysis on Deep Learning based Pan-Sharpening of Very High-Resolution Satellite Images. , 2021, ss.150 - 165. 10.30897/ijegeo.834760 |
| AMA | WANG p,Alganci U,Sertel E Comparative Analysis on Deep Learning based Pan-Sharpening of Very High-Resolution Satellite Images. . 2021; 150 - 165. 10.30897/ijegeo.834760 |
| Vancouver | WANG p,Alganci U,Sertel E Comparative Analysis on Deep Learning based Pan-Sharpening of Very High-Resolution Satellite Images. . 2021; 150 - 165. 10.30897/ijegeo.834760 |
| IEEE | WANG p,Alganci U,Sertel E "Comparative Analysis on Deep Learning based Pan-Sharpening of Very High-Resolution Satellite Images." , ss.150 - 165, 2021. 10.30897/ijegeo.834760 |
| ISNAD | WANG, peijuan vd. "Comparative Analysis on Deep Learning based Pan-Sharpening of Very High-Resolution Satellite Images". (2021), 150-165. https://doi.org/10.30897/ijegeo.834760 |
| APA | WANG p, Alganci U, Sertel E (2021). Comparative Analysis on Deep Learning based Pan-Sharpening of Very High-Resolution Satellite Images. International Journal of Environment and Geoinformatics, 8(2), 150 - 165. 10.30897/ijegeo.834760 |
| Chicago | WANG peijuan,Alganci Ugur,Sertel Elif Comparative Analysis on Deep Learning based Pan-Sharpening of Very High-Resolution Satellite Images. International Journal of Environment and Geoinformatics 8, no.2 (2021): 150 - 165. 10.30897/ijegeo.834760 |
| MLA | WANG peijuan,Alganci Ugur,Sertel Elif Comparative Analysis on Deep Learning based Pan-Sharpening of Very High-Resolution Satellite Images. International Journal of Environment and Geoinformatics, vol.8, no.2, 2021, ss.150 - 165. 10.30897/ijegeo.834760 |
| AMA | WANG p,Alganci U,Sertel E Comparative Analysis on Deep Learning based Pan-Sharpening of Very High-Resolution Satellite Images. International Journal of Environment and Geoinformatics. 2021; 8(2): 150 - 165. 10.30897/ijegeo.834760 |
| Vancouver | WANG p,Alganci U,Sertel E Comparative Analysis on Deep Learning based Pan-Sharpening of Very High-Resolution Satellite Images. International Journal of Environment and Geoinformatics. 2021; 8(2): 150 - 165. 10.30897/ijegeo.834760 |
| IEEE | WANG p,Alganci U,Sertel E "Comparative Analysis on Deep Learning based Pan-Sharpening of Very High-Resolution Satellite Images." International Journal of Environment and Geoinformatics, 8, ss.150 - 165, 2021. 10.30897/ijegeo.834760 |
| ISNAD | WANG, peijuan vd. "Comparative Analysis on Deep Learning based Pan-Sharpening of Very High-Resolution Satellite Images". International Journal of Environment and Geoinformatics 8/2 (2021), 150-165. https://doi.org/10.30897/ijegeo.834760 |
