Adaptive High-Frequency Clipping For Improved Image Quality Assessment

ABSTRACT

It is widely known that the human visual system (HVS) applies multi-resolution analysis to the scenes we see. In fact, many of the best image quality metrics, e.g. MS-SSIM and IW-PSNR/SSIM are based on multi-scale models. However, in existing multi-scale type of image quality assessment (IQA) methods, the resolution levels are fixed. In this paper, we examine the problem of selecting optimal levels in the multi-resolution analysis to preprocess the image for perceptual quality assessment. According to the contrast sensitivity function (CSF) of the HVS, the sampling of visual information by the human eyes approximates a low- pass process. For images, the amount of information we can extract depends on the size of the image (or the object(s) inside) as well as the viewing distance.

Therefore, we proposed a wavelet transform based adaptive high- frequency clipping (AHC) model to approximate the effective visual information that enters the HVS. After the high- frequency clipping, rather than processing separately on each level, we transform the filtered images back to their original resolutions for quality assessment. Extensive experimental results show that on various databases (LIVE, IVC, and Toyama-MICT), performance of existing image quality algorithms (PSNR and SSIM) can be substantially improved by applying the metrics to those AHC model processed images.

INTRODUCTION

Image quality assessment (IQA) is an important research area due to its possible application in the design and optimization of various image processing algorithms. Generally, IQA refers to both subjective assessment and objective assessment. The subjective assessment aims to obtain mean opinion scores (MOSs) from subjective viewing test. Although the subjective assessment is always known as the ultimate image quality gauge, it is often too complicated and expensive for practical applications. Last decades have witnessed the rise of a large number of objective image quality metrics (IQMs), among which PSNR and SSIM [1] are perhaps the most popular methods that have been integrated into many practical image processing systems. In the current IQA research, the so called multi-scale type of methods, e.g. MS-SSIM [2], IFC [3], VIF [4], IW-PSNR/SSIM [5]and MIS-SSIM [6], as inspired by the multi-resolution property of the HVS, achieved some of the best performance.

In the traditional study of IQA, the influence of subjective viewing environments in different databases on the prediction accuracies of objective IQA methods has been largely overlooked. Lin et al pointed out in [7] that objective image quality metrics should take into account some significant external factors, such as ambient illumination, display resolution, and viewing distance. However, as a matter of fact, existing image databases are quite diverse in terms of image sizes and viewing distances used in the subjective experiments [7]. In ITU-R BT.500 [8], although the ratio between viewing distance and image height is suggested, there is no recommendation of appropriate image size, or resolution (DPI).

Realizing the impact of viewing distance and image size on perceptual quality assessment, in our former work, we proposed a self-adaptive scale transform (SAST) model [9] for IQA metrics. The SAST model estimates the best scale parameter from the image size and viewing distance. And the images are then resized (by low-pass filtering and sampling) accordingly for better IQA performance. This approach is effective because as the viewing distance increases, the viewing angel shrinks and less image details can be noticed. On the other hand, it is obvious that by resizing the images, we essentially discard part of the high resolution information. Therefore, in this paper, we try to directly remove part of image details using adaptive high- frequency clipping in wavelet subbands.

The AHC model filtered subband coefficients are then synthesized back to an image at its original resolution to be used by IQA methods.The rest of this paper is organized as follows. In Section 2, we first review some related researches and then propose the AHC model. In Section 3, the AHC model based PSNR/SSIM are compared to some competitive IQAalgorithms on the LIVE database [10], IVC database [11], and Toyama-MICT database [12]. Finally, Section 4 concludes this paper.

THE PROPOSED AHC MODEL

It is suggested in [7] that the external factors, including image size and viewing distance, can have considerable impacts on the prediction performance of IQA algorithms. A simple, empirical method [7] has been exploited for SSIM to determine the downsampling scale S for evaluating images viewed from a typical distance:SI = max(1,round(H/256)) (1) with H being the image height.As mentioned, human perception of images details islimited by both image size and viewing distance. The SAST model [9] was recently proposed using the concept of human visual angle and angle of gaze. The scale transform coefficient is defined as follows: (2)where d is the viewing distance between the viewers and the image X, and W is the image width. θH and θW represent the human visual angles for horizontal and vertical directions, respectively.

Accordingly, we can compute the scale transformed image from X by low-pass filtering and sampling. We denote the results with Z (= I or S) depending on the scale parameter SI or SS.When the ratio between image height and viewing distance (H/d) is small, image details cannot be observed. This research aims to find the correspondence among the if i ∈ LH or HL 1 if i ∈ HH. (4)Then, each computed weight is compared with a threshold (thr = 1 in this work). If the weight is smaller than the threshold, the corresponding subband is clipped out. Wavelet reconstruction is used to get final result XA0. Multi-resolution analysis is used inside the AHC model, the final output is a single scale image with part of high- frequency details properly erased.

Eventually, three pairs of improved PSNRZ and SSIMZ (Z =(5)SSIMZ = SSIM(XZ0 ,YZ0 ).(6)viewing distance, image size, and the visible spatial frequency. To that end, we first apply wavelet transform to where XZ0 and YZ0 are the preprocessed original anddecompose the image X, as exemplified in Fig. 1 on the “lighthouse” image. A weighting function is then introduced to assign different weights to all the LH, HL and HH subbands: (3)where L indicates the decomposition layer, and we set L = 4 in this paper. Namely, the number of total subbands is 12 excluding the LL subband. The coefficient i (= [LH, HL, HH]) represents the subband being processed. The model parameters a (= 10), k (= 10), t (= 2), and d0 (= 512) are empirically assigned. The coefficient b favors the retaining of LH and HL subbands over the HH subband on the same level:distorted images, respectively.

EXPERIMENTAL RESULTS

The experiments are conducted on LIVE [10], IVC [11] and Toyama-MICT [12] databases, because quite different viewing distances and image sizes were used in the subjective experiments, as highlighted in Table 1.We adopt the four-parameter logistic function suggested by VQEG [13] as the nonlinear regression between the sub-Database nameImage size (W×H)d / HNumber768×512480×720640×512632×505LIVE634×5053∼3.75779618×453610×488627×482634×438IVC512×5124185Toyama-MICT768×5126168Acknowledgmentjective scores and the prediction scores of ten metrics PSNR, PSNRI, PSNRS [9], IW-PSNR [5], PSNRA, SSIM, SSIMI, SSIMS [9], IW-SSIM [5] and SSIMA: (7)where x indicates the input score, q(x) is the mapped score, and α1 to α4 are free parameters to be determined during the curve fitting process.

Three commonly used performance metrics, Pearson Linear Correlation Coefficient (PLCC), Spearman Rank-Order Correlation Coefficient (SROCC), and RMSE as suggested by VQEG [13], are employed to further evaluate the proposed AHC model based PSNR/SSIM metric and the other eight IQA methods on LIVE, IVC, and Toyama-MICT databases. It can be found from those results that the proposed AHC model can significantly improve the performance of PSNR and SSIM. The improved metrics PSNRA and SSIMA have prediction accuracy comparable to the most sophisticated metrics of IW-PSNR and IW-SSIM [5].

CONCLUSION

In this paper, we propose a new adaptive high-frequency clipping (AHC) model to improve the performance of IQA metrics. The AHC model is designed within the framework of wavelet transform to remove part of the high-frequency details of the image that is believed to be indiscernible to the audiences at the viewing distance. Experimental results on LIVE, IVC, and Toyama-MICT databases are provided to demonstrate that the proposed AHC model can improve the performance of PSNR and SSIM to the level of most advanced image quality metrics, such as IW-PSNR and IWSSIM in the literature.

References:

1. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE Trans. Image Process., vol. 13, no. 4, pp. 600-612, April 2004.
2. Z. Wang, E. P. Simoncelli, and A. C. Bovik, “Multi-scale structural similarity for image quality assessment,” IEEE
3. H. R. Sheikh, A. C. Bovik, and G. de Veciana, “An information fidelity criterion for image quality assessment using natural scene statistics,” IEEE Trans. Image Process., vol. 14, no. 12, pp. 2117-2128, 2005.
4. H. R. Sheikh and A. C. Bovik, “Image information and visual quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, February 2006.
5. Z. Wang and Q. Li, “Information content weighting for perceptual image quality assessment,” IEEE Trans. Image Process., vol. 20, no. 5, pp. 1185-1198, 2011.
6. M. Liu, G. Zhai, K. Gu, Q. Xu, X. Yang, X. Sun, W. Chen, and Y. Zuo, “A new image quality metric based on mix- scale transform,” Proc. IEEE Workshop on Signal Processing Systems, 2013.
7. W. Lin and C.-C. Jay Kuo, “Perceptual visual quality metrics: A survey,” J. Vis. Commun. Image R., vol. 22, no. 4, pp. 297-312, 2011. ITU, “Methodology for the subjective assessment of the quality of television pictures,” Recommendation, International Telecommunication Union/ITU Radiocommunication Sector, 2009.
8. K. Gu, G. Zhai, X. Yang, and W. Zhang, “Self-adaptive scale transform for IQA metric”, Proc. IEEE International Symposium on Circuits and Systems, 2013.
9. H. R. Sheikh, Z. Wang, L. Cormack, and A. C. Bovik, “LIVE image quality assessment Database Release 2,” [Online]. Available: http://live.ece.utexas.edu/research/quality.
10. A. Ninassi, P. Le Callet, and F. Autrusseau, “Subjective quality assessment-IVC database,” [Online]. Available: http://www2.irccyn.ec-nantes.fr/ivcdb
11. Y. Horita, K. Shibata, Y. Kawayoke, and Z. M. P. Sazzad, “MICT image quality evaluation database,” [Online]. Available: http://mict.eng.utoyama.ac.jp/mict/index2.html
12. VQEG, “Final report from the video quality experts group on the validation of objective models of video quality assessment,” March 2000, http://www.vqeg.org/.View publication stats