Document Type : Research Articles

Authors

University of Sistan and Baluchestan

Abstract

Videos directly captured from a computer or smartphone screen have certain characteristics that differentiate them from camera-captured (CC) ones. These videos are called screen content (SC) videos whose specific encoder has been introduced as a new extension of the HEVC standard called screen content coding (SCC). Most screen content applications are real-time with low delay requiring an accurate rate control. The difference in the characteristics and use of special coding tools such as palette mode, intra block copy, and adaptive color transform in this standard, have affected the mechanism of bit rate generation and control. This paper presents methods and criteria to evaluate the controllability of the bit rate of SC videos and compare it with that of CC counterparts. Furthermore, the requirements of SC video rate control are studied. The experimental results indicate that the bit rate of SC videos is much less controllable than the conventional ones so that the conventional rate-distortion models and bit rate control algorithms are not effective in coding the SC videos.

Keywords

Main Subjects

[1].W. Peng, F. Walls, R. Cohen, J Xu, J. Ostermann, A.
MacInnis, and T. Lin, “Overview of Screen Content Video
Coding: Technologies, Standards, and Beyond,” IEEE
Journal on Emerging and Selected Topics in Circuits and
Systems, vol. 6, No.4, pp. 393 408, 2016.

[2]J. Xu, R. Joshi and R. A. Cohen, "Overview of the
Emerging HEVC Screen Content Coding Extension," in
IEEE Transactions on Circuits and Systems for Video
Technology, vol. 26, no. 1, pp. 50-62, Jan. 2016, doi:
10.1109/TCSVT.2015.2478706.

[3].S. Liu, X. Xu, S. Lei, & K. Jou, (2015). Overview of
HEVC extensions on screen content coding. APSIPA
Transactions on Signal and Information Processing, 4, E10.
doi:10.1017/ATSIP.2015.11.

[4].Y. Sun et al., "Improvements of HEVC SCC Palette Mode
and Intra Block Copy," in IEEE Journal on Emerging and
Selected Topics in Circuits and Systems, vol. 6, no. 4, pp.
433-445, Dec. 2016, doi: 10.1109/JETCAS.2016.2598193.

[5].X. Xu et al., "Intra Block Copy in HEVC Screen Content
Coding Extensions," in IEEE Journal on Emerging and
Selected Topics in Circuits and Systems, vol. 6, no. 4, pp.
409-419, Dec. 2016, doi: 10.1109/JETCAS.2016.2597645.

[6].L. Zhang et al., "Adaptive Color-Space Transform in
HEVC Screen Content Coding," in IEEE Journal on
Emerging and Selected Topics in Circuits and Systems, vol.
6, no. 4, pp. 446-459, Dec. 2016, doi:
10.1109/JETCAS.2016.2599860.

[7].B. Li et al, "Adaptive Motion Vector Resolution for Screen
Content, document", JCTVC-S0085, Joint Collaborative
Team Video Coding, Strasbourg, France, Oct. 2014.

[8] H. Guo, C. Zhu, S. Li and Y. Gao, "Optimal Bit Allocation
at Frame Level for Rate Control in HEVC," in IEEE
Transactions on Broadcasting, vol. 65, no. 2, pp. 270-281,
June 2019.

[9].Ch. W. Seo, J. H. Moon, J. K. Han, “Rate control for
consistent objective quality in high efficiency video
coding,” IEEE Trans. Image Process. vol. 22, no. 6, pp.
24422454, 2013.

[10].Sh. Wang, S. Ma, Sh. Wang, D. Zhao and W. Gao,”
Rate-GOP Based Rate Control for High Efficiency Video
Coding,” IEEE Journal of Selected Topics in Signal
Processing. vol. 7, No. 6, pp. 1101-1111, 2013.

[11].B. Lee, M. Kim, and T. Q. Nguyen,” A Frame-Level Rate
Control Scheme Based on Texture and Nontexture Rate
Models for High Efficiency Video Coding,” IEEE
Transactions on Circuits and Systems for Video
Technology, vol. 24, No. 3, pp. 465 479, 2014.

[12].B. Li, H. Li, L. Li, and J. Zhang,” λ Domain Rate Control
Algorithm for High Efficiency Video Coding,” IEEE
Transactions on Image Processing, vol. 23, No. 9, pp.
3841-3854, 2014.

[13].M. Wang, K. N. Ngan and H. Li,” Low-Delay Rate
Control for Consistent Quality Using Distortion-Based
Lagrange Multiplier,” IEEE Transactions on Image
Processing, vol. 25, No. 7, pp. 2943-2955, 2016.

[14].D. Fani and M. Rezaei,” A Novel PID-Fuzzy Video Rate
Controller for High-Delay Applications of HEVC
Standard,” IEEE Transactions on Circuits and Systems for
Video Technology, vol. PP, No. 99, p. 1, 2017.

[15].Z. Zhang, T. Jing, J. Han, Y. Xu, and F. Zhang,” A New
Rate Control Scheme for Video Coding Based On Region
of Interest,” IEEE Access, vol. 5, pp. 13677-13688, 2017.

[16].W. Gao, S. Kwong, and Y. Jia,” Joint Machine Learning
and Game Theory for Rate Control in High Efficiency
Video Coding,” IEEE Transactions on Image Processing,
vol. 26, No.12, pp. 6074-6089, 2017.

[17] Y. Guo, B. Li, S. Sun, and J. Xu, “Rate control for screen
content coding in HEVC,” Circuits and Systems (ISCAS),
2015 IEEE International Symposium on, Lisbon, 2015.

[18] Y. Guo, B. Li, S. Sun, and J. Xu, “Rate control for screen
content coding based on picture classification,” Visual
Communications and Image Processing (VCIP), Singapore,
2015.

[19].J. Xiao, B. Li, S. Sun and J. Xu, "Rate control with delay
constraint for screen content coding," 2017 IEEE Visual
Communications and Image Processing (VCIP), St.
Petersburg, FL, 2017, pp. 1-4, doi:
10.1109/VCIP.2017.8305100.

[20].Y. Yang, L. Shen, H. Yang, & P. An (2019). "A
content-based rate control algorithm for screen content
video coding". Journal of Visual Communication and
Image Representation, 60, 328338.

[21].S. Wang, J. Li, S. Wang, S. Ma and W. Gao, "A Frame
Level Rate Control Algorithm for Screen Content Coding,"
2018 IEEE International Symposium on Circuits and
Systems (ISCAS), Florence, 2018, pp. 1-4, doi:
10.1109/ISCAS.2018.8351596.

[22].T. Tang, & L. Li (2019). "A low delay rate control method
for screen content coding". Multimedia Tools and
Applications, 78(19), 2823128256.

[23].T. Tang, J. Yang, B. Du and L. Tang, "Down-Sampling
Based Rate Control for Mobile Screen Video Coding," in
IEEE Access, vol. 7, pp. 139560-139570, 2019, doi:
10.1109/ACCESS.2019.2943887.

[24].H. Yang, L. Shen, Y. Yang and W. Lin, "A Novel Rate
Control Scheme for Video Coding in HEVC-SCC," in
IEEE Transactions on Broadcasting, vol. 66, no. 2, pp.
333-345, June 2020, doi: 10.1109/TBC.2019.2954062.

[25].Y. Li, B. Li, D. Liu and Z. Chen, "A convolutional neural
network-based approach to rate control in HEVC intra
coding," 2017 IEEE Visual Communications and Image
Processing (VCIP), St. Petersburg, FL, 2017, pp. 1-4, doi:
10.1109/VCIP.2017.8305050.

[26].X. Lu, B. Zhou, X. Jin and G. Martin, "A Rate Control
Scheme for HEVC Intra Coding Using Convolution Neural
Network (CNN)," 2020 Data Compression Conference
(DCC), Snowbird, UT, USA, 2020, pp. 382-382, doi:
10.1109/DCC47342.2020.00055.

[27].M. Rezaei et al., “Analyzed Rate Distortion Model in
Standard Video Codecs for Rate Control”, IEEE Workshop
on Signal Processing Systems Design and Implementation,
Athens, Greece, Nov. 2005.

[28].M. Wang, K. N. Ngan and H. Li, "An Efficient
Frame-Content Based Intra Frame Rate Control for High
Efficiency Video Coding," in IEEE Signal Processing
Letters, vol. 22, no. 7, pp. 896-900, July 2015, doi:
10.1109/LSP.2014.2377032.

[29].L. Sun, O. C. Au, W. Dai, Y. Guo and R. Zou, "An
adaptive frame complexity based rate quantization model
for intra-frame rate control of High Efficiency Video
Coding (HEVC)," Proceedings of The 2012 Asia Pacific
Signal and Information Processing Association Annual
Summit and Conference, Hollywood, CA, 2012, pp. 1-6.

[30].W. Li, P. Ren, E. Zhang, and F. Zhao, "Rate control for
HEVC intra-coding with a CTU-dependent distorti
jfv,mhngn xyon model,". SIViP 13, 1725 (2019).
https://doi.org/10.1007/s11760-018-1323-8.