Full title
Quality of Experience in 3DTV
Duration of project
From March 2011 to February 2013
Funding source
COST Action IC1003: European Network on Quality of Experience in Multimedia Systems and Services (QUALINET)
Description
From all the emerging multimedia modalities, 3D television (3DTV) is currently by far the one with the largest momentum, due to the availability of interesting 3D content and affordable 3D display technologies. However, one of the problems that may prevent 3DTV from becoming a world-wide success is that users expect an improved experience with respect to conventional TV and HDTV. Unfortunately, current 3DTV does not always offer a better Quality of Experience (QoE). In some situations, the perceived quality may become lower for the same content shown in 3D when compared to 2D. Reasons for that include technological limitations of current 3D technologies and lack of knowledge for 3D content creation. In order to ensure the adoption of 3DTV in the next decade, it is therefore crucial to optimize the 3D quality from a user perspective along the whole 3D processing chain. However, this requires reliable tools for 3D quality evaluation. While 2D quality evaluation is quite mature, 3D quality evaluation is a rather unexplored topic.
The goal of this project is to advance the state of the art in 3D video quality evaluation by developing more reliable subjective quality methods and objective quality metrics that consider quality from a user perspective. This will be achieved by analyzing the requirements of a home entertainment scenario and creating a representative dataset which considers not only the content, but also the context and the users. This dataset will be used to compare existing approaches for 3D quality evaluation and identify their strength and weaknesses, based on which more reliable methods will be developed.
Research activities
- Define scenarios and create datasets for the development and evaluation of 3D video objective metrics.
- Assess available 2D video quality evaluation methodologies and metric, and adapt them where possible, for components that make sense, to 3D video quality assessment and metrics.
- Study and develop reliable 3D video quality metrics.
- Evaluate and validate developed methods and metrics to facilitate the creation of a quality certification mechanism.
References
Related publications
Journal Articles
2020
Review Of Particle Physics
Progress Of Theoretical And Experimental Physics. 2020-08-01. Vol. 2020, num. 8, p. 083C01. DOI : 10.1093/ptep/ptaa104.2018
ICtCp Versus Y’CbCr
IEEE CONSUMER ELECTRONICS MAGAZINE. 2018. Vol. 7, num. 3, p. 38-47. DOI : 10.1109/MCE.0017.2714696.Quality assessment of an HDR dual-layer backward-compatible codec compared to uncompromised SDR and HDR solutions
IEEE Transactions on Broadcasting. 2018. Vol. 64, num. 2, p. 422-431. DOI : 10.1109/TBC.2018.2823913.Evaluation of ICtCp color space and an Adaptive Reshaper for HDR and WCG
IEEE CONSUMER ELECTRONICS MAGAZINE. 2018. Vol. 7, num. 3, p. 38-47. DOI : 10.1109/MCE.2017.2714696.2017
Comparison and evaluation of light field image coding approaches
IEEE Journal of Selected Topics in Signal Processing. 2017. Vol. 11, num. 7, p. 1092-1106. DOI : 10.1109/JSTSP.2017.2740167.Evaluation of JPEG XT for high dynamic range cameras
Signal Processing-Image Communication. 2017. Vol. 50, p. 9-20. DOI : 10.1016/j.image.2016.10.004.2016
REVIEW OF PARTICLE PHYSICS Particle Data Group
IOP Science: Chinese Physics C. 2016. Vol. 40, num. 10, p. 100001. DOI : 10.1088/1674-1137/40/10/100001.JPEG XT: A Compression Standard for HDR and WCG Images [Standards in a Nutshell]
IEEE Signal Processing Magazine. 2016. Vol. 33, num. 2, p. 118-124. DOI : 10.1109/MSP.2015.2506199.2015
Overview and Evaluation of the JPEG XT HDR Image Compression Standard
Journal of Real-Time Image Processing. 2015. p. 1-16. DOI : 10.1007/s11554-015-0547-x.Benchmarking of objective quality metrics for HDR image quality assessment
EURASIP Journal on Image and Video Processing. 2015. Vol. 2015, p. 39. DOI : 10.1186/s13640-015-0091-4.Modeling Immersive Media Experiences by Sensing Impact on Subjects
Multimedia Tools and Applications. 2015. p. 1-21. DOI : 10.1007/s11042-015-2980-z.Subjective Quality Evaluation of High Dynamic Range Video and Display for Future TV
SMPTE Motion Imaging Journal. 2015. Vol. 124, num. 4, p. 1-6. DOI : 10.5594/j18555.2014
Calculation of average coding efficiency based on subjective quality scores
Journal of Visual Communication and Image Representation. 2014. Vol. 25, num. 3, p. 555-564. DOI : 10.1016/j.jvcir.2013.11.008.2013
Paired comparison-based subjective quality assessment of stereoscopic images
Multimedia Tools and Applications. 2013. Vol. 67, num. 1, p. 31-48. DOI : 10.1007/s11042-012-1011-6.Conference Papers
2017
Impact of interactivity on the assessment of quality of experience for light field content
2017. 9th International Conference on Quality of Multimedia Experience (QoMEX), Erfurt, Germany, May 31 – June 02, 2017. DOI : 10.1109/QoMEX.2017.7965636.On the performance of objective metrics for omnidirectional visual content
2017. 9th International Conference on Quality of Multimedia Experience (QoMEX 2017), Erfurt, Germany, May 31 – June 2, 2017. DOI : 10.1109/QoMEX.2017.7965660.2016
Objective and subjective evaluation of light field image compression algorithms
2016. 32nd Picture Coding Symposium, Nuremberg, Germany, December 4-7, 2016. DOI : 10.1109/PCS.2016.7906379.A new approach to subjectively assess quality of plenoptic content
2016. SPIE Optical Engineering + Applications, San Diego, California, USA, August 28 – September 1, 2016. p. 99710X-1-99710X-13. DOI : 10.1117/12.2240279.JPEG backward compatible coding of omnidirectional images
2016. SPIE Optics + Photonics, San Diego, California, USA, August 28 – September 1, 2016. p. 99711O-1-99711O-12. DOI : 10.1117/12.2240281.Testbed for subjective evaluation of omnidirectional visual content
2016. 32nd Picture Coding Symposium (PCS), Nuremberg, Germany, December 4-7, 2016. DOI : 10.1109/PCS.2016.7906378.New Light Field Image Dataset
2016. 8th International Conference on Quality of Multimedia Experience (QoMEX), Lisbon, Portugal, June 6-8, 2016.How to Benchmark Objective Quality Metrics from Paired Comparison Data?
2016. 8th International Conference on Quality of Multimedia Experience (QoMEX), Lisbon, Portugal, June 6-8, 2016. DOI : 10.1109/QoMEX.2016.7498960.Subjective and Objective Evaluation of HDR Video Coding Technologies
2016. 8th International Conference on Quality of Multimedia Experience (QoMEX), Lisbon, Portugal, June 6-8, 2016. DOI : 10.1109/QoMEX.2016.7498943.Towards prediction of Sense of Presence in immersive audiovisual communications
2016. Human Vision and Electronic Imaging (HVEI) 2016, San Francisco, February 14-18, 2016. DOI : 10.2352/ISSN.2470-1173.2016.16.HVEI-128.2015
Towards high dynamic range extensions of HEVC: subjective evaluation of potential coding technologies
2015. SPIE Optical Engineering + Applications, San Diego, California, USA, August 10-13, 2015. p. 95990G. DOI : 10.1117/12.2193832.Multimodal Dataset for Assessment of Quality of Experience in Immersive Multimedia
2015. ACMMULTIMEDIA 2015 (ACM MM), Brisbane, Australia, October 26-30, 2015. DOI : 10.1145/2733373.2806387.Rate-Distortion Evaluation for Two-Layer Coding Systems
2015. IEEE International Conference on Image Processing, Québec City, Canada, September 27-30, 2015. p. 2884-2889. DOI : 10.1109/ICIP.2015.7351330.Subjective quality assessment database of HDR images compressed with JPEG XT
2015. 7th International Workshop on Quality of Multimedia Experience (QoMEX), Costa Navarino, Messinia, Greece, May 26-29, 2015. DOI : 10.1109/QoMEX.2015.7148119.Quality Evaluation of HEVC and VP9 Video Compression in Real-Time Applications
2015. 7th International Workshop on Quality of Multimedia Experience (QoMEX), Costa Navarino, Messinia, Greece, May 26-29, 2015. DOI : 10.1109/QoMEX.2015.7148088.Active crosstalk reduction system for multiview autostereoscopic displays
2015. IEEE International Conference on Multimedia and Expo, Torino, Italy, June 29-July 3, 2015. DOI : 10.1109/ICME.2015.7177519.Subjective and objective evaluation of HDR video compression
2015. 9th International Workshop on Video Processing and Quality Metrics for Consumer Electronics (VPQM), Chandler, Arizona, USA, February 5-6, 2015.Visual attention in LDR and HDR images
2015. 9th International Workshop on Video Processing and Quality Metrics for Consumer Electronics (VPQM), Chandler, Arizona, USA, February 5-6, 2015.2014
Evaluation of privacy in high dynamic range video sequences
2014. SPIE Optical Engineering + Applications, San Diego, California, USA, August 18-21, 2014. DOI : 10.1117/12.2065559.Comparison of compression efficiency between HEVC/H.265 and VP9 based on subjective assessments
2014. SPIE Optical Engineering + Applications, San Diego, California, USA, August 18-21, 2014. DOI : 10.1117/12.2065561.Subjective Evaluation of Higher Dynamic Range Video
2014. SPIE Optical Engineering + Applications, San Diego, California, USA, August 18-21, 2014. DOI : 10.1117/12.2076644.EyeC3D: 3D video eye tracking dataset
2014. 6th International Workshop on Quality of Multimedia Experience (QoMEX), Singapore, September 18-20, 2014. p. 55-56. DOI : 10.1109/QoMEX.2014.6982290.Cross-lab Subjective Evaluation of the MVC+D and 3D-AVC 3D Video Coding Standards
2014. 6th International Workshop on Quality of Multimedia Experience (QoMEX), Singapore, September 18-20, 2014. p. 183-188. DOI : 10.1109/QoMEX.2014.6982317.Crowdsourcing evaluation of high dynamic range compression
2014. SPIE Optical Engineering + Applications, San Diego, California, USA, August 18-21, 2014. DOI : 10.1117/12.2065560.Subjective Quality Evaluation Of High Dynamic Range Video And Display For Future TV
2014. International Broadcasting Convention (IBC), Amsterdam, Netherlands, September 11-15, 2014.Impact of Ultra High Definition on Visual Attention
2014. ACM International Conference on Multimedia, Orlando, Florida, USA, November 3-7, 2014. p. 247–256. DOI : 10.1145/2647868.2654917.HDR image compression: a new challenge for objective quality metrics
2014. 6th International Workshop on Quality of Multimedia Experience (QoMEX), Singapore, September 18-20, 2014. p. 159-164. DOI : 10.1109/QoMEX.2014.6982313.Ultra-Eye: UHD and HD images eye tracking dataset
2014. 6th International Workshop on Quality of Multimedia Experience (QoMEX), Singapore, September 18-20, 2014. p. 39-40. DOI : 10.1109/QoMEX.2014.6982284.Free-Viewpoint Video Sequences: a New Challenge for Objective Quality Metrics
2014. 16th International Workshop on Multimedia Signal Processing (MMSP), Jakarta, Indonesia, September 22-24, 2014.EEG correlates during video quality perception
2014. 22nd European Signal Processing Conference (EUSIPCO), Lisbon, Portugal, September 1-5, 2014.Survey of Web-based Crowdsourcing Frameworks for Subjective Quality Assessment
2014. 16th International Workshop on Multimedia Signal Processing, Jakarta, Indonesia, September 22-24, 2014. DOI : 10.1109/MMSP.2014.6958831.Predicting Subjective Sensation of Reality During Multimedia Consumption Based on EEG and Peripheral Physiological Signals
2014. IEEE International Conference on Multimedia and Expo (ICME), Chengdu, China, July 14-18, 2014. DOI : 10.1109/ICME.2014.6890239.A new database of still and moving High Dynamic Range pictures
2014. HDRi2014 -Second International Conference and SME Workshop on HDR imaging, Sarajevo, Bosnia and Herzegovina, March 04-05, 2014.Audiovisual focus of attention and its application to Ultra High Definition video compression
2014. SPIE electronic Imaging, San Francisco, California, USA, February 02, 2014. DOI : 10.1117/12.2047850.User-independent classification of 2D versus 3D multimedia experiences through EEG and physiological signals
2014. 8th International Workshop on Video Processing and Quality Metrics for Consumer Electronics – VPQM 2014, Chandler, Arizona, USA, January 30-31, 2014.Crowd-based quality assessment of multiview video plus depth coding
2014. IEEE International Conference on Image Processing, Paris, France, October 27-30, 2014. p. 743-747. DOI : 10.1109/ICIP.2014.7025149.Subjective evaluation of an active crosstalk reduction system for mobile autostereoscopic displays
2014. SPIE Electronic Imaging, San Francisco, California, USA, February 2-6, 2014. DOI : 10.1117/12.2040391.Subjective evaluation of two stereoscopic imaging systems exploiting visual attention to improve 3D quality of experience
2014. SPIE Electronic Imaging, San Francisco, California, USA, February 2-6, 2014. DOI : 10.1117/12.2040465.2013
On the evaluation of 3D codecs on multiview autostereoscopic display
2013. 4th IEEE International Workshop on Hot Topics in 3D (Hot3D), San Jose, California, USA, July 15, 2013.JPEG backward compatible format for 3D content representation
2013. 14th International Workshop on Image and Audio Analysis for Multimedia Interactive Services, Paris, France, July 3-5, 2013. DOI : 10.1109/WIAMIS.2013.6616156.Automatic defect detection in video archives – Application to Montreux Jazz Festival Digital Archives
2013. SPIE Optical Engineering + Applications, San Diego, California, USA, August 25-29, 2013. p. 88560R. DOI : 10.1117/12.2026875.A quality assessment protocol for Free-viewpoint video sequences synthesized from decompressed depth data
2013. Fifth International Workshop on Quality of Multimedia Experience, Klagenfurt am Wörthersee, Austria, July 3-5, 2013. DOI : 10.1109/QoMEX.2013.6603218.Predicting 3D Quality based on Content Analysis
2013. 11th IEEE IVMSP Workshop : 3D Image/Video Technologies and Applications, Seoul, Korea, June 10-12, 2013. DOI : 10.1109/IVMSPW.2013.6611916.Benchmarking of quality metrics on ultra-high definition video sequences
2013. 18th International Conference on Digital Signal Processing, Santorini, Greece, July 1-3, 2013. DOI : 10.1109/ICDSP.2013.6622760.Subjective evaluation of HEVC intra coding for still image compression
2013. Seventh International Workshop on Video Processing and Quality Metrics for Consumer Electronics – VPQM 2013, Scottsdale, Arizona, USA, January 30 – February 1, 2013.Quality Assessment of a Stereo Pair Formed From Two Synthesized Views Using Objective Metrics
2013. Seventh International Workshop on Video Processing and Quality Metrics for Consumer Electronics – VPQM 2013, Scottsdale, Arizona, USA, January 30 – February 1, 2013.2012
Subjective quality evaluation of the upcoming HEVC video compression standard
2012. SPIE Optical Engineering + Applications, San Diego, California, USA, August 13-16, 2012. DOI : 10.1117/12.946036.Comparison of 3D portable display restitution techniques based on stereo and motion parallax
2012. QoMEX 2012, Yarra Valley, 5-7 July 2012. p. 80-85. DOI : 10.1109/QoMEX.2012.6263858.Quality Assessment of a Stereo Pair Formed From Decoded and Synthesized Views Using Objective Metrics
2012. 3DTV-Conference: The True Vision – Capture, Transmission and Display of 3D Video (3DTV-CON), Zurich, Switzerland, October 15-17, 2012. DOI : 10.1109/3DTV.2012.6365478.Quality Assessment of Asymmetric Stereo Pair Formed From Decoded and Synthesized Views
2012. 4th International Workshop on Quality of Multimedia Experience (QoMEX), Yarra Valley, Australia, July 5-7, 2012. DOI : 10.1109/QoMEX.2012.6263854.Towards a Descriptive Depth Index for 3D Content: Measuring Perspective Depth Cues
2012. 6th International Workshop on Video Processing and Quality Metrics for Consumer Electronics (VPQM), Scottsdale, Arizona, USA, January 19-20, 2012.Towards certification of 3D video quality assessment
2012. 6th International Workshop on Video Processing and Quality Metrics for Consumer Electronics – VPQM 2012, Scottsdale, Arizona, USA, January 19-20, 2012.2011
Towards reliable and reproducible 3D video quality assessment
2011. Conference on Three-Dimensional Imaging, Visualization, and Display 2011, Orlando, FL, Apr 27-28, 2011. p. 804302. DOI : 10.1117/12.887037.Motion Parallax Based Restitution of 3D Images on Legacy Consumer Mobile Devices
2011. 13th international workshop on multimedia signal processing, Hangzhou, China, October 17-19, 2011. DOI : 10.1109/MMSP.2011.6093789.A new analysis method for paired comparison and its application to 3D quality assessment
2011. ACM Multimedia, Scottsdale, Arizona, November 28 – December 1, 2011. p. 1281–1284. DOI : 10.1145/2072298.2071994.2010
Temporal Synchronization in Stereoscopic Video: Influence on Quality of Experience and Automatic Asynchrony Detection
2010. International Conference on Image Processing, Hong Kong, September 26-29, 2010. p. 3241-3244. DOI : 10.1109/ICIP.2010.5651142.3D Quality is More Than Just the Sum of 2D And Depth
2010.Impact of Acquisition Distortion on the Quality of Stereoscopic Images
2010. International Workshop on Video Processing and Quality Metrics for Consumer Electronics, Scottsdale, Arizona, USA, January 13-15, 2010.A Comprehensive Database and Subjective Evaluation Methodology for Quality of Experience in Stereoscopic Video
2010. IS&T/SPIE Electronic Imaging, San Jose, January 17-21, 2009. DOI : 10.1117/12.839438.2009
Analysis of the Limits of Graph-Based Object Duplicate Detection
2009. The Fifth IEEE International Workshop on Multimedia Information Processing and Retrieval, San Diego, December 14-16, 2009. p. 600-605. DOI : 10.1109/ISM.2009.119.Reviews
2022
Review Of Particle Physics
Progress Of Theoretical And Experimental Physics. 2022-08-08. Vol. 2022, num. 8, p. 083C01. DOI : 10.1093/ptep/ptac097.2018
REVIEW OF PARTICLE PHYSICS Particle Data Group
Physical Review D. 2018-08-17. Vol. 98, num. 3, p. 030001. DOI : 10.1103/PhysRevD.98.030001.2014
REVIEW OF PARTICLE PHYSICS Particle Data Group
Chinese Physics C. 2014. Vol. 38, num. 9, p. 090001. DOI : 10.1088/1674-1137/38/9/090001.2012
Review of Particle Physics, Particle Data Group
Physical Review D. 2012. Vol. 86, num. 1, p. 010001. DOI : 10.1103/PhysRevD.86.010001.Theses
2016
Quality of Experience in Immersive Video Technologies
Lausanne, EPFL, 2016.Book Chapters
2014
Compression Performance Analysis in HEVC
High Efficiency Video Coding (HEVC); Springer International Publishing, 2014. p. 275–302.2013
3D Video Quality Assessment
Emerging Technologies for 3D Video: Creation, Coding, Transmission and Rendering; John Wiley & Sons Inc, 2013. p. 378-393.Working Papers
2016
[ISO/IEC JPEG contribution] New Test Images for JPEG XS Evaluations
2016
[JCT-VC contribution] HDR CE3: Benchmarking of objective metrics for HDR video quality assessment
2016
[JCT-VC contribution] HDR CE3: Results of subjective evaluations conducted with the DSIS method
2016
2015
[ISO/IEC JTC 1/SC 29/WG1 contribution] EPFL Light-Field Image Dataset
2015
[ISO/IEC MPEG contribution] Results of HDR CfE subjective evaluations conducted at EPFL
2015
[ISO/IEC MPEG contribution] HDR CfE Subjective Evaluations at EPFL
2015
[ISO/IEC MPEG contribution] Benchmarking of LDR metrics for HDR video quality assessment
2015
2014
[ISO/IEC MPEG contribution] Benchmarking of HDR-VDP-2 for HDR video quality assessment
2014
[ISO/IEC MPEG contribution] Correlation of subjective scores and objective metrics for HDR video quality assessment
2014
[ISO/IEC MPEG contribution] Report of the formal subjective assessment of the Submission in response to the Joint Call for Proposal (JCfP) for new technologies in the area of Screen Content Coding (SCC)
2014
2013
[JCT-3V contribution] AHG9: On the evaluation on stereoscopic and multiview autostereoscopic displays
2013
[JCT-VC contribution] AhG4: Subjective evaluation of HEVC intra coding for still image compression
2013
2012
[JCT-3V contribution] AHG9: Accuracy of 2D metrics for 3D video quality assessment
2012
[JCT-3V contribution] 3DV: Quality assessment of stereo pairs formed from two synthesized views
2012
[ISO/IEC MPEG contribution] 3DV: Alternative metrics to PSNR
2012
[ISO/IEC MPEG contribution] 3DV: Objective quality measurement for the 2-view case scenario
2012