Summary
Subjective quality evaluation methodologies are the most reliable way to measure the quality of experience of any multimedia service for a user. Unfortunately, subjective tests are usually quite time consuming and require a large number of users to provide reliable results. In order to obtain reliable and reproducible subjective results, it is fundamental to define and describe each of the following elements very carefully:
- Laboratory equipment, including details of the audio and video monitors and their arrangment, screen and viewing distance, illumination and reverberation characteristics of the room.
- Data set, including the original and processed contents and their distribution across different sessions.
- Test methodology, including the rating target (quality, comparison, or impairment), the scale (categorical or continuous) and the type of stimuli (single or double).
- Score processing, including score normalization, outlier detection, mean score and confidence intervals computation, and significance tests.
Test laboratory at MMSPG
Test campaigns at MMSPG
Our professional test laboratory was also used to perform subjective test campaigns for the international standardization communities: