The ETSI Speech and multimedia Transmission Quality (STQ) Workshop on "Quality of Emerging Services for Speech and Audio: A user-centred perspective" took place on 21 to 22 November 2022 at the Amazon facility in Bratislava, Slovakia.
About
The users’ Quality of Experience in mobile communications ecosystem entering the 5th and 6th generation is constantly evolving. This workshop will foster discussions on related services such as Augmented and Virtual Reality, and other Quality of Service aspects that will make future mobile generation more attractive. Innovations in 5G Network Slicing make it necessary to refine the technical approaches and related standards on network performance assessment and Quality of Service Benchmarking.
Standardization is crucial to ensure that the users can experience high-level acoustics and audio performance of new devices, e.g. in a smart home & IoT context. It is essential to evaluate the differences between human perception and automatic speech recognition, and to understand how machines respond to quality, as well as how users perceive the application quality of M2M communication.
Among other topics, this workshop highlighted technical innovations and new standards in prediction of perceived quality of speech and audio communications systems, including listening effort, loudness and echo perception.
Target Audience
The event was addressed to:
- Manufacturers,
- Research bodies,
- Platform providers,
- Regulators,
- Operators
- Users
bringing together a diverse participation from industry and academia to facilitate the exchange of knowledge and best practices for delivering high-quality user experiences in services and systems.
Agenda Monday 21 November 2022
| 09.30 | Setting the Scene Session Chair: Scott Isabelle, TC STQ Chair, Amazon |
| 09.35 | ETSI: a Dynamic & Innovative global ICT Standards Organization Carmine Rizzo, ETSI Technical Officer |
| 09.50 | ETSI TC STQ: Ongoing Work and Challenges Scott Isabelle, TC STQ Chair, Amazon |
| 10.05 | ETSI STQ Mobile: Ongoing Work and Challenges Ralf Pabst, Umlaut, TC STQ Mobile Chair |
| 10.20 | Coffee and Networking Break |
| 10.45 | Session 1: Acoustic And Audio Aspects Of Evaluating The Performance Of Smart Home & IoT Devices Session Chair: Jan Holub, Mesaqin.com Ltd |
| 10.45 | Measuring Acoustic Performance Of Voice Assistants – In Practice, Alexa David Berol,Surfaceink |
| 11.05 | Reproducing Reverberation for the Receiving Path Magnus Schäfer, HEAD acoustics GmbH |
| 11.25 | Recreating Complex Soundscapes For Audio Quality Evaluation Damian Koszewski – Intel Corporation |
| 11.45 | Single-Ended Prediction Of Listening Effort For Smart Speakers Jan Rennies-Hochmuth, Fraunhofer IDMT |
| 12.05 | Acoustic Aspects Of Sound Event Detection For IoT Based Monitoring Application Jakub Bajzík, University of Žilina |
| 12.25 | Questions and Answers Session with the speakers and the audience |
| 12.45 | Lunch - Courtesy of ETSI |
| 14.00 | Session 2 part 1: New Innovations In Prediction Of Perceived Quality Of Speech Communications Systems (incl. Listening Effort, Loudness And Echo Perception) Session Chair: Peter Pocta, University of Zilina |
| 14.00 | Can Gender Analysis Improve Quality? Yann Kowalczuk, Faculty of Electrical Engineering, Czech Technical University in Prague |
| 14.20 | Individual Differences In Quality Perception Robert Spang - TU-Berlin |
| 14.40 | ML-Based Noise Cancellation Algorithms And Quality Evaluation Challenges Ani Chilingaryan - Krisp |
| 15.00 | Coffee & Networking Break |
| 15.30 | Session 2 part 2: New Innovations In Prediction Of Perceived Quality Of Speech Communications Systems (Incl. Listening Effort, Loudness And Echo Perception) Session Chairs: Peter Pocta, University of Zilina |
| 15.30 | Auditory and Instrumental Listening Effort – Recent Work in ETSI STQ Jan Reimes - HEAD acoustics GmbH |
| 15.50 | Recent Advances in Non-Intrusive Speech Quality Prediction Wafaa Wardah – Technische Universität Berlin |
| 16.10 | New Horizons for practical metrics on speech intelligibility Wolfgang Balzer -Focus Infocom GmbH |
| 16.30 | Questions and Answers Session with the speakers and the audience |
| 17.00 | Close of the Day |
Agenda Tuesday 22 November 2022
| 09.30 | Session 3: Qos Aspects Of Services Making 5th And 6th Mobile Generation More Attractive (incl.Cloud-Based Gaming, Augmented And Virtual Reality Services) Session Chair : Ralf Pabst, Umlaut, TC STQ Mobile Chair |
| 09.30 | Quantifying Network Performance For Emerging Services And Non-Human Users Jens Berger, Rohde & Schwarz GmbH |
| 09.50 | A View On ML/AI Techniques For Mobile Networks Performance, Qos And Qoe Evaluation Irina Cotanis, Infovista SAS |
| 10.10 | Questions and Answers Session with the speakers and the audience |
| 10.30 | Coffee & Networking Break |
| 11.00 | Session 4: The Evolving Approach Of Quality Of Experience In Mobile Communications Ecosystem Entering The 5th And 6th Generation Session Chair: Hans W. Gierlich, HEAD acoustics GmbH |
| 11.00 | Utilizing Timeslice Kpis For AI/ML-Based Voice Service Automation Michael Wallbaum, Voipfuture GmbH |
| 11.20 | On Qoe of Videocall With A Misaligned Camera(S) Jan Holub, MESAQIN.com Ltd |
| 11.40 | Questions and Answers Session with the speakers and the audience |
| 12.00 | Lunch |
| 13.30 | Session 5: Consideration Of Signal Processing Differences Between Human Perception And Automatic Speech Recognition Session Chair: Scott Isabelle, ETSI TC STQ Chairman, Amazon |
| 13.30 | ASR Testing in Reverberant Environment - Word Error Rate and Supplementary Numbers Frank Kettler, HEAD acoustics GmbH |
| 13:50 | Subjective Evaluation Of Music Source Separation Peter Kasak, University of Žilina |
| 14.10 | Dependence of EER and MOS in the context of automatic speaker recognition and adverse conditions Maroš Jakubec, University of Zilina |
| 14.30 | Questions and Answers Session with the speakers and the audience |
| 14.50 | Wrap-up & Next Steps Scott Isabelle, Amazon, ETSI TC STQ Chair |
| 15.00 | Event Close |
- S. Isabelle, Amazon, STQ Chair
- H. W. Gierlich, HEAD acoustics, STQ vice-chair
- J. Holub, Czech Technical University, STQ expert
- J. Y. Monfort, AFUTT, SC User Chair
- P. Pocta, University of Zilina, STQ expert
- J. Pomy, Focus Infocom, STQ vice-chair
- R. Pabst, STQ Mobile vice Chair
- J. Reimes, HEAD acoustics
- C. Rizzo, ETSI, STQ Technical Officer
Access to presentations (pdf) after the event