The ETSI Speech and multimedia Transmission Quality (STQ) Workshop on "Quality of Emerging Services for Speech and Audio: A user-centred perspective" took place on 21 to 22 November 2022 at the Amazon facility in Bratislava, Slovakia. 

About 

The users’ Quality of Experience in mobile communications ecosystem entering the 5th and 6th generation is constantly evolving. This workshop will foster discussions on related services such as Augmented and Virtual Reality, and other Quality of Service aspects that will make future mobile generation more attractive. Innovations in 5G Network Slicing make it necessary to refine the technical approaches and related standards on network performance assessment and Quality of Service Benchmarking.

Standardization is crucial to ensure that the users can experience high-level acoustics and audio performance of new devices, e.g. in a smart home & IoT context. It is essential to evaluate the differences between human perception and automatic speech recognition, and to understand how machines respond to quality, as well as how users perceive the application quality of M2M communication.

Among other topics, this workshop highlighted technical innovations and new standards in prediction of perceived quality of speech and audio communications systems, including listening effort, loudness and echo perception.

 Target Audience

The event was addressed to:

  • Manufacturers,
  • Research bodies,
  • Platform providers,
  • Regulators,
  • Operators
  • Users

bringing together a diverse participation from industry and academia to facilitate the exchange of knowledge and best practices for delivering high-quality user experiences in services and systems. 

Agenda Monday 21 November 2022

 

09.30 Setting the Scene 
Session Chair: Scott Isabelle, TC STQ Chair, Amazon 
09.35 ETSI: a Dynamic & Innovative global ICT Standards Organization
Carmine Rizzo, ETSI Technical Officer
09.50  ETSI TC STQ: Ongoing Work and Challenges
Scott Isabelle, TC STQ Chair, Amazon
10.05 ETSI STQ Mobile: Ongoing Work and Challenges
Ralf Pabst, Umlaut, TC STQ Mobile Chair
10.20  Coffee and Networking Break
10.45 Session 1: Acoustic And Audio Aspects Of Evaluating The Performance Of Smart Home & IoT Devices
Session Chair: Jan Holub, Mesaqin.com Ltd
10.45 Measuring Acoustic Performance Of Voice Assistants – In Practice, Alexa
David Berol,Surfaceink
11.05 Reproducing Reverberation for the Receiving Path
Magnus Schäfer, HEAD acoustics GmbH
11.25 Recreating Complex Soundscapes For Audio Quality Evaluation
Damian Koszewski – Intel Corporation
11.45 Single-Ended Prediction Of Listening Effort For Smart Speakers
Jan Rennies-Hochmuth, Fraunhofer IDMT
12.05 Acoustic Aspects Of Sound Event Detection For IoT Based Monitoring Application
Jakub Bajzík, University of Žilina
12.25 Questions and Answers Session with the speakers and the audience
12.45  Lunch - Courtesy of ETSI
14.00 Session 2 part 1: New Innovations In Prediction Of Perceived Quality Of Speech Communications Systems
(incl. Listening Effort, Loudness And Echo Perception)
Session Chair: Peter Pocta, University of Zilina
14.00 Can Gender Analysis Improve Quality?
Yann Kowalczuk, Faculty of Electrical Engineering, Czech Technical University in Prague
14.20 Individual Differences In Quality Perception
Robert Spang - TU-Berlin 
14.40 ML-Based Noise Cancellation Algorithms And Quality Evaluation Challenges
Ani Chilingaryan - Krisp
15.00 Coffee & Networking Break
15.30 Session 2 part 2: New Innovations In Prediction Of Perceived Quality Of Speech Communications Systems
(Incl. Listening Effort, Loudness And Echo Perception)
Session Chairs: Peter Pocta, University of Zilina 
15.30 Auditory and Instrumental Listening Effort – Recent Work in ETSI STQ
Jan Reimes - HEAD acoustics GmbH
15.50 Recent Advances in Non-Intrusive Speech Quality Prediction
Wafaa Wardah – Technische Universität Berlin
16.10 New Horizons for practical metrics on speech intelligibility
Wolfgang Balzer -Focus Infocom GmbH
16.30 Questions and Answers Session with the speakers and the audience
17.00 Close of the Day

Agenda Tuesday 22 November 2022

 

09.30 Session 3: Qos Aspects Of Services Making 5th And 6th Mobile Generation More Attractive
(incl.Cloud-Based Gaming, Augmented And Virtual Reality Services)
Session Chair : Ralf Pabst, Umlaut, TC STQ Mobile Chair
09.30  Quantifying Network Performance For Emerging Services And Non-Human Users
Jens Berger, Rohde & Schwarz GmbH
09.50 A View On ML/AI Techniques For Mobile Networks Performance, Qos And Qoe Evaluation
Irina Cotanis, Infovista SAS
10.10 Questions and Answers Session with the speakers and the audience
10.30 Coffee & Networking Break
11.00 Session 4: The Evolving Approach Of Quality Of Experience In Mobile Communications Ecosystem Entering The 5th And 6th Generation
Session Chair: Hans W. Gierlich, HEAD acoustics GmbH
11.00  Utilizing Timeslice Kpis For AI/ML-Based Voice Service Automation
Michael Wallbaum, Voipfuture GmbH
11.20 On Qoe of Videocall With A Misaligned Camera(S)
Jan Holub, MESAQIN.com Ltd
11.40 Questions and Answers Session with the speakers and the audience
12.00 Lunch
13.30 Session 5: Consideration Of Signal Processing Differences Between Human Perception And Automatic Speech Recognition
Session Chair: Scott Isabelle, ETSI TC STQ Chairman, Amazon
13.30 ASR Testing in Reverberant Environment - Word Error Rate and Supplementary Numbers
Frank Kettler, HEAD acoustics GmbH
13:50 Subjective Evaluation Of Music Source Separation
Peter Kasak, University of Žilina
14.10 Dependence of EER and MOS in the context of automatic speaker recognition and adverse conditions
Maroš Jakubec, University of Zilina
14.30 Questions and Answers Session with the speakers and the audience
14.50 Wrap-up & Next Steps
Scott Isabelle, Amazon, ETSI TC STQ Chair
15.00 Event Close

  • S. Isabelle, Amazon, STQ Chair
  • H. W. Gierlich, HEAD acoustics, STQ vice-chair
  • J. Holub, Czech Technical University, STQ expert
  • J. Y. Monfort, AFUTT, SC User Chair
  • P. Pocta, University of Zilina, STQ expert
  • J. Pomy, Focus Infocom, STQ vice-chair
  • R. Pabst, STQ Mobile vice Chair
  • J. Reimes, HEAD acoustics
  • C. Rizzo, ETSI, STQ Technical Officer
Any Questions? Contact us