Overview
UCAAT is the pinnacle in the calendar of ETSI’s Technical Committee Methods for Testing and Specification. This well-established event addresses the practical challenges of testing and test automation faced by industry today. UCAAT is dedicated to all aspects of automated testing and is not limited in scope. Rapid technical advances in application areas such as information and communication technology, industrial automation, transportation (automotive, avionics, rail), enterprise information technology, healthcare, and biotechnology, as well as shifting operational landscapes such as DevOps practices and machine learning techniques applied to software development are ever increasing the demands on testing.
UCAAT 2022: Testing of Trustworthy Systems
Our society becomes increasingly dependent on complex software systems and widely interconnected systems of systems that take over more and more crucial tasks of our daily lives. These systems will be only accepted by their users if they offer a high level of trust in various facets reaching from safety, availability, business integrity, to robustness, security, and other areas. Moreover, these large systems integrate many different technologies, such as Artificial Intelligence or semantic networks, that are challenging on their own. Testing and certifying them becomes therefore an urgent task that needs to be addressed through appropriate methods, tools, and processes. UCAAT 2022 aims at discussing particularly all aspects concerning trustworthiness of systems and how it can be tackled best in testing. UCAAT 2022 provides a unique opportunity for practitioners, tool vendors, test service providers and researchers from many different application domains to come together.
Attendees will be able to exchange on:
- Experiences gained and lessons learned from the adoption of novel test automation techniques, methods, and tools
- The long-term application and the latest advances in the industrial use of test automation.
Target audience
The event is of particular interest for:
- Test engineers, designers or test automation experts
- Engineers working in test specification, automation, DevOps and RPA
- Software developers or architects
- Project leaders or managers in digital transformation
- Test tool or service providers
- Test and quality assurance analysts
- Academics and researchers in testing
Important dates
- Call for Presentation Deadline: 2 May 2022
- Notification of Acceptation: 30 May 2022
- Programme online: 15 June 2022
- Presentation submission: 29 August 2022
UCAAT 2022 AGENDA (13-15 September)
- 13 September: Pre-Conference Tutorials and Conference Day 1
PRE-CONFERENCE TUTORIALS
09.00 - 12.30: TUTORIAL 1: Bughunt! An interactive workshop in Model-Based Testing with the Axini Modeling Platform
Machiel Van der Bijl and Ulyana Tikhonova, AxiniThe problem we are addressing: How do you know that your system conforms to its requirements and specifications in an automated fashion? The state of practice is to automate test-cases. However, how does one create and maintain a high coverage (regression) test-set in an economically feasible fashion? Model-based testing is an interesting theory that brings specification and testing together. It makes it possible to test your systems with unprecedented precision in a maintainable way.
In this interactive workshop you get to know the Axini Modeling Platform. It is a platform to model and test systems. Tests are model-based, which means that the platform fully automates the testing process: test-case generation, execution and evaluation. Participants will take part in the bug hunt. We present 10 different implementations of the Smart Door IoT system, with several bugs in them. The person that finds the most bugs wins. In order to find the bugs, participants will model the specification of the Smart Door and then execute automatically generated tests against the implementations – all in the Axini Modeling Platform.
- Key Learning points:
- How to model a system to model-based test it. In this workshop we use the Axini Modeling Language, a rich modelling language with actions (inputs and outputs), data, time, non-deterministic choices and loops, etc.
- How to debug models
- How to execute tests against implementations
- How to analyze and diagnose the executed tests
10.30 - 11.00: Networking Coffee Break
11.00 - 12.30: TUTORIAL 2: Test Intelligence
Jakob Rott, CQSE- Many Teams must test more and more functionality in less and less time. Historically grown test suites are often not up to this challenge since they test too much and too little at the same time. Too much, since they contain redundant tests that cause execution and maintenance costs but provide little value over similar tests. Too little since important functionality remains untested.
In this talk, I present analysis techniques to uncover these problems in your own system: History analyses of the version control system show, where in the past most bugs occurred. Test gap analysis reveals, which code changes have not yet been tested. And pareto optimization of test suites and test impact analysis identifies tests that right now have the best cost-benefit ratio. For each analysis, I outline the research foundation and give examples and experiences from application in practice. - Key Learnings:
- Which data sources are available and which data they contain
- Which analyses exist that make use of the data
- How to detect test gaps (=untested changes)
- Where in the system most bugs remained undetected in the past and how this can help improve their discovery in the future
- How to find redundant tests in historically grown test suites using pareto-optimization
- Which test impact analysis approaches work and how they discover those tests, that find new bugs most quickly.
12.30 - 14.00: Networking Lunch
CONFERENCE DAY 1
14.00 - 14.30: Conference Opening
Andreas Ulrich, UCAAT PC Chair
Philip Makedonski, ETSI TC MTS Chair
Anthony Wiles14.30 - 15.45: SESSION 1: INDUSTRIAL SYSTEMS
- 14.30 - 14.55: A New Way for Testing in the Autonomous Factory – Simulation Based Testing
Matthias Eichenseher, Siemens AG - 14.55 - 15.20: Keyword-Driven Simulation Testing for Open-Source Robotics
Niels Hoppe and Steffen Lüdtke, Fraunhofer FOKUS - 15.20 - 15.45: Back-to-Back Testing Applied on Distributed PLC
Anais Guignard, Systerel
15.45 - 16.15: Networking Coffee Break
16.15 - 17.30: SESSION 2: FAILURE ANALYSIS
- 16.15 - 16.40: Automatic Log Analysis: Expert Knowledge for Everyone!
Graham Rawlings and Stephan Schulz, Giesecke+Devrient - 16.40 - 17.05: AI-Based Defect Analytics and Assignment
Rajesh Kumar, Gabor Pal Csaki and Michael Schmidts, Siemens AG - 17.05 - 17.30: ML-Based Test Prioritization “To Fail First” as a Service
Alexandre Vernotte and Aymeric Cretin, Smartesting, Femto-ST Institute
- 14.30 - 14.55: A New Way for Testing in the Autonomous Factory – Simulation Based Testing
- 14 September: Conference Day 2
CONFERENCE DAY 2
09.00 - 10.00: KEYNOTE SPEECH - Testing AI: A New Testing Specialism
Stuart Reid, CTO, STA Consulting, Korea- Artificial intelligence (AI) is now a mainstream technology that touches the lives of most of us, with a global market predicted to reach US$500 billion within two years.
AI-based systems, and machine-learning (ML) systems in particular, are different from traditional IT systems, as the underlying code is not written by a programmer but is inferred from patterns in data. This major change to classic software development has two significant knock-on effects. It has created the attractive new role of AI specialist. However, it has also created several new sources of specialised defects, such as the training data, the deliverable ML model and the ML development framework.
This talk proposes a new software testing specialism that is focused on addressing the new challenges introduced by AI and ML. To be able to test a ML system, testers need to understand the underlying technology and how it can go wrong. They also need to be able to address those system attributes, such as ethics and trust, which are so important with AI-based systems.
From now on, as AI-based systems grow in complexity and criticality, achieving acceptable levels of quality and trust will only be possible by employing independent, AI test specialists. This talk will explain why they are needed, what specialist tasks they will do, and suggest how we should handle the transition to a future where the AI test specialist is a recognized role.
10.00 - 10.25: Posters Introduction
5' per poster10.25 - 11.00: Networking Coffee Break
11.00 - 12.15: SESSION 3 - AI SYSTEMS
- 11.00 - 11.25: JanIA: Intelligent Practices for Automated Testing
Pablo Manso García-Mauriño, MTP - 11.25 - 11.50: Automating Adversarial Robustness Testing of DNN Models
Albert Negura and Kobus Grobler, NavInfo Europe B.V. - 11.50 - 12.15: Debugging Machine Learning Models
Emmanuel Charleson Dapaah and Jens Grabowski, University of Göttingen
12.15 - 13.45: Networking Lunch
13.45 - 15.00: SESSION 4 - REAL-TIME SYSTEMS
- 13.45 - 14.10: Identifying and Classifying Uncertainties to Support Testing of Industrial Elevators
Tao Yue, Simula Research Laboratory - 14.10 - 14.35: Benefits of Test Automation in Time Travel Testing
Aritra Majumdar, EPAM Systems - 14.35 - 15.00: Increase Robustness with Time Travel – Is It Possible and Reasonable?
Rosalinde Schuster, Victor Navratil and Stephan Schulz, Giesecke+Devrient
15.00 - 15.30: Networking Coffee Break
15.30 - 17.10: SESSION 5 - DEPENDABLE SYSTEMS
- 15.30 - 15.55: Overview of Work in Progress to Update IEEE 982, Standard for Measures of the Software Aspects of Dependability
Robert Binder, IEEE - 15.55 - 16.20: Assessment of GNSS-Based Equipment in the Context of AEBS-UNECE R152
Ernst Phillip Mrohs and Marcel Visser, NavCert GmbH - 16.20 - 16.45: Overview of the ETSI Electronic Signatures Testing Activities
Laurent Velez, ETSI - 16.45 - 17.10: Continuous Quality Assurance Strategy of an NFVI
Szilard Szell, Eficode Oy
- Artificial intelligence (AI) is now a mainstream technology that touches the lives of most of us, with a global market predicted to reach US$500 billion within two years.
- 15 September: Conference Day 3
CONFERENCE DAY 3
09.00 - 10.00: KEYNOTE SPEECH - Testing Challenges for Cyber Physical Systems
Jan Tretmans, Research fellow at ESI (TNO), EindhovenComplex high-tech cyber physical systems increasingly depend on software: software controls, connects, and monitors almost every aspect of system operation, be it a car, a 'smart' factory, a printer, an airplane, or a wafer lithography system. Consequently, ever more system failures are due to software bugs. Moreover, we build ever larger, more complex, and increasingly connected software systems, with more and more options and features, and evolving ever faster. There is a serious threat that testing methods cannot keep pace with these developments, and will endanger the fast and thorough testing of future generations of complex systems, thus hampering innovation. In this presentation we discuss trends, challenges, and research directions for testing future generations of high-tech cyber physical systems.
10.00 - 12.15: SESSION 6 - METHODOLOGY
- 10.00 - 10.25: Holistic Model-Based Approach for Test Generation from Safety Models
Philipp Dormeier, Siemens AG
10.25 - 11.00: Networking Coffee Break
- 11.00 - 11.25: Towards the Absence of Bugs: An Intelligent Combination of Static and Dynamic Analysis to Identify Vulnerabilities in the Development Process
Ramon Barakat, Fraunhofer FOKUS - 11.25 - 11.50: Systematic Selection of Testing Methodology for Low-Code Development
Shreyasi Warunkar, S&N CQM GmbH - 11.50 - 12.15: How to Make your Automation Last Forever
Martin Gijsen, Sogeti
12.15 - 13.30: Networking Lunch
13.30 - 15.55: SESSION 7 - TEST AUTOMATION
- 13.30 - 13.55: Isn’t Test Automation a Silver Bullet?
Mesut Durukal, - 13.55 - 14.20: Best Practices of Agile Testing in Cloud Software Products
Gemmilyn Chu and Daniel Ardelean, Nokia - 14.20 - 14.45: A Spring in Tester’s Life Using SpringBoot
Ashokkumar Rajendiran, PagoNxt
14.45 - 15.05: Networking Coffee Break
- 15.05 - 15.30: A Beginners Guide for Monkey Test Automation on Android & iOS Native Apps
Lakshminarayanan Krishnamurthy, EPAM Systems - 15.30 - 15.55: Complementing GUI Testing Scripts with Smart Monkey Testing
Pekka Aho, Tanja E. J. Vos, Fernando Pastor Ricos and Olivia Rodriguez Valdes
Universitat Politecnica de Valencia, Open Universiteit
15.55 - 16.15: BEST PAPER AWARD / CONFERENCE CLOSING
Andreas Ulrich, UCAAT PC Chair
Philip Makedonski, ETSI TC MTS Chair- 10.00 - 10.25: Holistic Model-Based Approach for Test Generation from Safety Models
The independent Programme Committee is composed mostly of industry stakeholders nominated by the appointed Programme Committee Chair.
The Programme Committee determines the call for presentations and evaluates the received presentation, poster, and tutorial proposals to select the best ones for the conference programme.
Programme Committee Chair:
- Andreas Ulrich, Siemens AG
Programme Committee Members:
- Jorge Asensio, ARHS Developments
- Sergio Borghese, NetResults Srl
- Carlos Cárdenas, Dekra
- Rémi Caudwell, Sogeti
- Tibor Csöndes, Ericsson Hungary
- Zhen Ru Dai, Hamburg University of Applied Sciences (HAW)
- Alexis Despeyroux, Testing Solutions and Services
- Muslim Elkotob, Vodafone
- Baris Güldali, S&N Invent
- Raquel Jimenez, MTP
- Martti Käärik Elvior, LLC
- Philip Makedonski, University of Göttingen
- Ultan Mulligan, ETSI
- Dejan Ničković , AIT
- Jorge Pascal, Audi
- Axel Rennoch, Fraunhofer FOKUS
- Stephan Schulz, Giesecke + Devrient
- Matthias Simon, Nokia
- Szilard Szell, Eficode Oy
- Dirk Tepelmann, Spirent
- Peter Zimmerer, Siemens AG
The following companies will exhibit at UCAAT 2022:
Axini, the next step in test automation
At Axini we improve the way software is made, in particular the quality of software. We offer a platform in which your domain experts specify the requirements and functionality of your system and our platform fully automatically takes care of the testing. This approach fits perfect with Agile development. Don’t wait until the system integration for errors to manifest – detect problems as early as possible in the development process.
Powered by formal techniques developed over 35 years of academic research, Axini is the leading company on model-based engineering and model-based testing in The Netherlands. We have over 10 years of experience applying these formal techniques with great success in high-tech, finance and rail.
Kalifast is a software solution whose development began in 2010, sponsor of UCAAT since 2016, visitors will discover the advances of this solution always at the forefront of innovation.
Kalifast aims to be an all-integrated solution that goes from requirements management, to test automation, to project management.
On the automation side, Kalifast has a powerful scheduler allowing it to perform in particular:
- Scenarios requiring several devices, which makes it possible to test, for example: chat conversations, user / administrator interactions, etc.
- The use of robots of different technologies in the same scenario
- Adaptable scenarios according to the results
- Variabilizations that can use Excel files of thousands lines
MTP is a Spanish company founded in 1997 and it is offering Quality Assurance and Quality Control Professional Services for Digital Business. In MTP, there are more than 600 employees distributed across Spain, Mexico, Brazil, and the United States. MTP clients are National and International organizations from all industries. MTP is totally independent of development manufacturers and most of the employees are ISTQB/SCRUM Certified.
MTP Digital Business Assurance business comprehends the following services:
• Quality Assurance (QA): Functional and performance testing and test automation/regression
• Cybersecurity (CS): Audits for SW applications, mobile applications, and IoT
• User Experience (UX): usability and accessibility testing and user interface re-engineering
• DevOps and Agile Development (DA): support for agile methodologies deployment
SMARTESTING develops cognitive test solutions that boost the efficiency of IT Quality Assurance teams.
Our products combine visual representations and AI technologies to facilitate the agile design of business relevant tests and accelerate their automation.
SMARTESTING enjoys a worldwide reputation of expert in Model-Based Testing. In 2017, SMARTESTING launched YEST®, a visual and Agile test design tool. YEST® is the fruit of our team’s unique know-how, experience and unrelenting determination to offer a simple MBT tool to the IT testing community.
SMARTESTING also develops GRAVITY, a usage-driven automation testing solution. GRAVITY automatically creates automated regression tests from the identification of key customer journeys on the application under test.
SMARTESTING is present in France and Germany. www.smartesting.com - contact@smartesting.com
Should you also wish to exhibit at UCAAT 2022, and introduce your organization, company, product or service to people interested in advanced automated testing
- complete and return the application form below for submission to events@etsi.org.
Exhibitor application form
Exhibition Stands are free of charge and consist of:
- A 4sqm stand with 1 table, 2 chairs, electricity and internet connection
- Possibility to display company’s own exhibition material on the stand (pop up, banner...)
Exhibitors Visibility
- UCAAT exhibitors will be listed on the website
- The exhibition area will be announced at the opening of conference
- The Conference programme will promote the exhibition area
- Encourage participants to visit the stands during coffee/lunch breaks
- Exhibition stands plan available in printed programme
- 10 lines company profile and logo in the printed programme
Should you need any further information, please contact events@etsi.org
German Testing Day Digital 2022
Be a part of the company-independent conference on software quality on 03 + 04 May 2022! For everyone involved in quality assurance of business IT systems as well as embedded systems, the German Testing Day Digital offers the perfect platform! The main focus is on the sharing of experiences & new trends as well as methods. Look forward to 25 cutting-edge sessions on testing including inside talks by Vodafone, Raiffeisen Bank, Siemens Healthineers and many more!
» More info
HUSTEF is one of Europe’s premier Software Testing & Quality conferences with 730+ attendees from 30+ countries preserving its very friendly and familiar atmosphere.This year we are coming back as a LIVE event on 4-6 October!
Would you like to join us as a Speaker? Submit your proposal and see you in Budapest, Hungary!
The ASQF is the network of experts for quality. With 1.200 members we have been developing and securing software and system quality for 25 years.
Upcoming event: The ASQF Safety & Security Day on September 7, 2022 invites you to discuss the topic “Your assets are at risk: processes, methods and tools to protect them” (from the perspective of functional safety, cybersecurity and safety at work).
» View all ASQF events
All presentations made at UCAAT 2022 are now available here
UCAAT is an established series of annual events. Previous editions of the UCAAT have focused on different aspects of test automation following emerging trends through the years. Information about previous editions of the UCAAT, including the programmes and presentation slides, are available from the archived websites linked below. Browse through the archives and discover past experiences shared at UCAAT!
- UCAAT 2021
- UCAAT 2020 (cancelled)
- UCAAT 2019
- UCAAT 2018
- UCAAT 2017
- UCAAT 2016
- UCAAT 2015
- UCAAT 2014
- UCAAT 2013