5G PoC White Paper: AI in Test Systems, Testing AI Models and ETSI GANA Model's Cognitive Decision Elements (DEs)

Rationale

The purpose of this 5G Proof of Concept (PoC) White paper #5, is to lay the groundwork of the standardization that has been jointly launched recently in ETSI by TC INT and TC MTS with the support of the Centre of Testing and Interoperability (CTI) on the topic of “Testing of AI and AI in Testing Systems” that will address the various aspects linked to this topic through the development of ETSI assets such as specifications to be used by the industry.

These specifications will include the definition of metrics pertaining to specific classes of AI models that can be targeted for testing and assessment, for such metrics definitions are currently missing in the work being done in the various standardization groups.

Moreover, the specifications will close a gap also identified in the 5G PoC White Paper #5 on the need for a “Test & Certification Framework for AI Models in AMC” (Autonomic Management & Control) to support the Industry in implementing and achieving Multi-Layer AMC for Autonomous Networks being specified by ETSI and other Standards Development Organizations (SDOs) / Fora.

It is noticeable that the framework being proposed by ETSI TC INT is aligned with the European Commission’s White Paper, published on 19 February 2020, on “Artificial Intelligence: a European approach to excellence and trust” that emphasizes the need for:

1. A Regulatory Framework
2. The Creation of an AI Testing Center, and
3. The Creation of a Certification Center

Looking at the topic of “Testing of AI and AI in Test Systems” as a journey, ETSI TC INT has identified, already in 2015, the need for a Test & Certification Framework for Adaptive Networks and their Associated Autonomic Functions using AI Components and published, in 2016, EG 203 341 “Approaches for Testing Adaptive Networks” to anticipate and prepare the Industry’s readiness in implementing Multi-Layer Autonomic (AMC) frameworks for evolving and future networks. 

This first ETSI Guide on “Testing Autonomic Networks” laid the groundwork to the next milestone that led to the 5G PoC White Paper #5 which was done in collaboration with ETSI TC MTS. The third milestone was reached in 2019 with the creation of the joint TC INT and TC MTS standardization request on “AI in Test Systems and Testing AI Models”. It will leverage on the outcome from White paper #5 and produce four deliverables in the 2020/2021 timeframe.

The four deliverables expected are:

1. A General Guide on the Benefits of Artificial Intelligence (AI) in Test Systems, with some illustrations of its usage in this scope
2. A General Guide on Testing of AI Models and the Definition of Standardized Metrics for Measurements and Assessments in Testing and Certification of AI Models, including the Certification of AI Models in Autonomic and Self-Adaptive Components/Systems
3. Testing ETSI GANA Model's Cognitive Decision Elements (DEs) as AI Models for Autonomic (Closed-Loop) Network Automation in the space of Autonomic Management & Control (AMC) of Networks and Services, with illustrations of the usage of AI Models for AMC of 5G Network Slices
4. A Generic Test Framework for Testing ETSI GANA Multi-Layer Autonomics and their AI Models for Closed-Loop Network Automation. An ETSI Technical Report (TR) will be produced in 2020/2021 to extend the Early Draft in ETSI EG 203 341 V1.1.1.