Icon Consulting Logo Management and Technology  Consulting
Managing ComplexityBusiness SectorsServicesOur ConsultantsStudy ReportsContactsNewsletters

Print Friendly version

 
  VALIDATION DATA REPOSITORY - A CASE STUDY  
   

Validation is the process through which a desired level of confidence in the ability of a deliverable to operate in a real-life environment may be demonstrated against a pre-defined level of functionality, operability and performance.

Source: European Organisation for the Safety of Air Traffic Management (EUROCONTROL); FAA/EUROCONTROL Cooperative R&D Action Plan 5; Operational Concept Validation Strategy Document


The development and implementation of new concepts in the field of Air Traffic Management is a protracted process that can take 15 to 20 years from concept formulation before benefits begin to be realised from the implemented changes. In that period, hundreds of millions of euros are typically spent on each of the concepts to develop them to an accepted and certified operational state.

In the latter part of the 1990's EUROCONTROL, the European Organisation for the Safety of Air Traffic Management, recognised that there was a lot of validation information available from a multiplicity of projects that was fragmented, unstructured and inconsistent. This made it difficult to obtain an overview of the validation activities being performed, or to understand the relevance of results in terms of decision making, or to reuse common useful information.

There were other issues as well, not least in the scope of projects being undertaken, which were too often overlapping or duplicating activities whilst covering some requirements inadequately or missing them altogether. To address these issues, EUROCONTROL, with the European Commission and the USA's Federal Aviation Authority (FAA), established a common understanding and framework for validation activities within the ATM industry, resulting in a joint validation strategy document covering the development of ATM operational concepts. This strategy document sets out some important principles for validation, namely:

  • an agreed overall validation plan for the different concept developments in order to avoid redundancies and to control the progress of work,
  • common methodology,
  • common ATM performance objectives, and
  • a tool to communicate all developments and results in a common structure: a shared Validation Data Repository (VDR).

The strategy sees the VDR as an integral part of a coherent ATM industry approach to validation providing a structured information framework for storing and analysing the findings of validation activities. EUROCONTROL has developed the VDR as an internet based information framework for use by strategic planners, programme and project managers and R&D experts. The VDR information framework has four main elements:

  • Project specific information - the core data of the VDR, which detail the validation activities, their design, objectives, results, conclusions and recommendations. These are owned by the project;
  • Technical reference information - industry wide reference lists of technical data related to the mechanisms of validation activity such as standard techniques, tools, methods, indictors and metrics. These are maintained by the VDR team;
  • Operational reference information - industry wide accepted descriptions of the operational context of the validation activities, for example lists of strategic improvements, of internationally accepted future operational concepts. These are maintained by the VDR team;
  • Formal classification schemes - to enable indexing and facilitate search algorithms. Most data sets within the VDR have classifications, from simple project status (planning, reporting, complete etc.), to objective type (aim, detailed objective, hypothesis), to results type (quantitative, qualitative etc.).

Development of the full VDR started in 2001. It has been fully operational via the internet since February 2003 (See the VDR Home Page). To date (November 2005) it contains details of approximately 60 projects with over 300 individual validation exercises covering simulations, pre-operational trials, cost-benefit analyses, safety cases, environmental studies, paper based feasibility studies and system architecture studies.

From a portfolio management perspective, the VDR provides overviews of project coverage with respect to strategic change items and key performance areas. The VDR is regularly used in large multi-partner R&D programmes to help coordinate and organise validation activities around a formal hierarchical structure of objectives, ensuring that appropriate techniques and tools are planned for validation, relevant metrics are measured and that findings are clearly related back to original validation objectives. The VDR has been a key tool in compiling summary results of projects by collecting, collating, and packaging relevant data. For example it has been used to produce material for expert workshops where the findings from over 20 individual validation simulations and trials were structured according to the key performance area validation objective (e.g. safety enhancement, capacity increase etc.) and the specific proposed operational concept change (e.g. free routing).

One of the most important uses of the VDR is for information dissemination because it acts as a permanent, web-based, indexed record of a project's achievements that will exist long after the project's own web site has been closed down.

Key principles of VDR operation are that the data stored is in the public domain and use of the web site is free of charge. The data captured in the VDR is taken from published project documentation with hyperlinks provided to the source documents. Each item of data has a nominated owner and the VDR has a suite of security facilities to protect information ownership, in particular with relation to entering and updating data.

The public web site is designed for ease of use, hiding a complex data schema, therefore the functionality offered over the web is necessarily limited. To supplement the on-line facilities and to ensure quality assurance and consistency of the captured data, the EUROCONTROL VDR team provides comprehensive data capture and analysis support for users.

Icon has been directly involved in the project leadership of the VDR for EUROCONTROL since 2000, initially supporting the business case for its development by producing a full working prototype to demonstrate the benefits of such a tool for a typical project. Icon consultants are responsible for the day-to-day running of the VDR development project and provision of services to users. The technical development is carried out by an in-house EUROCONTROL team and, in line with corporate policy, the IT infrastructure provision is outsourced.

It is important to reiterate that the VDR is an integral part of a validation philosophy. It is most effectively used in support of a formalised, structured validation method and similarly a structured method is most effectively applied when supported by a formalised information management approach.

The VDR development by EUROCONTROL is a success story. It has significantly improved validation information management for the many co-operative international ATM development activities. With both the European Commission and FAA committing to the VDR's long-term use by the projects they fund, the story hasn't finished. Visit the VDR web site at www.eurocontrol.int/eatmp/vdr

.

Return to the Icon Winter Newsletter 2005

 
 


Icon Consulting, 2Fl, Berkeley Square House, Berkeley Square, London W1J 6BD
tel: +44 (0) 207 494 9994 | email: enquiries@icon-consulting.com