Numerous networks have been set up—the European network for Health Technology Assessment (EUnetHTA) (Reference Kristensen, Lampe and Chase7) among them—to foster links between HTA agencies (Reference Garrido, Kristensen and Nielsen4). Such a network gives members the opportunity to share evidence about the outcomes and effectiveness of different healthcare technologies (5) and to make better use of existing HTA reports. The process of adapting HTA reports from other agencies has clear advantages in terms of both development and resource management of healthcare systems (Reference Chase, Rosten and Turner1;Reference Turner, Chase and Milne10;Reference Turner, Chase and Milne11).
One aim of the EUnetHTA Project was to develop practical tools, through a series of work packages, for use within the network to avoid duplication and ensure better use of resources (Reference Kristensen, Lampe and Chase7). It is hoped that this process will encourage the sharing and adaptation of HTA reports between countries. However, the sharing and adaptation of HTA reports between different countries, settings, and contexts presents certain challenges (Reference Kristensen, Lampe and Chase7). Among them is the issue of language. Although many similarities exist between European countries, confusion can still arise about the exact meaning of different words. Readers may interpret the meaning of words used in an HTA report as they would had it been written for their own healthcare setting, without realizing that the authors’ setting is different and the words may have a different and distinct meaning. For example, the use of the word “guidance” in HTA reports in the United Kingdom often refers to a specific set of reports produced by the UK's National Institute for Health and Clinical Excellence (NICE). Likewise, in France the term “advice,” when used in an HTA context, refers to whether health insurers are required to reimburse the cost of a health technology.
To help deal with the problems of confusion and misunderstanding that can arise, a glossary of HTA adaptation terms was developed by a partnership of HTA organizations in the EUnetHTA Project. A glossary is, in essence, simply a list of terms and their associated meanings. However, because this glossary aims to help users recognize and deal with the variety of meanings of different HTA terms, it was decided that the glossary should provide several descriptions of how the various terms are used in different settings across Europe as opposed to a single definition. The glossary focuses on HTA terms specifically relating to adaptation, where the purpose of adaptation is to enable an HTA agency in one country (or region or setting) to make use of an HTA report produced elsewhere, thus saving time and money.
This study aims to describe the development of the glossary of HTA adaptation terms, part of EUnetHTA Work Package 5 (2), for use by the European Union (EU) and Member States and how such a glossary might be used to help faciliate the understanding of HTA reports.
METHODS
The National Coordinating Centre for Health Technology Assessment (NCCHTA), based at the University of Southampton in the United Kingdom, led a partnership of twenty-eight HTA agencies and networks from across Europe, all EUnetHTA Work Package 5 (WP5) Partners (Reference Chase, Rosten and Turner1;Reference Turner, Chase and Milne11). It was tasked with developing the glossary.
NCCHTA used several methods to understand Partners’ experiences of adaptation, to consider the purpose, and to develop the content of the glossary. Figure 1, Developmental Stages of the Glossary, below presents these methods, detailing the stages of glossary development and the methods used at each stage. Stage 1 involved developing a list of terms for possible inclusion in the glossary from a variety of sources. Descriptions for these terms were then gathered from the Partners (Stage 2). These descriptions were collated (Stage 3), and the Partners were asked to comment on them (Stage 4). Final editing of the glossary was done (Stage 5) before a final review by all EUnetHTA Partners (Stage 6). To further enhance the glossary, NCCHTA coordinated the development of EUnetHTA descriptions for certain terms (Stage 7), and additional descriptions were gathered for the remaining terms (Stage 8).
Figure 1. Developmental stages of the glossary.
Stage 1. Developing the List of Adaptation Terms for the Glossary
This stage involved identifying terms that would be suitable for inclusion in the glossary. Potential terms were identified from a variety of sources (Reference Chase, Rosten and Turner1), for example, the INAHTA glossary (5;6) and the HTA resource in the National Library of Medicine (9), in addition to the Partners themselves who attended a meeting to discuss the work and who responded to round 1 of a Delphi questionnaire (Reference Chase, Rosten and Turner1;Reference Turner, Chase and Milne10;Reference Turner, Chase and Milne11).
From these sources, NCCHTA assembled a list of adaptation terms to be included in the glossary. It was decided to include terms if they satisfied the following characteristics: (i) were concerned with adaptation (i.e., the process by which an HTA report from one setting can be made appropriate for use in a different setting); and (ii) were subject to considerable confusion; and/or (iii) were used differently by different countries.
Thirty-eight terms were ultimately identified for inclusion in the initial draft of the glossary, including terms such as adaptation, policy, and transferability. A pragmatic decision was taken at this stage to produce the glossary in English, as this was the most shared language among the Partners and the working language of EUnetHTA.
Stage 2. Gathering Descriptions and Examples of Term Usage
The second stage of development involved obtaining input from twelve EUnetHTA Partners, all of whom had committed 30 days or more to this WP5 work, on the adaptation terms identified in Stage 1. Each organization was allocated three or four terms. Some terms were stand-alone (e.g., affordability) others were grouped terms (e.g., efficacy and effectiveness). Terms were grouped if NCCHTA believed that they were closely related and, therefore, were more likely to be prone to confusion. For these terms, the Partners were asked to highlight the differences between them in their descriptions.
It is important to note that the Partners were not asked to provide definitions for the terms. Rather, they were specifically asked to provide descriptions; to discuss the possible interpretations of each of the terms and to provide examples of how these terms are used in different countries. The organizations were encouraged to use their experience with, and understanding of, HTA to guide them.
To help the Partner organizations with this task, NCCHTA developed a description of the term adaptation for the glossary. This description was initially drafted by NCCHTA and was then distributed to all partners for comment. The finalized description was circulated with the instructions for this stage of development to give the Partner organizations an example of what was required of them.
Stage 3. Collating the Descriptions
Of the twelve Partners contacted to aid with Stage 2 of development, eleven responded. Their descriptions were collated by NCCHTA, and a draft glossary was developed. Minimal editing was done to the glossary at this stage.
Stage 4. Obtaining Comments on Collated Descriptions from Partners
The draft glossary was placed on the Members Only section of the EUnetHTA Web site. All Partners were contacted and asked to review and comment on the glossary. Partners were given 2 weeks to review the draft glossary and make comments. Of the eight Partners that replied to this request, six registered comments.
Stage 5. Collation and Editing of the Glossary by NCCHTA
The next stage of development was to review the various descriptions, examples, and comments provided by the Partners. This was done by NCCHTA, the Lead Partner of WP5. This process involved reviewing the descriptions, examples, and comments with a view to identifying the best way to display all of this information in the glossary.
NCCHTA considered whether to integrate the descriptions and comments into a single entry for each term, but decided against this approach with the glossary. It was believed that attempting to merge the descriptions and comments might result in the loss of some detail. This would also involve a lengthy checking process, requiring going back to each Partner to obtain their approval for each of the merged descriptions. Hence, it was decided to leave the different descriptions of each term as part of the glossary format. These descriptions were subject to minor editing, eliminating any contradictory information within a single description. Any similarity between the descriptions was left untouched to highlight areas of strong agreement.
The descriptions were then placed in the following order: (i) EUnetHTA definition, like that for the term “adaptation,” which had been developed by the network; (ii) INAHTA definition, if one existed; (iii) HTA organization descriptions, placed in alphabetical order by the first letter of each organization's common name.
NCCHTA believed that the comments made on the first draft of the collated glossary (Stage 4) made a valuable contribution. Therefore, it was decided to link these comments to each of the glossary terms. These comments were not edited by NCCHTA, but the name of the contributing HTA organization was removed.
Stage 6. Final Review
The glossary was delivered to EUnetHTA. Partners were given 4 weeks to register final comments on the glossary before it was made available on the general EUnetHTA extranet page and promoted as a product from the partnership.
In light of the eight comments received, new descriptions were added for some of the terms and it was decided to explore the possibility of creating EUnetHTA definitions for some of the terms. The glossary of HTA Adaptation Terms was then published on the main EUnetHTA Web site (3).
Stage 7. Developing EUnetHTA Definitions for Certain Terms
To enhance the glossary, some convergence on glossary entries was reached for a few specific terms. These form EUnetHTA definitions for these terms and, as is described in Stage 5, are placed as the first entry for that term in the glossary. The terms were selected because they were considered to be particularly relevant to the issues of adapting HTA reports. The terms selected were clinical and policy question, context specific and setting, domain, speedy sifting and toolkit, relevance and reliability, and generalizability and transferability.
NCCHTA drafted initial definitions for these terms, based in part on the various descriptions already in the glossary. These definitions were then circulated to Partners who were given one week to make comments and suggest amendments to the proposed EUnetHTA definitions. All comments and suggestions received by NCCHTA were considered. Based on this feedback, the EUnetHTA definitions were re-drafted and integrated into the glossary.
Stage 8. Gathering Additional Descriptions
It was decided that additional descriptions should be gathered for the terms that did not have a drafted EUnetHTA definition. This resulted in eleven of the Partners who had already submitted descriptions being allocated a further three glossary terms and asked to write descriptions for these. They were given 4 weeks to draft additional descriptions for the terms they had been allocated. Of the eleven Partners contacted, ten were able to provide descriptions for these terms. Once the new descriptions had been received, they were reviewed by NCCHTA and incorporated into the glossary.
RESULTS
The completed glossary provides a comprehensive range of descriptions, examples, and comments for forty-two potentially confusing HTA terms related to HTA reports and their adaptation from one setting to another. Table 1, List of the Forty-Two HTA Adaptation Terms Included in the Glossary, below lists all of the terms included, and the full glossary can be found at: http://www.eunethta.net/upload/WP5/Glossary%20of%20HTA%20Adaptation%20Terms%20November%202007.pdf. Beside each description is the name of the HTA organization that provided it and the country from which it originates.
Table 1. List of the Forty-Two HTA Adaptation Terms Included in the Glossary
The aim of the glossary is to identify and highlight key words and concepts that are easily misunderstood between countries. Hence, it includes several descriptions for each term, which attempt to clarify areas of confusion by demonstrating the range of ways the terms may be used depending on the setting. It also contains examples of where the usage of these terms may differ between countries. This is done to raise awareness that these terms may mean different things in different countries.
The glossary attempts to highlight areas of confusion and seeks to clarify them. Hence, certain terms have been grouped together if they are closely related and therefore more likely to cause confusion. For these terms in particular, the various descriptions attempt to bring out the differences between the terms. In each description, various possible interpretations of each of term are provided, along with examples of how the term is used in different countries. Therefore, the glossary provides descriptions of the various uses of each term in different contexts rather than one prescriptive definition.
To enhance the glossary, some convergence on glossary entries was reached for a few specific terms. These form EUnetHTA definitions for these terms and, as is described in Stage 5, are placed as the first entry for that term in the glossary. The terms were selected because they were considered to be particularly relevant to the issues of adapting HTA reports. The terms selected were: clinical and policy question, context specific and setting, domain, speedy sifting and toolkit, relevance and reliability, and generalizability and transferability.
By referring to the glossary, users can see the wide range of usage of the various terms in different countries and HTA agencies across Europe. It can also be used by all stakeholders involved in the HTA process of help their understanding of HTA reports from different contexts and as a tool to aid in the adaptation of such reports.
DISCUSSION
This study describes the development of a glossary of HTA adaptation terms for use by the EU and Member States. The resulting glossary is available on the Web (http://www.eunethta.net/upload/WP5/Glossary%20of%20HTA%20Adaptation%20Terms%20November%202007.pdf) and can be used by HTA agencies to improve their understanding of how various HTA terms are used in different contexts. The glossary can also act as an aid to help adapt information and data from an HTA report written for another context into material relevant for their own contextual report.
The initial work by NCCHTA involved examining existing resources available to HTA agencies interested in adaptation. This research fed into the initial list of terms for inclusion in the glossary. A vital part of the development was the substantial contribution from other EUnetHTA Partners. These contributions included suggestions for additional terms, detailed descriptions of how each term is used in their specific setting, examples of this usage, and comments on descriptions provided by other Partners. The variety of HTA agencies that contributed is noteworthy. Contributors ranged from established agencies to those just starting out, and from across the EU.
The glossary that the Partners produced is fundamentally different from other glossaries that are available, for example, the INAHTA glossary. First, it deals only with terms relating to adaptation. Second, it provides numerous descriptions of these terms from different HTA organizations in the EUnetHTA Project, rather than simply prescribing a single definition. One of the major benefits of the glossary is that it can help clarify how the same term can be used in different contexts and settings. Although certain terms have internationally recognized meanings, the glossary highlights the fact that many terms are used in ways that are slightly, but importantly, different. In particular, the context in which a term is used can have important consequences on its precise meaning.
Users, however, should be aware of several limitations of the glossary. For instance, the present list of terms is limited. Furthermore, the descriptions, examples, and comments in the glossary currently come only from a European context. There is considerable scope for further development of the glossary. Input from elsewhere, for example, North America, Australasia, and South America, would be extremely useful for broadening the scope of the glossary. This input would ideally involve suggestions for additional terms for inclusion and the provision of additional descriptions/examples.
Presently, the glossary is available on the Web as a downloadable PDF, complete with hyperlinks and bookmarks to aid navigation. However, the glossary would be further enhanced by providing it in the format of an interactive tool, which users could search freely and to which they could add their own HTA terms and descriptions.
By developing the glossary, the EUnetHTA Partners have provided the creators and users of HTA with a tool that helps highlight where confusion in language can arise. We anticipate that the glossary will continue to grow and develop as more terms, descriptions, and examples are contributed from HTA organizations worldwide. Therefore, the next step is to provide the glossary in a format that will encourage this process. As described above, NCCHTA is designing a Web site to host the glossary and enable users to make suggestions and additions to the list of terms and/or descriptions. In this way, the glossary will continue to evolve as the language of HTA develops and as organizations from yet more settings add their understanding of the HTA adaptation terms.
It is hoped that the glossary will encourage the sharing and adaptation of HTA reports within Europe and beyond. Its use and continued development will help retain the network established by the development process and foster links between agencies to share work on HTA. Furthermore, the glossary will help to bring about a greater understanding of HTA reports from different contexts.
CONTACT INFORMATION
Claire Rosten, BA, MA, PhD (c.e.rosten@brighton.ac.uk), Research Fellow, Research Design Service South East, University of Brighton, Mayfield House, Brighton BN1 9PH, UK
Deborah L. Chase, BSc, MSc, PhD (dla1@soton.ac.uk), Senior Researcher, Nicholas J. Hicks, MBBS, BA (Hons), FFPH (hicks@soton.ac.uk), Consultant in Public Health Medicine, Ruairidh Milne, MB, BS, FFPH (rm2@soton.ac.uk), Senior Lecturer in Public Health, National Coordinating Centre for Health Technology Assessment, University of Southampton, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK
