Objectives: As health technology assessments (HTA) may have considerable impact on health-care decisions, it is essential to guarantee the expected quality of these assessments. Variations in the methods used or lack of transparency can affect the important role of HTA reports.

Methods: In our study, we analyzed the methods used in two corresponding HTA reports to assess the validity of two key papers, which were included in both reports. Also the discussions and the final conclusions of both reports were compared.

Results: The authors of the two HTA reports used different instruments to assess the validity of the original studies. A minor problem is differences that were found in the assessments of identical validity aspects of the included studies and in the information provided in the HTA reports. A more serious problem was found in both HTA reports which identified the weakness of the key papers and expressed these limitations in the discussions (read mainly by scientists) but not in the conclusions (read mainly by policy-makers).

Conclusions: The results of this study may be important for any institution that prepares recommendations for policy-makers. In the case of HTA reports, no new checklists are necessary as the checklist offered by the INAHTA contains the needed information. It may be necessary, however, to consider an “intramural quality board,” which helps the member organizations guarantee both the transparency of the original studies included in HTA reports and the transparency of the HTA report itself.