To assess convergent validity of stated preference methods in studies where they were used to elicit patient preferences for informing medical product decisions.

In four studies, two stated preference methods were used to elicit preferences of patients with neuromuscular diseases (NMD; n = 140, Discrete Choice Experiment [DCE] and Best-Worst Scaling [BWS] case 2), diabetes (n = 495, DCE and swing weighting [SW]), myocardial infarction (MI; n = 335, DCE and BWS case 1), and rheumatoid arthritis (RA; n = 982, DCE and probabilistic threshold technique [PTT]). In each study, results of the two methods were compared using a normalized preference measure for which confidence intervals (CIs) were estimated using nonparametric bootstrapping of 500 samples. Normalized preference measures comprised of mean relative attribute importance weights (NMD and diabetes studies), attribute uptake probability (MI study), or maximum acceptable risk (RA study).

In all four studies, attribute ranking showed similar patterns between DCE and other methods for the most important attributes. The same attribute had highest importance in three out of four studies. Significant differences were found in ranges of normalized preference measures of each study between DCE and the other methods: 4.1–43.4 versus 8.9–24.7 for DCE and BWS case 2 in NMD; 3.8–49.7 versus 11.9–16.8 for DCE and SW in diabetes; 2.0–85.5 versus 0.2–69.0 for DCE and BWS case 1 in MI; -3.5–49.2 versus 1.1–18.1 for DCE and PTT in RA.

Preferences differed significantly between DCE and other preference methods implying limited convergent validity. The substantially larger ranges in normalized outcome measures in DCE compared to other methods, are likely due to differences in mechanics and bias related to the methods. Since none of the methods is considered the golden standard for measuring stated preferences as true preferences are unknown, further studies are necessary to compare stated preference methods, determine internal validity and data quality, and potentially measure external validity.

Understanding patient preferences and the demand for healthcare interventions and technology is critical for health technology assessment (HTA). New health technologies have potential for savings and increased efficiency but even the most cost-effective and efficacious interventions can fail if patient preferences are not properly accounted for. Patient preferences in HTA are primarily limited to representation in appraisal committees; however, more robust methods are available and should be incorporated into the assessment of interventions.

Using data from three discrete choice experiments (DCEs), we reflect on the importance of patient preferences in the design of healthcare interventions. We draw insights from three studies which investigated preferences relating to HIV self-testing amongst long distance truck drivers in Kenya; differentiated antiretroviral therapy services amongst stable HIV patients in Zimbabwe; and tuberculosis preventive therapy for children in Eswatini.

We highlight three key findings. First, understanding patient preferences is crucial when designing services, and providers sometimes underestimate behavioural barriers and overestimate the extent to which people are motivated simply by health benefits. Optimism is often driven by evidence showing high acceptability, but when preference structures are incorporated in intervention design, there are important insights into how patients plan to utilize services. Second, trade-offs matter in determining which characteristics are perceived to be most important to patients – a key strength of the DCE methodology. Understanding of these trade-offs can help prioritize which characteristics of interventions to target. Finally, disentangling the effect of different characteristics of service delivery models on preferences is important for rethinking how interventions are delivered. If services are designed to better align with preferences, implementers can ensure new interventions have the desired effect on health and economic outcomes.

These findings highlight the value of behavioural economic approaches for investigating preferences for health interventions and providing insights into the demand for services, which must feed into the HTA analyses. Incorporating DCEs into HTA is inexpensive and provides robust data for improving HTA.

The evolution of breast cancer treatments over the last decade has resulted in tailored therapies for the different types and stages of breast cancer. Each treatment has a profile of benefits and adverse effects which are taken into consideration when planning a treatment pathway. The objective of this study is to examine whether patients’ preferences are in line with what is considered important from policy-makers viewpoint.

An online discrete choice experiment (DCE) was conducted in six European countries (France, Germany, Ireland, Poland, Spain, UK) with breast cancer (BC) patients. The DCE comprised of six attributes: overall survival (OS), hyperglycaemia, rash, pain, functional well-being (FWB) and out-of-pocket payment (OOP). Sixteen choice sets with two hypothetical treatments and a “no treatment” option were presented. Sociodemographic and disease related data were collected. Heteroscedastic conditional and mixed logistic models accounted for scale and preference heterogeneity between countries and patients respectively. Latent class analysis categorized patients in classes. Marginal rates of substitution (MRS) were estimated for OOP versus the rest of attributes to establish the ranking of preferences for each attribute.

Two hundred and forty-seven patients with advanced or metastatic BC and 314 with early-stage BC responded. Forty-nine percent of patients were less than 44 years old and 65 percent had completed university education. The MRS of the analysis demonstrated that “severe pain” is the highest dis-preferred attribute level, followed by “severe impairment in FWB” and OS. Four classes of patients as “decision-makers” were identified. Additionally, there is sensitivity in preferences for both levels of pain and FWB depending on the stage of the disease.

This study suggests that there is heterogeneity in treatment preferences of breast cancer patients depending on their sociodemographic and disease related characteristics. In combination with clinical guidelines, patient preferences can support the selection and tailoring of treatment options.

When Cochlear implants (CI) were first introduced, only postlingually, profoundly hearing impaired individuals were implanted unilaterally. As experience grew, eligibility was followed by prelingual deaf children, and a second contralateral CI was being considered. Due to surgical and technological improvements, eligibility criteria for CI are now shifting, encompassing patients with more residual hearing. We aimed to explore, ex ante, whether such shift is warranted.

A dynamic, population-based Markov modeling study was conducted. Model parameters were based on available evidence, expert opinion, and calibration. The model mimics Dutch demographic development in three age categories over a period of 20 years. Impact of changing eligibility was explored in terms of number of CI recipients, costs, quality of life and cost-effectiveness from a societal perspective.

If those with severe hearing loss would qualify and opt for CI similar to those with profound hearing loss, this would lead to a fourfold increase of CI recipients (from 8,815 to 35,630) over a 20 year period, resulting in an increase in costs (EUR 550 million) and QALYs (54,000), with an Incremental Cost Utility Ratio of EUR 10,771/QALY (2.5–97.5 percentiles: 1,252–23,171).

Results suggest that expected health gains could be such, that the investment may be considered cost-effective against the backdrop of currently prevailing criteria. However, for this, a substantial increase in operating capacity, follow-up care and rehabilitation are required. Further inquiries are needed to investigate whether such increased capacity can be achieved, to ensure equitable access to those services.

In 2012, the Netherlands introduced a conditional reimbursement (CR) program to give patients earlier access to promising treatments (i.e., drugs and medical devices). After completion of the CR-study the results are used to (re)assess whether the treatment fulfills the criteria for reimbursement.

Promising treatments were identified via a bottom-up (physicians/researchers initiated) or top-down process (after the Dutch National Health Care Institute (Zorginstituut Netherland; ZIN) concluded that the intervention did not fulfill the criteria for reimbursement but the initial results seemed promising). A CR-grant was only approved if a committee determined the treatment had an added (health, social, ethical or organizational) benefit compared to Standard of Care (SoC) and the grant proposal was of good quality and fit for purpose. After approval from the Dutch Ministry of Health, all Dutch insurance companies were obliged to reimburse the treatment for patients participating in the CR-study. Researchers could also apply for a research grant (maximum EUR 400,000) from the Netherlands Organization for Health Research and Development (ZonMw), if there was no ‘wealthy’ manufacturer.

Currently, there are 23 (ongoing and completed) CR-studies. All of them are closely monitored by ZIN, ZonMw, and stakeholders. The results of all completed CR-studies (n = 11) have been used in a (re)assessment. ZIN concluded that five treatments were not effective compared to SoC. Six interventions were effective and cost-effective compared to SoC and are now reimbursed. In most cases (>80%) the physician and patient groups agreed with the conclusion about reimbursement. In some cases there were additional requirements to maintain the clinical effectiveness and cost-effectiveness in clinical practice (such as training of new physicians).

Data from CR-studies are important for reassessments. Factors with a positive influence are: a maximum duration for a CR-study, close monitoring, possibility to adapt the study design (only with approval from ZIN and ZonMw), and active involvement of stakeholders (physician and patient groups). A negative influence was: the legal requirements to ensure only reimbursement for patients participating in a CR-study.

The standard surgical technique for lumbosacral radicular syndrome in the Netherlands is open microdiscectomy (OM). An alternative technique, preferred by some Dutch physicians, is percutaneous transforaminal endoscopic discectomy (PTED). However, in 2006 the Dutch National Health Care Institute (Zorginstituut Netherland [ZIN]) concluded that the available evidence was insufficient, and a high quality randomized controlled trial (RCT) was needed to assess the cost-effectiveness of PTED compared to OM. The relevant physician group agreed with this conclusion, but they were unable to perform this RCT due to lack of funding and high treatment costs.

In 2012, the Netherlands introduced a conditional reimbursement (CR) program to give patients earlier access to promising treatments. Researchers, in collaboration with physicians and patients, submitted a grant proposal and in 2016 the Dutch Ministry of Health approved the CR of PTED. Due to this decision, insurance companies were obliged to reimburse PTED for patients participating in the PTED-study (NCT02602093). The Netherlands Organization for Health Research and Development (ZonMw) also provided a research grant to fund the PTED-study. In total, 682 adult patients with greater than 10 weeks of radiating pain, or greater than 6 weeks of excessive radiating pain and an indication for surgery were included. After 4 years and 5 months the PTED-study was completed.

Outcomes of published studies and the unpublished PTED-study were used in the HTA reassessment. Results showed that PTED was noninferior to OM with regards to leg pain (Visual Analogue Scale: mean difference (MD) -0.73; 95% confidence interval [CI] -5.04, 3.59), functional status (Oswestry Disability Index: MD -2.07; 95% CI -3.61, -0.53), and rate of complications (relative risk 0.45; 95% CI 0.18, 1.12) after 6 months (GRADE level ‘moderate’). Furthermore PTED was, after the surgeons’ learning-curve, cost-effective.

This CR project was successful and PTED is now reimbursed as part of the Dutch healthcare package. However, in order to maintain high quality care in clinical practice, safeguards should be developed (including the appropriate training of surgeons). This example shows that CR programs are essential for promising treatments without ‘wealthy’ manufacturers. Additionally, all stakeholders are needed to make a CR-study successful.

In early stages, the consequences of innovations are often unknown or deeply uncertain. This complicates health economic modelling. The field of decision-making under deep uncertainty (DMDU) uses exploratory modelling (EM) to help decision-makers make sound decisions under conditions of deep uncertainty (i.e., when stakeholders do not know, or cannot agree on, the system model, the probability distributions to place over the inputs to these models, which consequences to consider, and their relative importance). The aim of this research was to evaluate the potential of EM for the early evaluation of health technologies.

EM and early health economic modelling (EHEM) were applied to an early evaluation of minimally invasive surgery (MIS) for acute intracerebral hemorrhage, and were compared to derive differences, merits, and drawbacks of EM.

The approaches fundamentally differ in the way uncertainty is handled. Where in EHEM the focus is on the value of the technology, while accounting for the uncertainty, EM focuses on the uncertainty. EHEM aims to assess whether the innovative strategy is potentially cost-effective, often using assumptions. EM on the other hand focuses on finding robust strategies (i.e., strategies that give relatively good outcomes over a wide range of plausible futures). This was also reflected in our case study. For example, EHEM provided cost-effectiveness thresholds for MIS effectiveness, assuming fixed MIS costs. EM showed that a strategy with a population in which most patients had severe intracerebral hemorrhage was most robust, regardless of MIS effectiveness, complications, and costs.

EM seems most suited in the very early phases of innovation (i.e., when a problem is signaled). Here, it can explore the robustness of many potential strategies under uncertainty. When potential strategies are selected, EHEM seems useful to optimize these strategies. Yet, EM methods are complex and might only be fully effective when a policy window exists that facilitates flexible research and adoption strategies.

Access to affordable direct-acting antiviral (DAA) remains limited in developing countries, often due to high treatment cost. This study aimed to elaborate the initiatives taken by the Ministry of Health Malaysia (MoHM) to provide early access to ravidasvir, a new DAA for hepatitis C treatment, in Malaysia.

MoHM collaborated with Drugs for Neglected Diseases initiative (DNDi) to develop ravidasvir, a new chemical entity of oral non-structural protein 5A (NS5A) inhibitor. MoHM co-sponsored and participated the DNDi-led Phase II/III study (STORM-C-1 trial) to assess the efficacy and safety of ravidasvir-sofosbuvir combination therapy. Agreement was signed between Pharmaniaga, Pharco Pharmaceuticals and DNDi to register and supply affordable hepatitis C treatment in Malaysia and South-East Asia. MoHM and Pharmaniaga mutually worked on the registration of ravidasvir in Malaysia. Series of pre-submission meetings took place, rolling submission was allowed and conditional registration pathway was used. As a separate initiative, MoHM partnered with the Foundation for Innovative New Diagnostics (FIND) to implement decentralization and test-and-treat strategies for screening of hepatitis C virus (HCV).

First stage of the STORM-C-1 trial reported that the combination of ravidasvir-sofosbuvir was highly effective across all genotypes and safe. The Drug Control Authority (DCA) Malaysia has granted a conditional registration for ravidasvir hydrochloride 200mg tablet (Ravida®) in June 2021, making Malaysia as the first country in the world to approve ravidasvir. Registration process expedited and took place within 15 months. The supply of Ravida® in Malaysia is expected in near future. Meanwhile, MoHM also implemented nationwide HCV screening using rapid diagnostic test kit in private hospitals, community clinics, prisons and rehabilitation centers which previously was done only in hospitals for outreach to the targeted group.

Ravidasvir-sofosbuvir has potential as a tool to eliminate hepatitis C in Malaysia by 2030, the WHO’s global elimination targets. This alternative new drug development model was successful due to strong leadership, public-private partnership and collaborative strategies. This could also be exercised in other disease area.

Transcatheter aortic valve implantation (TAVI) is a minimally invasive therapy for patients with severe aortic stenosis. In Ontario, increases in capacity have not matched the rapidly growing demand for TAVI. As a result, wait-times for TAVI in Ontario exceed guideline targets, and waitlist morbidity is consequently considerable. The objective of this study was to evaluate the clinical implications of expanded TAVI capacity.

We performed a decision analysis using an open, parallel, resource-constrained microsimulation from the Ontario Ministry of Health perspective. Simulated patients entered the model during a five-year period, and stayed in the model until death or end of time horizon. Referral numbers increased annually according to historical trends. The additional capacity required to meet wait-time benchmarks in five years was identified by a sensitivity analysis. Clinical outcomes were estimated for three strategies: (i) current practice with annual capacity increases; (ii) accelerated capacity increases achieving benchmarks after five years; and (iii) no increase in capacity. Outcomes included pre-procedural mortality and hospitalization, and the proportion of TAVIs performed urgently.

Over the five years, we estimated that TAVI referrals would increase from 1,980/year to 3,268/year. To achieve wait-time benchmarks during this period, TAVI rates must be increased by approximately 6.3 percent annually, for a total of 12,220 procedures performed over the 5 years. Compared to current TAVI capacity increase, an accelerated increase in capacity achieving wait-time benchmarks led to a reduction of 29.36 percent in pre-procedural deaths, as well as 26.38 percent in pre-procedural hospitalizations and 30.31 percent in nonelective TAVIs.

Increases in TAVI capacity in Ontario must be accelerated to meet wait-time benchmarks in five years. Expansion of TAVI care in Ontario would be associated with considerable reductions in mortality and hospitalizations. Without intervention, both wait-times and adverse outcomes on the waitlist are expected to continue increasing. Prioritization strategies to mitigate the adverse effects of long wait-times must be used until wait-time targets are achieved.

Conducting joint health technology assessments (HTA) is one of the main goals of the Beneluxa Initiative. To strengthen this collaboration, this study aimed to assess similarities and differences between past assessments of Beneluxa Initiative member countries (Austria, Belgium, Ireland and the Netherlands).

A retrospective comparative analysis was performed that investigated the similarities and differences in drug assessments in the period 2016 to 2020 in (i) the number and type of assessed indications; (ii) the conclusions within assessments performed by at least two member countries; and (iii) the main arguments leading to the conclusions through a qualitative analysis of selected cases, looking into the patient population, the intervention, comparator, outcome, timing, and included evidence.

The scope of HTA differs between the countries, with Belgium and Ireland assessing most, the Netherlands focusing on drugs above a budget impact threshold and Austria on outpatient drugs. Furthermore, indications might slightly differ between countries. Therefore, only 44 (10%) of the 444 included drug-indication combinations were assessed through a full HTA by all four countries. Between any pair of countries, the overlap was higher, from 63 (Austria-the Netherlands) to 188 (Belgium-Ireland). Added benefit conclusions matched exactly in 62 to 76 percent of the indications, depending on the compared countries. In the remaining cases, often a difference of one added benefit level was observed (e.g., higher versus equal relative effect). Contradictory outcomes were very rare. Differences were observed with regards to whether a cost-effectiveness analysis was performed. When assessing the underlying arguments within the reports for nine cases with different outcomes, it became clear that organizations agree on almost all aspects, and that differences are mostly attributable to slight differences in weighing of some aspects and uncertainties.

Overall, which indications are assessed differs, but for those indications that are assessed by multiple member countries, considerations and assessment outcomes are similar.

The need for timely access to innovative technologies has placed a special focus on the development of policies and practices that can guarantee the availability whilst ensuring the safety of these technologies after launch or licensure. The aim of this paper is to present and discuss Post-Launch Evidence Generation (PLEG) practices among health technology assessment (HTA) bodies at the European level to explore cross-border collaboration opportunities.

In December 2019, a survey composed of nine closed-ended questions with multiple choice answers about the PLEG practices in each country was sent to 25 partners of the European Network of Health Technology Assessment (EUnetHTA) Joint Action 3. In addition to the survey, the national practices were discussed during a face-to-face meeting with all partners of the dedicated work package. A quantitative analysis and a qualitative synthesis of the results was carried out.

Twelve HTA bodies completed the survey. Of these, 11 reported procedures in place for official PLEG requests. In nine of the agencies, the requests are made at the time of the assessment/appraisal. Data collection and analysis mainly lies with companies for pharmaceuticals (60%) while it is more the responsibility of the HTA body for medical devices (75%). Only one agency reported owning the data and being able to exchange the data without asking permission. During the face-to face discussions, it was acknowledged that PLEG practices differ between countries depending on the topic concerned, but most rely on the usage of registries (mainly disease registries) for data collection. Most agencies estimated that a European collaboration could take place.

PLEG practices are in the remit of many European HTA bodies. Data sharing should be anticipated as only some own the data and can exchange them without asking permission. European collaboration on PLEG could commence once the evidence gaps have been defined or during the production of the HTA reports in the case of joint assessments.

The European Network for Health Technology Assessment (EUnetHTA) Joint Action 3 (JA3) aimed to develop a sustainable European model for scientific and technical collaboration on HTA. It succeeded EUnetHTA JA2, which focused on strengthening practical applications of approaches and tools in the European HTA collaboration. Compared to JA2, several changes in procedures and processes were undertaken throughout JA3 in order to improve the different steps in joint HTA production.

Findings and identified challenges regarding the assessment production processes from JA2 were considered as a basis. In JA3, vast majority of structured and informal feedback was gathered from the assessment teams and project managers via feedback surveys and meetings. Only limited informal feedback from stakeholders (such as patients, health care professionals, and health technology developers) that were involved in EUnetHTA assessments was collected. To this end, experiences were documented and recommendations for a future production process were developed.

During the course of JA3, the joint production resulted in 16 pharmaceutical assessments and 27 assessments of other technologies. The latter included medical devices, diagnostics, interventions, and screening. Due to the different context of pharmaceuticals and other technologies, some technology-specific changes needed to be made in their production process. However, the majority of implemented changes were made for both types of technologies to ensure maximum possible alignment in processes. The implemented changes affected several steps in the production process as well as the involvement of stakeholders in EUnetHTA assessments. The production and related project management of assessments was fine-tuned and resulted in clearer, standardized, and comprehensible processes that facilitated transparency and inclusiveness.

The procedural changes led to further standardization and elaboration of assessment production processes in preparation for a future European HTA system under the EU HTA Regulation. However, some methodological challenges remained to be tackled further in the currently ongoing EUnetHTA 21 service contract.

Patient and public involvement (PPI) is an expectation when conducting a health technology assessment (HTA), but there is little guidance for those wishing to embed PPI when developing an HTA proposal. We wanted to ensure PPI was central in preparing a proposal for an HTA potentially of any intervention for pelvic organ prolapse (POP) in women.

We conducted an open process to recruit two PPI co-applicants who, after induction to the project, were jointly responsible for governance of PPI in partnership with the PPI Lead throughout project planning. We facilitated an online workshop with the PPI co-applicants and other women with a lived experience of POP to: develop our question and scope; decide interventions and outcomes for the evidence synthesis; discuss the care pathway for the economic evaluation component; and plan dissemination. The PPI co-applicants were encouraged to comment on the full proposal, while workshop attendees were invited to comment on the plain language summary. Our work adhered to United Kingdom (UK) Standards for Public Involvement. We obtained funding to facilitate PPI within the proposal and reimburse those with lived experience for their time.

Involving the co-applicants and workshop participants strengthened the HTA proposal by: solidifying the rationale based on lived experience; adding interventions to our evidence synthesis not previously considered; and highlighting dissemination outlets that appealed to the public. Comments on the full proposal and plain language summary ensured the proposal was accessible. However, we were unable to discuss everything we originally planned even though researcher time spent on embedding PPI into the proposal was substantial.

Including PPI can be valuable for developing HTA proposals. However, research is required to explore the appropriate level of involvement at the HTA proposal stage, particularly given the large amount of researcher time and additional resource needed to incorporate meaningful PPI.

The cyclic appropriate care program of the National Health Care Institute aims at enhancing quality of care in the Netherlands. A full cycle consists of four phases: (i) screening each ICD-10 chapter and selecting a care trajectory, (ii) in-depth analysis of the care trajectory and formulating actions for improvement, (iii) implementation, and (iv) evaluation. Collaboration with and between relevant parties from the field is key to its success. We describe the 2nd and 3th phase of this cycle, aimed at osteoporosis.

First, nine clinicians, one nurse practitioner, two physiotherapists and one pharmacist representing scientific organizations, two patient representatives and one health insurer, defined research questions. Second, by in-depth analysis on declaration data of health care activities, and comparing the results to the guidelines, research questions were addressed. Last, results were discussed among all parties and actions for making wiser choices and improving the appropriateness of care were formulated. For each action, the parties defined who would take the lead and who would be involved.

The analyses showed that only 26 percent of Dutch fracture patients 50 years of age and above underwent dexa-scanning, as opposed to more than 80 percent in the guideline. All parties underlined this percentage should be increased. Possible actions were defined: to describe the care trajectory in the update of the multidisciplinary guideline; to better inform fracture patients about the importance of dexa-scanning; and to make adjustments to the hospital electronic system in order to facilitate the ordering of dexa-scans. For these actions, medical specialists were in the lead. Other data-analyses showed that half of osteoporosis patients stopped using medication within two years. For the jointly defined actions aimed at raising medication adherence, such as better patient education, the general practitioner and pharmacist were in the lead. Currently, the National Health Care Institute facilitates implementation and monitors improvement.

Close collaboration with relevant parties led to well-considered actions on improvement of osteoporosis care.

Involving patients and community organizations in the health technology assessment (HTA) lifecycle is a core principle at the National Institute for Health and Care Excellence (NICE) and helps build public confidence in the healthcare decision-making. From 2019 to2021 NICE assessed a HIV-1 medicine for the first time. This presented new opportunities and challenges for engaging with Human Immunodeficiency Virus (HIV) community organizations who hadn’t participated in a NICE HTA before. To understand their experience, we collected feedback on the impact and experience of community involvement throughout the assessment.

We used a mixed-method approach using a survey and qualitative feedback from the committee lay member, community experts, and the community organizations. An impact survey was sent to key committee members, the NICE technical team and Associate Director. It included Likert scale questions and open text boxes to capture both quantitative and qualitative data. Additionally, qualitative feedback was gathered throughout the assessment’s lifecycle from the NICE team and the community stakeholders.

Results from the impact survey showed that the community input: had significant impact on the evaluation (100% (n=5)); helped interpret the other evidence and information (80% (n=4)); provided new evidence (60% (n=3)); and, was consistent with the other evidence (40% (n=2)). Examples of impact included highlighting the population heterogeneity, stigma, side effects and the effects of frequent clinic attendance. The key feedback from the qualitative data from the community organizations and experts in terms of their experience were: early support and support throughout from the NICE team; NICE’s flexibility in involving them; and, meaningful inclusion of their evidence in the committee slides.

Both the NICE committee and the community stakeholders recognized that the community input was valued and had an impact on the decision-making. To ensure meaningful community engagement, support and flexibility from NICE were required throughout the assessment. This level of engagement will be adopted in future for community organizations new to medicines HTAs at NICE.

Robot-assisted surgery (RAS) is being adopted rapidly internationally across a wide range of surgical procedures. Although a great deal of evidence of the clinical effectiveness of RAS has been generated, it is possible that the evidence base is not complete or persuasive in some areas where adoption is being considered. This review seeks to summarize systematic reviews (SRs) undertaken to date to illustrate the weight of evidence across specialties. We then take an in depth look at the quality of evidence across several indications where the adoption of RAS is currently underway.

A comprehensive literature search was conducted using Ovid Medline, Embase, and Cochrane Central Register of Systematic Reviews from January 2017 to April 2021 for SRs describing clinical effectiveness outcomes. The body of evidence was mapped across all specialties. For a selected number of indications currently under consideration in Scotland, results were comparatively summarized, and the quality of the reviews was evaluated with the AMSTAR-2 tool.

A total of 451 SRs were found. Most were in urology (n = 130) where RAS is well established, followed by colorectal (n = 63), hepatology (n = 58), and gynecology (n = 41). From within these latter three specialties, we selected six indications in which RAS is currently being considered for adoption in Scotland for in depth review (colorectal cancer surgery, hysterectomy, gastrointestinal oncological resection, hepatic, pancreatic and biliary surgery). Evidence for the clinical effectiveness of RAS versus conventional laparoscopic surgery is mixed across indications and outcomes. In colorectal cancer surgery, for example, evidence was positive for conversion rate and neutral for length of hospital stays, blood loss and postoperative complication and negative for operative time. For hysterectomy, evidence was positive for the length of hospital stays and neutral for operative time, blood loss, conversion rate and postoperative complication. The quality of the included reviews was judged to be critically low.

The currently available evidence of clinical effectiveness is mixed across indications and of low quality.

The purpose of this systematic review is to evaluate whether implantable versus injectable bulking agents (second-line therapies) are equal/superior in terms of effectiveness (severity, quality of life [QoL], sustainability) and safety (adverse events) for fecal incontinence (FI).

A systematic review was conducted and five databases were searched (Medline via Ovid, Embase, Cochrane Library, University of York Centre for Reviews and Dissemination, and International Network of Agencies for Health Technology Assessment database). In-/exclusion criteria were predefined according to the PICOS scheme. The Institute of Health Economics risk of bias (RoB) tool assessed studies’ internal validity. According to the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach, the strength of evidence for safety outcomes was rated. A qualitative synthesis of the evidence was used to analyse the data.

Six prospective uncontrolled trials (143 patients) were included. The evidence consists of six prospective single-arm, before-after studies fulfilling the inclusion criteria for assessing clinical effectiveness and safety for implantable bulking agents. FI severity (Cleveland Clinic FI Score) statistically significantly improved to three months (p<0.01) and six months (p<0.05) follow-up (five studies). Improvements in severity sustainability were reported after 12, 14 (p<0.01), and 36 (p<0.0001) months postoperatively. Improved disease-related QoL (FI QoL Score) was found (p<0.05) 12 months after surgery, and statistically significant improvements in QoL’s sustainability after 12 months (one study).

Procedure-related adverse events (n=3) occurred, where prostheses extruded during surgery, and anal discomfort/pain was felt (n=11). Device-related adverse events, i.e., prostheses’ dislodgement (n=31) and removed/extruded prostheses (n=3), occurred. Studies were judged with moderate/high RoB. The strength of evidence for safety was judged to be very low.

Implantable bulking agents might be an effective and safe minimally invasive option in FI treatment if conservative therapies fail. FI severity significantly improved, but not QoL, which needs to be explored in further studies. Due to the uncontrolled nature of the case series, comparative studies need to be awaited.

Treatment options for men with moderate-to-severe lower urinary tract symptoms due to benign prostatic hyperplasia (BPH) include medical therapy, minimally invasive surgical therapies (MISTs), and invasive surgical procedures. While these treatments are recommended by American Urological Association Guidelines, they have different clinical profiles impacting both efficacy and durability outcomes. Using an indirect comparison approach, this study assessed the clinical effects of combination therapy (CT) using alpha-blockers and 5-alpha reductase inhibitors, two emerging MISTs (prostatic urethral lift [PUL] and water vapor thermal therapy [WVTT]), and two invasive surgical procedures (photoselective vaporization of the prostate [PVP] and transurethral resection of the prostate [TURP]).

A systematic search of Medline, Embase, Cochrane Library, and relevant health technology assessment (HTA) databases was conducted to identify randomized and non-randomized clinical trials of the five treatments published prior to December 2020. Trials were included if they reported changes in International Prostate Symptom Score (IPSS) and retreatment rates, without any country or language restrictions. A random-effects network meta-analysis (NMA) with an aggregate regression model was performed to account for the baseline BPH severity and characteristic differences among men from the different trials.

A total of 237 of 3,104 retrieved abstracts were included for full-text review. Of these, 16 randomized and four non-randomized clinical trials were included in the NMA. The random-effects NMA showed among medical and minimally invasive therapies, WVTT had the greatest one-year IPSS improvement (-∆11.7), followed by PUL (-∆10.4) and CT (-∆10.3). The one-year IPSS improvement for TURP and PVP was comparable (-∆14.1 vs. -∆13.8, respectively; p-value=0.675). The one-year retreatment rates were lowest for WVTT (3.0%), followed by CT (3.6%), TURP (6.3%), PVP (7.8%), and PUL (8.0%).

WVTT provided greater clinical and durability benefits compared to other less invasive treatment options for men with BPH. Given NMA is increasingly used in HTA processes, this study provided systematically synthesized evidence that could facilitate decision-makers in determining new technology coverage decisions globally.

Recently, there have been concerns regarding a trend toward poorer quality evidence being accepted by regulatory institutions and the consequent impact on health technology assessment (HTA) decision-making. This study aimed to determine whether there has been a change in the quality of evidence provided on cancer drugs proposed for listing on the Pharmaceutical Benefits Scheme, using data solely extracted from public summary documents (PSD) published by the Australian government.

PSDs published from July 2005–2020 were reviewed. Metrics associated with quality of evidence were extracted, including the directness of comparison, study design, sample size, and risk of bias (RoB). Additional data were extracted to provide greater context to any observed trends in quality of evidence. Analyses were performed across different time periods. Associations between the quality of evidence and time periods were explored using logistic regression analysis.

In total, 214 PSDs were included in the analysis. Only 13 percent of submissions provided a single arm study or observational study as the key evidence; however, 37 percent of submissions did not contain a direct (‘head-to-head’) comparison relevant to Pharmaceutical Benefits Advisory Committee (PBAC) decision-making. Among all submissions containing direct evidence, about half had findings of a moderate/high/unclear RoB. Among all submissions containing indirect comparisons, over half had transitivity issues. In submissions containing direct comparisons, there was an increase in the RoB over time even after adjusting for trial data maturity and the rareness of the drug indication (odds ratio [OR] 1.30; 95% confidence interval [CI] 0.99, 1.70). There were no clear time trends observed in sample size, directness, study design, or transitivity issues during any of the observed time periods.

In the last 7 years, a high proportion of cancer drug submissions presented findings with a high RoB and transitivity issues. As the evidence dossiers provided to the PBAC are often congruent with submissions made elsewhere, this poor evidence quality is of concern and can only lead to higher levels of decision-maker uncertainty.