Original Article
Using GRADE for evaluating the quality of evidence in hyperbaric oxygen therapy clarifies evidence limitations

https://doi.org/10.1016/j.jclinepi.2013.08.004Get rights and content

Abstract

Objectives

The current evidence rating for hyperbaric oxygen therapy indications uses the American Heart Association system, which mainly depends on the study design.

Study Design and Setting

We systematically reviewed the literature and applied the Grading of Evidence, Assessment, Development and Evaluation (GRADE) approach to the main patient-important outcomes in each indication.

Results

We included 17 systematic reviews that synthesized 44 randomized trials and 131 observational studies enrolling 8,145 participants. The quality of evidence for seven indications with category A was high (1), moderate (2), low (2), and very low (2); for 10 indications with category B, it was moderate (1), low (5), and very low (4); and for 1 indication with category C, it was high. The quality of evidence was rated down for the risk of bias and imprecision for most indications and rated up because of large effect size for some indications. Most discrepant ratings were in the indications of decompression illness (C, high), carbon monoxide poisoning (A, very low), and later presentations of idiopathic sudden hearing loss (A, very low).

Conclusion

The GRADE approach uncovered factors affecting the quality of evidence that were otherwise implicit. Knowing these factors can influence clinicians' confidence in applying hyperbaric oxygen therapy and orient the research agenda.

Introduction

What is new?

Key findings

  1. The Grading of Evidence, Assessment, Development and Evaluation (GRADE) approach uncovered the factors that affect the quality of evidence supporting the use of hyperbaric oxygen therapy that were otherwise implicit.

  2. Hyperbaric oxygen therapy is thought to be beneficial for the treatment of various conditions (decompression sickness, carbon monoxide poisoning, central retinal artery occlusion, clostridial myonecrosis, gas gangrene, crush injury, compartment syndrome, enhancement of healing of certain wounds, exceptional blood loss anemia, intracranial abscess, necrotizing soft tissue infections, osteomyelitis, delayed radiation injury, compromised skin grafts and flaps, and thermal burns).

What this adds to what was known?
  1. The quality of evidence of the main patient-important outcomes in approved hyperbaric oxygen therapy indications is commonly rated down for the risk of bias and imprecision but rated up because of large effect size for a few indications.

  2. Most discrepant quality of evidence was in the indications of decompression illness and air/gas embolism (level C, high), carbon monoxide poisoning (level A, very low), and later presentations of idiopathic sudden sensorineural hearing loss and tinnitus (level A, very low).

What is the implication and what should change now?
  1. The main implications of this study relate to agenda setting of future research, which should focus on improving the limitations that became more recognizable using the GRADE approach.

Hyperbaric oxygen therapy is thought to be beneficial and recommended for the treatment of several conditions: air or gas embolism, carbon monoxide poisoning (with and without concomitant cyanide poisoning), central retinal artery occlusion, clostridial myositis and myonecrosis, gas gangrene, crush injury, compartment syndrome, and other acute traumatic ischemias, decompression sickness, enhancement of healing of certain wounds, exceptional blood loss anemia, intracranial abscess, necrotizing soft tissue infections, osteomyelitis, delayed radiation injury, compromised skin grafts, and flaps and thermal burns [1].

Biological rationale and mechanistic plausibility support a therapeutic effect in these conditions. Hyperbaric oxygen increases oxygen transport capacity of the plasma (independent from red blood cells, which is important if a vascular obstruction or hemoglobinopathy exists), generates free radicals that oxidize bacterial cellular structures, improves leukocyte oxygen-dependent peroxidase system, inhibits bacterial growth (particularly anaerobes), reduces tissue edema due to the osmotic effect of oxygen, stimulates the growth of new blood vessels and improves fibroblast function, mobilizes stem/progenitor cells from the bone marrow, and improves the oxygen-dependent transport of antibiotics [2], [3]. Clinical and policy decision-making, however, requires comparative effectiveness evidence warranting high confidence in quantitative estimates of the effect of hyperbaric oxygen therapy on patient-important outcomes.

The Undersea and Hyperbaric Medical Society has been proactive in synthesizing and appraising the quality of evidence that supports the use of hyperbaric oxygen therapy. The Society uses the guideline framework developed by the American Heart Association. In this framework, the “certainty in evidence” is primarily dependent on the study design; level A evidence is derived from multiple randomized controlled trials (RCTs) or meta-analyses, level B is derived from a single RCT or nonrandomized studies, and level C is derived from consensus opinion of experts, case studies, or standard of care [4]. A limitation of this system is the rigidity imposed by the primacy of the study design. Study design is one factor that contributes to the quality of evidence, but it is not the only factor. There are many examples of RCTs that provide evidence warranting low confidence in the estimates of effect and, conversely, examples of observational studies warranting high confidence in the estimates of effect. Another problem is the ambiguous terms used to describe study design. In the American Heart Association classification, for example, level A evidence may be derived from “meta-analysis,” but there is no caveat that defines the design of the studies that contribute to that meta-analysis, which can greatly affect the quality of the meta-analysis evidence.

Clearly, there are factors other than study design that should affect our confidence in the estimates of effect (the quality of evidence). Rigorous frameworks such as Grading of Evidence, Assessment, Development and Evaluation (GRADE) and other guideline systems [5], [6], [7] that use components of the GRADE system allow for the explicit use of factors that can increase or decrease the quality of the evidence. The GRADE approach has been adopted by more than 70 national and international organizations [8]. A summit report by the American Heart Association Clinical Practice Guideline Methodology Group implied that they will incorporate some of the GRADE elements in future guidelines [9]. Similarly, the Cochrane Collaboration now requires authors to use GRADE for all important outcomes in their systematic reviews. The GRADE approach describes five factors that can lead to rating down the quality of evidence and three factors that can lead to rating up the quality of the evidence, from a starting point determined by study design [10].

In this study, we applied the GRADE approach to appraise the quality of evidence supporting hyperbaric oxygen therapy indications. Although GRADE has been used in a few Cochrane systematic reviews on hyperbaric oxygen therapy, this is the first comprehensive use of GRADE across all indications approved by the Undersea and Hyperbaric Medical Society or the Center for Medicare and Medicaid Services. The aim of this study was to determine if using GRADE provides more discrimination and less ambiguous description of the quality of evidence than the American Heart Association method. A more discriminative system not only improves the clarity, transparency, and credibility of the hyperbaric oxygen therapy guidelines but also facilitates the identification of research priorities and strategies in this field.

Section snippets

Methods

In this review, we defined hyperbaric oxygen therapy as a treatment in which patients breathe 100% oxygen intermittently while inside a treatment chamber at a pressure higher than sea level pressure (i.e., >1.4 atm absolute) [1]. Hyperbaric oxygen therapy by definition is systemic, not topical, therapy.

Search results

The literature search yielded 426 eligible references; of which, 61 were selected to be reviewed in full text and 17 systematic reviews were eventually included (carbon monoxide poisoning [14], clostridial myositis and myonecrosis [15], crush injuries and compartment syndrome [16], decompression illness [17], central retinal artery occlusion [18], enhancement of healing in selected problem wounds [19], [20], severe anemia [21], necrotizing soft tissue infections [15], refractory osteomyelitis

Main findings

We systematically searched the literature and used the rigorous GRADE approach to rate the quality of evidence supporting the effect of hyperbaric oxygen therapy on the main outcomes for indications approved by the Undersea and Hyperbaric Medical Society and the Centers for Medicaid and Medicare Services. We found that the American Heart Association method, based on study design, was not able to discriminate evidence that merited higher or lower confidence in the estimates judged using the

Conclusion

The GRADE approach uncovered the factors that affect the quality of evidence that were otherwise collapsed into fewer categories using the American Heart Association system. The GRADE approach can identify reasons to reduce or increase the quality of evidence, which can influence clinicians' confidence in applying hyperbaric oxygen therapy across approved indications and orient the research agenda.

References (36)

  • P. Tricoci et al.

    Scientific evidence underlying the ACC/AHA clinical practice guidelines

    JAMA

    (2009)
  • U.S. Preventive Services Task Force. Grade Definitions. Accessed 9/12/2013. Available at...
  • M. Faraday et al.

    Staying at the cutting edge: a review and analysis of evidence reporting and grading; the recommendations of the American Urological Association

    BJU Int

    (2009)
  • A.K. Jacobs et al.

    ACCF/AHA clinical practice guideline methodology summit report: a report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines

    Circulation

    (2013 Jan 15)
  • D. Thomson et al.

    The evolution of a new publication type: steps and challenges of producing overviews of reviews

    Res Synth Methods

    (2010)
  • V. Smith et al.

    Methodology in conducting a systematic review of systematic reviews of healthcare interventions

    BMC Med Res Methodol

    (2011)
  • G.Y. Gandhi et al.

    Patient-important outcomes in registered diabetes trials

    JAMA

    (2008)
  • N.A. Buckley et al.

    Hyperbaric oxygen for carbon monoxide poisoning

    Cochrane Database Syst Rev

    (2011 Apr 13)
  • Cited by (0)

    Conflict of interest: The authors report no financial conflicts of interest. M.H.M., V.M.M., and G.H.G. are members of the GRADE Working Group.

    View full text