GRADE home (2024)

One of the aims of the GRADE Working Group was to reduce unnecessary confusion arising from multiple systems for grading evidence and recommendations. To avoid adding to this confusion by having multiple variations of the GRADE system we suggest that the criteria below should be met when stating that the GRADE approach was used to assess evidence or develop recommendations. Also, while users may believe there may be good reasons for modifying the GRADE system, we discourage the use of "modified GRADE approaches" that differ from the approach described by the GRADE Working Group.

On the other hand, we encourage and welcome constructive criticism of the GRADE approach, suggestions for improvements, and involvement in the GRADE Working Group. As most scientific approaches to advancing healthcare, the GRADE approach will continue to evolve in response to new evidence and to meet the needs of systematic review authors, guideline developers and other users.

Suggested minimal requirements for stating that the GRADE system or approach was used (updated 2023-05)

1. The certainty in the evidence (also known as quality of evidence or confidence in the estimates) should be consistent with the operationalizations used by the GWG.[1, 9-11] Conceptually, GRADE has provided the following definition “The certainty that the true effect, accuracy measure, or association lies on one side of a particular threshold, or in a particular range.”[12, 13] And for evidence from reviews of qualitative research: “an assessment of the extent to which the review finding is a reasonable representation of the phenomenon of interest.”
2. Explicit consideration should be given to each of the GRADE domains for assessing the certainty in the evidence (i.e., risk of bias, imprecision, inconsistency, indirectness, publication bias, large effects, dose response gradients, and residual plausible opposing bias), although different terminology may be used to describe these domains (e.g., limitations in the detailed study design or execution instead of risk of bias).
3. The certainty in the body of evidence should be assessed for each critical or important outcome for a decision (also called main outcomes) using GRADE’s categories (e.g., high, moderate, low and/or very low), with definitions for each category that are consistent with the definitions published by the GWG.[9] These outcomes should describe both desirable and undesirable health effects (also called benefits and harms or efficacy and safety outcomes).
4. GRADE evidence tables (e.g., evidence profiles or summary of findings tables) should be used to present the results of evidence synthesis and an assessment of certainty in evidence, and these should be based on systematic reviews (we refer to systematic reviews as a methodology that is also used in health technology assessments and other evidence synthesis following such methodology).[14] At a minimum, the evidence that was assessed and the methods that were used to identify and appraise that evidence using GRADE should be clearly described.

The next three requirements apply only to GRADE for making decisions or recommendations:

5. Explicit criteria (both health related and contextual), as described in the GRADE Evidence to Decision (EtD) frameworks, should form the basis for recommendations or decisions. Explicit judgments should be made for each criterion that is chosen, including those that may be added, and the evidence used to inform each of those judgments should be provided. Additional considerations that influence the recommendation or decision should also be documented and be accessible.[5]
6. Guideline developers should describe the magnitude and balance of desirable and undesirable health effects of the options that are considered, the value of each outcome, and the certainty of the evidence for the aforementioned EtD criteria. Other EtD criteria, such as resources use, equity, acceptability, and feasibility that are used, may vary, depending on the type of recommendation or decision[5, 15, 16], and the mandate of the group that is making the recommendation or decision.
7. A recommendation should have one of two strengths (strong or conditional, also called weak) and one of two directions (for or against). The definitions for each category should be consistent with the definitions used by the GWG (although different terminology may be used, such as strong and discretionary).[17, 18]

How to cite this GGG guidance: Schunemann HJ, Brennan S, Akl EA, Hultcrantz M, Alonso Coello P, Xia J, et al. The development methods of official GRADE articles and requirements for claiming the use of GRADE - a statement by the GRADE Guidance Group. J Clin Epidemiol. 2023 May 19;159:79-84. doi: 10.1016/j.jclinepi.2023.05.010.

Citations:

1. Atkins D, Best D, Briss PA, Eccles M, Falck-Ytter Y, Flottorp S, et al. Grading quality of evidence and strength of recommendations. BMJ. 2004;328:1490.
2. Schunemann HJ, Best D, Vist G, Oxman AD, Group GW. Letters, numbers, symbols and words: how to communicate grades of evidence and recommendations. CMAJ : Canadian Medical Association journal - journal de l'Association medicale canadienne. 2003;169:677-80.
3. Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336:924-6.
4. Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. Journal of clinical epidemiology. 2011;64:383-94.
5. Alonso-Coello P, Schunemann HJ, Moberg J, Brignardello-Petersen R, Akl EA, Davoli M, et al. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction. BMJ. 2016;353:i2016.
6. Zhang Y, Akl EA, Schunemann HJ. Using systematic reviews in guideline development: the GRADE approach. Res Synth Methods. 2018
7. Dixon C, Dixon PE, Sultan S, Mustafa R, Morgan RL, Murad MH, et al. Guideline developers in the United States were inconsistent in applying criteria for appropriate Grading of Recommendations, Assessment, Development and Evaluation use. Journal of clinical epidemiology. 2020;124:193-9.
8. McGowan J, Akl EA, Coello PA, Brennan S, Dahm P, Davoli M, et al. Update on the JCE GRADE series and other GRADE article types. Journal of clinical epidemiology. 2021;140:163-4.
9. Hultcrantz M, Rind D, Akl EA, Treweek S, Mustafa RA, Iorio A, et al. The GRADE Working Group clarifies the construct of certainty of evidence. J Clin Epidemiol. 2017;87:4-13.
10. Guyatt GH, Oxman AD, Kunz R, Vist GE, Falck-Ytter Y, Schunemann HJ. What is "quality of evidence" and why is it important to clinicians? BMJ. 2008;336:995-8.
11. Guyatt G, Oxman AD, Sultan S, Brozek J, Glasziou P, Alonso-Coello P, et al. GRADE guidelines 11-making an overall rating of confidence in effect estimates for a single outcome and for all outcomes. Journal of clinical epidemiology. 2012.
12. Schünemann HJ, Neumann I, Hultcrantz M, Brignardello-Petersen R, Zeng L, Murad MH, et al. GRADE Guidance article 35: Update on rating imprecision for assessing contextualized certainty of evidence and making decisions. Journal of clinical epidemiology. 2022.
13. Schunemann HJ. Interpreting GRADE's levels of certainty or quality of the evidence: GRADE for statisticians, considering review information size or less emphasis on imprecision? Journal of clinical epidemiology. 2016;75:6-15.
14. Schunemann HJ, Reinap M, Piggott T, Laidmae E, Kohler K, Pold M, et al. The ecosystem of health decision making: from fragmentation to synergy. Lancet Public Health. 2022;7:e378-e90.
15. Alonso-Coello P, Oxman AD, Moberg J, Brignardello-Petersen R, Akl EA, Davoli M, et al. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 2: Clinical practice guidelines. BMJ. 2016;353:i2089.
16. Schunemann HJ, Mustafa R, Brozek J, Santesso N, Alonso-Coello P, Guyatt G, et al. GRADE Guidelines: 16. GRADE evidence to decision frameworks for tests in clinical practice and public health. Journal of clinical epidemiology. 2016;76:89-98.
17. Andrews JC, Schunemann HJ, Oxman AD, Pottie K, Meerpohl JJ, Coello PA, et al. GRADE guidelines: 15. Going from evidence to recommendation-determinants of a recommendation's direction and strength. Journal of clinical epidemiology. 2013;66:726-35.
18. Andrews J, Guyatt G, Oxman AD, Alderson P, Dahm P, Falck-Ytter Y, et al. GRADE guidelines: 14. Going from evidence to recommendations: the significance and presentation of recommendations. Journal of clinical epidemiology. 2013;66:719-25.