DISCUSSION
Our study found that CENETEC guideline production had a low score by the AGREE II tool. Using this instrument, we identified the following pitfalls in eligible guidelines i) Most of the health related questions were not created using the explicit elements of the PICO format ii) The target audience was excessively broad – aiming to reach nurses, doctors and other healthcare professional, across the different levels of health care – iii) The views and preferences of the target population were not included iv) The systematic review methods were inconsistent across the guidelines v) The CPGs mixed different grading systems, vi) no formal external peer review was conducted vi) The transition from the evidence to the recommendations was not transparent and clear vii) and the recommendations were sometimes were unclear and ambiguous viii)
We also found that guideline adherence to the GRADE framework was very low. Only one of the approximately 30% of the guidelines that purported to use GRADE included SoF tables, nor did they transparently address the GRADE domains related to reasons rating the quality of the evidence down or up.
The strengths of this study include the intensive training in the AGREE II tool provided to the reviewers, the experience of the team in used of the GRADE framework, and the relatively large number of CPGs included. Our study is limited by the following: although we did provide intensive training before the CPGs assessment, most of the reviewers were not experts in guideline development, and so possibilities of incomplete understanding remain. A standardized and valid tool to assess adherence to the GRADE framework is unavailable. To address this issue, we created an ad hoc instrument, the reliability and validity of which remains untested.
In 2019 Cabrera et al. reviewed the guideline development process with respect to the GRADE framework in guidelines developed in Latin America and Caribbean countries.(17) The CPGs were developed by Colombia (68%), Peru (13%), Chile (9%), Argentina, Costa Rica, Brazil, Honduras and Dominican Republic. The authors focused their analysis in detecting the methods for grading the quality of evidence and topic prioritization without formally assessing the adherence to the GRADE framework. The authors conducted a literature search on MEDLINE, SciELO and Embase, also, they search within guideline repositories and other governmental websites. They did not, however, include the “Catalogo maestro”, and thus excluded all the CENETEC CPGs. Of the 1,370 CPGs the authors assessed, only 98 (8.9%) followed the GRADE framework. In our findings we identify more CPGs using the GRADE approach (29.1%), but found very poor adherence to GRADE, an issue unaddressed in this previous work.
In general, our findings shown that despite the impressive and admirable production of CPGs by CENETEC, they in general have a number of methodological limitations.