DISCUSSION
Our study found that CENETEC guideline production had a low score by the
AGREE II tool. Using this instrument, we identified the following
pitfalls in eligible guidelines i) Most of the health related questions
were not created using the explicit elements of the PICO format ii) The
target audience was excessively broad – aiming to reach nurses,
doctors and other healthcare professional, across the different levels
of health care – iii) The views and preferences of the target
population were not included iv) The systematic review methods were
inconsistent across the guidelines v) The CPGs mixed different grading
systems, vi) no formal external peer review was conducted vi) The
transition from the evidence to the recommendations was not transparent
and clear vii) and the recommendations were sometimes were unclear and
ambiguous viii)
We also found that guideline adherence to the GRADE framework was very
low. Only one of the approximately 30% of the guidelines that purported
to use GRADE included SoF tables, nor did they transparently address the
GRADE domains related to reasons rating the quality of the evidence down
or up.
The strengths of this study include the intensive training in the AGREE
II tool provided to the reviewers, the experience of the team in used of
the GRADE framework, and the relatively large number of CPGs included.
Our study is limited by the following: although we did provide intensive
training before the CPGs assessment, most of the reviewers were not
experts in guideline development, and so possibilities of incomplete
understanding remain. A standardized and valid tool to assess adherence
to the GRADE framework is unavailable. To address this issue, we created
an ad hoc instrument, the reliability and validity of which remains
untested.
In 2019 Cabrera et al. reviewed the guideline development process with
respect to the GRADE framework in guidelines developed in Latin America
and Caribbean countries.(17) The CPGs were
developed by Colombia (68%), Peru (13%), Chile (9%), Argentina, Costa
Rica, Brazil, Honduras and Dominican Republic. The authors focused their
analysis in detecting the methods for grading the quality of evidence
and topic prioritization without formally assessing the adherence to the
GRADE framework. The authors conducted a literature search on MEDLINE,
SciELO and Embase, also, they search within guideline repositories and
other governmental websites. They did not, however, include the
“Catalogo maestro”, and thus excluded all the CENETEC CPGs. Of the
1,370 CPGs the authors assessed, only 98 (8.9%) followed the GRADE
framework. In our findings we identify more CPGs using the GRADE
approach (29.1%), but found very poor adherence to GRADE, an issue
unaddressed in this previous work.
In general, our findings shown that despite the impressive and admirable
production of CPGs by CENETEC, they in general have a number of
methodological limitations.