Table 9.
Assessment of methodological quality within evaluations
| Author, year | Methodological quality assessment |
|---|---|
| BMJ economic guidelines, 1996 | |
| Herman, 2005122 | Evaluated economic evaluations on four criteria: randomization; prospective economic data collection; comparison group was usual care; and study was not blinded or mandatory regarding participation. Both studies in endorsing arm met all four criteria compared with 5/11 studies in non-endorsing arm |
| CONSORT extension for herbal interventions, 2006 | |
| Ernst, 2011127 | Assessed studies by using Cochrane risk of bias tool. Only study from endorsing journal was assessed as at moderate risk of bias. Studies from non-endorsing journals were assessed at high (n=2) or moderate (n=2) risk of bias |
| PRISMA, 2009 | |
| Tunis, 2013143 | Assessed reviews by using AMSTAR. Using data provided by author, studies (n=13) from only endorsing journal scored mean of 9.2 of 11 points, and studies (n=48) from non-endorsing journals scored 7.6 of 11 points |
| Panic, 2013145 | Assessed reviews by using AMSTAR. Data by item are not presented. Endorsing versus non-endorsing journals: using data provided by author, mean summed score from studies (n=30) from endorsing journals was 7.2 (range 2 to 9), and those (n=30) from non-endorsing journals scored 6.4 (range 1-9). After versus before journal endorsement: using data provided by author, mean summed score was 7.3 (range 3-9, n=27 articles) after journal endorsement and 6.0 (range 0-9, n=26 articles) before endorsement |
| Fleming, 2013144 | Authors assessed reviews by using AMSTAR tool but analyzed across all included studies162 |
| QUOROM, 1999 | |
| Biondi-Zoccai, 2006129 | Assessed studies by using the Oxman and Guyatt index (range of 1 (minimal flaws) to 7 (extensive flaws)). Only study from endorsing journal scored 2 on index; studies (n=6) from non-endorsing journals scored range of 1-6 points |
| Poolman, 2007130 | Used the Oxman and Guyatt index (maximum score 7 points). Only study from endorsing journal scored 7 points. Studies from non-endorsing journals (n=6) scored range of 1-6 points; four studies scoring 1 or 2 points are considered to have “major flaws” according to index |
| STARD, 2003 | |
| Freeman, 2009131 | Assessed eight aspects that authors state address internal and external validity of included studies: selective participant sampling; lack of reporting ethnicity and/or sensitization status of participants; lack of reporting number of replicates, if done, that were used for overall study outcome; lack of reporting failure rate; lack of including reported failure rate in analysis; difference in reported and adjusted accuracy; lack of controlling for presence of fetal DNA; and lack of known genotypes in study as control. Raw data provided in tabular form without summary in text. Studies (n=3) from endorsing journals ranged from 2 to 4 of 8 flaws. Studies (n=8) from non-endorsing journals ranged from 2 to 6 flaws, and information from one study was not interpretable |
| Krzych, 2009137 | Authors assessed studies by using QUADAS tool but analyzed across all included studies |