Bivariate Random-effects Meta-analysis of Sensitivity and Specificity with SAS PROC GLIMMIX

 

 Buy Article Permissions and Reprints

Summary

Objectives: Meta-analysis allows to summarize pooled sensitivities and specificities from several primary diagnostic test accuracy studies. Often these pooled estimates are indirectly obtained from a hierarchical summary receiver operating characteristics (HSROC) analysis. This article presents a generalized linear random-effects model with the new SAS PROC GLIMMIX that obtains the pooled estimates for sensitivity and specificity directly.

Methods: Firstly, the formula of the bivariate random-effects model is presented in context with the literature. Then its implementation with the new SAS PROC GLIMMIX is empirically evaluated in comparison to the indirect HSROC approach, utilizing the published 2 x 2 count data of 50 meta-analyses.

Results: According to the empirical evaluation the meta-analytic results from the bivariate GLIMMIX approach are nearly identical to the results from the indirect HSROC approach.

Conclusions: A generalized linear mixed model with PROC GLIMMIX offers a straightforward method for bivariate random-effects meta-analysis of sensitivity and specificity.

• References

• 1 Armitage P, Berry G. Statistical methods in medical research. 3rd edition. Oxford: Blackwell Science; 1994
  Google Scholar
• 2 van Houwelingen HC. et al. Advanced methods in meta-analysis: multivariate approach and meta-regression. Stat Med 2002; 21: 589-624.
  CrossrefPubMedGoogle Scholar
• 3 Macaskill P. Empirical Bayes estimates generated in a hierarchical summary ROC analysis agreed closely with those of a full Bayesian analysis. J Clin Epidemiol 2004; 57: 925-932.
  CrossrefPubMedGoogle Scholar
• 4 Wang MC, Bushman BJ. Integrating results through meta-analytic review using SAS software. Cary, NC: SAS Institute Inc.; 1999
  Google Scholar
• 5 Reitsma JB. et al. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol 2005; 58: 982-990.
  CrossrefPubMedGoogle Scholar
• 6 Harbord RM, Deeks JJ, Egger M, Whiting P, Sterne JA. A unification of models for meta-analysis of diagnostic accuracy studies. Biostatistics 2007; 8: 239-251.
  CrossrefPubMedGoogle Scholar
• 7 Leeflang MM, Deeks JJ, Gatsonis C, Bossuyt PM. Systematic reviews of diagnostic test accuracy. Ann Intern Med 2008; 149: 889-897.
  CrossrefPubMedGoogle Scholar
• 8 van Houwelingen HC, Zwinderman KH, Stijnen T. A bivariate approach to meta-analysis. Stat Med 1993; 12: 2273-2284.
  CrossrefPubMedGoogle Scholar
• 9 Komaroff E, Wolfinger RD. Meta-analysis with linear and nonlinear multilevel models using Proc Mixed and Proc Nlmixed. SAS Conference Proceedings: PharmaSUG 2000. http://www.lexjansen.com/pharmasug/2000/stats/st09.pdf. Accessed September 3, 2008
  PubMedGoogle Scholar
• 10 Rutter CM, Gatsonis CA. A hierarchical regression approach to meta-analysis of diagnostic test accuracy evaluations. Stat Med 2001; 20: 2865-2884.
  CrossrefPubMedGoogle Scholar
• 11 Chu H, Cole SR. Bivariate meta-analysis of sensitivity and specificity with sparse data: a generalized linear mixed model approach. J Clin Epidemiol 2006; 59: 1331-1332.
  CrossrefPubMedGoogle Scholar
• 12 Hamon M, Champ-Rigot L, Morello R, Riddell JW, Hamon M. Diagnostic accuracy of in-stent coronary restenosis detection with multislice spiral computed tomography: a meta-analysis. Eur Radiol 2008; 18: 217-225.
  CrossrefPubMedGoogle Scholar
• 13 Kwee TC, Kwee RM. Combined FDG-PET/CT for the detection of unknown primary tumors: systematic review and meta-analysis. Eur Radiol 2009; 19: 731-744.
  CrossrefPubMedGoogle Scholar
• 14 Kuss O, Gromann C. An exact test for meta-analysis with binary endpoints. Methods Inf Med 2007; 46: 662-668.
  Article in Thieme ConnectPubMedGoogle Scholar
• 15 Littell RC, Milliken GA, Stroup WW, Wolfinger RD, Schabenberger O. SAS for mixed models. Second edition. Cary, NC: SAS Institute Inc; 2006
  Google Scholar
• 16 Jones A, Huddleston E. SAS/STAT 9.2 User’s Guide. 1st electronic book. Cary, NC: SAS Institute Inc.; 2008. http://support.sas.com. Accessed June 27, 2008
  Google Scholar
• 17 Schabenberger O. Introducing the Glimmix procedure for generalized linear mixed models. SUGI 30 Proceedings 2005, Paper 196-30. www2.sas.com/proceedings/sugi30/196-30.pdf. Accessed August 25, 2008
  PubMedGoogle Scholar
• 18 Scheidler J, Hricak H, Yu KK, Subak L, Segal MR. Radiological evaluation of lymph node metastases in patients with cervical cancer. A meta-analysis. JAMA 1997; 278: 1096-1101.
  CrossrefPubMedGoogle Scholar
• 19 Review Manager (RevMan) version 5.0.16.. Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration, 2008. http://www.ccims.net RevMan/RevMan5. Accessed October 1, 2008
  PubMedGoogle Scholar
• 20 van Zaane B, Zuithoff NP, Reitsma JB, Bax L, Nierich AP, Moons KG. Meta-analysis of the diagnostic accuracy of transesophageal echocardiography for assessment of atherosclerosis in the ascending aorta in patients undergoing cardiac surgery. Acta Anaesthesiol Scand 2008; 52: 1179-1187.
  CrossrefPubMedGoogle Scholar
• 21 Christou MA, Siontis GC, Katritsis DG, Ioannidis JP. Meta-analysis of fractional flow reserve versus quantitative coronary angiography and noninvasive imaging for evaluation of myocardial ischemia. Am J Cardiol 2007; 99: 450-456.
  CrossrefPubMedGoogle Scholar
• 22 Geleijnse ML, Krenning BJ, Soliman OI, Nemes A, Galema TW, ten Cate FJ. Dobutamine stress echo cardiography for the detection of coronary artery disease in women. Am J Cardiol 2007; 99: 714-717.
  CrossrefPubMedGoogle Scholar
• 23 Gisbert JP, Abraira V. Accuracy of Helicobacter pylori diagnostic tests in patients with bleeding pepticulcer: a systematic review and meta-analysis. Am J Gastroenterol 2006; 101: 848-863.
  CrossrefPubMedGoogle Scholar
• 24 Gisbert JP, de la Morena F, Abraira V. Accuracy of monoclonal stool antigen test for the diagnosis of H. pylori infection: a systematic review and meta-analysis. Am J Gastroenterol 2006; 101: 1921-1930.
  CrossrefPubMedGoogle Scholar
• 25 Rosman AS, Korsten MA. Meta-analysis comparing CT colonography, air contrast barium enema, and colonoscopy. Am J Med 2007; 120: 203-210.
  CrossrefPubMedGoogle Scholar
• 26 Stein PD, Yaekoub AY, Matta F, Sostman HD. 64-slice CT for diagnosis of coronary artery disease: a systematic review. Am J Med 2008; 121: 715-725.
  CrossrefPubMedGoogle Scholar
• 27 Alvarez S, Anorbe E, Alcorta P, Lopez F, Alonso I, Cortes J. Role of sonography in the diagnosis of axillary lymph node metastases in breast cancer: a systematic review. Am J Roentgenol 2006; 186: 1342-1348.
  CrossrefPubMedGoogle Scholar
• 28 Niemann T, Kollmann T, Bongartz G. Diagnostic performance of low-dose CT for the detection of urolithiasis: a meta-analysis. Am J Roentgenol 2008; 191: 396-401.
  CrossrefPubMedGoogle Scholar
• 29 Meijer AB, YL O, Geleijns J, Kroft LJ. Meta-analysis of 40- and 64-MDCT angiography for assessing coronary artery stenosis. Am J Roentgenol 2008; 191: 1667-1675.
  CrossrefPubMedGoogle Scholar
• 30 Jones AE, Fiechtl JF, Brown MD, Ballew JJ, Kline JA. Procalcitonin test in the diagnosis of bacteremia: a meta-analysis. Ann Emerg Med 2007; 50: 34-41.
  CrossrefPubMedGoogle Scholar
• 31 Shiga T, Wajima Z, Apfel CC, Inoue T, Ohe Y. Diagnostic accuracy of transesophageal echocardiography, helical computed tomography, and magnetic resonance imaging for suspected thoracic aortic dissection: systematic review and meta-analysis. Arch Intern Med 2006; 166: 1350-1356.
  CrossrefPubMedGoogle Scholar
• 32 Vanhoenacker PK, Decramer I, Bladt O, Sarno G, Bevernage C, Wijns W. Detection of non-ST-elevation myocardial infarction and unstable angina in the acute setting: meta-analysis of diagnostic performance of multi-detector computed tomo-graphic angiography. BMC Cardiovasc Disord 2007; 7: 39.
  CrossrefPubMedGoogle Scholar
• 33 Vanhoenacker PK, Decramer I, Bladt O, Sarno G, van Hul E, Wijns W, Dwamena BA. Multidetector computed tomography angiography for assessment of in-stent restenosis: meta-analysis of diagnostic performance. BMC Med Imaging 2008; 8: 14.
  CrossrefPubMedGoogle Scholar
• 34 Pewsner D, Juni P, Egger M, Battaglia M, Sundstrom J, Bachmann LM. Accuracy of electrocardiography in diagnosis of left ventricular hypertrophy in arterial hypertension: systematic review. BMJ 2007; 335: 711.
  CrossrefPubMedGoogle Scholar
• 35 van Vliet EP, Heijenbrok-Kal MH, Hunink MG, Kuipers EJ, Siersema PD. Staging investigations for oesophageal cancer: a meta-analysis. Br J Cancer 2008; 98: 547-557.
  CrossrefPubMedGoogle Scholar
• 36 Vestergaard ME, Macaskill P, Holt PE, Menzies SW. Dermoscopy compared with naked eye examination for the diagnosis of primary melanoma: a meta-analysis of studies performed in a clinical setting. Br J Dermatol 2008; 159: 669-676.
  CrossrefPubMedGoogle Scholar
• 37 Jiang J, Shi HZ, Liang QL, Qin SM, Qin XJ. Diagnostic value of interferon-gamma in tuberculous pleurisy: a metaanalysis. Chest 2007; 131: 1133-1141.
  CrossrefPubMedGoogle Scholar
• 38 Jing JY, Huang TC, Cui W, Xu F, Shen HH. Should FEV1/FEV6 replace FEV1/FVC ratio to detect airway obstruction? A metaanalysis. Chest 2009; 135: 991-998.
  CrossrefPubMedGoogle Scholar
• 39 Pfeiffer CD, Fine JP, Safdar N. Diagnosis of invasive aspergillosis using a galactomannan assay: a meta-analysis. Clin Infect Dis 2006; 42: 1417-1427.
  CrossrefPubMedGoogle Scholar
• 40 Leeflang MM, Debets-Ossenkopp YJ, Visser CE, Scholten RJ, Hooft L, Bijlmer HA, Reitsma JB, Bossuyt PM, Vandenbroucke-Grauls CM. Galactomannan detection for invasive aspergillosis in immuno-compromised patients. Cochrane Database Syst Rev 2008; 4: CD007394.
  PubMedGoogle Scholar
• 41 Purkayastha S, Tekkis PP, Athanasiou T, Tilney HS, Darzi AW, Heriot AG. Diagnostic precision of magnetic resonance imaging for preoperative prediction of the circumferential margin involvement in patients with rectal cancer. Colorectal Dis 2007; 9: 402-411.
  CrossrefPubMedGoogle Scholar
• 42 de Bondt RB, Nelemans PJ, Hofman PA, Casselman JW, Kremer B, van Engelshoven JM, Beets-Tan RG. Detection of lymph node metastases in head and neck cancer: a meta-analysis comparing US, USg FNAC CT and MR imaging. Eur J Radiol 2007; 64: 266-272.
  CrossrefPubMedGoogle Scholar
• 43 Lameris W, van Randen A, Bipat S, Bossuyt PM, Boermeester MA, Stoker J. Graded compression ultrasonography and computed tomography in acute colonic diverticulitis: meta-analysis of test accuracy. Eur Radiol 2008; 18: 2498-2511.
  CrossrefPubMedGoogle Scholar
• 44 Liu JL, Wyatt JC, Deeks JJ, Clamp S, Keen J, Verde P, Ohmann C, Wellwood J, Dawes M, Altman DG. Systematic reviews of clinical decision tools for acute abdominal pain. Health Technol Assess 2006; 10 (47) 1-167.
  PubMedGoogle Scholar
• 45 Martin JL, Williams KS, Abrams KR, Turner DA, Sutton AJ, Chapple C, Assassa RP, Shaw C, Cheater F. Systematic review and evaluation of methods of assessing urinary incontinence. Health Technol Assess 2006; 10 (06) 1-132.
  PubMedGoogle Scholar
• 46 Arbyn M, Sankaranarayanan R, Muwonge R, Keita N, Dolo A, Mbalawa CG, Nouhou H, Sakande B, Wesley R, Somanathan T, Sharma A, Shastri S, Basu P. Pooled analysis of the accuracy of five cervical cancer screening tests assessed in eleven studies in Africa and India. Int J Cancer 2008; 123: 153-160.
  CrossrefPubMedGoogle Scholar
• 47 Ewald B, Ewald D, Thakkinstian A, Attia J. Meta-analysis of B type natriuretic peptide and N-terminal pro B natriuretic peptide in the diagnosis of clinical heart failure and population screening for left ventricular systolic dysfunction. Intern Med J 2008; 38: 101-113.
  CrossrefPubMedGoogle Scholar
• 48 Virgili G, Menchini F, Dimastrogiovanni AF, Rapizzi E, Menchini U, Bandello F, Chiodini RG. Optical coherence tomography versus stereoscopic fundus photography or biomicroscopy for diagnosing diabetic macular edema: a systematic review. Invest Ophthalmol Vis Sci 2007; 48: 4963-4973.
  CrossrefPubMedGoogle Scholar
• 49 Hamon M. et al. Diagnostic performance of multi-slice spiral computed tomography of coronary arteries as compared with conventional invasive coronary angiography: a meta-analysis. J Am Coll Cardiol 2006; 48: 1896-1910.
  CrossrefPubMedGoogle Scholar
• 50 Nandalur KR. et al. Diagnostic performance of stress cardiac magnetic resonance imaging in the detection of coronary artery disease: a meta-analysis. J Am Coll Cardiol 2007; 50: 1343-1353.
  CrossrefPubMedGoogle Scholar
• 51 Morisson P, Neves DD. Evaluation of adenosine deaminase in the diagnosis of pleural tuberculosis: a Brazilian meta-analysis. J Bras Pneumol 2008; 34: 217-224.
  CrossrefPubMedGoogle Scholar
• 52 Gu P, Huang G, Chen Y, Zhu C, Yuan J, Sheng S. Diagnostic utility of pleural fluid carcinoembryonic antigen and CYFRA 21-1 in patients with pleural effusion: a systematic review and meta-analysis. J Clin Lab Anal 2007; 21: 398-405.
  CrossrefPubMedGoogle Scholar
• 53 Houssami N. et al. Accuracy and surgical impact of magnetic resonance imaging in breast cancer staging: systematic review and meta-analysis in detection of multifocal and multicentric cancer. J Clin Oncol 2008; 26: 3248-3258.
  CrossrefPubMedGoogle Scholar
• 54 Terasawa T. et al. Fluorine-18-Fluorodeoxyglucose Positron Emission Tomography for Interim Response Assessment of Advanced-Stage Hodgkin’s Lymphoma and Diffuse Large B-Cell Lymphoma: A Systematic Review. J Clin Oncol. 2009; 27 (11) 1906-1914.
  CrossrefPubMedGoogle Scholar
• 55 Morselli-Labate AM, Pezzilli R. Usefulness of serum IgG4 in the diagnosis and follow up of autoimmune pancreatitis: A systematic literature review and meta-analysis. J Gastroenterol Hepatol 2009; 24: 15-36.
  CrossrefPubMedGoogle Scholar
• 56 Sanders S, Barnett A, Correa-Velez I, Coulthard M, Doust J. Systematic review of the diagnostic accuracy of C-reactive protein to detect bacterial infection in nonhospitalized infants and children with fever. J Pediatr 2008; 153: 570-574.
  CrossrefPubMedGoogle Scholar
• 57 Met R. et al. Diagnostic performance of computed tomography angiography in peripheral arterial disease: a systematic review and meta-analysis. JAMA 2009; 301: 415-424.
  CrossrefPubMedGoogle Scholar
• 58 Wang P. et al. A meta-analysis of the accuracy of prostate cancer studies which use magnetic resonance spectroscopy as a diagnostic tool. Korean J Radiol 2008; 9: 432-438.
  CrossrefPubMedGoogle Scholar
• 59 Wardlaw JM. et al. Non-invasive imaging compared with intra-arterial angiography in the diagnosis of symptomatic carotid stenosis: a meta-analysis. Lancet 2006; 367: 1503-1512.
  CrossrefPubMedGoogle Scholar
• 60 Vos MJ. et al. Systematic review of the diagnostic accuracy of 201Tl single photon emission computed tomography in the detection of recurrent glioma. Nucl Med Commun 2007; 28: 431-439.
  CrossrefPubMedGoogle Scholar
• 61 Dong MJ. et al. Role of fluorodeoxyglucose-PET versus fluorodeoxyglucose-PET/computed tomography in detection of unknown primary tumor: a meta-analysis of the literature. Nucl Med Commun 2008; 29: 791-802.
  CrossrefPubMedGoogle Scholar
• 62 Doria AS. et al. US or CT for Diagnosis of Appendicitis in Children and Adults? A Meta-Analysis. Radiology 2006; 241: 83-94.
  CrossrefPubMedGoogle Scholar
• 63 Heijenbrok-Kal MH. et al. Lower extremity arterial disease: multidetector CT angiography meta-analysis. Radiology 2007; 245: 433-439.
  CrossrefPubMedGoogle Scholar
• 64 Hamon M. et al. Coronary arteries: diagnostic performance of 16- versus 64-section spiral CT compared with invasive coronary angiography – meta-analysis. Radiology 2007; 245: 720-731.
  CrossrefPubMedGoogle Scholar
• 65 Peters NH. et al. Meta-analysis of MR imaging in the diagnosis of breast lesions. Radiology 2008; 246: 116-124.
  CrossrefPubMedGoogle Scholar
• 66 Horsthuis K, Bipat S, Bennink RJ, Stoker J. Inflammatory bowel disease diagnosed with US, MR, scintigraphy, and CT: meta-analysis of prospective studies. Radiology 2008; 247: 64-79.
  CrossrefPubMedGoogle Scholar
• 67 Shi HZ. et al. Diagnostic value of carcinoembryonic antigen in malignant pleural effusion: a meta-analysis. Respirology 2008; 13: 518-527.
  CrossrefPubMedGoogle Scholar
• 68 Al-Khayal KA, Al-Omran MA. Computed tomography and ultrasonography in the diagnosis of equivocal acute appendicitis. A meta-analysis. Saudi Med J 2007; 28: 173-180.
  PubMedGoogle Scholar
• 69 Des Guetz G. et al. Is sentinel lymph node mapping in colorectal cancer a future prognostic factor? A meta-analysis. World J Surg 2007; 31: 1304-1312.
  CrossrefPubMedGoogle Scholar
• 70 Gerke O, Vach W, Hoilund-Carlsen PF. PET/CT in cancer: Methodological considerations for comparative diagnostic phase II studies with paired binary data. Methods Inf Med 2008; 47: 470-479.
  Article in Thieme ConnectPubMedGoogle Scholar
• 71 Reitsma JB, Zwinderman AH. Response to Chu and Cole: Bivariate meta-analysis of sensitivity and specificity with sparse data. J Clin Epidemiol 2006; 59: 1332-1333.
  CrossrefPubMedGoogle Scholar
• 72 Hamza TH, van Houwelingen HC, Stijnen T. . The binomial distribution of meta-analysis was preferred to model within-study variability. J Clin Epidemiol 2008; 61: 41-51.
  CrossrefPubMedGoogle Scholar
• 73 Cronin P. et al. Solitary pulmonary nodules and masses: a meta-analysis of the diagnostic utility of alternative imaging tests. Eur Radiol 2008; 18: 1840-1856.
  CrossrefPubMedGoogle Scholar
• 74 Higgins JP. et al. Measuring inconsistency in meta-analyses. BMJ 2003; 327: 557-560.
  CrossrefPubMedGoogle Scholar
• 75 Meta-DiSc version 1.4.. http://www.hrc.es investigacion/metadisc.html. Accessed June 2, 2008
  PubMed
• 76 Harbord RM. et al. An empirical comparison of methods for meta-analysis of diagnostic accuracy showed hierarchical models are necessary. J Clin Epidemiol 2008; 61: 1095-1103.
  CrossrefPubMedGoogle Scholar