Aspiring to Unintended Consequences of Natural Language Processing: A Review of Recent Developments in Clinical and Consumer-Generated Text Processing

 

 Permissions and Reprints

Summary

Objectives: This paper reviews work over the past two years in Natural Language Processing (NLP) applied to clinical and consumer-generated texts.

Methods: We included any application or methodological publication that leverages text to facilitate healthcare and address the health-related needs of consumers and populations.

Results: Many important developments in clinical text processing, both foundational and task-oriented, were addressed in community-wide evaluations and discussed in corresponding special issues that are referenced in this review. These focused issues and in-depth reviews of several other active research areas, such as pharmacovigilance and summarization, allowed us to discuss in greater depth disease modeling and predictive analytics using clinical texts, and text analysis in social media for healthcare quality assessment, trends towards online interventions based on rapid analysis of health-related posts, and consumer health question answering, among other issues.

Conclusions: Our analysis shows that although clinical NLP continues to advance towards practical applications and more NLP methods are used in large-scale live health information applications, more needs to be done to make NLP use in clinical applications a routine widespread reality. Progress in clinical NLP is mirrored by developments in social media text analysis: the research is moving from capturing trends to addressing individual health-related posts, thus showing potential to become a tool for precision medicine and a valuable addition to the standard healthcare quality evaluation tools.

^* Both authors contributed equally

• References

• 1 Friedman C, Alderson PO, Austin JH, Cimino JJ, Johnson SB. A general natural-language text processor for clinical radiology.. J Am Med Inform Assoc 1994; 1: 161-74.
  CrossrefPubMedGoogle Scholar
• 2 Maddox TM, Matheny MA. Natural Language Processing and the Promise of Big Data Small Step Forward, but Many Miles to Go.. Circ Cardiovasc Qual Outcomes 2015; 8: 463-5.
  CrossrefPubMedGoogle Scholar
• 3 Névéol A, Zweigenbaum P. Clinical Natural Language Processing in 2014: Foundational Methods Supporting Efficient Healthcare.. Yearb Med Inform 2015; 10: 194-8.
  Article in Thieme ConnectPubMedGoogle Scholar
• 4 Uzuner O, Bodnari A, Shen S, Forbush T, Pestian J, South BR. Evaluating the state of the art in coreference resolution for electronic medical records.. J Am Med Inf Assoc 2012; 19: 786-91.
  CrossrefPubMedGoogle Scholar
• 5 Uzuner Ö, Stubbs A. Practical applications for natural language processing in clinical research: The 2014 i2b2/UTHealth shared tasks.. J Biomed Inform 2015; Dec 58 Suppl: S1-5.
  CrossrefPubMedGoogle Scholar
• 6 Pradhan S, Elhadad N, South BR, Martinez D, Christensen L, Vogel A. et al. Evaluating the state of the art in disorder recognition and normalization of the clinical narrative.. J Am Med Inform Assoc 2015; 22: 143-54.
  CrossrefPubMedGoogle Scholar
• 7 Pradhan S, Elhadad N, Chapman W. et al. SemEval-2014 Task 7: Analysis of clinical text.. Proceedings of the International Workshop on Semantic Evaluation (SemEval); 2014 p. 54-62.
  PubMedGoogle Scholar
• 8 Elhadad N, Pradhan S, Lipsky Gorman S. et al. SemEval-2015 Task 14: Analysis of clinical text.. Proceedings of the International Workshop on Semantic Evaluation (SemEval); 2015 p. 303-10.
  PubMedGoogle Scholar
• 9 Text REtrieval Conference (TREC) Proceedings. http://trec.nist.gov/proceedings/proceedings.html.
  PubMed
• 10 Roberts K, Simpson M, Demner-Fushman D, Voorhees E, Hersh W. State-of-the-art in biomedical literature retrieval for clinical cases: a survey of the TREC 2014 CDS track.. Inf Retr J 2015; 19 1-2 113-48.
  PubMedGoogle Scholar
• 11 Tognola G, Fiszman M, Del Fiol G, Roberts A, Taboada M, Zweigenbaum P. Current perspectives on NLP for electronic medical records.. In: Medinfo 2015, São Paulo, Brasil: 2015
  Google Scholar
• 12 Neveol A, Dalianis H, Savova G, Zweigenbaum P. Clinical Natural Language Processing in Languages Other Than English.. AMIA Annu Symp Proc 2014 p. 183-5.
  PubMedGoogle Scholar
• 13 Safran C, Bloomrosen M, Hammond WE, Labkoff S, Markel-Fox S, Tang PC. et al. Toward a National Framework for the Secondary Use of Health Data: An American Medical Informatics Association White Paper.. J Am Med Inform Assoc 2007; 14: 1-9.
  CrossrefPubMedGoogle Scholar
• 14 Jensen PB, Jensen LJ, Brunak S. Mining electronic health records: towards better research applications and clinical care.. Nat Rev Genet 2012; 13: 395-405.
  CrossrefPubMedGoogle Scholar
• 15 Chen ES, Sarkar IN. Mining the electronic health record for disease knowledge.. Methods Mol Biol Clifton NJ 2014; 1159: 269-86.
  PubMedGoogle Scholar
• 16 Shivade C, Raghavan P, Fosler-Lussier E, Embi PJ, Elhadad N, Johnson SB. et al. A review of approaches to identifying patient phenotype cohorts using electronic health records.. J Am Med Inform Assoc 2014; 21: 221-30.
  CrossrefPubMedGoogle Scholar
• 17 Liao KP, Ananthakrishnan AN, Kumar V, Xia Z, Cagan A, Gainer VS et al.. Methods to Develop an Electronic Medical Record Phenotype Algorithm to Compare the Risk of Coronary Artery Disease across 3 Chronic Disease Cohorts.. PLoS ONE 2015; Aug 24 10 (Suppl. 08) e0136651.
  CrossrefPubMedGoogle Scholar
• 18 Castro V, Shen Y, Yu S, Finan S, Pau CT, Gainer V. et al. Identification of subjects with polycystic ovary syndrome using electronic health records.. Reprod Biol Endocrinol 2015; Oct 29 13: 116.
  CrossrefPubMedGoogle Scholar
• 19 Liao KP, Cai T, Savova GK, Murphy SN, Karlson EW, Ananthakrishnan AN. et al. Development of phenotype algorithms using electronic medical records and incorporating natural language processing.. BMJ 2015; Apr 24 350: h1885.
  CrossrefPubMedGoogle Scholar
• 20 Wei W-Q, Teixeira PL, Mo H, Cronin RM, Warner JL, Denny JC. Combining billing codes, clinical notes, and medications from electronic health records provides superior phenotyping performance.. J Am Med Inform Assoc 2016; Apr 23 (Suppl. 01) e20-7.
  CrossrefPubMedGoogle Scholar
• 21 Yu S, Liao KP, Shaw SY, Gainer VS, Churchill SE, Szolovits P. et al. Toward high-throughput phenotyping: unbiased automated feature extraction and selection from knowledge sources.. J Am Med Inform Assoc 2015; Sep 22 (Suppl. 05) 993-1000.
  CrossrefPubMedGoogle Scholar
• 22 Luo Y, Sohani AR, Hochberg EP, Szolovits P. Automatic lymphoma classification with sentence subgraph mining from pathology reports.. J Am Med Inform Assoc 2014; Sep-Oct 21 (Suppl. 05) 824-32.
  CrossrefPubMedGoogle Scholar
• 23 Halpern Y, Choi Y, Horng S, Sontag D. Using Anchors to Estimate Clinical State without Labeled Data.. AMIA Annu Symp Proc 2014; Nov 14 2014: 606-15.
  PubMedGoogle Scholar
• 24 Doshi-Velez F, Ge Y, Kohane I. Comorbidity clusters in autism spectrum disorders: an electronic health record time-series analysis.. Pediatrics 2014; 133: e54-63.
  CrossrefPubMedGoogle Scholar
• 25 Hidalgo CA, Blumm N, Barabási A-L, Christakis NA. A Dynamic Network Approach for the Study of Human Phenotypes.. PLoS Comput Biol 2009; Apr 5 (Suppl. 04) e1000353.
  CrossrefPubMedGoogle Scholar
• 26 Ho JC, Ghosh J, Sun J. Marble: High-throughput Phenotyping from Electronic Health Records via Sparse Nonnegative Tensor Factorization.. Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; 2014 p. 115-24.
  PubMedGoogle Scholar
• 27 Pivovarov R, Perotte AJ, Grave E, Angiolillo J, Wiggins CH, Elhadad N. Learning probabilistic phenotypes from heterogeneous EHR data.. J Biomed Inform 2015; Dec 58: 156-65.
  CrossrefPubMedGoogle Scholar
• 28 Weng C, Tu SW, Sim I, Richesson R. Formal representation of eligibility criteria: a literature review.. J Biomed Inform 2010; Jun 43 (Suppl. 03) 451-67.
  CrossrefPubMedGoogle Scholar
• 29 Pfiffner PB, Oh J, Miller TA, Mandl KD. ClinicalTrials.gov as a data source for semi-automated point-of-care trial eligibility screening.. PloS One 2014; Oct 21 9 (Suppl. 010) e111055.
  CrossrefPubMedGoogle Scholar
• 30 Petkov VI, Penberthy LT, Dahman BA, Poklepovic A, Gillam CW, McDermott JH. Automated determination of metastases in unstructured radiology reports for eligibility screening in oncology clinical trials.. Exp Biol Med Maywood NJ 2013; Dec 238 (Suppl. 12) 1370-8.
  PubMedGoogle Scholar
• 31 Kreuzthaler M, Schulz S, Berghold A. Secondary use of electronic health records for building cohort studies through top-down information extraction.. J Biomed Inform 2015; 53: 188-95.
  CrossrefPubMedGoogle Scholar
• 32 Ni Y, Kennebeck S, Dexheimer JW, McAneney CM, Tang H, Lingren T. et al. Automated clinical trial eligibility prescreening: increasing the efficiency of patient identification for clinical trials in the emergency department.. J Am Med Inform Assoc 2015; 22 (Suppl. 01) 166-78.
  CrossrefPubMedGoogle Scholar
• 33 Raghavan P, Fosler-Lussier E, Elhadad N, Lai AM. Cross-narrative Temporal Ordering of Medical Events.. Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics; 2014 p. 998-1008.
  PubMedGoogle Scholar
• 34 Li Q, Zhai H, Deleger L, Lingren T, Kaiser M, Stoutenborough L. et al. A sequence labeling approach to link medications and their attributes in clinical notes and clinical trial announcements for information extraction.. J Am Med Inform Assoc 2013; 20 (Suppl. 05) 915-21.
  CrossrefPubMedGoogle Scholar
• 35 Shivade C, Hebert C, Lopetegui M, de Marneffe MC, Fosler-Lussier E, Lai AM. Textual inference for eligibility criteria resolution in clinical trials.. J Biomed Inform 2015; Dec 58 Suppl: S211-8.
  CrossrefPubMedGoogle Scholar
• 36 Dagan I, Glickman O, Magnini B. The PASCAL Recognising Textual Entailment Challenge.. In: Quiñonero-Candela J, Dagan I, Magnini B. et al., editors. Machine Learning Challenges.. Evaluating Predictive Uncertainty, Visual Object Classification, and Recognising Tectual Entailment. Berlin Heidelberg: Springer; 2006. p. 177-90.
  Google Scholar
• 37 Miotto R, Weng C. Case-based reasoning using electronic health records efficiently identifies eligible patients for clinical trials.. J Am Med Inform Assoc 2015; 22: e141-50.
  CrossrefPubMedGoogle Scholar
• 38 Poulin C, Shiner B, Thompson P, Vepstas L, Young-Xu Y, Goertzel B. et al. Predicting the Risk of Suicide by Analyzing the Text of Clinical Notes.. PLoS ONE 2014; 9 (Suppl. 01) e85733.
  CrossrefPubMedGoogle Scholar
• 39 Goodwin T, Harabagiu SM. A Probabilistic Reasoning Method for Predicting the Progression of Clinical Findings from Electronic Medical Records.. AMIA Summits Transl Sci Proc 2015; 61-5.
  PubMedGoogle Scholar
• 40 Caruana R, Lou Y, Gehrke J, Koch P, Sturm M, Elhadad N. Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-day Readmission.. Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; 2015 p. 1721-30.
  PubMedGoogle Scholar
• 41 Ghassemi M, Naumann T, Doshi-Velez F, Brim-mer N, Joshi R, Rumshisky A. et al. Unfolding physiological state.. Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; 2014 p. 75-84.
  PubMedGoogle Scholar
• 42 Perotte A, Ranganath R, Hirsch JS, Blei D, Elhadad N. Risk prediction for chronic kidney disease progression using heterogeneous electronic health record data and time series analysis.. J Am Med Inform Assoc 2015; 22 (Suppl. 04) 872-80.
  CrossrefPubMedGoogle Scholar
• 43 Wang G, Jung K, Winnenburg R, Shah NH. A method for systematic discovery of adverse drug events from clinical notes.. J Am Med Inform Assoc 2015; 22 (Suppl. 06) 1196-204.
  CrossrefPubMedGoogle Scholar
• 44 Iyer SV, Harpaz R, LePendu P, Bauer-Mehren A, Shah NH. Mining clinical text for signals of adverse drug-drug interactions.. J Am Med Inform Assoc 2014; 21 (Suppl. 02) 353-62.
  CrossrefPubMedGoogle Scholar
• 45 Henriksson A, Kvist M, Dalianis H, Duneld M. Identifying adverse drug event information in clinical notes with distributional semantic representations of context.. J Biomed Inform 2015; 57: 333-49.
  CrossrefPubMedGoogle Scholar
• 46 Liu X, Chen H. A research framework for pharmacovigilance in health social media: Identification and evaluation of patient adverse drug event reports.. J Biomed Inform 2015; Dec 58: 268-79.
  CrossrefPubMedGoogle Scholar
• 47 Lardon J, Abdellaoui R, Bellet F, Asfari H, Souvignet J, Texier N. et al. Adverse Drug Reaction Identification and Extraction in Social Media: A Scoping Review.. J Med Internet Res 2015; Jul 10 17 (Suppl. 07) e171.
  CrossrefPubMedGoogle Scholar
• 48 Sarker A, Ginn R, Nikfarjam A, O’Connor K, Smith K, Jayaraman S. et al. Utilizing social media data for pharmacovigilance: A review.. J Biomed Inform 2015; Apr 54: 202-12.
  CrossrefPubMedGoogle Scholar
• 49 Abhyankar S, Demner-Fushman D, Callaghan FM, McDonald CJ. Combining structured and unstructured data to identify a cohort of ICU patients who received dialysis.. J Am Med Inform Assoc 2014; 21 Sep-Oct;21 (Suppl. 05) 801-7.
  CrossrefPubMedGoogle Scholar
• 50 Vijayakrishnan R, Steinhubl SR, Ng K, Sun J, Byrd RJ, Daar Z. et al. Prevalence of Heart Failure Signs and Symptoms in a Large Primary Care Population Identified Through the Use of Text and Data Mining of the Electronic Health Record.. J Card Fail Jul;20 7: 459-64.
  PubMedGoogle Scholar
• 51 Patterson OV, Forbush TB, Saini SD, Moser SE, DuVall SL. Classifying the Indication for Colonoscopy Procedures: A Comparison of NLP Approaches in a Diverse National Healthcare System.. Stud Health Technol Inform 2015; 216: 614-8.
  PubMedGoogle Scholar
• 52 Kerr GS, Richards JS, Nunziato CA, Patterson OV, DuVall SL, Aujero M. et al. Measuring Physician Adherence With Gout Quality Indicators: A Role for Natural Language Processing.. Arthritis Care Res 2015; 67 (Suppl. 02) 273-9.
  CrossrefPubMedGoogle Scholar
• 53 Carrell DS, Cronkite D, Palmer RE, Saunders K, Gross DE, Masters ET. et al. Using natural language processing to identify problem usage of prescription opioids.. Int J Med Inf 2015; 84 (Suppl. 12) 1057-64.
  CrossrefPubMedGoogle Scholar
• 54 Wang Y, Luo J, Hao S, Xu H, Shin AY, Jin B. et al. NLP based congestive heart failure case finding: A prospective analysis on statewide electronic medical records.. Int J Med Inf 2015; 84 (Suppl. 12) 1039-47.
  CrossrefPubMedGoogle Scholar
• 55 Sullivan R, Yao R, Jarrar R, Buchhalter J, Gonzalez G. Text Classification towards Detecting Misdiagnosis of an Epilepsy Syndrome in a Pediatric Population.. AMIA Annu Symp Proc 2014; 2014: 1082-7.
  PubMedGoogle Scholar
• 56 McCoy TH, Castro VM, Rosenfield HR, Cagan A, Kohane IS, Perlis RH. A Clinical Perspective on the Relevance of Research Domain Criteria in Electronic Health Records.. Am J Psychiatry 2015; 172 (Suppl. 04) 316-20.
  CrossrefPubMedGoogle Scholar
• 57 Perotte A, Pivovarov R, Natarajan K, Weiskopf N, Wood F, Elhadad N. Diagnosis code assignment: models and evaluation metrics.. J Am Med Inform Assoc 2014; 21 (Suppl. 02) 231-7.
  CrossrefPubMedGoogle Scholar
• 58 Subotin M, Davis A. A System for Predicting ICD-10-PCS Codes from Electronic Health Records.. Proceedings of the 2014 Workshop on Biomedical Natural Language Processing (BioNLP); 2015 p. 59-67.
  PubMedGoogle Scholar
• 59 Haas SW, Travers D, Waller A, Mahalingam D, Crouch J, Schwartz TA. et al. Emergency Medical Text Classifier: New system improves processing and classification of triage notes.. Online J Public Health Inform 2014; Oct 16 6 (Suppl. 02) e178.
  PubMedGoogle Scholar
• 60 López Pineda A, Ye Y, Visweswaran S, Cooper GF, Wagner MM, Tsui FR. Comparison of machine learning classifiers for influenza detection from emergency department free-text reports.. J Biomed Inform 2015; 58: 60-9.
  CrossrefPubMedGoogle Scholar
• 61 Yetisgen M, Klassen P, Tarczy-Hornoch P. Automating Data Abstraction in a Quality Improvement Platform for Surgical and Interventional Procedures.. EGEMS (Wash DC) 2014; 2 (Suppl. 01) 1114.
  PubMedGoogle Scholar
• 62 Raju GS, Lum PJ, Slack RS, Thirumurthi S, Lynch PM, Miller E. et al. Natural language processing as an alternative to manual reporting of colonoscopy quality metrics.. Gastrointest Endosc 2015; 82 (Suppl. 03) 512-9.
  CrossrefPubMedGoogle Scholar
• 63 Gawron AJ, Thompson WK, Keswani RN, Rasmussen LV, Kho AN. Anatomic and advanced adenoma detection rates as quality metrics determined via natural language processing.. Am J Gastroenterol 2014; 109 (Suppl. 12) 1844-9.
  CrossrefPubMedGoogle Scholar
• 64 Dentler K, Numans ME, ten Teije A, Cornet R, de Keizer NF. Formalization and computation of quality measures based on electronic medical records.. J Am Med Inform Assoc 2014; 21 (Suppl. 02) 285-91.
  CrossrefPubMedGoogle Scholar
• 65 Pai VM, Rodgers M, Conroy R, Luo J, Zhou R, Seto B. Workshop on using natural language processing applications for enhancing clinical decision making: an executive summary.. J Am Med Inform Assoc 2014; Feb 21 (Suppl. 01) e2-5.
  CrossrefPubMedGoogle Scholar
• 66 Dean NC, Jones BE, Ferraro JP, Vines CG, Haug PJ. Performance and utilization of an emergency department electronic screening tool for pneumonia.. JAMA Intern Med 2013; 173 (Suppl. 08) 699-701.
  CrossrefPubMedGoogle Scholar
• 67 Dean NC, Jones BE, Jones JP, Ferraro JP, Post HB, Aronsky D. et al. Impact of an Electronic Clinical Decision Support Tool for Emergency Department Patients With Pneumonia.. Ann Emerg Med 2015; 66 (Suppl. 05) 511-20.
  CrossrefPubMedGoogle Scholar
• 68 Demner-Fushman D, Seckman C, Fisher C, Thoma GR. Continual development of a personalized decision support system.. Stud Health Technol Inform 2013; 192: 175-9.
  PubMedGoogle Scholar
• 69 Devarakonda M, Tsou C-H. Automated Problem List Generation from Electronic Medical Records in IBM Watson.. Twenty-Seventh IAAI Conference; 2015. http://www.aaai.org/ocs/index.php/IAAI/IAAI15/paper/view/9516.
  PubMedGoogle Scholar
• 70 Tsou C-H, Devarakonda M, Liang JJ. Toward Generating Domain-Specific / Personalized Problem Lists from Electronic Medical Records.. 2015 AAAI Fall Symposium Series. http://www.aaai.org/ocs/index.php/FSS/FSS15/paper/view/11733.
  PubMedGoogle Scholar
• 71 Hirsch JS, Tanenbaum JS, Lipsky Gorman S, Liu C, Schmitz E, Hashorva D. et al. HARVEST, a longitudinal patient record summarizer.. J Am Med Inform Assoc 2015; 22 (Suppl. 02) 263-74.
  PubMedGoogle Scholar
• 72 Pivovarov R, Elhadad N. Automated methods for the summarization of electronic health records.. J Am Med Inform Assoc 2015; 22 (Suppl. 05) 938-47.
  CrossrefPubMedGoogle Scholar
• 73 Harrison C, Jorder M, Stern H, Stavinsky F, Reddy V, Hanson H. et al. Using Online Reviews by Restaurant Patrons to Identify Unreported Cases of Foodborne Illness — New York City, 2012–2013.. Morb Mortal Wkly Rep MMWR 2014; 63 (Suppl. 20) 441-5.
  PubMedGoogle Scholar
• 74 Smith CA. Consumer language, patient language, and thesauri: a review of the literature.. J Med Libr Assoc JMLA 2011; 99 (Suppl. 02) 135-44.
  PubMedGoogle Scholar
• 75 McCormack L, Craig Lefebvre R, Bann C, Taylor O, Rausch P. Consumer Understanding, Preferences, and Responses to Different Versions of Drug Safety Messages in the United States: A Randomized Controlled Trial.. Drug Saf 2016; 39 (Suppl. 02) 171-84.
  CrossrefPubMedGoogle Scholar
• 76 Polepalli Ramesh B, Houston T, Brandt C, Fang H, Yu H. Improving patients’ electronic health record comprehension with NoteAid.. Stud Health Technol Inform 2013; 192: 714-8.
  PubMedGoogle Scholar
• 77 Zhou X, Zheng A, Yin J, Chen R, Zhao X, Xu W. et al. Context-Sensitive Spelling Correction of Consumer-Generated Content on Health Care.. JMIR Med Inform 2015; 3 (Suppl. 03) e27.
  CrossrefPubMedGoogle Scholar
• 78 Kilicoglu H, Fiszman M, Roberts K, Demner-Fushman D. An Ensemble Method for Spelling Correction in Consumer Health Questions.. AMIA Annu Symp Proc 2015; 727-36.
  PubMedGoogle Scholar
• 79 Park A, Hartzler AL, Huh J, McDonald DW, Pratt W. Automatically Detecting Failures in Natural Language Processing Tools for Online Community Text.. J Med Internet Res 2015; 17 (Suppl. 08) e212.
  CrossrefPubMedGoogle Scholar
• 80 Liu Y, Xu S, Yoon H-J, Tourassi G. Extracting patient demographics and personal medical information from online health forums.. AMIA Annu Symp Proc 2014; Nov 14 2014: 1825-34.
  PubMedGoogle Scholar
• 81 Gupta S, MacLean DL, Heer J, Manning CD. Induced lexico-syntactic patterns improve information extraction from online medical forums.. J Am Med Inform Assoc 2014; Sep-Oct 21 (Suppl. 05) 902-9.
  CrossrefPubMedGoogle Scholar
• 82 Maramba ID, Davey A, Elliott MN, Roberts M, Roland M, Brown F. et al. (2015) Web-Based Textual Analysis of Free-Text Patient Experience Comments From a Survey in Primary Care.. JMIR Med Inform 2015; May 6 3 (Suppl. 02) e20.
  CrossrefPubMedGoogle Scholar
• 83 Yin Z, Fabbri D, Rosenbloom ST, Malin B. A Scalable Framework to Detect Personal Health Mentions on Twitter.. J Med Internet Res 2015; Jun 5 17 (Suppl. 06) e138.
  CrossrefPubMedGoogle Scholar
• 84 Doing-Harris KM, Zeng-Treitler Q. Computer-Assisted Update of a Consumer Health Vocabulary Through Mining of Social Network Data.. J Med Internet Res 2011; May 17 13 (Suppl. 02) e37.
  CrossrefPubMedGoogle Scholar
• 85 MacLean DL, Heer J. Identifying medical terms in patient-authored text: a crowdsourcing-based approach.. J Am Med Inform Assoc 2013; NovDec 20 (Suppl. 06) 1120-7.
  CrossrefPubMedGoogle Scholar
• 86 Elhadad N, Zhang S, Driscoll P, Brody S. Characterizing the Sublanguage of Online Breast Cancer Forums for Medications, Symptoms, and Emotions.. AMIA Annu Symp Proc 2014; Nov 14 2014: 516-25.
  PubMedGoogle Scholar
• 87 Vydiswaran VGV, Mei Q, Hanauer DA, Zheng K. Mining Consumer Health Vocabulary from Community-Generated Text.. AMIA Annu Symp Proc 2014; Nov 14 2014: 1150-9.
  PubMedGoogle Scholar
• 88 Seedor M, Peterson KJ, Nelsen LA, Cocos C, McCormick JB, Chute CG. et al. Incorporating Expert Terminology and Disease Risk Factors into Consumer Health Vocabularies.. Pac Symp Biocomput 2013; 421-32.
  PubMedGoogle Scholar
• 89 Lee H, McAuley JH, Hübscher M, Allen HG, Kamper SJ, Moseley GL. Tweeting back: predicting new cases of back pain with mass social media data.. J Am Med Inform Assoc 2016; May 23 (Suppl. 03) 644-8.
  CrossrefPubMedGoogle Scholar
• 90 Driscoll P, Lipsky Gorman S, Elhadad N. Learning Attribution Labels for Disorder Mentions in Online Health Forums.. Proceedings of the SIGIR Workshop on Health Search and Discovery; 2013 p. 3-6.
  PubMedGoogle Scholar
• 91 Hekler EB, Dubey G, McDonald DW, Poole ES, Li V, Eikey E. Exploring the Relationship Between Changes in Weight and Utterances in an Online Weight Loss Forum: A Content and Correlational Analysis Study.. J Med Internet Res 2014; Dec 8 16 (Suppl. 12) e254.
  CrossrefPubMedGoogle Scholar
• 92 Wang Y-C, Kraut RE, Levine JM. Eliciting and Receiving Online Support: Using Computer-Aided Content Analysis to Examine the Dynamics of Online Social Support.. J Med Internet Res 2015; Apr 20 17 (Suppl. 04) e99.
  CrossrefPubMedGoogle Scholar
• 93 Vlahovic TA, Wang Y, Kraut RE, Levine ML. Support Matching and Satisfaction in an Online Breast Cancer Support Community.. Proc 32nd Annu ACM Conf Hum Factors Comput Syst (CHI’2014). New York: ACM Press.; p. 1625-34.
  Google Scholar
• 94 Lewallen AC, Owen JE, Bantum EO, Stanton AL. How language affects peer responsiveness in an online cancer support group: implications for treatment design and facilitation.. Psychooncology 2014; Jul 23 (Suppl. 07) 766-72.
  CrossrefPubMedGoogle Scholar
• 95 Zhang S, Bantum E, Owen J, Elhadad N. Does sustained participation in an online health community affect sentiment?. AMIA Annu Symp Proc 2014; Nov 14 2014: 1970-9.
  PubMedGoogle Scholar
• 96 Zhao K, Yen J, Greer G, Qiu B, Mitra P, Portier K. Finding influential users of online health communities: a new metric based on sentiment influence.. J Am Med Inform Assoc 2014; Oct 21 (Suppl. 02) e212-8.
  CrossrefPubMedGoogle Scholar
• 97 Wallace BC, Paul MJ, Sarkar U, Trikalinos TA, Dredze M. A large-scale quantitative analysis of latent factors and sentiment in online doctor reviews.. J Am Med Inform Assoc 2014; NovDec 21 (Suppl. 06) 1098-103.
  CrossrefPubMedGoogle Scholar
• 98 Greaves F, Laverty AA, Cano DR, Moilanen K, Pulman S, Darzi A. et al. Tweets about hospital quality: a mixed methods study.. BMJ Qual Saf 2014; Oct 23 (Suppl. 10) 838-46.
  CrossrefPubMedGoogle Scholar
• 99 Hawkins JB, Brownstein JS, Tuli G, Runels T, Broecker K, Nsoesie EO. et al. (2015) Measuring patient-perceived quality of care in US hospitals using Twitter.. BMJ Qual Saf 2016; Jun 25 (Suppl. 06) 404-13.
  CrossrefPubMedGoogle Scholar
• 100 Jung Y, Hur C, Jung D, Kim M. Identifying Key Hospital Service Quality Factors in Online Health Communities.. J Med Internet Res 2015; Apr 7 17 (Suppl. 04) e90.
  CrossrefPubMedGoogle Scholar
• 101 Roberts K, Demner-Fushman D. Interactive use of online health resources: A comparison of consumer and professional questions.. J Am Med Inform Assoc 2016 May 4.
  PubMedGoogle Scholar
• 102 Cohen R, Elhadad M, Birk O. Analysis of free online physician advice services.. PLoS One 2013; 8 (Suppl. 03) e59963.
  CrossrefPubMedGoogle Scholar
• 103 Luo J, Zhang G-Q, Wentz S, Cui L, Xu R. SimQ: Real-Time Retrieval of Similar Consumer Health Questions.. J Med Internet Res 2015; Feb 17 17 (Suppl. 02) e43.
  CrossrefPubMedGoogle Scholar
• 104 Carroll RJ, Thompson WK, Eyler AE, Mandelin AM, Cai T, Zink RM. et al. Portability of an algorithm to identify rheumatoid arthritis in electronic health records.. J Am Med Inform Assoc 2012; Jun 19 (Suppl. 01) e162-9.
  CrossrefPubMedGoogle Scholar
• 105 Khalifa A, Meystre S. Adapting existing natural language processing resources for cardiovascular risk factors identification in clinical notes.. J Biomed Inform 2015; 58 Suppl: S128-32.
  CrossrefPubMedGoogle Scholar