Censoring Weighted Separate-and-Conquer Rule Induction from Survival Data

 

 Buy Article Permissions and Reprints

Summary

Objectives: Rule induction is one of the major methods of machine learning. Rule-based models can be easily read and interpreted by humans, that makes them particularly useful in survival studies as they can help clinicians to better understand analysed data and make informed decisions about patient treatment. Although of such usefulness, there is still a little research on rule learning in survival analysis. In this paper we take a step towards rule-based analysis of survival data.

Methods: We investigate so-called covering or separate-and-conquer method of rule induction in combination with a weighting scheme for handling censored observations. We also focus on rule quality measures being one of the key elements differentiating particular implementations of separate-and-conquer rule induction algorithms. We examine 15 rule quality measures guiding rule induction process and reflecting a wide range of different rule learning heuristics.

Results: The algorithm is extensively tested on a collection of 20 real survival datasets and compared with the state-of-the-art survival trees and random survival forests algorithms. Most of the rule quality measures outperform Kaplan-Meier estimate and perform at least equally well as tree-based algorithms.

Conclusions: Separate-and-conquer rule induction in combination with weighting scheme is an effective technique for building rule-based models of survival data which, according to predictive accuracy, are competitive with tree-based representations.

• References

• 1 Cox DR. Regression models and life-tables. Journal of the Royal Statistical Society Series B (Methodological) 1972; 34 (02) 187-220.
  PubMedGoogle Scholar
• 2 Fürnkranz J, Gamberger D, Lavrac N. Foundations of Rule Learning. Springer-Verlag 2012
  PubMedGoogle Scholar
• 3 Holmes G, Hall M, Frank E. Generating Rule Sets from Model Trees. In: Twelfth Australian Joint Conference on Artificial Intelligence. Springer 1999; pp 1-12.
  PubMedGoogle Scholar
• 4 Janssen F, Fürnkranz J. Heuristic Rule-Based Regression via Dynamic Reduction to Classification. In Walsh T. editor. Proceedings of the 22nd International Joint Conference on Artificial Intelligence (IJCAI-11) 2011; pp 1330-1335.
  PubMed
• 5 Sikora M, Skowron A, Wróbel Ł. Rule Quality Measure-Based Induction of Unordered Sets of Regression Rules. In Ramsay A, Agre G. editors. Artificial Intelligence: Methodology, Systems, and Applications. Vol. 7557 of Lecture Notes in Computer Science.. Berlin /Heidelberg: Springer; 2012. pp 162-171.
  Google Scholar
• 6 Michalski RS. Discovering classification rules using variable-valued logic system VL. In: Proceedings of the 3rd international joint conference on Artificial intelligence. Morgan Kaufmann Publishers Inc. 1973; pp 162-172.
  PubMedGoogle Scholar
• 7 Fürnkranz J. Separate-and-Conquer Rule Learning. Artificial Intelligence Review 1999; 13 (01) 3-54.
  CrossrefPubMedGoogle Scholar
• 8 Pattaraintakorn P, Cercone N. A foundation of rough sets theoretical and computational hybrid intelligent system for survival analysis. Computers & Mathematics with Applications 2008; 56 (07) 1699-1708.
  CrossrefPubMedGoogle Scholar
• 9 Bazan JG, Osmólski A, Skowron A. Ślęzak D, Szczuka MS, Wróblewski J. Rough Set Approach to the Survival Analysis. In: Alpigini JJ, Peters JF, Skowronek J, Zhong N. editors. Rough Sets and Current Trends in Computing. Vol. 2475 of Lecture Notes in Computer Science.. Springer: 2002. pp 522-529.
  Google Scholar
• 10 Sikora M, Wróbel Ł, Mielcarek M. Kawłak K. Application of rule induction to discover survival factors of patients after bone marrow transplantation. Journal of Medical Informatics & Technologies 2013; 22: 35-53.
  PubMedGoogle Scholar
• 11 Kronek LP, Reddy A. Logical analysis of survival data: prognostic survival models by detecting high-degree interactions in right-censored data. Bioinformatics 2008; 24 (016) i248-i253.
  CrossrefPubMedGoogle Scholar
• 12 Chikalov I, Lozin V, Lozina I, Moshkov M, Nguyen H, Skowron A. et al Logical Analysis of Data: Theory, Methodology and Applications. In Three Approaches to Data Analysis. Vol. 41 of Intelligent Systems Reference Library.. Berlin/Heidelberg; Springer: 2013. pp 147-192.
  Google Scholar
• 13 Crama Y, Hammer PL, Ibaraki T. Cause-effect relationships and partially defined Boolean functions. Annals of Operations Research 1988; 16 (01) 299-325.
  CrossrefPubMedGoogle Scholar
• 14 Liu X, Minin V, Huang Y, Seligson DB, Horvath S. Statistical methods for analyzing tissue microarray data. Journal of Biopharmaceutical Statistics 2004; 14 (03) 671-685.
  CrossrefPubMedGoogle Scholar
• 15 LeBlanc M, Crowley J. Relative risk trees for censored survival data. Biometrics 1992; 48 (02) 411-425.
  CrossrefPubMedGoogle Scholar
• 16 Therneau TM, Grambsch PM, Fleming TR. Martingale-based residuals for survival models. Biometrika 1990; 77 (01) 147-160.
  CrossrefPubMedGoogle Scholar
• 17 Breiman L, Friedman JH, Olshen RA, Stone CJ. Classification and Regression Trees. Wadsworth 1984
  PubMedGoogle Scholar
• 18 Bou-Hamad I, Larocque D, Ben-Ameur H. A review of survival trees. Statistics Surveys 2011; 5: 44-71.
  CrossrefPubMedGoogle Scholar
• 19 Radespiel-Troger M, Gefeller O, Rabenstein T, Hothorn T. Association between split selection instability and predictive error in survival trees. Methods Inf Med 2006; 45 (05) 548-556.
  Article in Thieme ConnectPubMedGoogle Scholar
• 20 Ciampi A, Thiffault J, Nakache JP, Asselain B. Stratification by stepwise regression, correspondence analysis and recursive partition: a comparison of three methods of analysis for survival data with covariates. Computational statistics & data analysis 1986; 4 (03) 185-204.
  CrossrefPubMedGoogle Scholar
• 21 LeBlanc M, Crowley J. Survival trees by goodness of split. Journal of the American Statistical Association 1993; 88 (422) 457-467.
  CrossrefPubMedGoogle Scholar
• 22 Segal MR. Regression trees for censored data. Biometrics 1988; 44 (01) 35-47.
  CrossrefPubMedGoogle Scholar
• 23 Wróbel Ł. Tree-based induction of decision list from survival data. Journal of Medical Informatics & Technologies 2012; 20: 73-78.
  PubMedGoogle Scholar
• 24 Frank E, Witten IH. Generating Accurate Rule Sets without Global Optimization. In: Proceedings of the Fifteenth International Conference on Machine Learning. Morgan Kaufmann Publishers Inc. 1998; pp 144-151.
  PubMedGoogle Scholar
• 25 An A, Cercone N. Rule Quality Measures for Rule Induction Systems: Description and Evaluation. Computational Intelligence 2001; 17 (03) 409-424.
  CrossrefPubMedGoogle Scholar
• 26 Bruha I, Tkadlec J. Rule quality for multiple-rule classifier: Empirical expertise and theoretical methodology. Intelligent Data Analysis 2003; 7 (02) 99-124.
  CrossrefPubMedGoogle Scholar
• 27 Janssen F, Fürnkranz J. On the quest for optimal rule learning heuristics. Machine Learning 2010; 78 (03) 343-379.
  CrossrefPubMedGoogle Scholar
• 28 Sikora M. Rule Quality Measures in Creation and Reduction of Data Rule Models. In. Greco S, Hata Y, Hirano S, Inuiguchi M, Miyamoto S, Nguyen H. et al. editors. Rough Sets and Current Trends in Computing. Vol. 4259 of Lecture Notes in Computer Science.. Berlin/Heidelberg; Springer: 2006. pp 716-725.
  Google Scholar
• 29 Sikora M, Wróbel Ł. Data-driven adaptive selection of rule quality measures for improving rule induction and filtration algorithms. International Journal of General Systems 2013; 42 (06) 594-613.
  CrossrefPubMedGoogle Scholar
• 30 Yao Y, Zhong N. An Analysis of Quantitative Measures Associated with Rules. In. Zhong N, Zhou L. editors. PAKDD. Vol. 1574 of Lecture Notes in Computer Science. Springer 1999; pp 479-488.
  PubMedGoogle Scholar
• 31 Geng L, Hamilton HJ. Interestingness measures for data mining: A survey. ACM Computing Surveys 2006; 38 (03) 9.
  CrossrefPubMedGoogle Scholar
• 32 Ohsaki M, Abe H, Tsumoto S, Yokoi H, Yamaguchi T. Evaluation of rule interestingness measures in medical knowledge discovery in databases. Artificial Intelligence in Medicine 2007; 41 (03) 177-196.
  CrossrefPubMedGoogle Scholar
• 33 Kaplan EL, Meier P. Nonparametric Estimation from Incomplete Observations. Journal of the American Statistical Association 1958; 53 (282) 457-481.
  CrossrefPubMedGoogle Scholar
• 34 Parkin DM, Bray F, Ferlay J, Pisani P. Estimating the world cancer burden: Globocan 2000. International Journal of Cancer 2001; 94 (02) 153-156.
  CrossrefPubMedGoogle Scholar
• 35 Zupan B, Demšar J, Kattan MW, Beck JR, Bratko I. Machine learning for survival analysis: a case study on recurrence of prostate cancer. Artificial Intelligence in Medicine 2000; 20 (01) 59-75.
  CrossrefPubMedGoogle Scholar
• 36 Štajduhar I. Dalbelo-Bašic B. Learning Bayesian networks from survival data using weighting censored instances. Journal of Biomedical Informatics 2010; 43 (04) 613-622.
  CrossrefPubMedGoogle Scholar
• 37 Fürnkranz J, Widmer G. Incremental reduced error pruning. In: International Conference on Machine Learning 1994; pp 70-77.
  PubMedGoogle Scholar
• 38 Little RJ, Rubin DB. Statistical analysis with missing data. Vol. 539. New York: Wiley; 1987
  Google Scholar
• 39 Wohlrab L, Fürnkranz J. A review and comparison of strategies for handling missing values in separate-and-conquer rule learning. Journal of Intelligent Information Systems 2011; 36 (01) 73-98.
  CrossrefPubMedGoogle Scholar
• 40 R Core eam R: A Language and Environment for Statistical Computing. Vienna, Austria 2013 ISBN 3-900051-07-0
  PubMedGoogle Scholar
• 41 Hosmer DW, Lemeshow S, May S. Applied Survival Analysis: Regression Modeling of Time to Event Data. Wiley-Interscience 2008
  PubMedGoogle Scholar
• 42 Lange N, Ryan L, Billard L, Brillinger D, Conquest L, Greenhouse J. editors. Case studies in biometry. Wiley series in probability and mathematical statistics: Applied probability and statistics. Wiley 1994
  PubMedGoogle Scholar
• 43 Therneau T, Atkinson B, Ripley B. rpart: Recursive Partitioning; 2013. R package version 4. 1-1.
  PubMedGoogle Scholar
• 44 Hothorn T, Hornik K, Zeileis A. Unbiased recursive partitioning: A conditional inference framework. Journal of Computational and Graphical Statistics 2006; 15 (03) 651-674.
  CrossrefPubMedGoogle Scholar
• 45 Breiman L. Random Forests. Machine Learning 2001; 45 (01) 5-32.
  CrossrefPubMedGoogle Scholar
• 46 Ishwaran H, Kogalur UB, Blackstone EH, Lauer MS. Random survival forests. Ann Appl Statist 2008; 2 (03) 841-860.
  CrossrefPubMedGoogle Scholar
• 47 Hothorn T, Buehlmann P, Dudoit S, Molinaro A, Laan MVD. Survival Ensembles. Biostatistics 2006; 7 (03) 355-373.
  PubMedGoogle Scholar
• 48 Graf E, Schmoor C, Sauerbrei W, Schumacher M. Assessment and comparison of prognostic classification schemes for survival data. Statistics in medicine 1999; 18 (017) (018) 2529-2545.
  CrossrefPubMedGoogle Scholar
• 49 Schumacher M, Graf E, Gerds T. et al How to assess prognostic models for survival data: a case study in oncology. Methods Inf Med 2003; 42 (05) 564-571.
  Article in Thieme ConnectPubMedGoogle Scholar
• 50 Demšar J. Statistical comparisons of classifiers over multiple data sets. The Journal of Machine Learning Research 2006; 7: 1-30.
  PubMedGoogle Scholar
• 51 García S, Herrera F. An Extension on “Statistical Comparisons of Classifiers over Multiple Data Sets” for all Pairwise Comparisons. Journal of Machine Learning Research 2008; 9: 2677-2694.
  PubMedGoogle Scholar
• 52 Schumacher M, Bastert G, Bojar H, Hübner K, Olschewski M, Sauerbrei W. et al Randomized 2 x 2 trial evaluating hormonal treatment and the duration of chemotherapy in node-positive breast cancer patients. German Breast Cancer Study Group. Journal of clinical oncology: official journal of the American Society of Clinical Oncology 1994: 12-10 2086
  PubMedGoogle Scholar