Yandong Liu
ABSTRACT
Question answering communities such as Naver and Yahoo! Answers have emerged as popular, and often effective, means of information seeking on the web. By posting questions for other participants to answer, information seekers can obtain specific answers to their questions. Users of popular portals such as Yahoo! Answers already have submitted millions of questions and received hundreds of millions of answers from other participants. However, it may also take hours --and sometime days-- until a satisfactory answer is posted. In this paper we introduce the problem of predicting information seeker satisfaction in collaborative question answering communities, where we attempt to predict whether a question author will be satisfied with the answers submitted by the community participants. We present a general prediction model, and develop a variety of content, structure, and community-focused features for this task. Our experimental results, obtained from a largescale evaluation over thousands of real questions and user ratings, demonstrate the feasibility of modeling and predicting asker satisfaction. We complement our results with a thorough investigation of the interactions and information seeking patterns in question answering communities that correlate with information seeker satisfaction. Our models and predictions could be useful for a variety of applications such as user intent inference, answer ranking, interface design, and query suggestion and routing.
- E. Agichtein, E. Brill, S. Dumais, and R. Ragno. Learning user interaction models for predicting web search result preferences. In Proc. of SIGIR, 2006. Google Scholar
Digital Library
- E. Agichtein, C. Castillo, D. Donato, A. Gionis, and G. Mishne. Finding high-quality content in social media with an application to community-based question answering. In Proceedings of WSDM, 2008. Google ScholarDigital Library
- N. Belkin, R. N. Oddy, and H. M. Brooks. Information retrieval: Part ii. results of a design study. Journal of Documentation, 38(3):145--164, 1982.Google ScholarCross Ref
- N. J. Belkin. User modeling in information retrieval. Tutorial presented at the Sixth International Conference on User Modelling (UM97).Google Scholar
- E. Brill, S. Dumais, and M. Banko. An analysis of the askmsr question-answering system. In Proceedings of EMNLP, 2002. Google ScholarDigital Library
- E. Cutrell and Z. Guan. Eye tracking in MSN Search: Investigating snippet length, target position and task types, MSR-TR-2007.Google Scholar
- H. T. Dang, D. Kelly, and J. Lin. Overview of the TREC 2007 question answering track. In Proc.of TREC, 2007.Google Scholar
- D. Demner-Fushman and J. Lin. Answering clinical questions with knowledge-based and statistical techniques. Computational Linguistics, 33(1):63--103, 2007. Google ScholarDigital Library
- D. Downey, S. T. Dumais, and E. Horvitz. Models of searching and browsing: Languages, studies, and applications. In Proc. of IJCAI, 2007. Google ScholarDigital Library
- Y. Freund and R. Schapire. Experiments with a new boosting algorithm. In Proc. of the 13th international conference on machine learning (ICML1996), 1996.Google Scholar
- S. P. Harter and C. A. Hert. Evaluation of information retrieval systems: Approaches, issues, and methods.Google Scholar
- J. Jeon, W. Croft, and J. Lee. Finding similar questions in large question and answer archives. In Proceedings of CIKM, 2005. Google ScholarDigital Library
- J. Jeon, W. Croft, J. Lee, and S. Park. A framework to predict the quality of answers with non-textual features. In Proceedings of SIGIR, 2006. Google ScholarDigital Library
- T. Joachims, L. Granka, B. Pan, H. Hembrooke, F. Radlinski, and G. Gay. Evaluating the accuracy of implicit feedback from clicks and query reformulations in web search. ACM Trans. Inf. Syst., 25(2), 2007. Google ScholarDigital Library
- M. Kobayashi and K. Takeda. Information retrieval on the web. ACM Computing Surveys, 32(2), 2000. Google ScholarDigital Library
- J. Lin and D. Demner-Fushman. Methods for automatically evaluating answers to complex questions. Information Retrieval, 9(5):565--587, 2006. Google ScholarDigital Library
- J. Lin and P. Zhang. Deconstructing nuggets: the stability and reliability of complex question answering evaluation. In Proceedings of SIGIR, pages 327--334, 2007. Google ScholarDigital Library
- J. C. Platt. Fast training of support vector machines using sequential minimal optimization. Advances in Kernal Methods -- Support Vector Learning, pages 185--208, 1998. Google ScholarDigital Library
- J. Quinlan. Improved use of continuous attributes in c4.5. In Journal of Artificial Intelligence Research, 1996. Google ScholarDigital Library
- D. E. Rose and D. Levinson. Understanding user goals in web search. In Proceedings of WWW, 2004. Google ScholarDigital Library
- I. Ruthven, L. A. Glasgow, M. Baillie, R. Bierig, E. Nicol, S. Sweeney, and M. Yakici. Intra-assessor consistency in question answering. In Proceedings of SIGIR, pages 727--728, 2007. Google ScholarDigital Library
- R. Soricut and E. Brill. Automatic question answering: Beyond the factoid. In HLT-NAACL, 2004.Google Scholar
- Q. Su, D. Pavlov, J. Chow, and W. Baker. Internet-scale collection of human-reviewed data. In Proc. of the 16th international conference on World Wide Web (WWW), 2007. Google ScholarDigital Library
- E. M. Voorhees. The philosophy of information retrieval evaluation. In Proceedings of (CLEF), 2001. Google ScholarDigital Library
- E. M. Voorhees. Overview of the TREC 2003 question answering track. In Text REtrieval Conference, 2003.Google Scholar
- R. White, M. Bilenko, and S. Cucerzan. Studying the use of popular destinations to enhance web search interaction. In Proc. of SIGIR, 2007. Google ScholarDigital Library
- R. W. White and S. M. Drucker. Investigating behavioral variability in web search. In Proc. of WWW, 2007. Google ScholarDigital Library
- I. Witten and E. Frank. Data Mining: Practical machine learning tools and techniques. Morgan Kaufman, 2nd edition, 2005. Google ScholarDigital Library
- J. Zobel. How reliable are the results of large-scale information retrieval experiments? In Proceedings of SIGIR, pages 307--314, 1998. Google ScholarDigital Library
Index Terms
- Predicting information seeker satisfaction in community question answering
Recommendations
Evaluating and predicting answer quality in community QA
SIGIR '10: Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrievalQuestion answering (QA) helps one go beyond traditional keywords-based querying and retrieve information in more precise form than given by a document or a list of documents. Several community-based QA (CQA) services have emerged allowing information ...
Analyzing and predicting question quality in community question answering services
WWW '12 Companion: Proceedings of the 21st International Conference on World Wide WebUsers tend to ask and answer questions in community question answering (CQA) services to seek information and share knowledge. A corollary is that myriad of questions and answers appear in CQA service. Accordingly, volumes of studies have been taken to ...
Modeling information-seeker satisfaction in community question answering
Question Answering Communities such as Naver, Baidu Knows, and Yahoo! Answers have emerged as popular, and often effective, means of information seeking on the web. By posting questions for other participants to answer, information seekers can obtain ...
Comments