[CHI] How Do Humans Assess the Credibility of Weblogs:Qualifying and Verifying Human Factors withMachine Learning

Abstract

The purpose of this paper is to understand the factors involved when a human judges the credibility of information and to develop a classification model for weblogs, a primary source of information for many people. Considering both computational and human-centered approaches, we conducted a user study designed to consider two cognitive procedures--(1) visceral, behavioral and (2) reflective assessments--in the evaluation of information credibility. The results of the 80-participant study highlight that human cognitive processing varies according to an individual's purpose and that humans consider the structures and styles of content in their reflective assessments. We experimentally proved these findings through the development and analysis of classification models using 16,304 real blog posts written by 2,944 bloggers. Our models yield greater accuracy and efficiency than the models with well-known best features identified in prior research.

Researcher

• Researcher
  • 조용걸 [아주대학교 데이터사이언스학과 / [email protected]]
• Research Assistant
  • 김민우 [아주대학교 미디어학과 / [email protected]]
• Adviser
  • 한경식 [아주대학교 데이터사이언스학과 / [email protected]]

Research Questions

• RQ: What is a reader's cognitive processing of information credibility in blogs and how does the understanding of such processing influence a computational analysis of information credibility in blogs?

Data Set

• Naver
• Dbdbdeep

참고 논문

• [1] Michael A Banks. 2008. Blogging heroes: interviews with 30 of the world’s top bloggers. John Wiley & Sons.
• [2] Mikhail Bautin, Lohit Vijayarenu, and Steven Skiena. 2008. Interna- tional Sentiment Analysis for News and Blogs.. In ICWSM.
• [3] Fabricio Benevenuto, Gabriel Magno, Tiago Rodrigues, and Virgilio Almeida. 2010. Detecting spammers on twitter. In Collaboration, elec- tronic messaging, anti-abuse and spam conference (CEAS), Vol. 6. 12.
• [4] David Boud, Rosemary Keogh, and David Walker. 2013. Reflection: Turning experience into learning. Routledge.
• [5] Evelyn M Boyd and Ann W Fales. 1983. Reflective learning: Key to learning from experience. Journal of humanistic psychology 23, 2 (1983), 99–117.
• [6] Judee K Burgoon and Jerold L Hale. 1984. The fundamental topoi of relational communication. Communication Monographs 51, 3 (1984), 193–214.
• [7] Brady Butterfield and Janet Metcalfe. 2006. The correction of errors committed with high confidence. Metacognition and Learning 1, 1 (2006), 69–84.
• [8] Carlos Castillo, Marcelo Mendoza, and Barbara Poblete. 2011. Infor- mation credibility on twitter. In Proceedings of the 20th international conference on World wide web. ACM, 675–684.
• [9] Thomas Chesney and Daniel KS Su. 2010. The impact of anonymity on weblog credibility. International journal of human-computer studies 68, 10 (2010), 710–718.
• [10] Dietrich Dörner and Harald Schaub. 1994. Errors in planning and decision-making and the nature of human information processing. Applied psychology 43, 4 (1994), 433–453.
• [11] Jonathan St BT Evans. 2008. Dual-processing accounts of reasoning, judgment, and social cognition. Annu. Rev. Psychol. 59 (2008), 255–278.
• [12] Jonathan St BT Evans and Keith E Stanovich. 2013. Dual-process theories of higher cognition: Advancing the debate. Perspectives on psychological science 8, 3 (2013), 223–241.
• [13] Emilio Ferrara, Onur Varol, Clayton Davis, Filippo Menczer, and Alessandro Flammini. 2016. The rise of social bots. Commun. ACM 59,7 (2016), 96–104.
• [14] Andrew J Flanagin and Miriam J Metzger. 2003. The perceived credi-bility of personal Web page information as influenced by the sex of the source. Computers in human behavior 19, 6 (2003), 683–701.
• [15] BJ Fogg and Hsiang Tseng. 1999. The elements of computer credibility. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems. ACM, 80–87.
• [16] Brian J Fogg. 2003. Prominence-interpretation theory: Explaining how people assess credibility online. In CHI’03 extended abstracts on human factors in computing systems. ACM, 722–723.
• [17] Brian J Fogg, Cathy Soohoo, David R Danielson, Leslie Marable, Ju-lianne Stanford, and Ellen R Tauber. 2003. How do users evaluate the credibility of Web sites?: a study with over 2,500 participants. In Proceedings of the 2003 conference on Designing for user experiences. ACM, 1–15.
• [18] Namrata Godbole, Manja Srinivasaiah, and Steven Skiena. 2007. Large- Scale Sentiment Analysis for News and Blogs. Icwsm 7, 21 (2007), 219–222.
• [19] KyungsikHan.2018.Howdoyouperceivethisauthor?Understanding and modeling authors’ communication quality in social media. PloS one 13, 2 (2018), e0192061.
• [20] Thomas J Johnson and Barbara K Kaye. 2009. In blog we trust? Deci- phering credibility of components of the internet among politically interested internet users. Computers in Human Behavior 25, 1 (2009), 175–182.
• [21] Andreas Juffinger, Michael Granitzer, and Elisabeth Lex. 2009. Blog credibility ranking by exploiting verified content. In Proceedings of the 3rd workshop on Information credibility on the web. ACM, 51–58.
• [22] Daniel Kahneman and Shane Frederick. 2002. Representativeness revisited: Attribute substitution in intuitive judgment. Heuristics and biases: The psychology of intuitive judgment 49 (2002), 81.
• [23] Andreas M Kaplan and Michael Haenlein. 2010. Users of the world, unite! The challenges and opportunities of Social Media. Business horizons 53, 1 (2010), 59–68.
• [24] Barbara K Kaye. 2007. Blog use motivations: An exploratory study. Blogging, citizenship, and the future of media (2007), 127–148.
• [25] Gary A Klein. 2017. Sources of power: How people make decisions. MIT press.
• [26] KP Krishna Kumar and G Geethakumari. 2014. Detecting misinforma- tion in online social networks using cognitive psychology. Human- centric Computing and Information Sciences 4, 1 (2014), 14.
• [27] Kyumin Lee, James Caverlee, and Steve Webb. 2010. Uncovering social spammers: social honeypots+ machine learning. In Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval. ACM, 435–442.
• [28] Stephan Lewandowsky, Ullrich KH Ecker, Colleen M Seifert, Norbert Schwarz, and John Cook. 2012. Misinformation and its correction: Continued influence and successful debiasing. Psychological Science in the Public Interest 13, 3 (2012), 106–131.
• [29] Fangtao Li, Minlie Huang, Yi Yang, and Xiaoyan Zhu. 2011. Learn- ing to identify review spam. In IJCAI Proceedings-International Joint Conference on Artificial Intelligence, Vol. 22. 2488.
• [30] Yuqing Lu, Lei Zhang, Yudong Xiao, and Yangguang Li. 2013. Simul- taneously detecting fake reviews and review spammers using factor graph model. In Proceedings of the 5th annual ACM web science confer- ence. ACM, 225–233.
• [31] UniversalMcCann.2008.UniversalMcCannsocialmediatrackerwave 3, March 2008. Universal McCann, New York (2008).
• [32] Meredith Ringel Morris, Scott Counts, Asta Roseway, Aaron Hoff, and Julia Schwarz. 2012. Tweeting is believing?: understanding microblog credibility perceptions. In Proceedings of the ACM 2012 conference on computer supported cooperative work. ACM, 441–450.
• [33] ArjunMukherjee,VivekVenkataraman,BingLiu,andNatalieSGlance. 2013. What yelp fake review filter might be doing?. In ICWSM. 409– 418.
• [34] DonNorman.2013.Thedesignofeverydaythings:Revisedandexpanded edition. Constellation.
• [35] Myle Ott, Yejin Choi, Claire Cardie, and Jeffrey T Hancock. 2011. Find- ing deceptive opinion spam by any stretch of the imagination. In Proceedings of the 49th Annual Meeting of the Association for Computa- tional Linguistics: Human Language Technologies-Volume 1. Association for Computational Linguistics, 309–319.
• [36] Rebecca B Rubin and Michael P McHugh. 1987. Development of parasocial interaction relationships. (1987).
• [37] Richard Samuels. 2006. The magical number two, plus or minus: Some comments on dual-process theories. In Proceedings of the 28 th Annual Conference of the Cognitive Science Society, Vol. 202. 207.
• [38] Herbert A Simon. 1956. Rational choice and the structure of the environment. Psychological review 63, 2 (1956), 129.
• [39] Hyeonjin Soh. 2013. Attributes of trusted blog contents: through analysis of product-reviews in powerblogs and consumer survey. The Journal of the Korea Contents Association 13, 1 (2013), 73–82.
• [40] Gianluca Stringhini, Christopher Kruegel, and Giovanni Vigna. 2010. Detecting spammers on social networks. In Proceedings of the 26th annual computer security applications conference. ACM, 1–9.
• [41] Peter M Visscher, Jian Yang, and Michael E Goddard. 2010. A commen- tary on ‘common SNPs explain a large proportion of the heritability for human height’ by Yang et al.(2010). Twin Research and Human Genetics 13, 6 (2010), 517–524.
• [42] Chih-Chien Wang and Hung-Yu Chien. 2012. Believe or Skepticism? An Empirical Study onIndividuals’ Attitude to Blog Product Review. International Journal of Innovation, Management and Technology 3, 4 (2012), 343.
• [43] Yingxu Wang and Vincent Chiew. 2010. On the cognitive process of human problem solving. Cognitive systems research 11, 1 (2010), 81–92.
• [44] Debbie Weil. 2006. The corporate blogging book. Piatkus.
• [45] Sung-Un Yang and Joon Soo Lim. 2009. The effects of blog-mediated public relations (BMPR) on relational trust. Journal of Public Relations Research 21, 3 (2009), 341–359.
• [46] YimingYangandJanOPedersen.1997.Acomparativestudyonfeature selection in text categorization. In Icml, Vol. 97. 412–420.