The dimensions of argumentative texts and their assessment

Title: The dimensions of argumentative texts and their assessment
Source document: Studia paedagogica. 2019, vol. 24, iss. 4, pp. [11]-44
Extent
[11]-44
  • ISSN
    1803-7437 (print)
    2336-4521 (online)
Type: Article
Language
License: Not specified license
 

Notice: These citations are automatically created and might not follow citation rules properly.

Abstract(s)
The definition and the assessment of the quality of argumentative texts has become an increasingly crucial issue in education, classroom discourse, and argumentation theory. The different methods developed and used in the literature are all characterized by specific perspectives that fail to capture the complexity of the subject matter, which remains ill-defined and not systematically investigated. This paper addresses this problem by building on the four main dimensions of argument quality resulting from the definition of argument and the literature in classroom discourse: dialogicity, accountability, relevance, and textuality (DART). We use and develop the insights from the literature in education and argumentation by integrating the frameworks that capture both the textual and the argumentative nature of argumentative texts. This theoretical background will be used to propose a method for translating the DART dimensions into specific and clear proxies and evaluation criteria.
Note
This project has received funding from the European Union's HORIZON 2020 research and innovation program under grant agreement 770045. Fabrizio Macagno would like to thank Fundação para a Ciência e a Tecnologia for the grant no. PTDC/FER-FIL/28278/2017.
References
[1] Akiguet, S., & Piolat, A. (1996). Insertion of connectives by 9-to 11-year-old children in an argumentative text. Argumentation, 10(2), 253–270. | DOI 10.1007/BF00180728

[2] Alexander, R. (2008). Culture, dialogue and learning: Notes on an emerging pedagogy. In N. Mercer & S. Hodgkinson (Eds.), Exploring Talk in School: Inspired by the Work of Douglas Barnes (pp. 91–114). London: Sage Publications.

[3] Anderson, R. C., Chinn, C., Chang, J., Waggoner, M., & Yi, H. (1997). On the logical integrity of children's arguments. Cognition and Instruction, 15(2), 135–167. | DOI 10.1207/s1532690xci1502_1

[4] Anthony, R., & Kim, M. (2015). Challenges and remedies for identifying and classifying argumentation schemes. Argumentation, 29(1), 81–113. | DOI 10.1007/s10503-014-9335-1

[5] Aristotle. (1991). Rhetoric. In J. Barnes (Ed.), The complete works of Aristotle, vol. II. Princeton: Princeton University Press.

[6] Azar, M. (1999). Argumentative text as rhetorical structure: An application of rhetorical structure theory. Argumentation, 13(1), 97–114. | DOI 10.1023/A:1007794409860

[7] Cavagnetto, A. (2010). Argument to foster scientific literacy: A review of argument interventions in K–12 science contexts. Review of Educational Research, 80(3), 336–371. | DOI 10.3102/0034654310376953

[8] Chinn, C. (2006). Learning to argue. In A. O'Donnell, C. Hmelo-Silver, & G. Erkens (Eds.), Collaborative learning, reasoning, and technology (pp. 355–383). Mahwah: Lawrence Erlbaum Publishers.

[9] Choi, Y. H. (1988). Text structure of Korean speakers' argumentative essays in English. World Englishes, 7(2), 129–137. | DOI 10.1111/j.1467-971X.1988.tb00226.x

[10] Clark, D., & Sampson, V. (2008). Assessing dialogic argumentation in online environments to relate structure, grounds, and conceptual quality. Journal of Research in Science Teaching, 45(3), 293–321. | DOI 10.1002/tea.20216

[11] Coirier, P., & Golder, C. (1993). Writing argumentative text: A developmental study of the acquisition of supporting structures. European Journal of Psychology of Education, 8(2), 169–181. | DOI 10.1007/BF03173160

[12] Cranton, P. (2001). Becoming an authentic teacher in higher education. Professional practices in adult education and human resource development series. Malabar: Krieger Publishing Company.

[13] Erduran, S., Ozdem, Y., & Park, J.-Y. (2015). Research trends on argumentation in science education: a journal content analysis from 1998–2014. International Journal of STEM Education, 2(1), 1–12. | DOI 10.1186/s40594-015-0020-1

[14] Erduran, S., Simon, S., & Osborne, J. (2004). TAPping into argumentation: Developments in the application of Toulmin's Argument Pattern for studying science discourse. Science Education, 88(6), 915–933. | DOI 10.1002/sce.20012

[15] Felton, M. (2004). The development of discourse strategies in adolescent argumentation. Cognitive Development, 19(1), 35–52. | DOI 10.1016/j.cogdev.2003.09.001

[16] Felton, M., & Kuhn, D. (2001). The development of argumentive discourse skill. Discourse Processes, 32(2), 135–153.

[17] Garcia-Mila, M., & Andersen, C. (2007). Cognitive foundations of learning argumentation. In S. Erduran & M. P. Jiménez-Aleixandre (Eds.), Argumentation in science education (pp. 29–45). Amsterdam: Springer.

[18] Giddens, A. (1991). Modernity and self-identity: Self and society in the late modern age. Stanford: Stanford University Press.

[19] Giora, R. (1985). Notes towards a theory of text coherence. Poetics Today, 6(4), 699–715. | DOI 10.2307/1771962

[20] Giora, R. (1988). On the informativeness requirement. Journal of Pragmatics, 12(5–6), 547–565. | DOI 10.1016/0378-2166(88)90048-3

[21] Giora, R. (1997). Discourse coherence and theory of relevance: Stumbling blocks in search of a unified theory. Journal of Pragmatics, 27(1), 17–34. | DOI 10.1016/0378-2166(95)00065-8

[22] Giora, R. (1998). Discourse coherence is an independent notion: A reply to Deirdre Wilson. Journal of Pragmatics, 29(1), 75–86. | DOI 10.1016/S0378-2166(97)00045-3

[23] Glassner, A., & Schwarz, B. (2007). What stands and develops between creative and critical thinking? Argumentation? Thinking Skills and Creativity, 2(1), 10–18. | DOI 10.1016/j.tsc.2006.10.001

[24] Golder, C., & Coirier, P. (1994). Argumentative text writing: Developmental trends. Discourse Processes, 18(2), 187–210.

[25] Govier, T. (1992). A practical study of argument. Belmont: Wadsworth.

[26] Graff, G. (2003). Clueless in academe: how schooling obscures the life of the mind. New Haven & London: Yale University Press.

[27] Grice, P. (1975). Logic and conversation. In P. Cole & J. Morgan (Eds.), Syntax and semantics 3: Speech acts (pp. 41–58). New York: Academic Press.

[28] Halliday, M. A. K., & Hasan, R. (1976). Cohesion in English. London: Longman.

[29] Hanks, W. F. (1989). Text and textuality. Annual Review of Anthropology, 18(1), 95–127. | DOI 10.1146/annurev.an.18.100189.000523

[30] Hitchcock, D. (1998). Does the traditional treatment of enthymemes rest on a mistake? Argumentation, 12(1), 15–37. | DOI 10.1023/A:1007738519694

[31] Jimenez-Aleixandre, M. P., Rodriguez, A. B., & Duschl, R. (2000). "Doing the lesson" or "doing science": Argument in high school genetics. Science Education, 84(6), 757–792. | DOI 10.1002/1098-237X(200011)84:6<757::AID-SCE5>3.0.CO;2-F

[32] Johnson, R. (2000). Manifest Rationality: A Pragmatic Theory of Argument. Hillsdale: Lawrence Erlbaum Associates.

[33] Kelly, G., Druker, S., & Chen, C. (1998). Students' reasoning about electricity: Combining performance assessments with argumentation analysis. International Journal of Science Education, 20(7), 849–871. | DOI 10.1080/0950069980200707

[34] Kelly, G. J., & Takao, A. (2002). Epistemic levels in argument: An analysis of university oceanography students' use of evidence in writing. Science Education, 86(3), 314–342. | DOI 10.1002/sce.10024

[35] Kinneavy, J. (2002). Kairos in classical and modern rhetorical theory. In P. Sipiora & J. Baumlin (Eds.), Rhetoric and kairos: Essays in history, theory, and praxis (pp. 58–76). Albany: SUNY Press.

[36] Kuhn, D. (1991). The skills of argument. New York: Cambridge University Press.

[37] Kuhn, D. (2010). Teaching and learning science as argument. Science Education, 94(5), 810–824. | DOI 10.1002/sce.20395

[38] Kuhn, D., & Crowell, A. (2011). Dialogic argumentation as a vehicle for developing young adolescents' thinking. Psychological Science, 22(4), 545–552. | DOI 10.1177/0956797611402512

[39] Kuhn, D., Hemberger, L., & Khait, V. (2014). Argue with me: Argument as a path to developing students' thinking and writing. New York: Wessex Press.

[40] Kuhn, D., & Park, S.-H. (2005). Epistemological understanding and the development of intellectual values. International Journal of Educational Research, 43(3), 111–124. | DOI 10.1016/j.ijer.2006.05.003

[41] Kuhn, D., & Udell, W. (2003). The development of argument skills. Child Development, 74(5), 1245–1260. | DOI 10.1111/1467-8624.00605

[42] Leech, G. (1983). Principles of pragmatics. London: Longman.

[43] Leitão, S. (2000). The potential of argument in knowledge building. Human Development, 43(6), 332–360. | DOI 10.1159/000022695

[44] Levinson, S. (2012). Action formation and ascription. In J. Sidnell & T. Stivers (Eds.), The handbook of conversation analysis (pp. 101–130). Chichester: John Wiley & Sons, Ltd.

[45] Long, M. (1996). Authenticity and learning potential in L2 classroom discourse. University of Hawai'i Working Papers in English as a Second Language 114(2), 127–149.

[46] Macagno, F. (2016). Argument relevance and structure. Assessing and developing students' uses of evidence. International Journal of Educational Research, 79(1), 180–194. | DOI 10.1016/j.ijer.2016.07.002

[47] Macagno, F. (2018). Assessing relevance. Lingua, 210–211, 42–64. | DOI 10.1016/j.lingua.2018.04.007

[48] Macagno, F. (2019). Coding relevance. Learning, Culture and Social Interaction, Advance online publication.

[49] Macagno, F., & Capone, A. (2016). Uncommon ground. Intercultural Pragmatics, 13(2), 151–180. | DOI 10.1515/ip-2016-0007

[50] Macagno, F., & Damele, G. (2013). The dialogical force of implicit premises. Presumptions in enthymemes. Informal Logic, 33(3), 365–393. | DOI 10.22329/il.v33i3.3679

[51] Macagno, F., & Konstantinidou, A. (2013). What students' arguments can tell us: Using argumentation schemes in science education. Argumentation, 27(3), 225–243. | DOI 10.1007/s10503-012-9284-5

[52] Macagno, F., Mayweg-Paus, E., & Kuhn, D. (2015). Argumentation theory in education studies: Coding and improving students' argumentative strategies. Topoi, 34(2), 523–537. | DOI 10.1007/s11245-014-9271-6

[53] Macagno, F., & Walton, D. (2015). Classifying the patterns of natural arguments. Philosophy and Rhetoric, 48(1), 26–53. | DOI 10.5325/philrhet.48.1.0026

[54] Macagno, F., & Walton, D. (2017). Interpreting straw man argumentation. The pragmatics of quotation and reporting. Amsterdam: Springer.

[55] Manz, E. (2016). Examining evidence construction as the transformation of the material world into community knowledge. Journal of Research in Science Teaching, 53(7), 1113–1140. | DOI 10.1002/tea.21264

[56] Mayweg-Paus, E., & Macagno, F. (2016). How dialogic settings influence evidence use in adolescent students. Zeitschrift Für Pädagogische Psychologie, 30(2–3), 121–132. | DOI 10.1024/1010-0652/a000171

[57] Mayweg-Paus, E., Macagno, F., & Kuhn, D. (2016). Developing argumentation strategies in electronic dialogs: Is modeling effective? Discourse Processes, 53(4), 280–297.

[58] Metaxas, N., Potari, D., & Zachariades, T. (2016). Analysis of a teacher's pedagogical arguments using Toulmin's model and argumentation schemes. Educational Studies in Mathematics, 93(3), 383–397. | DOI 10.1007/s10649-016-9701-z

[59] Michaels, S., Connor, C., & Resnick, L. (2008). Deliberative discourse idealized and realized: accountable talk in the classroom and in civic life, 27(4), 283–297.

[60] Newell, G. E., Beach, R., Smith, J., & Van Der Heide, J. (2011). Teaching and learning argumentative reading and writing: A review of research. Reading Research Quarterly, 46(3), 273–304.

[61] Nussbaum, M. (2003). Appropriate appropriation: Functionality of student arguments and support requests during small-group classroom discussions. Journal of Literacy Research, 34(4), 501–544.

[62] Nussbaum, M. (2008). Using argumentation vee diagrams (AVDs) for promoting argumentcounterargument integration in reflective writing. Journal of Educational Psychology, 100(3), 549–565. | DOI 10.1037/0022-0663.100.3.549

[63] Nussbaum, M. (2011). Argumentation, dialogue theory, and probability modeling: Alternative frameworks for argumentation research in education. Educational Psychologist, 46(2), 84–106. | DOI 10.1080/00461520.2011.558816

[64] Nussbaum, M., & Edwards, O. V. (2011). Critical questions and argument stratagems: A framework for enhancing and analyzing students' reasoning practices. Journal of the Learning Sciences, 20(3), 443–488. | DOI 10.1080/10508406.2011.564567

[65] Nussbaum, M., & Schraw, G. (2007). Promoting argument- counterargument integration in students' writing. The Journal of Experimental Education, 76(1), 59–92. | DOI 10.3200/JEXE.76.1.59-92

[66] O'Keefe, D. (1977). Two concepts of argument. Journal of the American Forensic Society, 13(3), 121–128.

[67] Osborne, J. (2010). Arguing to learn in science: The role of collaborative, critical discourse. Science, 328(5977), 463–466. | DOI 10.1126/science.1183944

[68] Osborne, J., Erduran, S., & Simon, S. (2004). Enhancing the quality of argumentation in school science. Journal of Research in Science Teaching, 41(10), 994–1020. | DOI 10.1002/tea.20035

[69] Paglieri, F. (2015). Bogency and goodacies: On argument quality in virtue argumentation theory. Informal Logic, 35(1), 65–87. | DOI 10.22329/il.v35i1.4209

[70] Perkins, D., Farady, M., & Bushey, B. (1991). Everyday reasoning and the roots of intelligence. In J. Voss, D. Perkins, & J. Segal (Eds.), Informal reasoning and education (pp. 83–105). Hillsdale: Lawrence Erlbaum Associates.

[71] Rapanta, C., Garcia-Mila, M., & Gilabert, S. (2013). What is meant by argumentative competence? An integrative review of methods of analysis and assessment in education. Review of Educational Research, 83(4), 483–520. | DOI 10.3102/0034654313487606

[72] Rapanta, C., & Macagno, F. (2016). Argumentation methods in educational contexts: Introduction to the special issue. International Journal of Educational Research, 79(1), 142–149. | DOI 10.1016/j.ijer.2016.03.006

[73] Rapanta, C., & Walton, D. (2016a). Identifying paralogisms in two ethnically different contexts at university level/Identificación de paralogismos en dos contextos universitarios diferenciados étnicamente. Infancia y Aprendizaje, 39(1), 119–149. | DOI 10.1080/02103702.2015.1111610

[74] Rapanta, C., & Walton, D. (2016b). The use of argument maps as an assessment tool in higher education. International Journal of Educational Research, 79(1), 211–221. | DOI 10.1016/j.ijer.2016.03.002

[75] Reinhart, T. (1980). Conditions for text coherence. Poetics Today, 1(4), 161–180. | DOI 10.2307/1771893

[76] Reznitskaya, A., & Wilkinson, I. (2015). Professional development in dialogic teaching: Helping teachers promote argument literacy in their classrooms. In D. Scott & E. Hargreaves (Eds.), The SAGE handbook of learning (pp. 219–232). London: SAGE Publications Ltd.

[77] Sampson, V., & Clark, D. (2006). Assessment of argument in science education: A critical review of the literature. In S. Barab, K. Hay, & D. Hickey (Eds.), Proceedings of the 7th international conference on learning sciences (pp. 655–661). Bloomington: ACM Press.

[78] Sampson, V., & Clark, D. (2008). Assessment of the ways students generate arguments in science education: Current perspectives and recommendations for future directions. Science Education, 92(3), 447–472. | DOI 10.1002/sce.20276

[79] Sandoval, W. A., & Millwood, K. A. (2005). The quality of students' use of evidence in written scientific explanations. Cognition and Instruction, 23(1), 23–55. | DOI 10.1207/s1532690xci2301_2

[80] Schank, R., Collins, G., Davis, E., Johnson, P., Lytinen, S., & Reiser, B. (1982). What's the point? Cognitive Science, 6(3), 255–275. | DOI 10.1207/s15516709cog0603_2

[81] Schwarz, B., Yair, N., Julia, G., & Merav, I. (2003). Construction of collective and individual knowledge in argumentative activity. The Journal of the Learning Sciences, 12(2), 219–256. | DOI 10.1207/S15327809JLS1202_3

[82] Stab, C., & Gurevych, I. (2014). Identifying argumentative discourse structures in persuasive essays. In A. Moschitti, B. Pang, & W. Daelemans (Eds.), Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 46–56). Doha: Association for Computational Linguistics.

[83] Toulmin, S. (1958). The uses of argument. Cambridge: Cambridge University Press.

[84] Toulmin, S., Rieke, R., & Janik, A. (1984). An introduction to reasoning. New York: Macmillan Publishing Company.

[85] Van Dijk, T. (1977a). Sentence topic and discourse topic. Papers in Slavic Philology, 1, 49–61.

[86] Van Dijk, T. (1977b). Pragmatic macrostructures in discourse and cognition. In M. De Mey, R. Pinxten, M. Poriau, & F. Vandamme (Eds.), CC77. International Workshop on the Cognitive Viewpoint (pp. 99–113). Ghent: University of Ghent Press.

[87] Van Dijk, T. (1980). Text and context explorations in the semantics and pragmatics of discourse. London & New York: Longman.

[88] Van Dijk, T., & Kintsch, W. (1983). Strategies of discourse comprehension. New York: Academic Press.

[89] van Eemeren, F., & Grootendorst, R. (1984). Speech acts in argumentative discussions: A theoretical model for the analysis of discussions directed towards solving conflicts of opinion. Dordrecht: Floris Publications.

[90] van Eemeren, F., & Grootendorst, R. (2004). A systematic theory of argumentation: The pragmadialectical approach. Cambridge: Cambridge University Press.

[91] Van Lier, L. (1996). Interaction in the language classroom: Awareness, autonomy and authenticity. London: Longman.

[92] Vatz, R. (1973). The myth of the rhetorical situation. Philosophy & Rhetoric, 6(3), 154–161.

[93] Von Aufschnaiter, C., Erduran, S., Osborne, J., & Simon, S. (2008). Arguing to learn and learning to argue: Case studies of how students' argumentation relates to their scientific knowledge. Journal of Research in Science Teaching, 45(1), 101–131. | DOI 10.1002/tea.20213

[94] Walton, D. (1989). Informal logic. New York: Cambridge University Press.

[95] Walton, D. (1990). What is reasoning? What is an argument? Journal of Philosophy, 87(8), 399–419. | DOI 10.2307/2026735

[96] Walton, D. (1992). Plausible argument in everyday conversation. Albany: State University of New York Press.

[97] Walton, D. (1995). Argumentation Schemes for Presumptive Reasoning. Mahwah: Routledge.

[98] Walton, D. (2004). Relevance in argumentation. Mahwah: Erlbaum.

[99] Walton, D. (2006). Fundamentals of critical argumentation. New York: Cambridge University Press.

[100] Walton, D. (2010a). Objections, rebuttals and refutations. In J. Ritola (Ed.), Proceedings from the 8th Conference of the Ontario Society for the Study of Argumentation, June 3–6, 2009, University of Windsor (pp. 1–11). Windsor: OSSA.

[101] Walton, D. (2010b). Why fallacies appear to be better arguments than they are. Informal Logic, 30(2), 159–184. | DOI 10.22329/il.v30i2.2868

[102] Walton, D., & Macagno, F. (2016). Profiles of dialogue for relevance. Informal Logic, 36(4), 523–556. | DOI 10.22329/il.v36i4.4586

[103] Walton, D., Reed, C., & Macagno, F. (2008). Argumentation schemes. New York: Cambridge University Press.

[104] Witte, S. P., & Faigley, L. (1981). Coherence, cohesion, and writing quality. College Composition and Communication, 32(2), 189–204. | DOI 10.2307/356693

[105] Zohar, A., & Nemet, F. (2002). Fostering students' knowledge and argumentation skills through dilemmas in human genetics. Journal of Research in Science Teaching, 39(1), 35–62. | DOI 10.1002/tea.10008