Title: Validation of the Teacher Decision-Making Inventory (TDMI): measuring data-based and intuitive dimensions in teachers' decision Process
Source document: Studia paedagogica. 2021, vol. 26, iss. 4, pp. [47]-66
Extent
[47]-66
-
ISSN1803-7437 (print)2336-4521 (online)
Persistent identifier (DOI): https://doi.org/10.5817/SP2021-4-3
Stable URL (handle): https://hdl.handle.net/11222.digilib/144768
Type: Article
Language
License: Not specified license
Notice: These citations are automatically created and might not follow citation rules properly.
Abstract(s)
Teacher decision making has a great impact on the quality of education in schools, yet we know little about how teachers make decisions in practice. It is assumed that teachers use both intuition and data in the different steps of the decision process. No reliable, valid scales are available to research both dimensions during the different steps of teachers' decision process (problem definition, data collection, sense making, and evaluatioof alternatives). Building on the integratedframework we constructed in earlier research, the main aim of this study was to develop and validate a Teacher Decision-Making Inventory fl'DMI). One hundred and one teachers in adult education participated voluntarily in a web-based survey. Based on the good UFA factor loadings, the CFA fit indices, and the internal consistency (Cronbach's alpha), we conclude that the TDMI is a valid psychometric tool that can be used to assess the intuitive and data-driven dimensions of teachers' decisions in large-scale quantitative research.
References
[1] Allal, L. (2013). Teachers' professional judgement in assessment: A cognitive act and a socially situated practice. Assessment in Education: Principles, Policy & Practice, 20(1), 20—34. https://doi.org/10.1080/0969594X.2012.736364 | DOI 10.1080/0969594X.2012.736364
[2] Benson, J., & Clark, F. (1982). A guide for instrument development and validation. American Journal of Occupational Therapy, 36(12), 789-800. https://doi.org/10.5014/ajot.36.12.789 | DOI 10.5014/ajot.36.12.789
[3] Bertrand, M., & Marsh, J. A. (2015). Teachers' sensemaking of data and implications for equity. American Educational Research journal, 52(5), 861-893. https://doi.org/10.3102/0002831215599251 | DOI 10.3102/0002831215599251
[4] Blackwell, R. D., Miniard, P. W., & Engel, J. F. (2006). Consumer behavior. Harcourt College Publishers.
[5] Bonvin, P. (2003). The role of teacher attitudes and judgement in decision-making: The case of grade retention. European Educational Research journal, 2(2), 277—294. https://doi.org/10.2304/eerj.2003.2.2.6 | DOI 10.2304/eerj.2003.2.2.6
[6] Bosker, R. J., Branderhorst, E . M., & Visscher, A. J. (2007). Improving the utilisation of management information systems in secondary schools. School Effectiveness and School Improvement, 18(A), 451-467. https://doi.org/10.1080/09243450701712577 | DOI 10.1080/09243450701712577
[7] Bromme, R., Prenzel, M., & Jager, M. (2014). Empirische Bildungsforschung und evidenzbasierte Bildungspolitik. Zeitschrift für Erziehungswissenschaft, 17(A), 3-54. https://doi.org/10.1007/sll618-014-0514-5 | DOI 10.1007/sll618-014-0514-5
[8] Brown, C. (2017). Research learning communities: How the RLC approach enables teachers to use research to improve their practice and the benefits for students that occur as a result. Research for All, 1(2), 387-405. | DOI 10.18546/RFA.01.2.14
[9] Burgess, P. W., Alderman, N., Evans, J. O. N.. Emslie, H., & Wilson, B. A. (1998). The ecological validity of tests of executive function. Journal of the International Neuropsychological Society, 4(6), 547-558. https://doi.org/10.1017/S1355617798466037 | DOI 10.1017/S1355617798466037
[10] Burgess, S., Greaves, E., Vignoles, A., & Wilson, D. (2009). What parents want: schoolpreferences and school choice. CMPO.
[11] Chase, W. G., & Simon, H. A. (1973). Perception in chess. Cognitive Psychology, 4(1), 55—81. https://doi.org/10.1016/0010-0285(73)90004-2 | DOI 10.1016/0010-0285(73)90004-2
[12] Coburn, C. E., & Turner, E. O. (2012). The practice of data use: An introduction. American Journal of Education, 118(2), 99-111. https://doi.org/10.1086/663272 | DOI 10.1086/663272
[13] Cohen, L., Manion, L., & Morrison, K. (2017). Action research. In L. Cohen, L. Manion, & K. Morrison (Eds.), Research methods in education (8th ed., pp. 440-456). Routledge.
[14] Datnow, A., & Hubbard, L. (2015). Teachers' use of assessment data to inform instruction: Lessons from the past and prospects for the future. Teachers College Record, 117(A), 1—26.
[15] Datnow, A., Park, V., & Wohlstetter, P. (2007). Achieving with data. University of Southern California, Center on Educational Governance.
[16] De Maeyer, S., & Kavadias, D. (2007). Openleerpakket beschrijvende statistiek. Academia Press.
[17] Earl, L. M., & Katz, S. (Eds.). (2006). Leading schools in a data-rich world: Harnessing data for school improvement. Corwin Press.
[18] Earl, L., & Louis, K. S. (2013). Data use: Where to from here? In K. Schildkamp, M. K. Lai, & L. Earl (Eds.), Data-based decision makingin education: Challenges and opportunities (pp. 193-204). Springer,
[19] Ehren, M. C, & Swanborn, M. S. (2012). Strategic data use of schools in accountability systems. School Effectiveness and School Improvement, 23(2), 257-280.https://doi.org/10.1080/09243453.2011.652127 | DOI 10.1080/09243453.2011.652127
[20] Eurydice. (2011). Grade retention during compulsory education in Europe: regulations and statistics. OECD.
[21] Evans, J. S. B. (2008). Dual-processing accounts of reasoning, judgment, and social cognition. Annual Review of Psychology, 59,255-278. https://doi.org/10.1146/annurev.psych.59.103006.093629 | DOI 10.1146/annurev.psych.59.103006.093629
[22] Field, A. (2009). Discovering statistics using SPSS. SAGE Publications Limited.
[23] Harteis, C , Koch, T., & Morgenthaler, B. (2008). How intuition contributes to high performance: An educational perspective. US-China Education Review, 5(1), 68-80.
[24] Hinkin, T. R. (1998). A brief tutorial on the development of measures for use in survey questionnaires. Organisational Research Methods, 7(1), 104—121. https://doi.org/10.1177/109442819800100106 | DOI 10.1177/109442819800100106
[25] Hogarth, R. M. (2001). Educating intuition. University of Chicago Press.
[26] Hubbard, L., Datnow, A., & Pruyn, L. (2014). Multiple initiatives, multiple challenges: The promise and pitfalls of implementing data. Studies in Educational Evaluation, 42, 54—62. https://doi.Org/10.1016/j.stueduc.2013.10.003 | DOI 10.1016/j.stueduc.2013.10.003
[27] Kahneman, D. (2003). A perspective on judgment and choice: Mapping bounded rationality. American Psychologist, 58(9), 697-720. https://doi.Org/10.1037/0003-066X.58.9.697 | DOI 10.1037/0003-066X.58.9.697
[28] Kahneman, D., & Frederick, S. (2002). Representativeness revisited: Attribute substitution in intuitive judgment. In T. Gilovich, D. Griffin, & D. Kahneman (Eds.), Heuristics and biases: The psychology of intuitive judgment (pp. 49-81). Cambridge University Press.
[29] Kahneman, D., & Klein, G. (2009). Conditions for intuitive expertise: A failure to disagree. American Psychologist, 64(6), 515-526. https://doi.org/10.1037/a0016755 | DOI 10.1037/a0016755
[30] Kaiser, J., Retelsdorf, J., Siidkamp, A., & Moller, J, (2013). Achievement and engagement: How student characteristics influence teacher judgments. Learning and Instruction, 28, 73—84. https://doi.Org/10.1016/j.learninstruc.2013.06.001 | DOI 10.1016/j.learninstruc.2013.06.001
[31] Klein, G. (2008). Naturalistic decision making. Human Factors, 50(3), 456-460. https://doi.org/10.1518/001872008X288385 | DOI 10.1518/001872008X288385
[32] Loehlin, J. C. (2004). Latent variable models: An introduction to factor, path, and structural equation analysis (4th ed.). Lawrence Erlbaum.
[33] Mandinach, E. B., & Jimerson, J. B. (2016). Teachers learning how to use data: A synthesis of the issues and what is known. Teaching and Teacher Education, 60, 452—457. https://doi.org/10.1016/j.tate.2016.07.009 | DOI 10.1016/j.tate.2016.07.009
[34] Mandinach, E. B., Honey, M., & Light, D. (2006, April 9). A theoretical framework for data-driven decision making. Paper presented at the annual meeting of the American Educational Research Association, San Francisco, CA.
[35] Mintzberg, H., & Westley, F. (2001). It's not what you think. MIT Sloan Management Review, 42(3), 89-93.
[36] Rosseel, Y. (2012). Lavaan: An R package for structural equation modeling and more. Version 0.5-12 (BETA), journal ofStatisticalSoftware, 48(2), 1-36.
[37] Rubie-Davies, C. M. (2010). Teacher expectations and perceptions of student attributes: Is there a relationship? British Journal of Educational Psychology, 80{\), 121-135.https://doi.org/10.1348/000709909X466334 | DOI 10.1348/000709909X466334
[38] Schildkamp, K., & Lai, M. K. (2013). Conclusions and a data use framework. In K. Schildkamp, M. K. Lai, & L. Earl (Eds.), Data-based decision making in education: Challenges and opportunities (pp. 177—191). Springer.
[39] Schildkamp, K., Poortman, C. L., & Handelzalts, A. (2016). Data teams for school improvement. School Effectiveness and School Improvement, 27(2), 228-254. https://doi.org/10.1080/09243453.2015.1056192 | DOI 10.1080/09243453.2015.1056192
[40] Strayhorn, T. L. (2009). Accessing and analyzing national databases. In T. Kowalski & T.J. Lasley (Eds.), Handbook of data-based decision making in education (pp. 121- 138). Routledge.
[41] Tversky, A., & Kahneman, D. (1981). The framing of decisions and the psychology of choice. Science, 21 /(4481), 453-458. https://doi.org/10.1007/978-l-4613-2391-4_2 | DOI 10.1126/science.7455683
[42] Urhahne, D., & Wijnia, L. (2020). A review on the accuracy of teacher judgments. Educational Research Review, 32, 100374. https://doi.Org/10.1016/j.edurev.2020.100374 | DOI 10.1016/j.edurev.2020.100374
[43] Vanlommel, K. (2018). Opening the black box of teacher judgement: The interplay of rational and intuitive processes [Dissertation]. University of Antwerp, https://www.researchgate.net/publication/324909244_Opening_the_black_box_of_teacher_judgement_the_interplay_of_rational_and_intuitive_processes_Doctoral_thesis
[44] Vanlommel, K., & Schildkamp, K. (2019). How do teachers make sense of data in the context of high-stakes decision making? American Educational Research journal, 56(3), 792-821. https://doi.org/10.3102/0002831218803891 | DOI 10.3102/0002831218803891
[45] Vanlommel, K., Van Gasse, R., Vanhoof, J., & Van Petegem, P. (2017). Teachers' decisionmaking: Data based or intuition driven? International Journal of Educational Research, 83, 75-83. https://doi.Org/10.1016/j.ijer.2017.02.013 | DOI 10.1016/j.ijer.2017.02.013
[46] Vanlommel, K., Van Gasse, R., Vanhoof, J., &c Van Petegem, P. (2018). Teachers' high-stakes decision making. How teaching approaches affect rational and intuitive data collection. Teaching and Teacher Education, 7/(1), 108-119. | DOI 10.1016/j.tate.2017.12.011
[47] Vanlommel, K., Van Gasse, R., Vanhoof, J., & Van Petegem, P. (2020). Sorting pupils into their next educational track: How strongly do teachers rely on data-based or intuitive processes when they make the transition decision? Studies in Educational Evaluation, 69, 100374.https://doi.Org/10.1016/j.stueduc.2020.100865 | DOI 10.1016/j.stueduc.2020.100865
[48] Zanting, A., Verloop, N., & Vermunt, J. D. (2001). Student teachers' beliefs about mentoring and learning to teach during teaching practice. British Journal of Educational Psychology, 7/(1), 57-80. https://doi.org/10.1348/000709901158398 | DOI 10.1348/000709901158398
[2] Benson, J., & Clark, F. (1982). A guide for instrument development and validation. American Journal of Occupational Therapy, 36(12), 789-800. https://doi.org/10.5014/ajot.36.12.789 | DOI 10.5014/ajot.36.12.789
[3] Bertrand, M., & Marsh, J. A. (2015). Teachers' sensemaking of data and implications for equity. American Educational Research journal, 52(5), 861-893. https://doi.org/10.3102/0002831215599251 | DOI 10.3102/0002831215599251
[4] Blackwell, R. D., Miniard, P. W., & Engel, J. F. (2006). Consumer behavior. Harcourt College Publishers.
[5] Bonvin, P. (2003). The role of teacher attitudes and judgement in decision-making: The case of grade retention. European Educational Research journal, 2(2), 277—294. https://doi.org/10.2304/eerj.2003.2.2.6 | DOI 10.2304/eerj.2003.2.2.6
[6] Bosker, R. J., Branderhorst, E . M., & Visscher, A. J. (2007). Improving the utilisation of management information systems in secondary schools. School Effectiveness and School Improvement, 18(A), 451-467. https://doi.org/10.1080/09243450701712577 | DOI 10.1080/09243450701712577
[7] Bromme, R., Prenzel, M., & Jager, M. (2014). Empirische Bildungsforschung und evidenzbasierte Bildungspolitik. Zeitschrift für Erziehungswissenschaft, 17(A), 3-54. https://doi.org/10.1007/sll618-014-0514-5 | DOI 10.1007/sll618-014-0514-5
[8] Brown, C. (2017). Research learning communities: How the RLC approach enables teachers to use research to improve their practice and the benefits for students that occur as a result. Research for All, 1(2), 387-405. | DOI 10.18546/RFA.01.2.14
[9] Burgess, P. W., Alderman, N., Evans, J. O. N.. Emslie, H., & Wilson, B. A. (1998). The ecological validity of tests of executive function. Journal of the International Neuropsychological Society, 4(6), 547-558. https://doi.org/10.1017/S1355617798466037 | DOI 10.1017/S1355617798466037
[10] Burgess, S., Greaves, E., Vignoles, A., & Wilson, D. (2009). What parents want: schoolpreferences and school choice. CMPO.
[11] Chase, W. G., & Simon, H. A. (1973). Perception in chess. Cognitive Psychology, 4(1), 55—81. https://doi.org/10.1016/0010-0285(73)90004-2 | DOI 10.1016/0010-0285(73)90004-2
[12] Coburn, C. E., & Turner, E. O. (2012). The practice of data use: An introduction. American Journal of Education, 118(2), 99-111. https://doi.org/10.1086/663272 | DOI 10.1086/663272
[13] Cohen, L., Manion, L., & Morrison, K. (2017). Action research. In L. Cohen, L. Manion, & K. Morrison (Eds.), Research methods in education (8th ed., pp. 440-456). Routledge.
[14] Datnow, A., & Hubbard, L. (2015). Teachers' use of assessment data to inform instruction: Lessons from the past and prospects for the future. Teachers College Record, 117(A), 1—26.
[15] Datnow, A., Park, V., & Wohlstetter, P. (2007). Achieving with data. University of Southern California, Center on Educational Governance.
[16] De Maeyer, S., & Kavadias, D. (2007). Openleerpakket beschrijvende statistiek. Academia Press.
[17] Earl, L. M., & Katz, S. (Eds.). (2006). Leading schools in a data-rich world: Harnessing data for school improvement. Corwin Press.
[18] Earl, L., & Louis, K. S. (2013). Data use: Where to from here? In K. Schildkamp, M. K. Lai, & L. Earl (Eds.), Data-based decision makingin education: Challenges and opportunities (pp. 193-204). Springer,
[19] Ehren, M. C, & Swanborn, M. S. (2012). Strategic data use of schools in accountability systems. School Effectiveness and School Improvement, 23(2), 257-280.https://doi.org/10.1080/09243453.2011.652127 | DOI 10.1080/09243453.2011.652127
[20] Eurydice. (2011). Grade retention during compulsory education in Europe: regulations and statistics. OECD.
[21] Evans, J. S. B. (2008). Dual-processing accounts of reasoning, judgment, and social cognition. Annual Review of Psychology, 59,255-278. https://doi.org/10.1146/annurev.psych.59.103006.093629 | DOI 10.1146/annurev.psych.59.103006.093629
[22] Field, A. (2009). Discovering statistics using SPSS. SAGE Publications Limited.
[23] Harteis, C , Koch, T., & Morgenthaler, B. (2008). How intuition contributes to high performance: An educational perspective. US-China Education Review, 5(1), 68-80.
[24] Hinkin, T. R. (1998). A brief tutorial on the development of measures for use in survey questionnaires. Organisational Research Methods, 7(1), 104—121. https://doi.org/10.1177/109442819800100106 | DOI 10.1177/109442819800100106
[25] Hogarth, R. M. (2001). Educating intuition. University of Chicago Press.
[26] Hubbard, L., Datnow, A., & Pruyn, L. (2014). Multiple initiatives, multiple challenges: The promise and pitfalls of implementing data. Studies in Educational Evaluation, 42, 54—62. https://doi.Org/10.1016/j.stueduc.2013.10.003 | DOI 10.1016/j.stueduc.2013.10.003
[27] Kahneman, D. (2003). A perspective on judgment and choice: Mapping bounded rationality. American Psychologist, 58(9), 697-720. https://doi.Org/10.1037/0003-066X.58.9.697 | DOI 10.1037/0003-066X.58.9.697
[28] Kahneman, D., & Frederick, S. (2002). Representativeness revisited: Attribute substitution in intuitive judgment. In T. Gilovich, D. Griffin, & D. Kahneman (Eds.), Heuristics and biases: The psychology of intuitive judgment (pp. 49-81). Cambridge University Press.
[29] Kahneman, D., & Klein, G. (2009). Conditions for intuitive expertise: A failure to disagree. American Psychologist, 64(6), 515-526. https://doi.org/10.1037/a0016755 | DOI 10.1037/a0016755
[30] Kaiser, J., Retelsdorf, J., Siidkamp, A., & Moller, J, (2013). Achievement and engagement: How student characteristics influence teacher judgments. Learning and Instruction, 28, 73—84. https://doi.Org/10.1016/j.learninstruc.2013.06.001 | DOI 10.1016/j.learninstruc.2013.06.001
[31] Klein, G. (2008). Naturalistic decision making. Human Factors, 50(3), 456-460. https://doi.org/10.1518/001872008X288385 | DOI 10.1518/001872008X288385
[32] Loehlin, J. C. (2004). Latent variable models: An introduction to factor, path, and structural equation analysis (4th ed.). Lawrence Erlbaum.
[33] Mandinach, E. B., & Jimerson, J. B. (2016). Teachers learning how to use data: A synthesis of the issues and what is known. Teaching and Teacher Education, 60, 452—457. https://doi.org/10.1016/j.tate.2016.07.009 | DOI 10.1016/j.tate.2016.07.009
[34] Mandinach, E. B., Honey, M., & Light, D. (2006, April 9). A theoretical framework for data-driven decision making. Paper presented at the annual meeting of the American Educational Research Association, San Francisco, CA.
[35] Mintzberg, H., & Westley, F. (2001). It's not what you think. MIT Sloan Management Review, 42(3), 89-93.
[36] Rosseel, Y. (2012). Lavaan: An R package for structural equation modeling and more. Version 0.5-12 (BETA), journal ofStatisticalSoftware, 48(2), 1-36.
[37] Rubie-Davies, C. M. (2010). Teacher expectations and perceptions of student attributes: Is there a relationship? British Journal of Educational Psychology, 80{\), 121-135.https://doi.org/10.1348/000709909X466334 | DOI 10.1348/000709909X466334
[38] Schildkamp, K., & Lai, M. K. (2013). Conclusions and a data use framework. In K. Schildkamp, M. K. Lai, & L. Earl (Eds.), Data-based decision making in education: Challenges and opportunities (pp. 177—191). Springer.
[39] Schildkamp, K., Poortman, C. L., & Handelzalts, A. (2016). Data teams for school improvement. School Effectiveness and School Improvement, 27(2), 228-254. https://doi.org/10.1080/09243453.2015.1056192 | DOI 10.1080/09243453.2015.1056192
[40] Strayhorn, T. L. (2009). Accessing and analyzing national databases. In T. Kowalski & T.J. Lasley (Eds.), Handbook of data-based decision making in education (pp. 121- 138). Routledge.
[41] Tversky, A., & Kahneman, D. (1981). The framing of decisions and the psychology of choice. Science, 21 /(4481), 453-458. https://doi.org/10.1007/978-l-4613-2391-4_2 | DOI 10.1126/science.7455683
[42] Urhahne, D., & Wijnia, L. (2020). A review on the accuracy of teacher judgments. Educational Research Review, 32, 100374. https://doi.Org/10.1016/j.edurev.2020.100374 | DOI 10.1016/j.edurev.2020.100374
[43] Vanlommel, K. (2018). Opening the black box of teacher judgement: The interplay of rational and intuitive processes [Dissertation]. University of Antwerp, https://www.researchgate.net/publication/324909244_Opening_the_black_box_of_teacher_judgement_the_interplay_of_rational_and_intuitive_processes_Doctoral_thesis
[44] Vanlommel, K., & Schildkamp, K. (2019). How do teachers make sense of data in the context of high-stakes decision making? American Educational Research journal, 56(3), 792-821. https://doi.org/10.3102/0002831218803891 | DOI 10.3102/0002831218803891
[45] Vanlommel, K., Van Gasse, R., Vanhoof, J., & Van Petegem, P. (2017). Teachers' decisionmaking: Data based or intuition driven? International Journal of Educational Research, 83, 75-83. https://doi.Org/10.1016/j.ijer.2017.02.013 | DOI 10.1016/j.ijer.2017.02.013
[46] Vanlommel, K., Van Gasse, R., Vanhoof, J., &c Van Petegem, P. (2018). Teachers' high-stakes decision making. How teaching approaches affect rational and intuitive data collection. Teaching and Teacher Education, 7/(1), 108-119. | DOI 10.1016/j.tate.2017.12.011
[47] Vanlommel, K., Van Gasse, R., Vanhoof, J., & Van Petegem, P. (2020). Sorting pupils into their next educational track: How strongly do teachers rely on data-based or intuitive processes when they make the transition decision? Studies in Educational Evaluation, 69, 100374.https://doi.Org/10.1016/j.stueduc.2020.100865 | DOI 10.1016/j.stueduc.2020.100865
[48] Zanting, A., Verloop, N., & Vermunt, J. D. (2001). Student teachers' beliefs about mentoring and learning to teach during teaching practice. British Journal of Educational Psychology, 7/(1), 57-80. https://doi.org/10.1348/000709901158398 | DOI 10.1348/000709901158398