Science of teaching and learning

J. Cape

The purpose of the proposed study is to conduct an environmental scan of research published by UM faculty over the past ten years that investigates an aspect of teaching and learning, and that would fall under the “big tent” (Chick, 2014) of SoTL research. While other kinds of teaching and learning research—formal and informal—certainly occur at UM without being disseminated in peer-reviewed contexts, publication suggests a level of value placed on the research. This work seeks to catalog the UM authors undertaking this work, their disciplines, the publications where the work is disseminated, and to examine the questions investigated. 

H. Malyk

Varied definitions and interpretations of SoTL-based concepts challenge the meaning of SoTL and its discourse (McKinney, 2002) and constrain SoTL engagement rather than enhance original inquiry and authentic teaching practices. Terminological inconsistency within the broad field of SoTL is an apparent barrier to its capacity building, as it pushes SoTL to the periphery of pedagogical research and makes it the “road less traveled” (Simmons, 2017) and “a borderland” in which scholars from different disciplinary cultures come to “trade their wares” (Chick, 2013; Huber & Morreale, 2002), transforming it into a vague undervalued “big tent” (Chick, 2013; Hewson & Easton, 2022) “under which a wide variety of work can thrive” (Huber & Hutchings, 2005, p. 4). This study attempts to address this inconsistency by exploring and analyzing the nature and substance of SoTL concepts, their definitions and terminological applications to facilitate a requisite cultural shift for enhanced educational practices within 4 M Framework of SoTL Impact (Simmons, 2020) and “dispel the fog of learning through SoTL” (Tagg, 2010). 

L. A. Doan & B. M. Stoesz

Self-reflection on teaching is a critical component of professional development for instructors in higher education (Szűcs, 2018). A reflective method that includes creative (fictionalizing some biographical events), expressive (writing about traumatic events and the emotions experienced), and reflective (reflecting on one’s life, including questions regarding one’s identity and beliefs) writing contributes positively to the development of professional identity (Lengelle et al., 2016). The reflective practices by teachers in K-12 education and community colleges have received much more attention in the research literature than those of university educators ​(Jaramillo Cherrez & Jin, 2020),​ or educational developers. Given the research on the benefits of self-reflective practice, and that limited research has been conducted with educational developers, our primary goal is to examine whether educational developers in higher education participate in self-reflection and how this practice may or may not impact their professional growth and identity development. 

B. M. Stoesz, M. Quesnel, J. Plohman, I. Arowolo, & W. Snow

Training postsecondary educators to implement pedagogies and deal with various teaching challenges more effectively supports the goal of striving for teaching excellence at the University of Manitoba. A novel way to prepare graduate students for success in teaching involves the use of extended/virtual reality (XR/VR). This project aims to determine the effectiveness of such an approach to teacher training. 

L. Scholle-Collen (Queen’s University) & B. M. Stoesz with support from L. Doan

Chapter in press

Many Canadian universities trace their origins to religious foundations with a focus on ministry education but have distanced themselves from this mission over the last century and now offer a broad range of academic programs. Examining these historical shifts within Canadian universities can reveal persisting tensions related to academic integrity. University governance documents from Queen’s University and the University of Manitoba, specifically 264 archived Senate meeting minutes over 16 years, were analyzed to uncover the prevailing culture of academic integrity within higher education. By using traditional human qualitative analysis methods, evidence of subtle shifts toward cultures of integrity at both universities emerged. The utility of generative artificial intelligence (genAI) for exploring text was also tested, which revealed significant limitations that should prevent its use as a stand-alone qualitative data analysis tool. Providing an overview of the changing perceptions, susceptibility, and attitudes toward integrity in historical documentation contributes to the growing body of literature on academic integrity culture. In addition, the findings that emerged from the document analysis and conclusions about genAI as a research tool have implications for university leaders as they revise and/or develop policies to align with the evolving integrity and technological landscapes to foster cultures of honesty, trust, and transparency. 

W. M. Snow & A. E. De Jaeger

Research in many disciplines, including higher education, relies heavily on self-report surveys. Increasingly, survey administration is moving away from in-person to online and digital environments, the advantages of which can include increased anonymity and convenience for participants as well as reduced research costs. Collecting survey data in digital and online environments, however, increases the capacity for fraudulent participation in research. We investigated the degree of suspected invalid responding in an online survey on barriers to entry and persistence in adult job-related training and education among Manitobans and extended upon an algorithm designed to facilitate detection of suspected fraudulent responding. This research will have important implications for designing more effective, rigorous methods of data collection to reduce potential illegitimate participation in online survey research and to ease the burden of detection of such instances for researchers going forward. We also argue for more standardized reporting in online survey research studies on the extent of suspected fraudulent responding and methods used to define data as fraudulent to increase the validity of findings and foster trust in survey data collected online. Such measures are essential to ensure data integrity in online survey research in education and beyond. 

B. M. Stoesz, R. Attas, & H. Sanni-Anibire

Within the topic of academic integrity, accommodations for students experiencing learning challenges have largely been ignored in research and practice. Instructors’ lack of knowledge and accompanying skills may result in misconceptions about the intersection between academic integrity and the learning needs of students; in particular, the misconception that accommodations are “situationally sanctioned cheating” that allow students to “gain an unfair advantage” rather than facilitating learning and success must be addressed. This chapter explores questions that students, instructors, and other postsecondary community members may have about accessibility and equity, and how these considerations show respect, fairness, trust, responsibility, honesty, and courage and contribute to building cultures of academic integrity. To read more, click here. 

M. Quesnel & B. M. Stoesz

Contract cheating is a significant concern in higher education, requiring a multifaceted teaching and learning approach to address it. Quizzing students about their writing to promote engagement, confirm authorship and detect cheating has not yet been investigated systematically. Therefore, in this study, our objective was to explore the validity and usability of the quizzing software Auth+ to verify assessment authorship and detect contract cheating. A total of 24 third-year computer science students submitted their assignments to Auth+ and were quizzed, then completed a working memory assessment and a user experience survey. Students reported that Auth+ was easy to use and would discourage cheating but increased anxiety and contributed little to their learning. Quiz scores were correlated with individual differences in working memory capacity but not with other contract cheating indicators. These findings suggest . . . To read more, click here.  

Student ratings of instruction (SRI) are commonly used to evaluate courses and teaching in higher education. Much debate about their validity in evaluating teaching exists, which is due to concerns of bias by factors unrelated to teaching quality (Spooren et al., 2013). Our objective was to identify peer-reviewed original research published in English from January 1, 2012, to March 10, 2021, on potential sources of bias in SRIs. Our systematic review of 63 articles demonstrated strong support for the continued existence of gender bias, favoring male instructors and bias against faculty with minority ethnic and cultural backgrounds. These and other biases must be considered when implementing SRIs and reviewing results. Critical practices for reducing bias when using SRIs include implementing bias awareness training and avoiding use of SRIs as a singular measure of teaching quality when making decisions for teaching development or hiring and promotion.

Stoesz, B., De Jaeger, A., Quesnel, M., Bhojwani, D. & Los, R. (2022). Bias in student ratings of instruction: A systematic review of research from 2012 to 2021. Canadian Journal of Educational Administration and Policy/Revue canadienne en administration et politique de l’éducation, (201), 39–62. https://doi.org/10.7202/1095482ar 

Approximately 2,034,957 Canadians were enrolled in courses at postsecondary institutions in 2015/2016 (Statistics Canada, 2017), and an estimated 610,487 (30%) are completing online courses in any given year (Bates, 2015). This project examined students’ visual perceptions of the learning management systems (LMS) they use and how it influences their learning experiences. Findings contribute to our knowledge of LMS interface design.  

Stoesz, B. M., & Niknam, M. (2023). Student perceptions of the visual design of learning management systems. Canadian Journal of Learning and Technology, 48(3), 1-22. https://cjlt.ca/index.php/cjlt/article/view/28154/20689  

Researchers from four Canadian universities and an industry partner are collaborating to better understand faculty perceptions and needs related to academic integrity in Canadian higher education. The survey designed for this project was informed by previous academic integrity research (e.g. McCabe, 1993), and the findings are expected to lead to the development of specific supports for faculty in Canadian higher education institutions to promote academic integrity.

Eaton, S. E., Stoesz, B. M., Crossman, K., Garwood, K., & McKenzie, A. (2023). Faculty Perspectives of Academic Integrity During COVID-19: A Mixed Methods Study of Four Canadian Universities. Canadian Journal of Higher Education, 52(3), 42–58. https://doi.org/10.47678/cjhe.vi0.189783 

Postsecondary educators and students were among the first affected by COVID-19 pandemic safety protocols and were required to transition quickly from face-to-face to unfamiliar remote teaching and learning environments. To support this transition, support staff at teaching and learning centres (TLCs) also pivoted their support strategies and developed and delivered more online resources and virtual professional development workshops. The purpose of this study was to examine whether the demands for online and remote teaching and learning support was evident in the Google Analytics data of the TLC webpages of a research-intensive Canadian university. To facilitate interpretation of observed trends, we aligned the timing of messages emailed to faculty and new resource launches on the TLC webpages to the analytics data. Results confirmed that TLC webpages are vital sources of information for faculty development , and targeted communications increased faculty engagement with teaching resources and professional development opportunities. As pandemic safety protocols ease and educators and students return to their postsecondary campuses, they will face new challenges. In response, TLCs must continue to monitor faculty engagement and their changing support needs, and they must continue to adjust approaches to offering information and professional development opportunities as necessary. The examination of website analytics data is just one measure that can support evidence-informed decision making for this purpose.

Stoesz, B. M. (2022). Using Google analytics to measure engagement with a teaching and learning centre during COVID-19. Journal on Centers for Teaching and Learning, 14, 39-57. https://openjournal.lib.miamioh.edu/index.php/jctl/article/view/245 

The topic of ethics has received much attention within the SoTL community; however, limited studies have examined both REB reviewers’ and SoTL researchers’ perspectives on partnerships when preparing or reviewing SoTL research. In this chapter, we describe the findings from our study that aimed to address this gap by gathering researchers’ and REB reviewers’ experiences with and perceptions of SoTL research and situate findings within the micro-meso-macro-mega (4M) framework.

De Jaeger, A. E., Stoesz, B. M., Doan, L. A. (2022). Tensions and Partnerships: Understanding Research Ethics in the Scholarship of Teaching and Learning (SoTL). In: Fedoruk, L.M. (eds) Ethics and the Scholarship of Teaching and Learning. Ethics and Integrity in Educational Contexts, vol 2. Springer, Cham. https://doi.org/10.1007/978-3-031-11810-4_4 

This study investigated the knowledge of academic integrity and associated emotions of a small sample of international students studying at Canadian postsecondary institutions (n = 60) using survey methodology. Depending on the survey item, 25–60 participants provided responses. Many respondents appeared knowledgeable about academic integrity and misconduct and reported that expectations in their home countries and in Canada were similar. There was, however, disagreement on the concept of duplicate submission/self-plagiarism, indicating an important gap in educating students about specific aspects of policy in postsecondary education in Canada. In addition, more than a third of respondents provided neutral responses to a situation involving contract cheating, suggesting a lack of certainty in how to respond when witnessing peers’ engagement in outsourcing academic work. Many respondents reported feeling confident upon reading the academic integrity and misconduct policies of their Canadian postsecondary institution, although nearly one third indicated feeling fearful, anxious, and/or confused. These negative feelings were associated with reduced knowledge of academic integrity and misconduct. Future research should further explore the experiences and emotions of international students related to academic integrity and misconduct to better understand the successes and challenges that they face in their postsecondary studies in Canada. Our findings have important implications for the delivery of academic integrity education, enhancing supports and resources, and refining academic integrity policies and procedures to improve the experience of students who come from abroad to study in Canada.

Sanni-Anibire, H., Stoesz, B. M., Gervais, L., & Vogt, L. (2021). International students’ knowledge and emotions related to academic integrity at Canadian postsecondary institutions. International Journal of Educational Integrity, 17, 21 (2021). https://doi.org/10.1007/s40979-021-00088-4 

Readiness for teaching online can be defined as having the competencies required to deliver a high-quality online course, where readiness can be equated with the qualities exemplified by the ideal online instructor. This project seeks to determine the characteristics that exemplify effective online instructors and create a first-of-its-kind online teaching readiness assessment with psychometric properties. The assessment will provide instructors with immediate feedback as to the areas they excel and can improve upon, assist in identifying faculty training/development opportunities, determine strengths and areas in need of enhancement in instructors’ online teaching skillsets, and further understand the barriers and attitudes towards online teaching.

Los, R., De Jaeger, A. & Stoesz, B. M. (2021). Development of the Online and Blended Teaching Readiness Assessment (OBTRA). Frontiers in Education, 6(673594). https://doi.org/10.3389/feduc.2021.673594 

Delivering courses online can be effective but learning via learning management systems (LMS) may be attenuated by poor visual design. This project aimed to determine if existing models of complexity for webpages can be extended to LMS and explore how individual differences influence ratings of complexity. Findings inform the development of online learning environments.  

Stoesz, B. M., Niknam, M., & Sutton, J. (2020). Defining the visual complexity of learning management systems using image metrics and subjective ratings. Canadian Journal of Learning and Technology, 46(2), 1-21. https://www.cjlt.ca/index.php/cjlt/article/view/27899/20497