IMPACT

Deutsch

Implementing AI-based feedback and assessment with Trusted Learning Analytics in higher education institutions. Change management, didactics for formative assessment and feedback

IMPACT is a CATALPA project.

Better higher education thanks to artificial intelligence - the IMPACT cooperation project not only researches the use of trusted learning analytics and AI in university teaching, but also provides scientific support for implementation. Along the Student Life Cycle, prospective students, first-year students, and undergraduates receive text-based, highly informative, and personalized feedback in higher education in order to determine, what supports study success the most.

Project goals and research questions

Artificial intelligence (AI) is currently changing opportunities in business, society and key areas of life. In order for Germany to become a global leader in the research, development and application of artificial intelligence, it needs a broad and highly trained skilled workforce. The federal-state initiative for the promotion of universities on artificial intelligence in higher education pursues the objective of achieving effective effects in studying and teaching through the use of AI. In this way, the improvement of the quality, performance and effectiveness of higher education is to be advanced through the use of AI.

In the cooperation project IMPACT, the implementation of trusted learning analytics and AI in higher education is implemented, scientifically accompanied and researched by the research focus CATALPA and the project management of Prof. Dr. Claudia de Witt together with other project partners.

IMPACT promotes the improvement of higher education through the scalable use of artificial intelligence (AI) methods for the (partially) automated analysis of texts as part of a trusted learning analytics approach. Along the Student Life Cycle, prospective students, first-year students, and students receive text-based, highly informative, and personalized feedback in higher education. At five German universities, text-based AI methods such as chatbots, personalized feedback systems for formative as well as summative assessment are being widely applied. The interdisciplinary consortium uses internationally proven open source software solutions as well as common standards and takes into account the interoperability in university teaching with common learning management systems (Moodle, Stud.IP, ILIAS). For the goal-oriented integration of the AI applications, (media-)didactic conceptions and adaptations are made. A prerequisite for the sustainable implementation of the AI applications in teaching and learning is a data-ethical change management based on the SHEILA process model. The model is applied in all participating universities and thus provides the network with a uniform, scientifically based framework concept for continuous cooperation with committees, teachers and students. The project results are made accessible throughout Germany by the network through workshops and according to Open Science principles.

  • The joint project IMPACT brings together five universities in Germany that have special expertise in the area of ethical, legal and social implications and numerous practical experiences in the didactic use of learning analytics. This expertise is to be used to consolidate the preliminary work of the collaborative universities on the application of AI and to make it accessible to the collaborative and other interested universities by means of open source software (OSS).

    The FernUniversität in Hagen is pursuing three central sub-goals in particular within the framework of the joint project:


    1. sub-goal of the FernUniversität in Hagen: implementation of the Sheila Framework.

    The scientifically developed Sheila Framework serves the a priori identification of conditions for success for the use of interventions or innovative technical solutions in six steps and the structured monitoring of the implementation process. In order to coordinate the implementation process in the respective institutions, needs and conditions for success are recorded and all departments of the institutions are involved. This includes continuous cooperation with representatives of students and teachers as well as relevant university committees and the development of further education material for the implemented Trusted Learning Analytics applications. Through the SHEILA process model, the universities participating in the network are accompanied by the FernUniversität in Hagen in questions of AI adaptation; through the interlinking of the universities in the network, impulses and different approaches are also informally exchanged nationwide. In this way, the universities support each other in the development of relevant processes for the data-ethical handling of students.

    2. partial goal of the FernUniversität in Hagen: media didactic conception

    In close cooperation with the Humboldt University of Berlin, the FernUniversität in Hagen is developing a basis on the basis of existing data, from which didactic concepts and adaptations for the design of AI-supported applications for formative feedback are derived. Formative feedback is concerned with timely feedback in the form of highly informative feedback during the course of study. Highly informative feedback contains task-level, process-level, and (occasionally) self-regulatory level information (Wisniewski, Zierer & Hattie, 2020). Based on the didactic concepts and data, AI components for processing text data are created or extended, integrated into the learning management system, and iteratively enhanced. In addition, the FeU team supports the didactic design of (1) AI-supported summative feedback and (2) the further development of Online Study Choice Assistants (OSA) at the other project partners.


    3. subgoal of the FernUniversität in Hagen - implementation of an AI-supported application for formative feedback.

    The provision and further development of specific AI applications for formative feedback will be based on text data. Under realistic conditions, the AI applications will be tested, evaluated for their contribution to identified needs, and their sustainable deployment will be realized through the use of selected open source software in the network. At FeU, the open source software OnTask will be deployed with the goal of integrating it into the regular course structure over the course of the project across several modules and degree programs at FeU and, if the evaluation results are positive, to continue it after the end of the project. Interoperability with other systems will be ensured on an ongoing basis.

  • Prof. Dr. Claudia de Witt

  • Bmbf LogoFoto: BMBF

    The project is funded by the Federal Ministry of Education and Research (BMBF) within the framework of the federal-state initiative "Promotion of Artificial Intelligence in Higher Education".

    Funding Announcement:

    https://www.bmbf.de/bmbf/shareddocs/bekanntmachungen/de/2021/02/3409_bekanntmachung.html

    Funding code: 16DHBKI043

    Nordrhein-westfalen 1500Foto: MKW

    Funded by the Ministry of Culture and Science of the State of North Rhine-Westphalia.

    • Goethe-Universität Frankfurt, Prof. Dr. Hendrik Drachsler (Konsortialführung)
    • Humboldt-Universität zu Berlin, Prof. Dr. Niels Pinkwart
    • Freie Universität Berlin, Prof. Dr. Tim Landgraf und Dipl. Soz. Alexander Schulz
    • Universität Bremen, Prof. Dr. Andreas Breiter

  • 1 December 2021 - 30 November 2025

  • 2023

    Journals

    • Ferguson, R., Khosravi, H., Kovanović, V., Viberg, O., Aggarwal, A., Brinkhuis, M., Buckingham Shum, S., Chen, L. K., Drachsler, H., Guerrero, V. A., Hanses, M., Hayward, C., Hicks, B., Jivet, I., Kitto, K., Kizilcec, R., Lodge, J. M., Manly, C. A., Matz, R. L., … Yan, V. X. (2023). Aligning the goals of learning analytics with its research scholarship: An open peer commentary approach. Journal of Learning Analytics, 10(2), 14–50. https://doi.org/10.18608/jla.2023.8197

    2022

    Conferences

    • van Rijn, L., Karolyi, H., & de Witt, C. (2022). Trusted Learning Analytics verstetigen. Mit Change Management zu didaktischen Innovationen. In B. Standl (Ed.), Digitale Lehre nachhaltig gestalten. Waxmann.

    Talks and Poster Presentations

    • Karolyi, H., & van Rijn, L. (2022). Im Team besser – worauf es bei der interdisziplinären Zusammenarbeit ankommt [Presentation]. Workshop zum Thema Interdisziplinäre Zusammenarbeit, Online.
    • Karolyi, H., & Wrede, S. (2022). Gestaltung formativer Feedbacks an Hochschulen mit Künstlicher Intelligenz und Trusted Learning Analytics [Presentation].
    • van Rijn, L., Karolyi, H., & de Witt, C. (2022, September 13). Trusted Learning Analytics verstetigen. Mit Change Management zu didaktischen Innovationen [Presentation]. 30. Jahrestagung der Gesellschaft für Medien in der Wissenschaft e.V., Karlsruhe.
Research Center CATALPA | 19.04.2024