Research Professorship of Computational Linguistics

Deutsch

Members of the research professorship Photo: Henrik Schipper

Language technology for improvements in higher education

Using artificial intelligence to automatically score essays and thus offering individual support to students, (partially) automating exam assessments and reducing lecturers workload with automatic handwriting recognition - the potential use cases of language technology in higher education are countless, as it can be found wherever there is linguistic interaction in digital studies.

Prof. Dr.-Ing. Torsten Zesch and his team conduct research in the field of language technology and natural language processing at CATALPA since March 2022. They are particularly interested in the analysis and processing of non-standardized language as found in the speech of learners. With the application possibilities of computational linguistics and in collaboration with the projects of CATALPA, Torsten Zesch and his research professorship contribute to supporting digital higher education with automated, intelligent solutions - both for students and teachers.

  • How can language technology and artificial intelligence (AI) support teaching and learning processes? This question is at the core of the research professorship of Computational Linguistics. We are everywhere where there are linguistic interactions and they are ubiquitous in (digital) learning contexts. Learners, for example, receive texts and tasks from teachers, teachers in turn, assess those written tasks and provide feedback. Our team is researching how many of these processes can be automated.

    This includes:

    • Automatic analysis of the (linguistic) properties of texts
    • Automatic generation of tasks, e.g., deriving reading comprehension questions from texts
    • Automatic evaluation of free text answers. These can be short answers, where the content is most important, but also longer essays, where argument structure or style play a role.
    • Automatic (adaptive) feedback on texts

    We are also interested in the peculiarities of learners' language, such as spelling mistakes or the language of non-native speakers.

    This is not always only about digital texts. Our research areas also include

    • Multimodal applications, e.g. description of images and analysis of memes
    • Processing of spoken language
    • Automatic handwriting recognition

    We do not see basic research as a contradiction to application-oriented research, but as an important basis for it. Therefore, we also deal with fundamental questions on topics such as

    • Robustness of models, e.g. with respect to language change
    • Explainability of AI
    • Ethical and legal implications of AI models.

    We are committed to reproducible and replicable research. We generally make all research software publicly available.

  • Prof. Dr. Torsten Zesch

    Torsten-zesch-schipper-2022Foto: Henrik Schipper

    Piush Aggarwal

    Piush-aggarwal-schipper-2022Foto: Henrik Schipper

    Jeanette Bewersdorff

    Jeanette-bewersdorff-schipper-2022Foto: Henrik Schipper

    Marie Bexte

    Marie BexteFoto: Hardy Welsch

    Christian Gold

    Christian-gold-schipper-2022Foto: Henrik Schipper

    Jasmin Grams

    jasmin gramsFoto: Hardy Welsch

    Dr. Josef Ruppenhofer

    Josef RuppenhoferFoto: Hardy Welsch
    		  

    • TrACE (2021 - 2024)
      "Training Assessment Competencies in English as a Second Language" led by Prof. Jens Möller (Institute for Pedagogical-Psychological Research on Teaching and Learning (IPL) Kiel University) and Prof. Stefan Keller (University of Applied Sciences Northwestern Switzerland). Our role as project partner: Automatic evaluation and analysis of English argumentative texts of German and Swiss 11th grade students
    • DAKODA (2022 - 2025)
      "Data competencies in DaF/DaZ: Exploring language technology approaches to the analysis of L2 acquisition levels in learner corpora". BMBF project
    • WisPerMed (2021 - 2025)
      "AI for personalized medicine"

    • KISS-Pro (2023 - 2025)
      "AI in Language Learning - Concepts for Teacher Professional Development and Utilization Perspectives of AI-based Feedback Systems and Chatbots for Language Learning in Schools". BMBF project

    • "Use of AI for early detection of criminal offences" (2020 - 2023). BMBF "Research for civil security".
    • "Exploration of digital technologies in public employment services using text mining as an example" (2019-2020). BMAS FIS (Fördernetzwerk Interdisziplinäre Sozialpolitikforschung).
    • "Automatic evaluation of free text answers" (2019-2020). TestDaF Institute
    • "DKPro - Language Technology Middleware" (2018-2020). DFG in-kind grant in the program "Sustainability of Research Software".
    • "ASSURE- Argument-based Decision Support for Recommender Systems" (2018-2020). DFG in-kind grant in the priority program "Ratio - Robust Argumentation Machines"
    • "Mobile Communications Database - MoCoDa" (2017). Word class recognition for chat data. Ministry of Culture and Science NRW
    • "Educational Equity in Focus II" (2016 - 2020). Understanding and modeling learner responses. BMBF
    • "Transferring the RTG experience in social media" (2016). German-Arab Transformation Partnership. DAAD
    • "User-centered Social Media" (2015 - 2020). Interdisciplinary DFG Research Training Group (Computer Science and Psychology)
    • "Text as Process" (2010 - 2013). Analysis of collaborative writing processes in Wikipedia and the influence of discussion pages on the quality of articles. Subproject in the LOEWE focus "Digital Humanities Hessian Excellence Program LOEWE".

  • 2024

    Journals

    • Flinz, C., & Ruppenhofer, J. (2024). Coreferenza e nuclei tematici nelle interviste del corpus IS. ANNALI. SEZIONE GERMANICA. Rivista Del Dipartimento Di Studi Letterari, Linguistici e Comparati dell’Università degli studi di Napoli L’Orientale, 383–414. https://doi.org/10.6093/GERMANICA.V0I33.10752

    Conferences

    • Aggarwal, P., Mehrabanian, J., Huang, W., Alacam, Ö., & Zesch, T. (2024). Text or image? What is more important in cross-domain generalization capabilities of hate meme detection models? In Y. Graham & M. Purver (Eds.), 18th conference of the european chapter of findings of the association for computational linguistics (pp. 104–117). Association for Computational Linguistics. https://openreview.net/forum?id=RZYrUlRGv1
    • Bexte, M., Horbach, A., & Zesch, T. (2024). Rainbow – a benchmark for systematic testing of how sensitive visio-linguistic models are to color naming. In Y. Graham & M. Purver (Eds.), 18th conference of the european chapter of the association for computational linguistics (pp. 1858–1875). Association for Computational Linguistics. https://openreview.net/forum?id=0x4Bg1sNDo

    2023

    Journals

    • Wisniewski, K., Zesch, T., Schwendemann, M., Ruppenhofer, J., & Portmann, A. (2023). Automatische Analysen von Erwerbsstufen in einer großen Lernerkorpus-Datenbank für DaF/DaZ. Das Forschungsprojekt DAKODA. Korpora Deutsch Als Fremdsprache, 3(2). https://doi.org/10.48694/kordaf.3845
    • Zesch, T., Horbach, A., & Zehner, F. (2023). To score or not to score: Factors influencing performance and feasibility of automatic content scoring of text responses. Educational Measurement: Issues and Practice, 42(1), 44–58. https://doi.org/10.1111/emip.12544

    Conferences

    • Aggarwal, P., Chawla, P., Das, M., Saha, P., Mathew, B., Zesch, T., & Mukherjee, A. (2023). HateProof: Are hateful meme detection systems really robust? Proceedings of the ACM Web Conference 2023, 3734–3743. https://doi.org/10.1145/3543507.3583356
    • Bexte, M., Horbach, A., & Zesch, T. (2023). Similarity-based content scoring - a more classroom-suitable alternative to instance-based scoring? Findings of the Association for Computational Linguistics: ACL 2023, 1892–1903. https://aclanthology.org/2023.findings-acl.119
    • Ding, Y., Bexte, M., & Horbach, A. (2023). Score it all together: A multi-task learning study on automatic scoring of argumentative essays. Findings of the Association for Computational Linguistics: ACL 2023, 13052–13063. https://aclanthology.org/2023.findings-acl.825
    • Gold, C., Laarmann-Quante, R., & Zesch, T. (2023a). Preserving the authenticity of handwritten learner language: Annotation guidelines for creating transcripts retaining orthographic features. 1st Computation and Written Language (CAWL) Workshop at ACL.
    • Gold, C., Laarmann-Quante, R., & Zesch, T. (2023b). Recognizing learner handwriting retaining orthographic errors for enabling fine-grained error feedback. Innovative Use of NLP for Building Educational Applications (BEA) Workshop at ACL.
    • Kupietz, M., Fankhauser, P., & Ruppenhofer, J. (2023). A distributional comparison between FOLK and DeReKo. The Twelfth International Corpus Linguistics Conference 2023. Lancaster University, Monday 3rd-Thursday 6th July, 2023. Book of Abstracts, 155–155.
    • Wiegand, M., Kampfmeier, J., Eder, E., & Ruppenhofer, J. (2023). Euphemistic abuse – a new dataset and classification experiments for implicitly abusive language. In H. Bouamor, J. Pino, & K. Bali (Eds.), Proceedings of the 2023 conference on empirical methods in natural language processing (pp. 16280–16297). Association for Computational Linguistics. https://doi.org/10.18653/v1/2023.emnlp-main.1012

    Proceedings

    • Kochmar, E., Burstein, J., Horbach, A., Laarmann-Quante, R., Madnani, N., Tack, A., Yaneva, V., Yuan, Z., & Zesch, T. (2023). Proceedings of the 18th workshop on innovative use of NLP for building educational applications (BEA 2023).

    Talks and Poster Presentations

    • Zehner, F., Zesch, T., & Horbach, A. (2023a, February 28–March 2). Mehr als nur Technologie- und Fairnessfrage: Ethische Prinzipien beim automatischen Bewerten von Textantworten aus Tests [Paper Presentation]. 10th GEBF Annual conference, Universität Duisburg-Essen.
    • Zehner, F., Zesch, T., & Horbach, A. (2023b, February 28–March 2). To score or not to score? Machbarkeits- und performanzfaktoren für automatisches scoring von textantworten [Paper Presentation]. 10th GEBF annual conference, Universität Duisburg-Essen.

    2022

    Journals

    • Sanguinetti, M., Bosco, C., Cassidy, L., Çetinoğlu, Ö., Cignarella, A. T., Lynn, T., Rehbein, I., Ruppenhofer, J., Seddah, D., & Zeldes, A. (2022). Treebanking user-generated content: A UD based overview of guidelines, corpora and unified recommendations. Language Resources and Evaluation, 57(2), 493–544. https://doi.org/10.1007/s10579-022-09581-9

    Conferences

    • Aggarwal, P., & Zesch, T. (2022a). Analyzing the real vulnerability of hate speech detection systems against targeted intentional noise. Proceedings of the Eighth Workshop on Noisy User-Generated Text (w-NUT 2022), 230–242. https://aclanthology.org/2022.wnut-1.25
    • Aggarwal, P., & Zesch, T. (2022b). Bye, bye, maintenance work? Using model cloning to approximate the behavior of legacy tools. Proceedings of the 18th Conference on Natural Language Processing (KONVENS 2022), 175–180. https://aclanthology.org/2022.konvens-1.21
    • Bexte, M., Horbach, A., & Zesch, T. (2022). Similarity-based content scoring - how to make S-BERT keep up with BERT. Proceedings of the 17th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2022), 118–123. https://aclanthology.org/2022.bea-1.16
    • Bexte, M., Laarmann-Quante, R., Horbach, A., & Zesch, T. (2022). LeSpell - a multi-lingual benchmark corpus of spelling errors to develop spellchecking methods for learner language. Proceedings of the Language Resources and Evaluation Conference, 697–706. https://aclanthology.org/2022.lrec-1.73
    • Ding, Y., Bexte, M., & Horbach, A. (2022). Don’t drop the topic - the role of the prompt in argument identification in student writing. Proceedings of the 17th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2022), 124–133. https://aclanthology.org/2022.bea-1.17
    • Gold, C., & Zesch, T. (2022, December). CNN-based ruled line removal in handwritten documents. Proceedings of the 18th International Conference on Frontiers of Handwriting Recognition (ICFHR 2022).
    • Horbach, A., Laarmann-Quante, R., Liebenow, L., Jansen, T., Keller, S., Meyer, J., Zesch, T., & Fleckenstein, J. (2022). Bringing automatic scoring into the classroom–measuring the impact of automated analytic feedback on student writing performance. Swedish Language Technology Conference and NLP4CALL, 72–83. https://ecp.ep.liu.se/index.php/sltc/article/view/580/550
    • Laarmann-Quante, R., Prepens, L., & Zesch, T. (2022). Evaluating automatic spelling correction tools on german primary school children’s misspellings. Swedish Language Technology Conference and NLP4CALL, 95–107. https://ecp.ep.liu.se/index.php/sltc/article/download/582/552
    • Laarmann-Quante, R., Schwarz, L., Horbach, A., & Zesch, T. (2022). ‘Meet me at the ribary’ – acceptability of spelling variants in free-text answers to listening comprehension prompts. Proceedings of the 17th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2022), 173–182. https://aclanthology.org/2022.bea-1.22
    • Ludwig, F., Dolos, K., Zesch, T., & Hobley, E. (2022). Improving generalization of hate speech detection systems to novel target groups via domain adaptation. Proceedings of the Sixth Workshop on Online Abuse and Harms (WOAH), 29–39. https://doi.org/10.18653/v1/2022.woah-1.4
    • Zesch, T., & Bewersdorff, J. (2022). German medical natural language processing–a data-centric survey. In C. Reich & U. Mescheder (Eds.), The upper-rhine artificial intelligence symposium UR-AI 2022 : AI applications in medicine and manufacturing, 19 october 2022, villingen-schwenningen, germany (pp. 137–145). Furtwangen University.
    • Zufall, F., Hamacher, M., Kloppenborg, K., & Zesch, T. (2022). A legal approach to hate speech – operationalizing the EU’s legal framework against the expression of hatred as an NLP task. Proceedings of the Natural Legal Language Processing Workshop 2022, 53–64. https://aclanthology.org/2022.nllp-1.5

    Proceedings

    • Kochmar, E., Burstein, J., Horbach, A., Laarmann-Quante, R., Madnani, N., Tack, A., Yaneva, V., Yuan, Z., & Zesch, T. (Eds.). (2022). Proceedings of the 17th workshop on innovative use of NLP for building educational applications (BEA 2022). Association for Computational Linguistics. https://aclanthology.org/2022.bea-1.0

    Software

    2021

    Conferences

    • Aggarwal, P., Liman, M. E., Gold, D., & Zesch, T. (2021). VL-BERT+: Detecting Protected Groups in Hateful Multimodal Memes. Proceedings of the 5th Workshop on Online Abuse and Harms (WOAH 2021), 207–214. https://doi.org/10.18653/v1/2021.woah-1.22
    • Bexte, M., Horbach, A., & Zesch, T. (2021). Implicit Phenomena in Short-answer Scoring Data. Proceedings of the First Workshop on Understanding Implicit and Underspecified Language.
    • Gold, C., Boom, D. van den, & Zesch, T. (2021). Personalizing Handwriting Recognition Systems with Limited User-Specific Samples. Proceedings of the 16th International Conference on Document Analysis and Recognition (ICDAR 2021).
    • Haring, C., Lehmann, R., Horbach, A., & Zesch, T. (2021). C-Test Collector: A Proficiency Testing Application to Collect Training Data for C-Tests. Proceedings of the 16th Workshop on Innovative Use of NLP for Building Educational Applications, 180–184. https://www.aclweb.org/anthology/2021.bea-1.19
    • Pham, D. D., Müller, J., Aggarwal, P., Khatri, A., Sharma, M., Zesch, T., & Pauli, J. (2021, January). Fully vs. Weakly Supervised Caries Localization in Smartphone Images with CNNs. Artificial Intelligence for Healthcare Applications International Workshop - ICPR 2020 Workshop Proceedings.
    • Schäfer, H., Idrissi-Yaghir, A., Galetzka, W., Bexte, M., & Friedrich, C. M. (2021). WisPerMed text at TREC clinical trials track 2021. In 30th Text REtrieval Conference, 7.

    Journals

    Proceedings

    • Burstein, J., Horbach, A., Kochmar, E., Laarmann-Quante, R., Leacock, C., Madnani, N., Pilán, I., Yannakoudakis, H., & Zesch, T. (Eds.). (2021). Proceedings of the 16th Workshop on Innovative Use of NLP for Building Educational Applications. Association for Computational Linguistics. https://www.aclweb.org/anthology/2021.bea-1.0

    2020

    Conferences

    • Ding, Y., Horbach, A., Wang, H., Song, X., & Zesch, T. (2020). Chinese Content Scoring: Open-Access Datasets and Features on Different Segmentation Levels. Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing(AACL-IJCNLP 2020). https://www.aclweb.org/anthology/2020.aacl-main.37.pdf
    • Ding, Y., Riordan, B., Horbach, A., Cahill, A., & Zesch, T. (2020). Don’t take "nswvtnvakgxpm" for an answer - The surprising vulnerability of automatic content scoring systems to adversarial input. Proceedings of the 28th International Conference on Computational Linguistics(COLING 2020). https://www.aclweb.org/anthology/2020.coling-main.76.pdf
    • Gold, C., & Zesch, T. (2020). Exploring the Impact of Handwriting Recognition on the Automated Scoring of Handwritten Student Answers. Proceedings of the 17th International Conference on Frontiers in Handwriting Recognition (ICFHR 2020). https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9257760
    • Horbach, A., Aldabe, I., Bexte, M., Lacalle, O. de, & Maritxalar, M. (2020). Appropriateness and Pedagogic Usefulness of Reading Comprehension Questions. Proceedings of the 12th International Conference on Language Resources and Evaluation (LREC-2020). https://www.aclweb.org/anthology/2020.lrec-1.217.pdf
    • Kovatchev, V., Gold, D., Marti, M. A., Salamo, M., & Zesch, T. (2020). Decomposing and Comparing Meaning Relations: Paraphrasing, Textual Entailment, Contradiction, and Specificity. Proceedings of the 12th International Conference on Language Resources and Evaluation (LREC-2020). https://www.aclweb.org/anthology/2020.lrec-1.709.pd

    Journals

    2019

    Conferences

    Journals

    2018

    Conferences

    • Gold, D., Bexte, M., & Zesch, T. (2018). Corpus of Aspect-based Sentiment in Political Debates. KONVENS, 89–99.
    • Hamed, O., & Zesch, T. (2018). The Role of Diacritics in Increasing the Difficulty of Arabic Lexical Recognition Tests. Proceedings of the 7th Workshop on NLP for Computer Assisted Language Learning at SLTC 2018 (NLP4CALL 2018), 23–31. http://www.ep.liu.se/ecp/152/003/ecp18152003.pdf
    • Horbach, A., Stennmanns, S., & Zesch, T. (2018). Cross-lingual Content Scoring. Proceedings of the Thirteenth Workshop on Innovative Use of NLP for Building Educational Applications, 410–419. http://www.aclweb.org/anthology/W18-0550
    • Horsmann, T., & Zesch, T. (2018). DeepTC–An Extension of DKPro Text Classification for Fostering Reproducibility of Deep Learning Experiments. Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC-2018). https://www.aclweb.org/anthology/L18-1403
    • Wojatzki, M., Horsmann, T., Gold, D., & Zesch, T. (2018). Do Women Perceive Hate Differently: Examining the Relationship Between Hate Speech, Gender, and Agreement Judgments. Proceedings of the 14th Conference on Natural Language Processing (KONVENS 2018). https://konvens.org/proceedings/2018/PDF/konvens18_13.pdf
    • Wojatzki, M., Mohammad, S., Zesch, T., & Kiritchenko, S. (2018). Quantifying qualitative data for understanding controversial issues. Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC-2018). https://www.aclweb.org/anthology/L18-1224
    • Zesch, T., & Horbach, A. (2018). ESCRITO - An NLP-Enhanced Educational Scoring Toolkit. Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC-2018). http://www.lrec-conf.org/proceedings/lrec2018/pdf/590.pdf
    • Zesch, T., Horbach, A., Goggin, M., & Wrede-Jackes, J. (2018). A flexible online system for curating reduced redundancy language exercises and tests. In P. Taalas, J. Jalkanen, L. Bradley, & S. Thouësny (Eds.), Future-proof CALL: Language learning as exploration and encounters - short papers from EUROCALL 2018 (pp. 319–324). https://doi.org/10.14705/rpnet.2018.26.857

    2017

    Conferences

    • Benikova, D., Wojatzki, M., & Zesch, T. (2017). What does this imply? Examining the Impact of Implicitness on the Perception of Hate Speech. Proceedings of the International Conference of the German Society for Computational Linguistics and Language Technology (GSCL-2017), 171–179. https://link.springer.com/content/pdf/10.1007%2F978-3-319-73706-5_14.pdf
    • Benikova, D., & Zesch, T. (2017). Same same, but different: Compositionality of paraphrase granularity levels. Proceedings of the Recent Advances in Natural Language Processing (RANLP-2017), 90–96. http://acl-bg.org/proceedings/2017/RANLP 2017/pdf/RANLP014.pdf
    • Hamed, O., & Zesch, T. (2017). The Role of Diacritics in Designing Lexical Recognition Tests for Arabic. 3rd International Conference on Arabic Computational Linguistics (ACLing 2017). http://www.ltl.uni-due.de/wp-content/uploads/ACLing-2017.pdf
    • Horbach, A., Ding, Y., & Zesch, T. (2017). The Influence of Spelling Error on Content Scoring Performance. Proceedings of the 4th Workshop on Natural Language Processing Techniques for Educational Applications, 45–53. http://www.aclweb.org/anthology/W17-5908
    • Horbach, A., Scholten-Akoun, D., Ding, Y., & Zesch, T. (2017). Fine-grained essay scoring of a complex writing task for native speakers. Proceedings of the Building Educational Applications Workshop at EMNLP, 357–366. http://aclweb.org/anthology/W17-5040
    • Horsmann, T., & Zesch, T. (2017). Do LSTMs really work so well for PoS tagging? – A replication study. Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), 738–747. https://www.aclweb.org/anthology/D17-1076
    • Riordan, B., Horbach, A., Cahill, A., Zesch, T., & Lee, C. M. (2017). Investigating neural architectures for short answer scoring. Proceedings of the Building Educational Applications Workshop at EMNLP, 159–168. http://www.aclweb.org/anthology/W17-5017
    • Wojatzki, M., Ruppert, E., Holschneider, S., Zesch, T., & Biemann, C. (2017). GermEval 2017: Shared Task on Aspect-based Sentiment in Social Media Customer Feedback. Proceedings of the GermEval 2017 Shared Task on Aspect-Based Sentiment in Social Media Customer Feedback, 1–12. https://www.ltl.uni-due.de/wp-content/uploads/germeval-2017.pdf
    • Wojatzki, M., & Zesch, T. (2017). Neural, Non-neural and Hybrid Stance Detection in Tweets on Catalan Independence. Stance and Gender Detection in Tweets on Catalan Independence at Ibereval 2017, 2. https://pdfs.semanticscholar.org/4682/101bd6c5a653f23c38726c5c046aed5c9e60.pdf

    Journals

    2016

    Conferences

    • Beinborn, L., Zesch, T., & Gurevych, I. (2016). Predicting the spelling difficulty of words for language learners. Proceedings of the Building Educational Applications Workshop at NAACL.
    • Bethard, S., Carpuat, M., Cer, D., Jurgens, D., Nakov, P., & Zesch, T. (2016). Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016). Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016).
    • Bethard, S., Cer, D., Carpuat, M., Jurgens, D., Nakov, P., & Zesch, T. (2016). Welcome to SemEval-2016. SemEval 2016-10th International Workshop on Semantic Evaluation, Proceedings, iii–iv.
    • Horsmann, T., & Zesch, T. (2016a). Assigning fine-grained PoS tags based on high-precision coarse-grained tagging. Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, 328–336.
    • Horsmann, T., & Zesch, T. (2016b). Building a social media adapted PoS tagger using FlexTag–a case study on italian tweets. Fifth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian - EVALITA 2016, Naples, Italy.
    • Horsmann, T., & Zesch, T. (2016c). LTLUDE@ EmpiriST 2015: Tokenization and PoS tagging of social media text. Proceedings of the 10th Web as Corpus Workshop (WAC-x) and the EmpiriST Shared Task. Association for Computational Linguistics, Berlin, 120–126.
    • Meyer, N., Wojatzki, M., & Zesch, T. (2016). Validating bundled gap filling–empirical evidence for ambiguity reduction and language proficiency testing capabilities. Proceedings of the Joint Workshop on NLP for Computer Assisted Language Learning and NLP for Language Acquisition at SLTC, Umeå, 16th November 2016, 51–59.
    • Pilán, I., Volodina, E., & Zesch, T. (2016). Predicting proficiency levels in learner writings by transferring a linguistic complexity model from expert-written coursebooks. Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, Osaka, Japan, pp–2101.
    • Wojatzki, M., Melamud, O., & Zesch, T. (2016). Bundled gap filling: A new paradigm for unambiguous cloze exercises. Proceedings of the Building Educational Applications Workshop at NAACL.
    • Zesch, T., & Horsmann, T. (2016). Flextag: A highly flexible pos tagging framework. Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), 4259–4263.

    Journals

    • Benikova, D., & Zesch, T. (2016). Bridging the gap between computable and expressive event representations in social media. EMNLP 2016, 6.
    • Nakov, P., & Zesch, T. (2016). Computational semantic analysis of language: SemEval-2014 and beyond. Language Resources and Evaluation, 50, 1–4.
    • Wojatzki, M., & Zesch, T. (2016a). Ltl. Uni-due at SemEval-2016 task 6: Stance detection in social media using stacked classifiers. Proceedings of SemEval, 428–433.
    • Wojatzki, M., & Zesch, T. (2016b). Stance-based argument mining–modeling implicit argumentation using stance. Bochumer Linguistische Arbeitsberichte, 313.

    2015

    Conferences

    • Beinborn, L., Zesch, T., & Gurevych, I. (2015a). Candidate evaluation strategies for improved difficulty prediction of language tests. Proceedings of the Tenth Workshop on Innovative Use of NLP for Building Educational Applications Held in Conjunction with NAACL 2015, 1–11.
    • Beinborn, L., Zesch, T., & Gurevych, I. (2015b). Factors of difficulty in german language proficiency tests. Book of Abstracts: Language, Learning, Technology Conference.
    • Hamed, O., & Zesch, T. (2015). Generating nonwords for vocabulary proficiency testing. Proceeding of the 7th Language and Technology Conference: Human Language Technologies as a Challenge for Computer Science and Linguistics, Poznań, Poland.
    • Horsmann, T., Erbs, N., & Zesch, T. (2015). Fast or accurate?–a comparative evaluation of PoS tagging models. Proceedings of the International Conference of the German Society for Computational Linguistics and Language Technology (GSCL-2015).
    • Horsmann, T., & Zesch, T. (2015). Effectiveness of domain adaptation approaches for social media PoS tagging. Proceedings of the Second Italian Conference on Computational Linguistics CLiC-It 2015, 166.
    • Nakov, P., Zesch, T., Cer, D., & Jurgens, D. (2015). Proceedings of the 9th international workshop on semantic evaluation (SemEval 2015). Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015).
    • Zesch, T., Heilman, M., & Cahill, A. (2015). Reducing annotation efforts in supervised short answer scoring. Proceedings of the Tenth Workshop on Innovative Use of NLP for Building Educational Applications, 124–132.
    • Zesch, T., Wojatzki, M., & Scholten-Akoun, D. (2015). Task-independent features for automated essay grading. Proceedings of the Tenth Workshop on Innovative Use of NLP for Building Educational Applications, 224–232.

    Journals

    • Erbs, N., Santos, P. B., Zesch, T., & Gurevych, I. (2015). Counting what counts: Decompounding for keyphrase extraction. ACL-IJCNLP 2015, 10.

    Talks and Poster Presentations

    • Bär, D., Zesch, T., & Gurevych, I. (2015). Composing measures for computing text similarity. Universitäts-und Landesbibliothek Darmstadt.

    2014

    Conferences

    • Daxenberger, J., Ferschke, O., Gurevych, I., Zesch, T., et al. (2014). DKPro TC: A java-based framework for supervised learning experiments on textual data. ACL (System Demonstrations), 61–66.
    • Erbs, N., Santos, P. B., Gurevych, I., & Zesch, T. (2014). DKPro keyphrases: Flexible and reusable keyphrase extraction experiments. ACL (System Demonstrations), 31–36.
    • Zesch, T., & Melamud, O. (2014). Automatic generation of challenging distractors using context-sensitive inference rules. Proceedings of the Ninth Workshop on Innovative Use of NLP for Building Educational Applications, 143–148.

    Journals

    • Beinborn, L., Zesch, T., & Gurevych, I. (2014a). Predicting the difficulty of language proficiency tests. Transactions of the Association for Computational Linguistics, 2, 517–529.
    • Beinborn, L., Zesch, T., & Gurevych, I. (2014b). Readability for foreign language learning: The importance of cognates. International Journal of Applied Linguistics, 165(2), 136–162.
    • Erbs, N., Gurevych, I., & Zesch, T. (2014). Sense and similarity: A study of sense-level similarity measures. Lexical and Computational Semantics (* SEM 2014), 30.
    • Horsmann, T., & Zesch, T. (2014). Towards automatic scoring of cloze items by selecting low-ambiguity contexts. NEALT Proceedings Series Vol. 22, 33–42.

    2013

    Conferences

    • Bär, D., Zesch, T., & Gurevych, I. (2013). DKPro similarity: An open source framework for text similarity. Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics: System Demonstrations, 121–126.
    • Beinborn, L., Zesch, T., & Gurevych, I. (2013). Cognate production using character-based machine translation. IJCNLP, 883–891.
    • Erbs, N., Gurevych, I., & Zesch, T. (2013). Hierarchy identification for automatically generating table-of-contents. Proceedings of the International Conference Recent Advances in Natural Language Processing RANLP 2013, 252–260.
    • Gurevych, I., Biemann, C., & Zesch, T. (2013). Language processing and knowledge in the web. LNCS/LNAI: 25th International Conference of the GSCL, 8105.
    • Korkontzelos, I., Zesch, T., Zanzotto, F. M., & Biemann, C. (2013). SemEval-2013 task 5: Evaluating phrasal semantics. Proceedings of the 7th International Workshop on Semantic Evaluation (SemEval 2013).
    • Levy, O., Zesch, T., Dagan, I., & Gurevych, I. (2013). Recognizing partial textual entailment. ACL (2), 451–455.
    • Miller, T., Erbs, N., Zorn, H.-P., Zesch, T., & Gurevych, I. (2013). DKPro WSD: A generalized UIMA-based framework for word sense disambiguation. ACL (Conference System Demonstrations), 37–42.
    • Zesch, T., Levy, O., Gurevych, I., & Dagan, I. (2013). UKP-BIU: Similarity and entailment metrics for student response analysis. Proceedings of the 17th International Workshop on Semantic Evaluation, 2, 285–289.

    Journals

    • Biemann, C., Bildhauer, F., Evert, S., Goldhahn, D., Quasthoff, U., Schäfer, R., Simon, J., Swiezinski, L., & Zesch, T. (2013). Scalable construction of high-quality web corpora. Journal for Language Technology and Computational Linguistics, 28(2), 23–60.
    • Gurevych, I., & Zesch, T. (2013). Collective intelligence and language resources: Introduction to the special issue on collaboratively constructed language resources. Language Resources and Evaluation, 47(1), 1.
    • Zesch, T. (2013). Detecting malapropisms using measures of contextual fitness. Special Issue of the TAL Journal on “Managing Noise in the Signal: Error Handling in Natural Language Processing.” 53:3, 53(3), 11–31.

    2012

    Conferences

    • Bär, D., Biemann, C., Gurevych, I., & Zesch, T. (2012). UKP: Computing semantic textual similarity by combining multiple content similarity measures. Proceedings of the 6th International Workshop on Semantic Evaluation, Held in Conjunction with the 1st Joint Conference on Lexical and Computational Semantics, 435–440.
    • Bär, D., Zesch, T., & Gurevych, I. (2012). Text reuse detection using a composition of text similarity measures. Proceedings of COLING 2012, 167–184.
    • Beinborn, L., Zesch, T., & Gurevych, I. (2012). Towards fine-grained readability measures for self-directed language learning. Proceedings of the SLTC 2012 Workshop on NLP for CALL, 80, 11–19.
    • Erbs, N., Agirre, E., Soroa, A., Barrena, A., Etxebarria, U., Gurevych, I., & Zesch, T. (2012). UKP-UBC entity linking at TAC-KBP. Proceedings of the Fifth Text Analysis Conference (TAC).
    • Miller, T., Biemann, C., Zesch, T., & Gurevych, I. (2012). Using distributional similarity for lexical expansion in knowledge-based word sense disambiguation. COLING, 1781–1796.
    • Zesch, T. (2012). Measuring contextual fitness using error contexts extracted from the wikipedia revision history. Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, 529–538.
    • Zesch, T., & Haase, J. (2012). HOO 2012 shared task: UKP lab system description. Proceedings of the Seventh Workshop on Building Educational Applications Using NLP.

    2011

    Conferences

    • Bär, D., Erbs, N., Zesch, T., & Gurevych, I. (2011). Wikulu: An extensible architecture for integrating natural language processing techniques with wikis. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: Systems Demonstrations, 74–79.
    • Bär, D., Zesch, T., & Gurevych, I. (2011). A reflective view on text similarity. Proceedings of the International Conference on Recent Advances in Natural Language Processing, 515–520.
    • Erbs, N., Bär, D., Gurevych, I., & Zesch, T. (2011). First aid for information chaos in wikis-collaborative information management enhanced through language technology. ISI, 501–502.
    • Erbs, N., Zesch, T., & Gurevych, I. (2011). Link discovery: A comprehensive analysis. 2011 IEEE Fifth International Conference on Semantic Computing, 83–86.
    • Ferschke, O., Zesch, T., & Gurevych, I. (2011). Wikipedia revision toolkit: Efficiently accessing wikipedia’s edit history. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: Systems Demonstrations, 97–102.
    • Stille, W., Erbs, N., Zesch, T., & Weihe, K. (2011). Aufbereitung und strukturierung von information mittels automatischer sprachverarbeitung. Proceedings of KnowTech, p–199.
    • Szarvas, G., Zesch, T., & Gurevych, I. (2011). Combining heterogeneous knowledge resources for improved distributional semantic models. International Conference on Intelligent Text Processing and Computational Linguistics, 289–303.
    • Zesch, T. (2011). Helping our own 2011: UKP lab system description. Proceedings of the Generation Challenges Session at the 13th European Workshop on Natural Language Generation, 260–262.

    2010

    Conferences

    • Gurevych, I., & Zesch, T. (2010). Proceedings of the 2nd workshop on the people’s web meets NLP: Collaboratively constructed semantic resources. Proceedings of the 2nd Workshop on the People’s Web Meets NLP: Collaboratively Constructed Semantic Resources.
    • Zesch, T. (2010). What’s the difference?–comparing expert-built and collaboratively-built lexical semantic resources. FLaReNet Forum. Barcelona.
    • Zesch, T., & Gurevych, I. (2010). The more the better? Assessing the influence of wikipedia’s growth on semantic relatedness measures. Proceedings of the Conference on Language Resources and Evaluation (LREC), Valletta, Malta.

    Journals

    • Zesch, T., & Gurevych, I. (2010). Wisdom of crowds versus wisdom of linguists–measuring the semantic relatedness of words. Natural Language Engineering, 16(1), 25.

    2009

    Conferences

    • Gurevych, I., & Zesch, T. (2009). Proceedings of the 2009 workshop on the people’s web meets NLP: Collaboratively constructed semantic resources (people’s web). Proceedings of the 2009 Workshop on the People’s Web Meets NLP: Collaboratively Constructed Semantic Resources (People’s Web).
    • Hoffart, J., Bär, D., Zesch, T., & Gurevych, I. (2009). Discovering links using semantic relatedness. Preproceedings of the INEX Workshop, 314–325.
    • Hoffart, J., Zesch, T., & Gurevych, I. (2009). An architecture to support intelligent user interfaces for wikis by means of natural language processing. Proceedings of the 5th International Symposium on Wikis and Open Collaboration, 12.
    • Zesch, T., & Gurevych, I. (2009). Approximate matching for evaluating keyphrase extraction. Proceedings of the 7th International Conference on Recent Advances in Natural Language Processing, 484–489.

    Journals

    • Čulo, O., Kunz, K., & Zesch, T. (2009). Semantic relations in a bilingual corpus of different registers.

    2008

    Conferences

    • Garoufi, K., Zesch, T., Gurevych, I., et al. (2008). Graph-theoretic analysis of collaborative knowledge bases in natural language processing. Poster and Demo Session Proceedings of the 7th International Semantic Web Conference.
    • Garoufi, K., Zesch, T., & Gurevych, I. (2008). Representational interoperability of linguistic and collaborative knowledge bases. Proceedings of the KONVENS Workshop on Lexical-Semantic and Ontological Resources–Maintenance, Representation, and Standards, Berlin, Germany.
    • Gurevych, I., & Zesch, T. (2008). Selbstorganisierende wikis. Proceedings of KnowTech, 317–324.
    • Hartmann, M., Zesch, T., Muhlhauser, M., & Gurevych, I. (2008). Using similarity measures for context-aware user interfaces. Semantic Computing, 2008 IEEE International Conference on, 190–197.
    • Müller, C., Zesch, T., Müller, M.-C., Bernhard, D., Ignatova, K., Gurevych, I., & Mühlhäuser, M. (2008). Flexible UIMA components for information retrieval research. Proceedings of the LREC 2008 Workshop’towards Enhanced Interoperability for Large HLT Systems: UIMA for NLP, 24–27.
    • Zesch, T., Müller, C., & Gurevych, I. (2008a). Extracting lexical semantic knowledge from wikipedia and wiktionary. Proceedings of the Conference on Language Resources and Evaluation (LREC), 15, 60.
    • Zesch, T., Müller, C., & Gurevych, I. (2008b). Using wiktionary for computing semantic relatedness. Proceedings of AAAI, 2008, 45.

    Journals

    • Gurevych, I., Müller, C., Zesch, T., & Mühlhäuser, M. (2008). Semantic information retrieval. Journal of Semantic Computing, 2(2), 253–272.

    2007

    Conferences

    • Gurevych, I., Mühlhäuser, M., Müller, C., Steimle, J., Weimer, M., & Zesch, T. (2007). Darmstadt knowledge processing repository based on UIMA. Proceedings of the First Workshop on Unstructured Information Management Architecture at Biannual Conference of the GSCL.
    • Gurevych, I., Muler, C., & Zesch, T. (2007). What to be?-electronic career guidance based on semantic relatedness. ANNUAL MEETING-ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, 45, 1032.
    • Gurevych, I., Müller, C., & Zesch, T. (2007). Teaching “unstructured information management: Theory and applications” to computational linguistics students. Proceedings of the First Workshop on Unstructured Information Management Architecture at Biannual Conference of the Society for Computational Linguistics and Language Technology.
    • Mohammad, S., Gurevych, I., Hirst, G., & Zesch, T. (2007). Cross-lingual distributional profiles of concepts for measuring semantic distance. Proceedings of the Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP/CoNLL-2007), 571–580.
    • Zesch, T., & Gurevych, I. (2007). Analysis of the wikipedia category graph for NLP applications. Proceedings of the TextGraphs-2 Workshop (NAACL-HLT 2007), 1–8.
    • Zesch, T., Gurevych, I., & Mühlhäuser, M. (2007). Comparing wikipedia and german wordnet by evaluating semantic relatedness on multiple datasets. Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Companion Volume, Short Papers, 205–208.

    Journals

    • Zesch, T., Gurevych, I., & Mühlhäuser, M. (2007). Analyzing and accessing wikipedia as a lexical semantic resource. Data Structures for Linguistic Resources and Applications, 197–205.

    2006

    Conferences

    • Zesch, T., & Gurevych, I. (2006). Automatically creating datasets for measures of semantic relatedness. Proceedings of the Workshop on Linguistic Distances, 16–24.

  • On this page you will find an overview of freely available data sets for the automatic scoring of free texts from the areas of automatic essay scoring and the assessment of content and short answers.

    List with records for Content Scoring / Essay Scoring

  • Thesis template

    Bachelor theses

    „Prediction of strong negative reactions to text posts in social medias“, Soriano Tapia, Isaac, abgeschlossen 2023

    „Design and implementation of a conversational AI to support foreign language acquisition“, Linder, Pia Selina, abgeschlossen 2023

    „Vergleichsstudie zu Grammatical Error Correction im Deutschen: Eine Analyse des Forschungsstands und ein Qualitätsvergleich mit Large Language Models“, Hüsing, Jonathan, abgeschlossen 2023

    Master theses

    „Evaluierung der Wirksamkeit von 'Hateful Meme Detection' durch gezielte Erweiterung des Testdatensatzes“ Mehrabanian, Jawar, abgeschlossen 2023

    „Automatische Erstellung von Trainingsdaten für die Bewertung von Freitextaufgaben mittels generative pre-trained language models“, Birkholz, Ulrich, abgeschlossen 2023

    „Truecasing German texts and its effect on downstream NLP tasks“, Go, Yongryul, 2024

    „From difficult to easy: developing a language model for translating official into easy language“, Müller, Max, 2024

    „Entwicklung eines Werkzeugs zur Interpretation der Ergebnisse von SBERT am Fallbeispiel von automatisch generierten Freitextbewertungen“, Schuljak, Jörg, 2024

    „LLM-based Systems for Research Question Assessment and Generation: A Conceptual Framework“, Hamberger, Mario, 2024

    „A Large Language Model approach to Oral Exam Simulations“, Kalle, Frederik, 2024

    „Recognizing specific groups in statements as an NLP classification task“, Kalleske, Stefanie, 2024

    „Automated Feedback Generation for Second Language Handwriting Exercises“, Koch, Anne, 2024

Sandra Kirschbaum | 19.04.2024