Andrea Horbach

Deutsch

Prof. Dr. Andrea Horbach Photo: Henrik Schipper

Prof. Dr. Andrea Horbach

Junior Research Group Leader "EduNLP"

Email: andrea.horbach


What is my role within CATALPA?

As a computational linguist, I lead the junior research group EduNLP. I find it fascinating how computers can analyze, understand and produce human language - even though language is so complex and ambiguous. I want to use natural language processing to help learners write better texts and enable teachers to evaluate texts more efficiently.

Why CATALPA?

As humans, communicating through language is the most natural choice in many situations. In online teaching, however, we often had to resort to "language-free" task formats because they can be evaluated more easily by a computer. In CATALPA, I am working towards online teaching that is driven by the requirements of the learners and not by technical feasibility.

    • Junior Research Group Leader, Center of Advanced Technology Assisted Learning and Predictive Analytics (CATALPA), FernUniversität in Hagen (since 12/2021)
    • Postdoctoral Research Associate, Language Technology Lab, University Duisburg-Essen (10/2016 - 11/2021)
    • PhD in Computational Linguistics, Saarland University, Saarbrücken (2018)
    • Research associate/ PhD student at the Department for Computational Linguistics, Saarland University, Saarbrücken (04/2010 – 09/2016)
    • Diplom in Computational Linguistics (equivalent to MSc), Saarland University, Saarbrücken (2008)

    • Natural Language Processing for educational applications
    • Automatic content and essay scoring
    • Feedback and exercise generation

  • 2024

    Conferences

    • Bexte, M., Horbach, A., & Zesch, T. (2024). Rainbow – a benchmark for systematic testing of how sensitive visio-linguistic models are to color naming. In Y. Graham & M. Purver (Eds.), 18th conference of the european chapter of the association for computational linguistics (pp. 1858–1875). Association for Computational Linguistics. https://openreview.net/forum?id=0x4Bg1sNDo

    Talks and Poster Presentations

    • Wehrhahn, F., Ding, Y., Gaschler, R., Zhao, F., & Horbach, A. (2024, June 26–28). Argumentative essay writing practice with automated feedback and highlighting. [Poster Presentation]. EARLI SIG WRITING 2024 – ways2write, Université Paris Nanterre, France.

    2023

    Journals

    • Zesch, T., Horbach, A., & Zehner, F. (2023). To score or not to score: Factors influencing performance and feasibility of automatic content scoring of text responses. Educational Measurement: Issues and Practice, 42(1), 44–58. https://doi.org/10.1111/emip.12544

    Conferences

    • Bexte, M., Horbach, A., & Zesch, T. (2023). Similarity-based content scoring - a more classroom-suitable alternative to instance-based scoring? Findings of the Association for Computational Linguistics: ACL 2023, 1892–1903. https://aclanthology.org/2023.findings-acl.119
    • Ding, Y., Bexte, M., & Horbach, A. (2023a). CATALPA_EduNLP at PragTag-2023. In M. Alshomary, C.-C. Chen, S. Muresan, J. Park, & J. Romberg (Eds.), Proceedings of the 10th workshop on argument mining (pp. 197–201). Association for Computational Linguistics. https://doi.org/10.18653/v1/2023.argmining-1.22
    • Ding, Y., Bexte, M., & Horbach, A. (2023b). Score it all together: A multi-task learning study on automatic scoring of argumentative essays. Findings of the Association for Computational Linguistics: ACL 2023, 13052–13063. https://aclanthology.org/2023.findings-acl.825
    • Ding, Y., Trüb, R., Fleckenstein, J., Keller, S., & Horbach, A. (2023). Sequence tagging in EFL email texts as feedback for language learners. Proceedings of the 12th Workshop on NLP for Computer Assisted Language Learning, 53–62.
    • Mousa, A., Laarmann-Quante, R., & Horbach, A. (2023). Manual and automatic identification of similar arguments in EFL learner essays. Proceedings of the 12th Workshop on NLP for Computer Assisted Language Learning, 85–93.

    Proceedings

    • Kochmar, E., Burstein, J., Horbach, A., Laarmann-Quante, R., Madnani, N., Tack, A., Yaneva, V., Yuan, Z., & Zesch, T. (2023). Proceedings of the 18th workshop on innovative use of NLP for building educational applications (BEA 2023).

    Talks and Poster Presentations

    • Zehner, F., Zesch, T., & Horbach, A. (2023a, February 28–March 2). Mehr als nur Technologie- und Fairnessfrage: Ethische Prinzipien beim automatischen Bewerten von Textantworten aus Tests [Paper Presentation]. 10th GEBF Annual conference, Universität Duisburg-Essen.
    • Zehner, F., Zesch, T., & Horbach, A. (2023b, February 28–March 2). To score or not to score? Machbarkeits- und performanzfaktoren für automatisches scoring von textantworten [Paper Presentation]. 10th GEBF annual conference, Universität Duisburg-Essen.

    2022

    Conferences

    • Bexte, M., Horbach, A., & Zesch, T. (2022). Similarity-based content scoring - how to make S-BERT keep up with BERT. Proceedings of the 17th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2022), 118–123. https://aclanthology.org/2022.bea-1.16
    • Bexte, M., Laarmann-Quante, R., Horbach, A., & Zesch, T. (2022). LeSpell - a multi-lingual benchmark corpus of spelling errors to develop spellchecking methods for learner language. Proceedings of the Language Resources and Evaluation Conference, 697–706. https://aclanthology.org/2022.lrec-1.73
    • Ding, Y., Bexte, M., & Horbach, A. (2022). Don’t drop the topic - the role of the prompt in argument identification in student writing. Proceedings of the 17th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2022), 124–133. https://aclanthology.org/2022.bea-1.17
    • Horbach, A., Laarmann-Quante, R., Liebenow, L., Jansen, T., Keller, S., Meyer, J., Zesch, T., & Fleckenstein, J. (2022). Bringing automatic scoring into the classroom–measuring the impact of automated analytic feedback on student writing performance. Swedish Language Technology Conference and NLP4CALL, 72–83. https://ecp.ep.liu.se/index.php/sltc/article/view/580/550
    • Laarmann-Quante, R., Schwarz, L., Horbach, A., & Zesch, T. (2022). ‘Meet me at the ribary’ – acceptability of spelling variants in free-text answers to listening comprehension prompts. Proceedings of the 17th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2022), 173–182. https://aclanthology.org/2022.bea-1.22

    Chapters in Edited Books

    • Horbach, A. (2022). Werkzeuge für die automatische Sprachanalyse. In M. Beißwenger, L. Lemnitzer, & C. Müller-Spitzer (Eds.), Forschen in der Linguistik. Eine Methodeneinführung für das Germanistik-Studium. Wilhelm Fink (UTB).

    2021

    Journals

    Conferences

    • Bexte, M., Horbach, A., & Zesch, T. (2021). Implicit Phenomena in Short-answer Scoring Data. Proceedings of the First Workshop on Understanding Implicit and Underspecified Language.
    • Haring, C., Lehmann, R., Horbach, A., & Zesch, T. (2021). C-Test Collector: A Proficiency Testing Application to Collect Training Data for C-Tests. Proceedings of the 16th Workshop on Innovative Use of NLP for Building Educational Applications, 180–184. https://www.aclweb.org/anthology/2021.bea-1.19

    2020

    Journals

    Conferences

    • Ding, Y., Horbach, A., Wang, H., Song, X., & Zesch, T. (2020). Chinese Content Scoring: Open-Access Datasets and Features on Different Segmentation Levels. Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing(AACL-IJCNLP 2020). https://www.aclweb.org/anthology/2020.aacl-main.37.pdf
    • Ding, Y., Riordan, B., Horbach, A., Cahill, A., & Zesch, T. (2020). Don’t take "nswvtnvakgxpm" for an answer - The surprising vulnerability of automatic content scoring systems to adversarial input. Proceedings of the 28th International Conference on Computational Linguistics(COLING 2020). https://www.aclweb.org/anthology/2020.coling-main.76.pdf
    • Horbach, A., Aldabe, I., Bexte, M., Lacalle, O. de, & Maritxalar, M. (2020). Appropriateness and Pedagogic Usefulness of Reading Comprehension Questions. Proceedings of the 12th International Conference on Language Resources and Evaluation (LREC-2020). https://www.aclweb.org/anthology/2020.lrec-1.217.pdf

    2019

    Journals

    2018

    Conferences

    • Horbach, A., & Pinkal, M. (2018). Semi-Supervised Clustering for Short Answer Scoring. LREC. http://www.lrec-conf.org/proceedings/lrec2018/pdf/427.pdf
    • Horbach, A., Stennmanns, S., & Zesch, T. (2018). Cross-lingual Content Scoring. Proceedings of the Thirteenth Workshop on Innovative Use of NLP for Building Educational Applications, 410–419. http://www.aclweb.org/anthology/W18-0550
    • Zesch, T., & Horbach, A. (2018). ESCRITO - An NLP-Enhanced Educational Scoring Toolkit. Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC-2018). http://www.lrec-conf.org/proceedings/lrec2018/pdf/590.pdf
    • Zesch, T., Horbach, A., Goggin, M., & Wrede-Jackes, J. (2018). A flexible online system for curating reduced redundancy language exercises and tests. In P. Taalas, J. Jalkanen, L. Bradley, & S. Thouësny (Eds.), Future-proof CALL: Language learning as exploration and encounters - short papers from EUROCALL 2018 (pp. 319–324). https://doi.org/10.14705/rpnet.2018.26.857

    2017

    Conferences

    • Horbach, A., Ding, Y., & Zesch, T. (2017). The Influence of Spelling Error on Content Scoring Performance. Proceedings of the 4th Workshop on Natural Language Processing Techniques for Educational Applications, 45–53. http://www.aclweb.org/anthology/W17-5908
    • Horbach, A., Scholten-Akoun, D., Ding, Y., & Zesch, T. (2017). Fine-grained essay scoring of a complex writing task for native speakers. Proceedings of the Building Educational Applications Workshop at EMNLP, 357–366. http://aclweb.org/anthology/W17-5040
    • Riordan, B., Horbach, A., Cahill, A., Zesch, T., & Lee, C. M. (2017). Investigating neural architectures for short answer scoring. Proceedings of the Building Educational Applications Workshop at EMNLP, 159–168. http://www.aclweb.org/anthology/W17-5017

    • For a complete list of my publications, please visit Google Scholar.

    • Zesch, T., Horbach, A., & Zehner, F. (2023) To Score or Not to Score: Factors Influencing Performance and Feasibility of Automatic Content Scoring of Text Responses. Educational Measurement: Issues and Practice. https://onlinelibrary.wiley.com/doi/full/10.1111/emip.12544

    • Horbach, A., Laarmann-Quante, R., Liebenow, L., Jansen, T., Keller, S., Meyer, J., ... & Fleckenstein, J. (2022, December). Bringing Automatic Scoring into the Classroom–Measuring the Impact of Automated Analytic Feedback on Student Writing Performance. In Swedish Language Technology Conference and NLP4CALL (pp. 72-83). https://ecp.ep.liu.se/index.php/sltc/article/view/580

    • Ding, Y., Bexte, M., & Horbach, A. (2022). Don’t Drop the Topic - The Role of the Prompt in Argument Identification in Student Writing. In Proceedings of the 17th Workshop on Innovative Use of NLP for Building Educational Applications. https://aclanthology.org/2022.bea-1.17/

    • Bexte, M., Horbach, A., & Zesch, T. (2022). Similarity-based Content Scoring - How to Make S-BERT Keep up with BERT. In Proceedings of the 17th Workshop on Innovative Use of NLP for Building Educational Applications.Laarmann-Quante, R., Schwarz, L., Horbach, A., & Zesch, T. (2022). https://aclanthology.org/2022.bea-1.16/

    • Meet me at the ribary’ – Acceptability of spelling variants in free-text answers to listening comprehension prompts. In Proceedings of the 17th Workshop on Innovative Use of NLP for Building Educational Applications. https://aclanthology.org/2022.bea-1.22/

    • Bexte, M., Laarmann-Quante, R., Horbach, A., & Zesch, T. (2022). LeSpell - A Multi-Lingual Benchmark Corpus of Spelling Errors to Develop Spellchecking Methods for Learner Language. In Proceedings of the 13th International Conference on Language Resources and Evaluation (LREC-2022). https://aclanthology.org/2022.lrec-1.73/

    • Bexte, M., Horbach, A., & Zesch, T. (2021). Implicit Phenomena in Short-answer Scoring Data. In Proceedings of the First Workshop on Understanding Implicit and Underspecified Language. https://aclanthology.org/2021.unimplicit-1.2/
    • Horbach, A., Aldabe, I., Bexte, M., Lopez de Lacalle, O., & Maritxalar, M. (2020). Appropriateness and Pedagogic Usefulness of Reading Comprehension Questions. In Proceedings of the 12th International Conference on Language Resources and Evaluation (LREC-2020). https://aclanthology.org/2020.lrec-1.217/
    • Ding, Y., Riordan, B., Horbach, A., Cahill, A., & Zesch, T. (2020). Don’t take “nswvtnvakgxpm” for an answer - The surprising vulnerability of automatic content scoring systems to adversarial input. In Proceedings of the 28th International Conference on Computational Linguistics(COLING 2020). https://aclanthology.org/2020.coling-main.76/
    • Ding, Y., Horbach, A., Wang, H., Song, X., & Zesch, T. (2020). Chinese Content Scoring: Open-Access Datasets and Features on Different Segmentation Levels. In Proceedings of the 1st conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing(AACL-IJCNLP 2020). https://aclanthology.org/2020.aacl-main.37/
    • Horbach, A., & Zesch, T. (2019). The Influence of Variance in Learner Answers on Automatic Content Scoring. Frontiers in Education, 4, 28. https://duepublico2.uni-due.de/servlets/MCRFileNodeServlet/duepublico_derivate_00047459/Horbach_Zesch_Influence_Variance.pdf
    • Zesch, T., Horbach, A., Goggin, M., & Wrede-Jackes, J. (2018). A flexible online system for curating reduced redundancy language exercises and tests. In P. Taalas, J. Jalkanen, L. Bradley, & S. Thouësny (Eds.), Future-proof CALL: language learning as exploration and encounters – short papers from EUROCALL 2018 (pp. 319–324). https://doi.org/10.14705/rpnet.2018.26.857
    • Horbach, A., Stennmanns, S., & Zesch, T. (2018). Cross-lingual Content Scoring. In Proceedings of the Thirteenth Workshop on Innovative Use of NLP for Building Educational Applications (pp. 410–419). New Orleans, LA, USA: Association for Computational Linguistics. http://www.aclweb.org/anthology/W18-0550
    • Horbach, A., & Pinkal, M. (2018). Semi-Supervised Clustering for Short Answer Scoring. In LREC. Miyazaki, Japan. http://www.lrec-conf.org/proceedings/lrec2018/pdf/427.pdf
    • Zesch, T., & Horbach, A. (2018). ESCRITO - An NLP-Enhanced Educational Scoring Toolkit. In Proceedings of the Language Resources and Evaluation Conference (LREC). Miyazaki, Japan: European Language Resources Association (ELRA). http://www.lrec-conf.org/proceedings/lrec2018/pdf/590.pdf
    • Horbach, A., Ding, Y., & Zesch, T. (2017). The Influence of Spelling Errors on Content Scoring Performance. In Proceedings of the 4th Workshop on Natural Language Processing Techniques for Educational Applications (NLPTEA 2017) (pp. 45–53). Taipei, Taiwan: Asian Federation of Natural Language Processing. https://www.aclweb.org/anthology/W17-5908
    • Horbach, A., Scholten-Akoun, D., Ding, Y., & Zesch, T. (2017). Fine-grained essay scoring of a complex writing task for native speakers. In Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications (pp. 357–366). Copenhagen, Denmark: Association for Computational Linguistics. https://doi.org/10.18653/v1/W17-5040
    • Riordan, B., Horbach, A., Cahill, A., Zesch, T., & Lee, C. M. (2017). Investigating neural architectures for short answer scoring. In Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications (pp. 159–168). Copenhagen, Denmark: Association for Computational Linguistics. https://aclanthology.org/W17-5017/
    • Keiper, L., Horbach, A., & Thater, S. (2016). Improving POS Tagging of German Learner Language in a Reading Comprehension Scenario. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016) (pp. 198–205). Portorož, Slovenia: European Language Resources Association (ELRA). Retrieved from https://www.aclweb.org/anthology/L16-1030
    • Horbach, A., & Palmer, A. (2016). Investigating Active Learning for Short-Answer Scoring. In Proceedings of the 11th Workshop on Innovative Use of NLP for Building Educational Applications (pp. 301–311). San Diego, CA: Association for Computational https://aclanthology.org/W16-0535/
    • Horbach, A., Thater, S., Steffen, D., Fischer, P. M., Witt, A., & Pinkal, M. (2015). Internet corpora: A challenge for linguistic processing. Datenbank-Spektrum, 15(1), 41–47. https://link.springer.com/article/10.1007%2Fs13222-014-0172-z
    • Ostermann, S., Horbach, A., & Pinkal, M. (2015). Annotating Entailment Relations for Shortanswer Questions. In Proceedings of the 2nd Workshop on Natural Language Processing Techniques for Educational Applications (pp. 49–58). Beijing, China: Association for Computational Linguistics. https://aclanthology.org/W15-4408/
    • Horbach, A., Poitz, J., & Palmer, A. (2015). Using Shallow Syntactic Features to Measure Influences of L1 and Proficiency Level in EFL Writings. In Proceedings of the fourth workshop on NLP for computer-assisted language learning (pp. 21–34). Vilnius, Lithuania: LiU Electronic Press. https://www.aclweb.org/anthology/W15-1903
    • Koleva, N., Horbach, A., Palmer, A., Ostermann, S., & Pinkal, M. (2014). Paraphrase Detection for Short Answer Scoring. In Proceedings of the third workshop on NLP for computer-assisted language learning (pp. 59–73). Uppsala, Sweden: LiU Electronic Press. https://www.aclweb.org/anthology/W14-3505
    • Horbach, A., Palmer, A., & Wolska, M. (2014). Finding a Tradeoff between Accuracy and Rater’s Workload in Grading Clustered Short Answers. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC-2014) (pp. 588–595). Reykjavik, Iceland: European Languages Resources Association (ELRA). http://www.lrec-conf.org/proceedings/lrec2014/pdf/887_Paper.pdf
    • Horbach, A., Palmer, A., & Pinkal, M. (2013). Using the text to evaluate short answers for reading comprehension exercises. In Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 1: Proceedings of the Main Conference and the Shared Task: Semantic Textual Similarity (pp. 286–295). Atlanta, Georgia, USA: Association for Computational Linguistics. https://www.aclweb.org/anthology/S13-1041

Research Center CATALPA | 22.04.2024