Beyond the Classroom: Exploring Mathematics Engagement in Online Communities with Natural Language Processing

Abstract

In an era where digital platforms increasingly shape the educational experiences of learners, this dissertation examines activity in the Mathematics Discord Server (MDS), an expansive online learning community used by hundreds of thousands of mathematics learners worldwide. Daily interactions, numbering in the tens of thousands, focused on mathematics problems brought by students in need of advice, comprise a dynamic environment for peer mentoring. The study investigated the phenomenon of online mathematics learning taking place in chat-based platforms by creating and analyzing MathConverse, a novel dataset of 200,000 structured conversations from the help channels on the MDS. This dataset, transformed from raw messages into a comprehensive repository of conversations with rich metadata, makes possible ways of understanding the complexity of real-time problem solving and cooperative learning that takes place when students look for help from others online. Beginning with tackling the complexities of transforming chat-based exchanges into analyzable data, this dissertation navigates the challenges of conversation disentanglement and contributes to the methodological and theoretical advancement of educational research in online spaces.

Central to this investigation are two primary objectives: First, to demonstrate and refine the application of methods from machine learning and natural language processing (NLP) to study text as data in educational research, addressing the methodological gap in analyzing voluminous, text-based datasets. Chapter 2 provides details of the work involved in transforming extensive conversational data into structured datasets for analysis. In Chapter 3 and Chapter 4, I provide case studies using MathConverse to illustrate how techniques from (NLP) can be used to draw rich qualitative insights from the texts we as social science researchers are surrounded by in our research. For example, once I determined a large language model could reliably categorize questions into question types, I used the model to classify a larger set of questions (