Much of what we know about collaboration is buried in free-form text—lab notebooks, attribution pages, self-reports, and narratives of group experience. This project investigates how Large Language Models (LLMs) can transform such unstructured text into analyzable data, allowing us to map not only team networks and task allocation structures, but also narrative accounts of how people experience collaboration and relational processes.
In our published work, we showed how GPT-4 could extract inter-team collaborations and intra-team task distributions from the digital lab notebooks of over 2,000 iGEM synthetic biology teams, achieving recall rates as high as 0.91 and preserving network properties such as centrality, assortativity, and nestedness. This demonstrates that LLMs can recover hidden patterns of collaboration with accuracy and scalability.
We are now extending this approach to more complex forms of text: survey responses, interviews, and reflective narratives. Here the goal is to create narrative maps that capture how participants describe trust, conflict, co-regulation, and empathy within their groups. By doing so, we can connect structural network data with the lived, relational experience of collaboration—opening new possibilities for a multi-layered science of collective intelligence.
Methodologically, the project emphasizes not just extraction but also rigorous validation. We build gold-standard datasets, apply metrics such as precision, recall, F1, and semantic similarity, and use bootstrapping to estimate confidence intervals. This ensures that LLM outputs are not treated as opaque but evaluated with the same standards of transparency and reproducibility as other scientific instruments.
The long-term aim is to develop generalizable pipelines for text-to-network and text-to-narrative mapping. These would allow researchers and practitioners to turn free-form text into reliable relational data, capturing both the formal structures of collaboration and the subjective experiences that animate them. In this way, we hope to bridge the gap between what groups do together and how they experience being together—providing new tools to study the emergence of collective intelligence.
📖 Read the article in Applied Network Science
📂 Data and code: GitHub Repository
Interaction Data Lab 