Computer Science Colloquia

Abstract

Knowledge is a prerequisite for intelligent behavior. Although modern language models acquire substantial knowledge and can leverage it to solve tasks with superior performance, the underlying mechanisms often remain opaque and fragile. This talk presents knowledgeable foundation models for robust intelligence and asks: What does the model know? When and why does it fail? How can we update it with minimal trade-off? And how do we use it to reason and ultimately to optimize decisions?

First, we make model knowledge explicit and testable and investigate why unreliable knowledge emerges, covering hallucination and staleness. We identify knowledge overshadowing, interactions among individually correct pieces of knowledge that are miscomposed and trigger hallucinations, and we quantify and even foresee such failures to gain greater model controllability and robustness. These diagnoses drive targeted, localized repairs that bound side effects. Second, we operationalize knowledge as interpretable, composable atomic skills, enabling modular reasoning that strengthens generalization and robustness. Finally, we translate interpretable knowledge into decision value by aligning reasoning with downstream utility.

Abstract

The leap from Large Language Models to Large Agent Models is to unfold reasoning as multi-turn interactions with the world. We take the first step by formalizing agent training as a Markov Decision Process (MDP), introducing an agent reasoning interface (RAGEN) to avoid “reasoning collapse”, where agents loop into repetitive reasoning and fail to explore. Extending to Partially Observable MDPs (POMDPs), we propose VAGEN, training agents to internalize world models for state estimation (“what is seen”) and transition modeling (“what comes next”).

We further show how divergence between Pass@1 and Pass@K reveals shallow exploitation over true exploration, and use self-play to inject world model knowledge to diversify exploration. Finally, we introduce cognitive maps as structured reasoning interfaces that integrate partial observations into coherent world beliefs. In the end, we lay out how these advances chart a path to agents that simulate, explore, and actively construct internal world models—a decisive step from LLMs to LAMs.

Abstract

Large language models (LLMs) are increasingly used in healthcare, yet most evaluations rely on clean, exam-style datasets that fail to capture the complexity of real-world clinical data, such as electronic health records (EHRs), and rarely keep pace with rapidly evolving LLMs. In this talk, we will present BRIDGE, a multilingual benchmark constructed from real clinical tasks and over one million EHR-derived samples, where we evaluated 95 leading LLMs through 24,000+ experiments and 39 million predictions. Our findings show wide variation across model families, tasks, and languages, with several open-source models matching proprietary ones. We also observe that chain-of-thought prompting often lowers accuracy for these clinical tasks, and we provide the first large-scale analysis of stigmatized language generated during model reasoning.

Abstract

This talk presents the data science foundations of the LabGenie project, focusing on the evaluation of large language models (LLMs) for laboratory test interpretation. I will discuss our empirical studies examining how LLMs answer lab-related questions and the challenges they face in numerical accuracy and contextual understanding. I will also highlight our findings on the role of lab data in improving LLM-based differential diagnosis. Finally, I will introduce LabQAR, a curated benchmark dataset for evaluating LLM performance on lab reference ranges and result interpretation. Together, this work positions LabGenie as a research testbed for advancing clinically grounded and trustworthy AI in laboratory medicine.