Program

The workshop will be held as a half-day workshop at ACM UMAP 2025 from 2:00 PM to 5:30 PM (time zone: UTC/GMT -4 hours)

Keynote Speaker Session 1: Shuai Zhang

Senior Applied Scientist at Amazon Web Services AI

Title: Steering Large Language Models: A Sustainable Way to Robustness and Personalization

Brief Biography

I am currently a senior applied scientist at Amazon Web Services AI, focusing on foundation models related research, product, and opensource projects. Before this role, I was a postdoctoral researcher in the Department of Computer Science at ETH Zurich, under the guidance of Prof. Ce Zhang. I received a PhD degree from the school of computer science and engineering of UNSW Sydney, advised by Prof. Lina Yao, and my PhD study was also supported by CSIRO’s Data61 PhD Scholarship. I obtained a Bachelor’s degree from Nanjing University. My primary areas of focus include LLMs, multimodal foundation models, information retrieval, time series forecasting, graph machine learning, etc. At Amazon, I have contributed to products such as Amazon Titan, AWS Glue, etc. I have served as the area chair for ACL Rolling Review, reviewer/guest editor/senior PC for a few leading AI/ML conferences/journals, and organizers for workshops in leading AI conferences.

Abstract of the Talk

As large language models (LLMs) are increasingly deployed in real-world applications, ensuring their robustness and adaptability to individual user preferences becomes important. In this talk, I will present a new perspective on achieving personalization and robustness in LLMs through model steering—a lightweight, sustainable alternative to fine-tuning or prompt engineering. I will showcase three of our recent works that demonstrate how inference-time steering of internal model representations can address critical challenges in LLM behavior. First, SteerFair enhances robustness by identifying and removing bias directions in the model’s representation space—such as spurious correlations between option position and correctness—to reduce performance variance across prompt variants. Second, CrEst uses weak supervision to estimate the credibility of retrieved documents through inter-document semantic agreement and steers the model toward more trustworthy content, either through prompt integration or activation-level editing. Finally, CONFST constructs confident steering directions aligned with user preferences and modifies LLM activations at inference time to support flexible, multi-directional control over style and topic—without requiring explicit prompts or layer-specific tuning. Together, these methods demonstrate how model steering can serve as a powerful, interpretable, and efficient framework for building more robust and personalized LLMs at inference time.

Keynote Speaker Session 2: Bamshad Mobasher

Professor of Computer Science and Director of the Center for Web Intelligence at DePaul University’s College of Computing and Digital Media in Chicago

Title: Agnotology in the Age of Artificial Intelligence: From Culturally Induced Ignorance to Algorithmic Knowledge Manipulation

Brief Biography

Bamshad Mobasher is a Professor of Computer Science and the Director of the Center for Web Intelligence at DePaul University’s College of Computing and Digital Media in Chicago. He also serves as the Director of DePaul’s AI Institute, which fosters interdisciplinary research and educational initiatives in the field of artificial intelligence. Dr. Mobasher’s primary research interests lie in artificial intelligence and machine learning, with a particular focus on recommender systems and algorithmic personalization. His scholarly contributions in these areas have been cited over 30,000 times. He has held leadership roles in several major international conferences, including the ACM Conference on Recommender Systems, the ACM SIGKDD Conference on Knowledge Discovery and Data Mining, and the ACM Conference on User Modeling, Adaptation, and Personalization. He currently serves as an associate editor for ACM Transactions on the Web, ACM Transactions on Intelligent Interactive Systems, and ACM Transactions on Recommender Systems. Additionally, he has served on the editorial boards of leading journals such as User Modeling and User-Adapted Interaction and the Journal of Web Semantics. As Director of the Center for Web Intelligence, Dr. Mobasher leads research efforts in these domains and actively collaborates with industry partners and community organizations on a variety of applied research projects.

Abstract of the Talk

The rapid proliferation of generative AI has reshaped traditional notions of truth, misinformation, and epistemic agency. In this presentation we examine the intersection of agnotology—the study of culturally and politically induced ignorance—and Artificial Intelligence. We explore how large language models (LLMs) and other generative AI systems function not only as engines of knowledge production but also as instruments of ignorance creation. Through empirical examples and recent case studies, we analyze how mechanisms such as persuasive content generation, deepfake synthesis, and algorithmic amplification of biases contribute to the erosion of epistemic clarity. These processes foster what some scholars describe as a “post-epistemic” condition, where distinctions between credible knowledge and fabricated information become increasingly ambiguous. These challenges are compounded by algorithmic opacity and emergent forms of epistemic injustice, including testimonial suppression and the exploitation of the “liar’s dividend.” We present the notion of algorithmic agnotology, a synthesis of agnotological theory and computational epistemology, as a conceptual framework for understanding ignorance at scale in AI-mediated environments. We briefly outline some research frontiers, including design paradigms for epistemic justice, cognitive impact assessment of AI-generated content, and strategies for cultivating collective epistemic resilience. As generative AI reshapes how information is created, personalized, and consumed, this work calls for interdisciplinary approaches to mitigating epistemological harms embedded within digital infrastructures.

Time Presentation
2:00 PM – 2:10 PM Welcome Address
2:10 PM – 2:45 PM Keynote by Shuai Zhang
02:45 PM – 03:05 PM Towards Automated Penetration Testing: Introducing LLM Benchmark, Analysis, and Improvements (Isamu Isozaki, Manil Shrestha, Rick Console, and Edward Kim)
03:05 PM – 03:15 PM  From Feedback to Formative Guidance: Leveraging LLMs for Personalized Support in Programming Projects (Fatemeh Ghoochani, Jonas Scharfenberger, Burkhardt Funk, Raoul Doublan, Mayur Jakharabhai Odedra, and Bennet Etsiwah) – (virtual presentation)
03:15 PM – 03:30 PM Towards the Embodied Conversational Interview Agentic Service ELIAS: Development and Evaluation of a First Prototype (Tobias Budig, Marcia Nißen, and Tobias Kowatsch)
03:30 PM – 04:00 PM Coffee break
04:00 PM – 04:40 PM Keynote by Bamshad Mobasher
04:40 PM – 04:55 PM “Which vocational training program is best for me?” – Design of a recommender system for school students using large language models (Alexander Piazza, Sigurd Schacht, and Michael Herzog)
04:55 PM – 05:05 PM Enhancing Mathematical Reasoning in GPT-J Through Topic-Aware Prompt Engineering (Lev Sukherman and Yetunde Folajimi) – (virtual presentation)
05:05 PM – 05:15 PM Exploring Responsible Use of Generative AI in Disaster Resilience for Indigenous Communities (Taranum Bano, Srishti Gupta, Yu-Che Chen, Edouardo Zendejas, Sarah Krafka and Chun-Hua Tsai)
05:15 PM – 05:30 PM Discussion and Wrap–up

Accepted Contributions:

Long Papers:

  • Towards Automated Penetration Testing: Introducing LLM Benchmark, Analysis, and Improvements (Isamu Isozaki, Manil Shrestha, Rick Console and Edward Kim)

Short Paper:

  • Towards the Embodied Conversational Interview Agentic Service ELIAS: Development and Evaluation of a First Prototype (Tobias Budig, Marcia Nißen and Tobias Kowatsch)
  • “Which vocational training program is best for me?” – Design of a recommender system for school students using large language models (Alexander Piazza, Sigurd Schacht and Michael Herzog)

Position Papers:

  • Exploring Responsible Use of Generative AI in Disaster Resilience for Indigenous Communities (Taranum Bano, Srishti Gupta, Yu-Che Chen, Edouardo Zendejas, Sarah Krafka and Chun-Hua Tsai)
  • Enhancing Mathematical Reasoning in GPT-J Through Topic-Aware Prompt Engineering (Lev Sukherman and Yetunde Folajimi)
  • From Feedback to Formative Guidance: Leveraging LLMs for Personalized Support in Programming Projects (Fatemeh Ghoochani, Jonas Scharfenberger, Burkhardt Funk, Raoul Doublan, Mayur Jakharabhai Odedra and Bennet Etsiwah)

Format

Length of the presentations:

  • Keynote: 35 mins including Q&A
  • Long papers: 15 + 5 mins Q&A 
  • Short papers: 10 + 5 mins Q&A
  • Position papers: 8 + 2 mins Q&A