Publications

Extending Minimal Pairs with Ordinal Surprisal Curves and Entropy Across Applied Domains

Andrew Katz

arXiv preprint

This paper extends surprisal-based minimal-pairs evaluation of language models from binary grammaticality judgments to ordinal-scaled classification and scoring tasks. Instead of asking models to generate answers, it measures the surprisal a model assigns to each position on a...

Thematic analysis with open-source generative AI and machine learning: A new method for inductive qualitative codebook development

Andrew Katz, Gabriella Coloyan Fleming, Joyce B. Main

Humanities and Social Sciences Communications

This paper introduces a novel method for developing qualitative codebooks using open-source generative AI and machine learning techniques, enabling more systematic and reproducible inductive thematic analysis.

Using generative AI for large-scale qualitative analysis of social media posts to understand why people leave computer science

Amanda Ross, Andrew Katz

Journal of Engineering Education

This study uses generative AI for large-scale qualitative analysis of over 10,000 Reddit posts to understand diverse reasons why people leave computer science, including job dissatisfaction and influential factors at different stages of departure.

Advancing Qualitative Analysis in Professional Disaster and Risk Communication: A Comparative Study of an OpenAI ChatGPT 3.5 Model-Enabled Method for Processing Complex Public Posts

Margaret Webb, Harman Singh, Rachel Inman, Sweta Baniya, Andrew Katz

International Journal of Disaster Risk Reduction

This paper explores the application of ChatGPT 3.5 to advance qualitative analysis in professional disaster and risk communication research, demonstrating a comparative method for processing complex public posts.

Expanding possibilities for generative AI in qualitative analysis: Fostering student feedback literacy through the application of a feedback quality rubric

Katherine Drinkwater Gregg, Olivia Ryan, Andrew Katz, Mark Huerta, Susan Sajadi

Journal of Engineering Education

This study investigates the reliability and utility of generative AI to apply a feedback quality rubric to peer feedback comments from first-year engineering students, exploring how LLMs can assist in qualitative analysis.

Automated Analysis of Knowledge Types in Computer Science Textbooks: A Natural Language Processing Approach to Understanding Epistemic Climate

Mitchell Gerhardt, Andrew Katz

Proceedings of the 2025 ASEE Annual Conference & Exposition

This paper uses a natural language processing approach to automatically analyze knowledge types in computer science textbooks, providing insights into the epistemic climate of educational materials.

Leveraging Generative Text Models and Natural Language Processing to Perform Traditional Thematic Data Analysis

Isil Anakok, Andrew Katz, Kai Jun Chew, Holly Matusovich

International Journal of Qualitative Methods

This paper explores how generative text models and natural language processing can be leveraged to perform traditional thematic data analysis in educational research contexts.

Understanding instructor decision-making in engineering education for sustainable development: a comparison of institutions in Denmark and the United States

Maya Menon, Marie C. Paretti, Jennifer M. Case, Andrew S. Katz

European Journal of Engineering Education

This comparative study examines instructor decision-making in engineering education for sustainable development across institutions in Denmark and the United States.

Curriculum comparison: chemical and mechanical engineering education in the United States and Turkey

Hatice Cilsalar-Sagnak, Isil Anakok, Andrew Katz

Australasian Journal of Engineering Education

This comparative study examines chemical and mechanical engineering education programs in the United States and Turkey, identifying similarities and differences in curriculum design.

Engineering students' interests in nonprofit and public policy careers: Applying a data-driven approach to identifying contributing factors

Dayoung Kim, Andrew Katz

Journal of Engineering Education

A Reinforcement Learning Framework for N-Ary Document-Level Relation Extraction

C. Yuan, R. Rossi, A. Katz, H. Eldardiry

IEEE Transactions on Big Data

From Manual Coding to Machine Understanding: Students' Feedback Analysis

A. Alsharif, A. Katz

Proceedings of the 131st Annual Conference of the American Society for Engineering Education

How do ethics and diversity, equity, and inclusion relate in engineering? A systematic review

J. L. Hess, A. Lin, A. Whitehead, A. Katz

Journal of Engineering Education

Inspiring Sustainability in Undergraduate Engineering Programs

T. Griesinger, K. Reid, D. Knight, A. Katz, J. Somers

Sustainability

Novel Approach Designing Interview Protocols with Generative Large Language Models to Study Mental Models and Engineering Design

F. Bilow, M. Paretti, T. Shealy, A. Katz

Proceedings of the 131st Annual Conference of the American Society for Engineering Education

Paradigm Shift? Preliminary Findings of Engineering Faculty Members’ Mental Models of Assessment in the Era of Generative AI

I. Anakok, K. Chew, H. Matusovich, A. Katz

Proceedings of the 131st Annual Conference of the American Society for Engineering Education

Skill Development of Engineering and Physical Science Doctoral Students: Understanding the Role of Advisor, Faculty, and Peer Interactions

A. Alsharif, M. Denton, D. Knight, M. Borrego, A. Katz

Proceedings of the 131st Annual Conference of the American Society for Engineering Education

Stumbling Our Way Through Finding a Better Prompt: Using GPT-4 to Analyze Engineering Faculty’s Mental Models of Assessment

A. Ross, A. Katz, K. Chew, H. Matusovich

Proceedings of the 131st Annual Conference of the American Society for Engineering Education

Thematic Analysis with Open-Source Generative AI and Machine Learning: A New Method for Inductive Qualitative Codebook Development

Andrew Katz, Gabriella Coloyan Fleming, Joyce Main

arXiv preprint

Using Generative Text Models to Create Qualitative Codebooks for Student Evaluations of Teaching

Andrew Katz, Mitch Gerhardt, Michelle Soledad

International Journal of Qualitative Methods

Using Generative Text Models to Create Qualitative Codebooks for Student Evaluations of Teaching

A. Katz, M. Gerhardt, M. Soledad

arXiv preprint

What engineering employers want: An analysis of technical and professional skills in engineering job advertisements

G. C. Fleming, M. Klopfer, A. Katz, D. Knight

Journal of Engineering Education

Advanced Considerations in Quantitative Methods for New Directions in Engineering Education Research

A. Katz, A. Godwin, C. Brozina

International Handbook of Engineering Education Research

Advancing qualitative analysis: An exploration of the potential of generative AI and NLP in thematic coding

Yasir Gamieldien, Jennifer M Case, Andrew Katz

Available at SSRN 4487768

Exploring Faculty Members' Conceptualizations of Diversity, Equity, and Inclusion in Engineering Education

I. Anakok, J. Hess, S. Panuganti, A. Katz

IEEE Frontiers in Education Conference (FIE)

Exploring NLP-based Methods for Generating Engineering Ethics Assessment Qualitative Codebooks

A. Hingle, A. Katz, A. Johri

IEEE Frontiers in Education Conference (FIE)

Exploring the Efficacy of ChatGPT in Analyzing Student Teamwork Feedback with an Existing Taxonomy

A. Katz, S. Wei, G. Nanda, C. Brinton, M. Ohland

arXiv preprint

Exploring the Impact of Engineering Projects in Community Service on Engineering Students’ Perspectives about Engineering as a Major

I. Anakok, M. Huerta, A. Katz

IEEE Frontiers in Education Conference (FIE)

How Participating in Extracurricular Activities Supports Dimensions of Student Wellness

B. Johnson, J. Main, A. Katz

IEEE Frontiers in Education Conference (FIE)

Promoting Research Quality to Study Mental Models of Ethics and Diversity, Equity, and Inclusion (DEI) in Engineering

J. Hess, A. Katz, I. Anakok, B. Jesiek, A. Whitehead, S. Panuganti

Proceedings of the 130th Annual Conference of the American Society for Engineering Education

Pushing Ethics Assessment Forward in Engineering: NLP-Assisted Qualitative Coding of Student Responses

U. Shakir, J. Hess, M. James, A. Katz

Proceedings of the 130th Annual Conference of the American Society for Engineering Education

The Utility of Large Language Models and Generative AI for Education Research

A. Katz, U. Shakir, B. Chambers

arXiv preprint

Utilizing Natural Language Processing to Examine Self-Reflections in Self-Regulated Learning

Yasir Gamieldien, Rachel McCord, Andrew Katz

Available at SSRN 4487795

WIP: Faculty Use of Metaphors When Discussing Assessment

A. Ross, A. Katz, H. Matusovich, K. Chew

Proceedings of the 130th Annual Conference of the American Society for Engineering Education

A Thematic and Trend Analysis of Engineering Education for Sustainable Development

M. Menon, A. Katz, M. Paretti

2022 ASEE Annual Conference & Exposition

An Empirical Study of Programming Languages Specified in Engineering Job Postings

L. Schibelius, A. Ross, A. Katz

2022 ASEE Annual Conference & Exposition

Analysis of Advances in Engineering Education Publications (2007-2020) to Examine Impact and Coverage of Topics

A. Katz, C. Brozina, A. Johri, L. Shuman

2022 IEEE Frontiers in Education Conference (FIE)

Applying concepts from political science and economics to advance the study of engineering education

A. Katz, J. Main

Studies in Engineering Education

Augmented Reality for Sustainable Collaborative Design

S. Yan, E. Kang, A. Katz, A. Hira

2022 ASEE Annual Conference & Exposition

Clustering-based unsupervised generative relation extraction

Chenhan Yuan, Ryan A Rossi, Andrew Katz, Hoda Eldardiry

2022 IEEE International Conference on Big Data (Big Data)

Collateral Damage: Investigating the Impacts of COVID on STEM Professionals with Caregiving Responsibilities

A. Maltese, K. Paul, J. Kim, A. Godwin, A. Katz, A. Diekman

2022 ASEE Annual Conference & Exposition

Defining Assessment: Foundation Knowledge Toward Exploring Engineering Faculty’s Assessment Mental Models

K. Chew, A. Ross, A. Katz, H. M. Matusovich

2022 IEEE Frontiers in Education Conference (FIE)

Development of hybrid laboratory sessions during the COVID-19 Pandemic

M. Szoke, A. Katz, A. Borgoltz, W. J. Devenport

Advances in Engineering Education

Students’ Feedback About Their Experiences in EPICS Using Natural Language Processing

I. Anakok, J. Woods, M. Huerta, J. Schoepf, H. Murzi, A. Katz

2022 IEEE Frontiers in Education Conference (FIE)

Understanding First-year Engineering Students’ Perceptions of Working with Real Stakeholders on a Design Project: A PBL Approach

H. Murzi, L. Fielding, M. Huerta, J. O. Alvarez, M. James, A. Katz, J. Grohs

2022 IEEE Frontiers in Education Conference (FIE)

Using Natural Language Processing to Explore Undergraduate Students’ Perspectives of Social Class, Gender, and Race

U. Shakir, S. Ovink, A. Katz

2022 ASEE Annual Conference & Exposition

Using Sentiment Analysis to Evaluate First-year Engineering Students Teamwork Textual Feedback

A. Alsharif, A. Katz, D. Knight, S. Alatwah

2022 ASEE Annual Conference & Exposition

Work-in-Progress: Using Latent Dirichlet Allocation to uncover themes in student comments from peer evaluations of teamwork

G. Nanda, S. Wei, M. Ohland, A. Katz, C. Brinton

2022 ASEE Annual Conference & Exposition

Civil Engineering Students’ Beliefs about Global Warming and Misconceptions about Climate Science

T. Shealy, A. Katz, A. Godwin, M. Bell

Journal of Civil Engineering Education

Engineering ethics in engineering design courses: A preliminary investigation

A. Katz, U. Shakir, I. Anakok, H. Murzi

American Society for Engineering Education Annual Conference

Factors associated with collaboration networks in ASEE conference papers

S. Abdelhamid, A. Katz

American Society for Engineering Education Annual Conference

Figurative language in computer education: Evidence from YouTube instructional videos

S. Abdelhamid, A. Katz

American Society for Engineering Education Annual Conference

Higher perceived design thinking traits and active learning in design courses motivate engineering students to tackle energy sustainability in their careers

J. Milovanovic, T. Shealy, A. Katz

Sustainability

Predicting engineering students’ desire to address climate change in their careers: An exploratory study using responses from a U.S. National survey

T. Shealy, A. Katz, A. Godwin

Environmental Education Research

Taking stock: An Analysis of IJEE publications from 1996--2020 to examine impact and coverage of topics

C. Brozina, A. Katz, A. Johri

International Journal of Engineering Education

The correlation between undergraduate student diversity and the representation of women of color faculty in engineering

J. Main, L. Tan, M. Cox, E. McGee, A. Katz

Journal of Engineering Education

Using natural language processing to facilitate student feedback analysis

A. Katz, M. Norris, A. Alsharif, M. Klopfer, D. Knight, J. Grohs

American Society for Engineering Education Annual Conference

Your views can be my views: Understanding differences in paradigms held by traditionally marginalized students in engineering

Q. Ketchum, M. C. Paretti, H. Murzi, A. Katz

American Society for Engineering Education Annual Conference

An Investigation of When and Where Ethics Appears in Undergraduate Engineering Curricula

A. Katz, U. Shakir

2020 ASEE Virtual Annual Conference

Civil Engineering Students’ Beliefs about the Technical and Social Implications of Global Warming and When Global Warming Will Impact Them Personally and Others

A. Katz, T. Shealy, A. Godwin

2020 ASEE Virtual Annual Conference

Clustering-based Unsupervised Generative Relation Extraction

C. Yuan, R. Rossi, A. Katz, H. Eldardiry

arXiv preprint

Harvesting tweets for a better understanding of engineering students' first-year experiences

S. Abdelhamid, A. Katz

2020 First-Year Engineering Experience Conference

Overcoming Challenges to Enhance a First Year Engineering Ethics Curriculum

A. Katz, K. Reid, D. Riley, N. Van Tyne

Advances in Engineering Education

Reinforcement Learning-based N-ary Cross-Sentence Relation Extraction

C. Yuan, R. Rossi, A. Katz, H. Eldardiry

arXiv preprint

Using Chatbots as Smart Teaching Assistants for First-Year Engineering Students

S. Abdelhamid, A. Katz

2020 First-Year Engineering Experience Conference

Monetizing Life May Be the Ethical Thing to Do

A. Salado, A. Katz

2019 American Society for Engineering Education Annual Conference

Investigating influences on first-year engineering students’ views of ethics and social responsibility

S. Nittala, T. Zephirin, S. M. J. Howland, D. Kim, A. Katz, B. K. Jesiek

American Society for Engineering Education Annual Conference

Learning from failures: Engineering education in an age of academic capitalism

A. Katz, D. Riley

American Society for Engineering Education Annual Conference

Factors related to faculty views of undergraduate engineering ethics education

A. Katz, D. Knight

American Society for Engineering Education Annual Conference

Telling it like it was: Histories of change in engineering ethics education

M. Davis, J. Herkert, M. Loui, I. Nair, S. Pfatteicher, M. Pritchard, A. Katz

Panel Session at the Association for Practical and Professional Ethics Annual Conference

Addressing the state of higher education in State of the Union Addresses

A. Katz

Association for the Study of Higher Education Annual Conference

Shadow codes of engineering ethics: An experiment in ethics imaginaries

A. Katz, Y. Lambrinidou, D. M. Riley

Association for Practical and Professional Ethics Annual Conference

The revealing effects of disaster: A case study from Tulane University

A. Katz

American Society for Engineering Education Annual Conference

Educating for a revolution: Discerning together the Highlander idea and its lessons for ESJP’s work

D. Riley, A. Katz

Engineering, Social Justice, and Peace Annual Conference