Evaluating AI-Generated Solution Ideas: A Comparative Study of AI and Human Assessments for Sustainable Process Design

Abstract

The integration of generative artificial intelligence (AI) in sustainable process design has gained substantial traction, with AI increasingly employed to generate innovative solutions. However, the efficacy of these AI-generated ideas requires rigorous evaluation to ensure their quality. This study examines the dual role of GPT-4o in generating and evaluating solution ideas for sustainable process design at the concept development stage. Focusing on process engineering, the research applies these methods to a case study involving froth flotation for nickel recovery. By comparing AI-driven assessments with those from human experts, the research aims to determine the alignment between AI and human evaluations across key criteria: novelty, feasibility, usefulness, and sustainability. The results reveal strong alignment in most areas, though notable discrepancies in novelty suggest that human expertise remains essential for nuanced judgments on uniqueness. These findings highlight GPT-4o’s potential as a preliminary evaluation tool, while also underscoring the need for a hybrid approach that combines AI insights with human expertise.