Files
Abstract
With large language models becoming more and more popular and their usage expanding into several different domains, this study explores how effective a Large Language Model like GPT4 would be in analyzing requirement specification documents and using relevant information from those to create a tradespace matrix. The research compares GPT-4’s performance with 30 human participants divided into three groups: engineering background, non-technical background, and computer science/artificial intelligence (CS/AI) background. Each participant and LLM completed a survey based on the provided materials. The analysis included within-group heatmaps, across-group comparisons, and human vs. GPT-4 heatmap evaluations. Results showed the CS/AI group had the closest responses to GPT-4. The study showcases how LLMs can be used to augment trade-space exploration and can be used as an alternative in cases where a team of human experts from different backgrounds is not feasible.