Performance Review and Improvement Metrics for Evaluating Project Delivery Methods with Large Language Models (LLMs)

Problem Statement

Alternative Delivery Methods (ADMs) such as Construction Manager/General Contractor (CM/GC) can improve project performance, risk management, and innovation compared to traditional Design-Bid-Build (DBB). However, the effectiveness of ADMs across cost, schedule, quality, and sustainability metrics is not fully assessed.

Figure 1 – Developed RAG framework for construction information retrieval.

Approach

This project integrates quantitative historical data and qualitative stakeholder insights to develop a comprehensive ADM performance evaluation framework. Open-source large language models (LLMs) such as LLaMA3 and Gemma 3, combined with vision-language models (VLMs), will analyze project reports, survey responses, and other documentation. A multi-modal Retrieval-Augmented Generation (RAG) system will be built to enable robust evaluation, prediction, and reporting of ADM performance. The final system will be packaged as a user-friendly, open-source tool for Caltrans, providing actionable insights and supporting decision-making on project delivery methods.

Ongoing Work

We are developing the multi-modal RAG framework, integrating LLMs and VLMs to process textual, visual, and tabular data. Future work will focus on building the user-friendly software interface for project stakeholders to evaluate and forecast ADM performance efficiently.

Students

Bozhou Zhuang

Yannik Hahn (VGR)

Nils Hütten (VGR)