Enabling Human-Robot Partnerships in Digitally-Driven Construction Work through Integration of Building Information Models, Interactive Virtual Reality, and Process-Level Digital Twins

Abstract

Human cognition plays a critical role in construction work, particularly in the context of high-level task planning and in-field improvisation. On the other hand, robots are adept at performing numerical computation and repetitive physical tasks with precise motion control. The unstructured and complex nature of construction environments and the inability to maintain tight tolerances in assembled workpieces pose several unique challenges to the wide application of robots in construction work. Thus, robotization of field construction processes is best achieved through human-robot partnerships that take advantage of both human and robot intelligence, as well as robots’ physical operational capabilities, to overcome uncertainties and successfully perform construction work. This dissertation explores the pathway of integrating building information models (BIM), interactive virtual reality (VR), and process-level digital twins to enable human-robot partnerships in digitally-driven construction through three levels. At the first level, an interactive and immersive process-level digital twin system in VR that serves as the human-robot collaboration platform is proposed. It integrates visualization and supervision, task planning and execution, and bi-directional communication to enable human workers to remotely collaborate with construction robots in field construction. A human-in-the-loop experiment based on a drywall installation case study was conducted for system verification and to collect user feedback for future improvements. Overall, the system enables human-robot partnerships and reduces the cognitive planning and physical workload of human workers. At the second level, Building Information Models (BIM) are integrated into the digital twin system to enable closed-loop BIM-driven Human-Robot Collaboration (HRC) in construction. BIM provides digital information to both the robot and its human partners to drive the construction process. In addition, deployment of the system to co-robotically performed construction work is studied. A physical drywall installation case study and three physical experiments (i.e., visual detection and end-effector movement) were conducted to verify the system workflow and to evaluate the system. Building on the previous level, the integration of BIM reduces human co-workers’ planning effort and improves construction work accuracy. Motivated by the programming and human instruction effort required to guide motion sequences in typical robotic work, the third level of this dissertation builds upon the BIM-driven digital twin system and explores how to enable robots to automatically plan their motion sequence. A Scene Distance Matrix (SDM) is proposed to guide robots’ sequential decisions in selecting modular construction skill primitives that lead to robot motions. Interactive Learning from Demonstration (LfD) is used to teach robots the mapping from the SDM to the skill primitives. The proposed approach is presented with a case study that contains three scenarios, including exterior wall sheathing, drywall installation, and timber frame construction. A wooden shelf construction task has been used to verify the proposed LfD module and its integration with the BIM-driven digital twin system. It further reduces the planning and programming effort of human workers. Overall, this research aims to create a scalable pathway to bring human workers in the loop of robotized construction and capitalize on human workers’ improvisation ability to handle uncertainties on construction sites. In addition, it explores the integration of BIM and LfD with the interactive digital twin to improve system autonomy in task planning and motion sequencing. This dissertation establishes the foundation of next-generation construction work by transitioning the role of construction workers from manual task performers to robot supervisors.