Cloud-Based Imitation Learning and On-Site Generative Models to Endow Robotic Apprentices with Construction Craft Skills

Abstract

With severe and persistent workforce shortages in the industry, construction robots have been increasingly promoted to enhance efficiency and productivity. However, the unpredictability of construction environments makes it impractical to rely solely on pre-programming for determining their motion trajectories. Human cognition is thus essential in construction, especially for supervision, planning and on-the-spot improvisation. Therefore, the most effective approach to integrating robots in construction is through human-robot collaborations that combines human insight with robotic operational capabilities and endurance, enabling them to navigate uncertainties and execute tasks successfully. This dissertation aims to explore how to equip construction co-robots with essential craft and collaboration skills, focusing on developing a programming and interaction system that demonstrates technical proficiency and finely tunes the complexity of human-robot interactions and mutual adaptations. First, this dissertation equips robots with the craft skills necessary to manipulate and maneuver construction materials to complete installation tasks accurately. Most construction tasks, such as arbitrary pick-and-place activities, require advanced object avoidance skills to ensure that materials are installed without interference from surrounding obstacles. To facilitate this, an Imitation Learning (IL) framework and a Virtual Reality (VR) environment were developed, allowing workers to demonstrate these skills with reduced effort and to effectively transfer these skills to robots. This dissertation also proposed to use hierarchical models to decompose construction skills and frame them within a more transparent and effective machine learning approach. Second, as robots progressively acquire craft skills, they can be deployed for increasingly complex tasks. Human workers will continue to play a crucial role in planning and overseeing these tasks. It is vital that robots are designed to understand human intent and to cooperate seamlessly and safely with human workers. This enhances safety during close-proximity interactions and reduces the mental strain on workers, who would otherwise need to remain perpetually vigilant and defensive around robotic counterparts. This dissertation used human physical state adaptive robot control to help robots replicate the human’s mutually adaptive and protective behavior. IL was also applied to transfer such behavior without additional hard-coding programming efforts on the human co-workers. Third, a fundamental requirement for construction workers is the ability to adapt and learn continuously, especially as the industry moves towards more modularized and organized approaches like prefabrication. Similarly, robots must be equipped with adaptive learning abilities to minimize the need for frequent reprogramming by humans and to alleviate both the physical and cognitive burdens on the workforce. This dissertation proposes a pipeline to retrieve such skills from the readily available web tutorials using modularized robot motions and Bayesian Network (BN) structure learning (SL). The robots will thus be able to learn from crowdsourced natural language resources to overcome the barrier of limited instructional resources arising from labor shortage. Fourth, this dissertation addressed the potential interactive behaviors affecting trust bonding between humans and robots in daily collaboration. Building on existing research that highlights differing expectations for collaborative robots, this dissertation investigates the behavior rules and norms preferred by workers. To gather concrete data, twelve workers were invited to interact with construction robots operating under various behavioral norms. Their assessments of the robots' trustworthiness across these different scenarios were meticulously recorded. The insights gained from these evaluations are instrumental in guiding the design of construction robots that are more likely to earn and retain the trust of their human counterparts.