Algorithms and Visualizations to Support Airborne Detection of Vertical Obstacles

Abstract

Slow or failed detection of low salience vertical obstacles and associated wires is one of today’s leading causes of fatal helicopter accidents. The risk of collisions with such obstacles is likely to increase as Advanced Aerial Mobility and broadening drone activity promises to increase the density of air traffic at low altitudes, while growing demand for electricity and communication will expand the number of vertical structures. The current ‘see-and-avoid’ detection paradigm relies on pilots to spend much of their visual attention looking outside for obstacles. This method is inadequate in low visibility conditions, cluttered environments and given the need for pilots to engage in multiple competing visual tasks. With the expected growing number of hazards and an increased traffic volume, the current approach to collision avoidance will become even less tenable. This dissertation provides methods for automatic detection and correlation of vertical obstacles and evaluates the effectiveness of sensor visualizations and graphic augmentations for supporting flight crews in noticing hazards.

The first contribution of this line of research is a modular set of algorithms which extract towers from raw point clouds. Vertical structures compose less than 0.2% of real world tiles containing over 100 million points. A mesh filter quickly removes large, flat surfaces. Next, a sphere detector finds and eliminates vegetation protrusions. Dense point clouds undergo clustering and a proportional height filter which increases the density of vertical structures over 2,000%. Sparse and cluttered point clouds pass through an overlap filter which effectively identifies vertical structures amid clutter.

The second contribution is an exploration of the current challenges and mitigations for obstacle detection, followed by a simulator study that compared tower detection times for combinations of sensor visualizations and graphic augmentations. A set of focus groups revealed that detecting obstacles remains a significant challenge and that current mitigation strategies are not sufficient to prevent collisions. A subsequent human-in-the-loop simulator study revealed that graphic augmentations led to faster tower detection time when ambient visibility and illumination was reduced close to the limit for visual flight. Bounding boxes around towers were detected first in all conditions but tended to mask the obstacle they were meant to highlight. Sensor visualization affected tower detection time only at night, where night vision goggles were more effective than the infrared thermal sensor.

The third contribution of this dissertation is a method to efficiently correlate vertical structure observations with existing databases and infer the presence of power lines. The method uses a spatial hash key which compares an observed tower to existing towers and updates similar objects based on height and position. When applied to Delaware’s Digital Obstacle File, average horizontal uncertainty decreased from 206 to 56 ft. Power line presence is inferred by automatically examining the arrangement of towers in the more accurate database. Over 87% of electrical transmission towers were correctly identified with no false negatives.

In summary, this thesis contributes to a better understanding of the current limitations of vertical obstacle detection and avoidance. It proposes and assesses modular methods to automatically detect, catalog, and categorize hazardous obstacles that are currently neglected, and it evaluates the effectiveness of current visualization technologies and sensor- and database-informed graphic augmentations for supporting pilots in the timely and reliable detection of towers. Taken together, this research will contribute to enhanced aviation safety in the low altitude environment.