Advancing Environmental Applications through Machine Learning and Computer Vision: Modeling, Algorithms, and Real-World Implementations

Abstract

The escalating concern over environmental challenges has spurred a growing interest in harnessing machine learning and computer vision techniques to represent scenes in environmental applications. Accurate and efficient scene representations play a pivotal role in addressing environmental issues, including air pollution, fire detection, and remote sensing analysis. This dissertation delves into the field of scene representations in machine learning and computer vision, with a specific focus on image-based approaches for environmental applications.

For vision-based air pollution applications, air quality can be estimated by observing haze effects in images; hence, digital cameras can be used to quantify pollutants across large areas. We propose to use vision-based air pollution algorithms to predict the level of air pollution within the environment. The prevalence of images suggests that images can be used to estimate high spatial resolution air pollutant concentrations. However, there are many challenges to develop a portable, inexpensive, and accurate method for pollutant analysis, such as image quality variability, sufficient data for training, and hardware and software optimizations to meet constraints.

I address those challenges by designing image-based air pollution prediction methods for sensing and forecasting, developing benchmark datasets to test and validate vision-based pollution estimation algorithms, and determining how sensing accuracy depends on point sensor density and use of cameras. My efforts can be divided into three categories: (1) We design an image-based multi-pollutant estimation algorithm that is capable of modeling atmospheric absorption in addition to scattering, spatial variation, and color dependence of pollution; (2) We use different spatial densities of sensors and vision-based algorithms to estimate air pollution concentrations and analyze hazy images; (3) We construct an image-based air quality forecasting model that fuses a history of PM2.5 measurements with colocated images (at the same spot); and (4) We develop an image-based air quality prediction model specifically tailored to the nighttime case.

All the techniques are evaluated and validated using real-world data. Experimental results show that our techniques can reduce sensing error significantly. For example, our multi-pollutant estimation technique reduces single-pollutant estimation RMSE (root mean square error) by 22% compared to previous existing vision-based techniques; for the images in our benchmarking dataset, using images decreases MAE (mean absolute error) by 8.4% on average; therefore, adding a camera to collect images helps more than adding more sensors. Finally, experiments on Shanghai data show that our forecasting model improves PM2.5 prediction accuracy by 15.8% in RMSE and 10.9% in MAE compared to previous forecasting methods.

Furthermore, two innovative deep learning models were introduced to address segmentation tasks in different environmental domains. The first model focused on fire segmentation in images, incorporating a multi-scale aggregation module and a context-oriented module to achieve accurate and rapid fire detection by extracting discriminative features from various receptive fields and capturing both local and global context information. The proposed fire segmentation network outperformed previous methods with a significant 2.7% improvement in Intersection over Union (IoU). The second model targeted remote sensing segmentation in aerial images, enhancing feature representation in the spatial and frequency domains through a Frequency Weighted Module and a Spatial Weighting Module, respectively. Additionally, a Multi-Domain Fusion Module was employed to combine features from different domains, leading to state-of-the-art performance on remote sensing datasets with a mean F1-score accuracy improvement of 1.9%.