Performance, Security, and Safety Requirements Testing for Smart Systems Through Systematic Software Analysis

Abstract

Smartphones, wearable devices and emerging autonomous vehicles (AVs) are significantly transforming our way of communication, networking, knowledge acquisition, healthcare and transportation. As our daily lives are increasingly relying on these smart end systems, certain guarantees on the performance, security and safety becomes critical requirements to the design and implementation of the software for these systems. To ensure such key requirements are met before shipping the software into users’ devices/vehicles, it is necessary to exhaustively test and verify the software at the development and testing stage. However, testing and verifying the performance, security and safety requirements for the software of these systems remains a research challenge. Due to the high mobility of these systems in the real world, the runtime environments faced by these systems vary significantly, which poses challenges to the testing and validation of performance requirements. Also, due to the layering design fashion and multi-party development process, software running on these systems is usually highly complex, potentially enlarging attack surface and posing challenges to the testing and validation of security and safety requirements.

To address this challenge, this dissertation focuses on developing systematic and automated software analysis tools for testing the performance, security and safety requirements of the software for smart end systems. Specifically, we demonstrate that automated program analyses based on 1) static program analysis and 2) runtime program profiling with certain system domain-specific customization, can lead to effective testing and validation of key performance, security and safety requirements for smart system software. This dissertation contributes to the performance, security and safety requirements testing of smart end systems in following aspects: (1) effectively test performance requirements and diagnose the cause of performance slowdown through lightweight monitoring of and systematic performance characterization based on cross-layer runtime events, (2) systematically detect noncompliance with important security principles (e.g., publish-subscribe overprivilege vulnerability) through systematic program analysis and mitigate security vulnerabilities through policy enforcement, and (3) systematically verify the compliance with safety requirements on the mission-critical components (e.g., AV’s driving decision control) of smart end systems.