Consumer Distributed File Systems.

Abstract

Local file systems file systems are no longer sufficient to manage consumer data due to the emergence of consumer electronics (CE) devices such as cell phones and digital cameras and the rapid increase of multimedia data. CE devices are a problem for local file systems because they store user data outside the purview of the local file system on heterogeneous devices. This dissertation demonstrates techniques to connect CE devices to a distributed file system with the goal of bringing all of a user’s storage devices into a single distributed storage system to ease management. This work also demonstrates ensembles, groups of disconnected devices that can band together to provide each other with data and updates, improving data consistency and availability. New, higher level management techniques are also needed to help manage the growing multimedia data. This work introduces persistent queries, a reliable notification mechanism for applications that allows them to provide type-specific behavior across the distributed file system. To allow the addition of type-aware features in the file system itself, we designed sprockets, a technique that allows code to be safely loaded and run in the address space of a distributed storage system. Finally, we built TrapperKeeper, a system that simplifies the extraction of file metadata by reusing the file parsing and display behavior of existing applications instead of requiring the special-purpose file-parsing code be built. By combining these contributions, we have extended the reach of distributed file systems to cover all of a user’s storage and we have simplified data management by providing mechanisms to extract and use type information to handle distributed data.