A Direct-Access File System for a New Generation of Flash Memory (thesis)

Abstract:

Flash memory has recently emerged as an important component of the storage
infrastructure, but presents several unique challenges to storage system designers:
individual blocks must be erased before they can be rewritten, block erasure is time
consuming, and individual blocks may be erased only a limited number of times. By
combining hardware parallelism and a log-structured approach, state of the art flash
storage systems can deliver two to three orders of magnitude more I/O operations
per second than existing high-performance fibre channel disk drives. Despite the
emergence of state of the art solid state disks (SSDs), the storage software stack has
changed little and is still optimized for magnetic disks; the basic abstraction remains
a linear array of fixed-size blocks together with a very large DRAM cache to convert
user I/O to a smaller number of large I/O transfers.

This thesis presents the design, implementation, and evaluation of the Direct File
System (DFS), describes the virtualized flash memory abstraction layer it depends
upon, and introduces some simple heuristics for improved flash-aware buffer cache
algorithms. We first examine the impact SSDs have had on the utility of the buffer
cache. As SSD performance and density improves, the value of very large DRAM
based buffer caches declines. We examine the change in tradeoffs through database
buffer cache traces and simulation and offer some simple heuristics to take advantage
of these changes. Second, we propose a richer interface more suitable for solid-state
storage systems. This interface provides for sparse block or object-based allocation,
atomic multi-block updates, and a block discard interface to facilitate reclamation
of unused storage. Finally, we present DFS, a file system designed specifically for
next generation flash memory systems that takes advantage of our proposed storage
interface. The result is a much-simplified file system implementation that runs under
Linux.

In both micro- and application benchmarks DFS shows consistent improvement
over ext3, a mature and widely deployed file system for Linux, both in throughput
and in CPU usage. For direct access, DFS delivers as much as a 20% performance
improvement in microbenchmarks. On an application level benchmark, DFS outperforms
ext3 by 7% to 250% while requiring less CPU power.