Big data applications on flash storage with accelerators

Abstract

Fast content-based searches and complex analytics of the vast amount of data collected via social media, cell phones, ubiquitous smart sensors, and satellites is likely to be the biggest economic driver for the IT industry over the next decade. Most such applications are performed on large clusters where the data resides in the collective DRAM of the cluster. Computing on such clusters is expensive and consumes enormous amount of power. We will present a cheaper and cooler alternative which provides high-performance, high-capacity, scalable random-access flash storage, and allows computation near the data via FPGA-based programmable flash controllers. We will discuss the preliminary results for a key-value store (KVS) implementation and several algorithms – Terabyte Sorts, PageRank and Breadth First Search, on BlueDBM consisting of 20 nodes and 20TB of flash.