Hardware-Software Co-design for Cost-efficient, High-performance (and Intelligent) Distributed Stores
A great deal of digital data is generated every day. These data are stored in the cloud and processed by large-scale distributed systems, running a myriad of applications in data centers. Storage servers require a significant capital and operating cost of data centers. There have been a lot of efforts to build cost-effective and energy-efficient stores. One promising approach is replacing power-hungry and costly x86-based storage servers with lightweight ARM-based storage devices. To make this possible, however, various technical challenges, including efficient integration between distributed storage pools and application servers, high-performance controller/system design, and compatibility with existing applications, must be properly addressed.
In this presentation, I introduce research efforts at DGIST DataLab to tackle these technical challenges. Firstly, I present LightStore (ASPLOS’19), which integrates network interface-based KVSSDs to create large-scale storage pools and evaluates their potential benefits in terms of power and cost. Secondly, I present PinK (USENIX ATC’20), a storage controller that provides fast response times and bandwidth with bounded tails. Thirdly, I introduce KEVIN (OSDI’21), which efficiently builds up a POSIX-compatible file system over the KV interface. Finally, I present BigKV (EuroSys’23), a high-performance caching system based on AFA for large-scale storage pools. If time permits, I also discuss technical issues for accelerating inference and training of machine learning models through near-data processing in distributed storage clusters.
Sungjin Lee received the BE degree in electrical engineering from Korea University in 2005 and the MS and PhD degrees in computer science and engineering from Seoul National University in 2007 and 2013, respectively. He is an associate professor at Daegu Institute of Science and Technology (DGIST). Before joining DGIST in 2017, he was an assistant professor at Inha University (2016–2017) and a postdoctoral associate with the Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology (MIT), Cambridge, Massachusetts (2013–2016). His current research interests include system software, operating systems, storage systems, and AI systems.