Efficient Hierarchical Storage Management Empowered by Reinforcement Learning Extended Abstract
- Resource Type
- Conference
- Authors
- Zhang, Tianru; Hellander, Andreas; Toor, Salman
- Source
- 2023 IEEE 39th International Conference on Data Engineering (ICDE) ICDE Data Engineering (ICDE), 2023 IEEE 39th International Conference on. :3869-3870 Apr, 2023
- Subject
- Computing and Processing
Cloud computing
Heuristic algorithms
Storage management
Software algorithms
Reinforcement learning
Big Data
Data engineering
Data Management
Cloud Computing
Hierarchical Storage System
Data Migration
Reinforcement Learning
- Language
- ISSN
- 2375-026X
With the rapid development of big data and cloud computing, data management has become increasingly challenging. A possible solution is to use an intelligent hierarchical (multi-tier) storage system (HSS). An HSS is a meta solution that consists of different storage frameworks organized as a jointly constructed storage pool. A built-in data migration policy that determines the optimal placement of the datasets in the hierarchy is essential. Placement decisions are a non-trivial task since they should be made according to the characteristics of the dataset, the tier status in a hierarchy, and access patterns. This paper presents an open-source hierarchical storage framework with a dynamic migration policy based on reinforcement learning (RL).