Accordion: Balancing Performance and Cost in Cloud-Native Data Analysis with Intra-Query Runtime Elasticity

Download PDF

Abstract

Cloud databases empower users to leverage vast computing resources for efficient data analysis. However, achieving cost-effective utilization of these resources remains a challenge. Users often struggle to balance computing resource allocation with their temporal and financial constraints. To address this, we propose the concept of Intra-Query Runtime Elasticity (IQRE), which allows a cloud-native OLAP engine to dynamically adjust the query Degree of Parallelism (DOP) during query execution. We introduce Accordion, the first IQRE engine. Accordion features a friendly user interface for parallelism adjustments. It includes an auto-tuner that supports both manual and automatic DOP tuning during query execution. In this demonstration, we present Accordion’s architecture and provide attendees with hands-on experience, allowing them to execute queries and adjust query parallelism during query execution based on their time or cost budgets.