Hermes: Off-the-Shelf Real-Time Transactional Analytics

Download PDF

Abstract

Many modern applications require real-time analytics, where analytical processing (AP) workloads needs access to the latest data updates from a transactional processing (TP) engine. However, managing separate TP and AP engines across teams complicates achieving real-time analytics without switching to specialized HTAP systems. To address this challenge, we introduce off-the-shelf real-time analytics , a system design that leverages the existing TP and AP engines to provide (1) the latest transactional updates for analytical queries and (2) support for efficient transactional analytics –transactions that combine transactional logic and analytical queries within a single ACID transaction–at various isolation levels. We demonstrate this concept with a new service called Hermes , which acts as a middleware that merges log records with analytical reads without altering existing engines. Our evaluation utilizes two AP engines, FlexPushdownDB and DuckDB , with MySQL as the TP engine. Using the HATtrick benchmark and a new workload called Transactional Analytics Workload (TAW), we compare Hermes with the leading HTAP solution, TiDB . Our results indicate that Hermes performs comparably to current HTAP solutions for real-time analytics and surpasses them by 3 × in transactional analytics performance.