Cassandra, a highly scalable NoSQL database, is renowned for its horizontal scaling capabilities, allowing seamless performance improvements by adding nodes. However, horizontal scaling often involves data migration, which can be time-consuming and complex. In contrast, vertical autoscaling offers a compelling alternative by adjusting resources on a single node, avoiding data migration and enabling flexible resource management. This article explores the nuances of vertical autoscaling in Cassandra, its technical challenges, algorithmic design, and practical applications, with a focus on optimizing resource utilization in Kubernetes environments.
Cassandra’s horizontal scaling model allows linear performance improvements by adding nodes, but this approach requires careful data redistribution and can introduce latency. Vertical autoscaling, on the other hand, adjusts CPU cores or memory on a single node, providing immediate performance gains without data migration. In Kubernetes, this is achieved by modifying Pod resource limits, enabling real-time adjustments without restarting containers. This method is particularly advantageous for workloads with predictable or periodic resource demands, such as those experiencing daily traffic peaks.
Adjusting CPU cores dynamically can disrupt application performance if not managed properly. For instance, SQL Server tests revealed that abrupt CPU core changes without application notification can reduce throughput. However, implementing resource hints (e.g., reducing thread counts) can improve transaction throughput by up to 30%. In Cassandra, the JVM’s availableProcessors method reads CPU cores at startup, and subsequent adjustments are not automatically recognized. To ensure accurate resource recognition, Cassandra must be launched with a high core count, followed by gradual reduction.
Unlike CPU, memory adjustment without restarting remains a technical challenge. Languages like Python and Java face hurdles in dynamically reallocating memory, making this area a focus for future research. Current efforts prioritize CPU optimization, but memory and I/O vertical scaling require further exploration.
The proposed algorithm combines reactive and predictive strategies to balance resource efficiency and performance. The reactive component monitors CPU usage trends using sliding windows and logarithmic calculations to determine scaling needs. Users can customize cost-benefit and performance priorities. The predictive component employs a simple time-series model, analyzing past 24-hour CPU patterns to forecast future demand. This modular design allows integration with more complex predictive models like ARIMA or LSTM.
Optimizing parameters is critical to minimizing resource waste (slack) and CPU throttling. Key considerations include:
To address the high cost of real-world testing, the Vertical Autoscaling Simulator (VASM) was developed. This tool replicates Kubernetes cluster resource characteristics, including core counts and node configurations, and supports replaying pre-recorded CPU usage traces. Key features include:
SQL Server tests showed that unannounced CPU adjustments degrade performance, but resource hints can boost throughput by 30%. In Cassandra, 6-node clusters exhibited periodic CPU fluctuations, with vertical scaling effectively aligning resources to demand. However, JVM resource recognition requires initial high-core configuration, as post-start adjustments are not automatically detected.
Traditional Kubernetes autoscaling algorithms perform single adjustments, struggling with periodic workloads. The new Casper algorithm, combining reactive and predictive components, improves resource utilization. Predictive models can preemptively reduce resources during low-load periods, minimizing waste.
Current systems lack memory adjustment without restarts, relying on application-level logic. Future research should explore memory and I/O vertical scaling, expanding beyond CPU-centric solutions.
Enhancing predictive models with advanced time-series analysis and automating parameter tuning will reduce manual intervention. Real-world validation of VASM results is also critical to ensure algorithm stability in production environments.
Vertical autoscaling in Cassandra offers a viable alternative to horizontal scaling, particularly for workloads with predictable resource patterns. While challenges like JVM resource recognition and memory adjustment persist, the integration of reactive-predictive algorithms and tools like VASM accelerates optimization. By balancing resource efficiency and performance, vertical autoscaling can enhance Cassandra’s scalability in Kubernetes environments. For developers, prioritizing parameter tuning and leveraging predictive models will maximize the benefits of this approach.