Engula

Engula
docs
docs

Release Notes

Release Notes: Engula v2.0

Release Window: December 2024 – March 2025 Focus: Core Memory Engine Integration, Basic Defragmentation, and Data Structure Support.

🚀 Highlights

This release introduces the eallocator memory engine, integrating it deeply with Valkey. It establishes the foundation for high-performance memory management, including ZSTD compression, memory block defragmentation, and specialized handling for complex data types (Hash, Set, Listpack).

✨ New Features

Core Engine Integration:

  • Integrated Valkey with eallocator and wrapped the key-value store into ekvstore.
  • Introduced eallocator support for basic memory arena management.
  • Added support for Mutable and Immutable memory blocks.

Compression & Encoding:

  • Added support for ZSTD compression in earena.
  • Implemented Variable Byte Encoding for integers.

Data Structure Support:

  • Added eallocator support for Hash objects and Set objects.
  • Refactored Listpack to support Set and Hash objects efficiently.
  • Implemented eobj serialization and deserialization.

Observability:

  • Introduced the engula info command to print internal statistics.

🛠 Improvements & Refactoring

Defragmentation:

  • Rolled out Memory Defragmentation Phase 2 (merging blocks) and Phase 3.
  • Added background threads specifically for zip and defragmentation tasks.
  • Optimization: Empty blocks are no longer added to the defrag request list.

Memory Management:

  • Supported up to 32KB record contents in dense/sparse memory blocks.
  • Optimized async deletion using a waitable queue in batch mode to reduce CPU usage.
  • Refactored elpobj pointers.

🐛 Bug Fixes

  • Fixed ZUNION errors when mixing Set and ZSet types.
  • Fixed bugs related to HINCRBY results not saving to listpack.
  • Resolved memory leaks in e-record and fixed "access after free" issues.
  • Fixed Universal Test (UT) errors for bit operations.

Release Notes: Engula v2.1

Release Window: April 2025 – November 2025 Focus: Performance Tuning, Large Value Support, Modules, and Upstream Sync (Valkey 8.1 / Redis 7.2).

🚀 Highlights

Version 2.1 focuses on significant performance optimizations (including IO threads and hashtable improvements) and massive scalability, adding support for values larger than 4GB. It also ensures compatibility with the latest upstream versions of Valkey (8.1.4) and Redis (7.2).

✨ New Features

Scalability & Storage:

  • Added support for values larger than 4GB.
  • Implemented Active Defragmentation.
  • Added support for compress groups.

Module System:

  • Added Engula module support.
  • Implemented Module GIL (Global Interpreter Lock) support.

Upstream Compatibility:

  • Merged Valkey 8.1.4 and Valkey 8.1.1.
  • Adapted codebase to support Redis 7.2 and related unit tests.

⚡ Performance Improvements

IO & Threading:

  • Performance improvements for 8 IO threads and 2 IO threads.
  • Added async IO notify functionality.

Hashtable Optimizations:

  • Improved overall performance for hashtable GET operations.
  • Optimized rehash performance and reduced decompression counts for HMGET.
  • Implemented memory prefetch for rehash operations.

Memory & Latency:

  • Reduced p99 latency.
  • Implemented quick async free.
  • Disabled lazyfree_lazy_user_flush by default to optimize flushing behavior.
  • Supported 64KB page size for ARM64 architecture.

🛠 Infrastructure & CI

Build System:

  • Added automated packaging for Linux EL7 (CentOS 7) tarballs.
  • Added montplex/ubuntu-build docker image for actions.
  • Updated GitHub Actions workflows significantly.

🐛 Bug Fixes

  • Fixed VM_HSet bug in the module system.
  • Fixed issues related to test scripts.
  • Addressed compilation tasks in eallocator.
ON THIS PAGE