SFVInt: Simple, Fast and Generic Variable-Length Integer Decoding using Bit Manipulation Instructions

التفاصيل البيبلوغرافية
العنوان: SFVInt: Simple, Fast and Generic Variable-Length Integer Decoding using Bit Manipulation Instructions
المؤلفون: Liao, Gang, Liu, Ye, Ding, Yonghua, Cai, Le, Chen, Jianjun
سنة النشر: 2024
المجموعة: Computer Science
مصطلحات موضوعية: Computer Science - Databases, Computer Science - Distributed, Parallel, and Cluster Computing
الوصف: The ubiquity of variable-length integers in data storage and communication necessitates efficient decoding techniques. In this paper, we present SFVInt, a simple and fast approach to decode the prevalent Little Endian Base-128 (LEB128) varints. Our approach effectively utilizes the Bit Manipulation Instruction Set 2 (BMI2) in modern Intel and AMD processors, achieving significant performance improvement while maintaining simplicity and avoiding overengineering. SFVInt, with its generic design, effectively processes both 32-bit and 64-bit unsigned integers using a unified code template, marking a significant leap forward in varint decoding efficiency. We thoroughly evaluate SFVInt's performance across various datasets and scenarios, demonstrating that it achieves up to a 2x increase in decoding speed when compared to varint decoding methods used in established frameworks like Facebook Folly and Google Protobuf.
Comment: DaMoN 2024
نوع الوثيقة: Working Paper
URL الوصول: http://arxiv.org/abs/2403.06898
رقم الأكسشن: edsarx.2403.06898
قاعدة البيانات: arXiv