Databases

As I am interating over the API design and the implementation of kelindar/column, I’ve started to build some toy examples. One of such toy examples I wanted to explore is how one might go about building a key-value cache using columnar storage. Now, let me first start by saying that doing this is not efficient and you are probably better off using a traditional key-value store organised as a B+Tree or a Hashmap, in fact, you’d be better of just using map[string]string than using the columnar store....

In the recent months, I’ve been working on a side-project of mine where I’m trying to build a high-performance, in-memory columnar datastore - kelindar/column. One of the initial builds had a huge lock around the every update and read, which is not really scalable since it’s rather prone to lock contentions. I’ve improved recently by introducing a sharded mutex, essentially an array of 128 locks with some padding around them to avoid false sharing....

Besides working on the new experimentation platform in Careem, over the past few weekends I’ve been toying with building a high-performance columnar, in-memory store in Go. Most importantly, I wondered if it is possible to build a nice API while having data laid out in a very cache-friendly way and support indexing. In addition, most of columnar data stores are designed primarily for OLAP and there’s not too many OLTP or HTAP stores built in a columnar fashion so can we build something for OLTP?...