61 points mfrw 1 day ago 7 comments

jpitz 1 day ago | parent

My introduction to probabilistic data structures was Pugh's skip list, which is probabilistic in structure and performance, but precise with answers.

aarjaneiro 1 day ago | parent

Note a lot of the diagrams seem broken

aarjaneiro 9 hours ago | parent

Seem fixed now

ncruces 1 day ago | parent

I'm in the process of adding a few of these to my Go SQLite driver.

https://pkg.go.dev/github.com/ncruces/go-sqlite3/ext/approx

They work as window functions, and they also support creating roll-up tables, e.g. you have a table storing a sketch of daily active users and can use that to get weekly and monthly active users.

Same for quantiles (e.g. median and tail latencies) and next up top-K (e.g. most popular content, highest spenders, etc).

donatj 22 hours ago | parent

I'm kind of sad there's no mention of Cuckoo filters. I've generally found them preferable to Bloom filters

kristianp 20 hours ago | parent

Now I've learnt about blocked Bloom filters:

> Unsure uses a blocked Bloom filter. The bit array is divided into 512-bit blocks, each matching the size of a CPU cache line. When you add or check an item, it is hashed once: part of the hash selects a block, and the rest selects k bits within that block.

The article also has a good quality list of references.

mfrw 16 hours ago | parent

A related CppCon talk: Fantastic Algorithms and Where To Find Them & the related GitHub Repo:

- https://youtu.be/YA-nB2wjVcI

- https://github.com/CppCon/CppCon2017