Description
PyData SF 2016
Smyte is blocking fraudsters, spammers, scammers and harassers through analysis of web and mobile event data. There are hundreds of new database options cropping up, and while most of them are great they all make a range of different design choices which may/may not align with what you need.
At Smyte we're solving infrastructure-heavy problems with a tiny engineering team. We've had to build custom databases in order to efficiently solve problems, but don't have the luxury of a ton of development time. In this talk I'll go through how we used Kafka & Python to implement a prototype Sliding HyperLogLog server. It held up for the six months we needed to, and some early decisions made it trivial to port the server to our new efficient C++ & RocksDB stack.