Description
Andrii Chaichenko - Building a RESTful real-time analytics system with Pyramid [EuroPython 2015] [23 July 2015] [Bilbao, Euskadi, Spain]
CeleraOne tries to bring its vision to Big Data by developing a unique platform for real-time Big Data processing. The platform is capable of personalizing multi-channel user flows, right-in time targeting and analytics while seamlessly scaling to billions of page impression. It is currently tailored to the needs of content providers, but of course not limited to.
- The platform’s architecture is based on four main layers:
- Proxy/Distribution -- OpenResty/LUA for dynamic request forwarding
- RESTful API -- several Python applications written using Pyramid web framework running under uWSGI server, which serve as an integration point for third party systems;
- Analytics -- Python API for Big Data querying and distributed workers performing heavy data collection.
- In-memory Engine -- CeleraOne’s NoSql database which provides both data storage and fast business logic.
In the talk I would like to give insights on how we use Python in the architecture, which tools and technologies were chosen, and share experiences deploying and running the system in production.