Description
Peter Hoffmann - SQLAlchemy as the backbone of a Data Science company [EuroPython 2016] [20 July 2016] [Bilbao, Euskadi, Spain] (https://ep2016.europython.eu//conference/talks/sqlalchemy-as-the-backbone-of-a-data-science-company)
In times of NoSQL databases and Map Reduce Algorithms it's surprising how far you can scale the relational data model. At Blue Yonder we use SQLAlchemy in all stages of our data science workflows and handle tenth of billions of records to feed our predictive algorithms. This talk will dive into SQLAlchemy beyond the Object Relational Mapping (ORM) parts and conentrate on the SQLAlchemy Core API, the Expression Language and Database Migrations with Alembic.
In times of NoSQL databases and Map Reduce Algorithms it's surprising how far you can scale the relational data model. At Blue Yonder we use SQLAlchemy in all stages of our data science workflows and handle tenth of billions of records to feed our predictive algorithms. This talk will dive into SQLAlchemy beyond the Object Relational Mapping (ORM) parts and conentrate on the SQLAlchemy Core API and the Expression Language:
- Database Abstraction: Statements are generated properly for different database vendor and type without you having to think about it.
- Security: Database input is escaped and sanitized prior to beeing commited to the database. This prevents against common SQL injection attacks.
- Composability and Reuse: Common building blocks of queries are expressed as SQLAlchemy selectables and can be reuesd in other queries.
- Testability: SQLAlchemy allows you to perform functional tests against a database or mock out queries and connections.
- Reflection: Reflection is a technique that allows you to generate a SQLAlchemy repesentation from an existing database. You can reflect tables, views, indexes, and foreign keys.
As a result of the usage of SQLAlchemy in Blue Yonder, we have implemented and open sourced a SQLAlchemy dialect for the in memory, column-oriented database system EXASolution