Description
PyData Berlin 2016
Rather than using big data in the cloud, at Transit App we send data sets to the client and process it there. To keep the data small, we’ve switched to using binary files instead of plain text formats like ᴊsᴏɴ. In this talk we’ll discuss how you can turn data into binary, how it works, and our own approach to store complete transit schedules on the device and show their data instantly. Abstract
Rather than using big data in the cloud, we at Transit App found it useful to to give as much data to client as possible and process it client-side. In order to keep transfer volume and performance reasonable, it makes sense to move from text based interchange formats like ᴊsᴏɴ or xᴍʟ back back in time to using binary file formats. In this talk we will introduce some common libraries to make data compact, to make it fast, and how these libraries work. We will also discuss the approach of our own library for creating schema-less binary file formats, and how it helped us store complete transit schedules in small files on the phone and display their data instantly.