Description
Day 2, 13:55-14:25
Abstract
PyMetheus is a project that the author started in order to monitor Raspberry PI units at home after the author found out that one of the RPis experiences intermittent failure. Although Prometheus can be a better choice in monitoring, however some RPis suffer crashing after installing the Prometheus' exporters. Hence the author started to learn about the Prometheus' architecture and began mimicking a Python based server-client monitoring system that can be installed on RPi zero, together with Grafana dashboard to display the monitoring data. Apart from that the author also integrates email and sms/phone notification system using external mail API and QPython on Android when there is an alert triggered in the monitoring system. The author is also looking in implementating PyMetheus as alternative for IoT system monitoring. The author likes to take the opportunity to share his experience in crafting and implementing the monitoring system using embedded SOC system.
Description
The Motivation The purpose of the talk is to demonstate how the author can design a monitoring system using small and portable embedded system with limited RAM such as Raspberry PI to monitor his home-made RPIs based data-centers.
The Architecture The client The client uses requests and posts the monitoring data to PyMetheus server via REST API. The client can also be crafted using urequests in MicroPython.
The server The server consists of several components namely:
Aggregator The Aggregator is responsible to collect and aggregate the data collected from clients via REST API, using flask framework. The Aggregator stores the data into the database, including the monitoring events from the client.
Database The Database is responsible to store the data collected from the client. The default database used in current version are sqlite3 and postgressql. The Database can be used as integration with the notification, reporting, as well as other third party applications via PyMetheus Open API.
Notification The current alert notification is integrated with external email API. The author also includes the Android integration system that the author had crafted previously using Qpython.
The reporting Grafana will be the current reporting dashboard but the author will be designing a new dashboard in coming version.
Modular - You choose what to monitor PyMetheus comes with some basic monitoring packages for monitoring, however it contains a base class that the user can design their own monitoring system, by overriding the class based on their requirements easily.
Third-party Integration Since PyMetheus contains Open API, therefore it allows the user to build their application using the PyMetheus Open API. Apart from that the user can also crafts their own monitoring system on the client side using the base class modular that the author builds.
Slides not uploaded by the speaker. HackMD: https://hackmd.io/@pycontw/2021/%2F%40pycontw%2FS15SHN5ft
Speaker: Ing Wei Tang
Ing Wei is the chair for PyCon MY 2019, the co-chair of PyCon MY 2018, and vice president of MyPOP. He has spoken in various PyCons, particularly in PyCon APAC 2018, as well as involving in PyCon communities actively in Malaysia. He uses python a lot in his daily work, especially coding the automation process and flow. During his past time, he likes to experience and perform hacking different things on operating system level. Apart from programming language, he can also speak 5 different types of languages concurrently in one sentence. Please ask for demo if time permits.