Contribute Media
A thank you to everyone who makes this possible: Read More

High Fidelity Web Crawling in Python

Description

Python modules such as Requests make it easy for Python to pull HTML from a webpage which you can feed to your parsing function. What becomes difficult is converting that process into an autonomous process to crawl webpages to parse their HTML for data. This talk covers the lessons learns and solutions I’ve found to create high fidelity autonomous web crawling scripts in Python.

Details

Improve this page