PyCon 2009: Scrape the Web: Strategies for programming websites that don't expect it (Part 1 of 3)
[VIDEO HAS ISSUES: Speaker walked away from the mic most of the time.] Do you find yourself faced with websites that have data you need to extract? Would your life be simpler if you could programmatically input data into web applications, even those tuned to resist interaction by bots? We'll discuss the basics of web scraping, and then dive into the details of different methods and where they are most applicable. You'll leave with an understanding of when to apply different tools, and learn about a "heavy hammer" for screen scraping that I picked up at a project for the Electronic Frontier Foundation. Atendees should bring a laptop, if possible, to try the examples we discuss and optionally take notes.