Using Selenium to Scrape ASP.NET Pages with AJAX Pagination

In my last post I went over the nitty-gritty details of how to scrape an ASP.NET AJAX page using Python mechanize. Since mechanize can’t process Javascript, we had to understand the underlying data formats used when sending form submissions, parsing the server’s response, and how pagination is handled. In this... Continue reading

Scraping ASP.NET Pages with AJAX Pagination

In a previous post I showed how to scrape a page that uses AJAX to return results dynamically. In that example, the results were easy to parse (XML) and the pagination scheme was straightforward (page number in the AJAX query JSON). In this post, I’ll show a more complicated example... Continue reading

Scraping with CasperJS

UPDATE 09/27/2018 - The site changed after this article was originally written. I’ve updated the code that waits for the jobs to load, along with the description in this article. Continue reading

Scraping AJAX Pages with Python

In this post I’ll show an example of how to scrape AJAX pages with Python. Continue reading

Getting Amazon Reviews for Library Books with Python

Often when I want to read about a new technical subject, I’ll see what’s available for free at the local library before buying a new book. To decide which book to check out, I browse through the library’s catalogue and look up the reviews for each book on Amazon to... Continue reading