The Call of the Web Scraper

Astrid, our Data Big Bang and Nektra content editor, is heading to Nepal on a birding and trekking quest. She needs birds sounds from xeno-canto and The Internet Bird Collection to identify the hundreds of species found in Nepal, but the site does not offer batch downloads. We could not pass up the opportunity to offer a useful scraper for birders. We found a blog post with code to download batches of recordings for specific species (not specific countries): Web Scraping with BeautifulSoup and Python. Like most script developers. we want to do things our own way. Our code allows simultaneous download of calls to speed up the process for specially diverse countries.

Web scraping is often associated with indecorous Internet behavior, but in fact, it is also a way to automate tedious manual work. Imagine that you want to have the complete schedule from EasyJet to choose a flight. It can take less than one hour to scrape all the desired routes. Right now there are no entry-level tools for scraping sites like there are for photo editing. Fortunately, script developers share their scraping code on sites like ScraperWiki.

If you liked this article, you might also like:


  • John Frey

    It is step by step and very informative as well. Very helpful for me. Everyone must give a try to this Amazon product scraper for once to know the benefits.

    For More :

  • William Mike

    Looks like I’m a little late on this topic. but I thought I’d like to recommend you try out a new launched free web scraper, Octoparse.

    It scrapes unstructured data from websites into structured data that can be stored in some data files. It’s powerful and free. It provides operations such as click, hover, pull-down switching, as well as the combination like cycling and branch judgment. No coding required. Octoparse extracts the data you need automatically after you configure the rule. You can not miss it.

    Check the video out and see how to scrape FB using Octoparse.