Log in
Log inBook a demo

Professional Web Scraping with Java

COURSE
Stone River eLearning
1 hr

Professional Web Scraping with Java

COURSE
Stone River eLearning
1 hr
Included in GO1 PremiumStarting from $12 per user for teamsLearn moreTry for free
Included in GO1 PremiumStarting from $12 per user for teamsLearn moreTry for free

Course Overview

In this short and concise course you will learn everything to get started with web scraping using Java.
You will learn the concepts behind web scraping that you can apply to practically any web page (static AND dynamic / AJAX).

Course Structure
We start with an overview of what web scraping is and what you can do with it.
Then we explain the difference in scraping static pages vs dynamic / AJAX pages. You learn how to classify a website in one of the two categories and then apply the right concept in order to scrape the data you want.
Now you will learn how to export the scraped data either as CSV or JSON. These are some popular formats that can be used for further processing.
Unfortunately many websites try to block scrapers or sometimes you just do not want to be detected. In the section going undercover you will learn how to stay undetected and avoid getting blocked.
At the end of the course you can download the full source code of all the lectures and we discuss an outlook to some advanced topics (private proxies, cloud deployment, multi threading ...). Those advanced topics are covered in a follow up course I am going to teach.

Why you should take this course

  • Stop imagining you can scrape data from websites and use the skills for your next web project, you can do it now.
  • Stay ahead of your competition
  • Be more efficient and automate tedious, manual tasks
  • Increase your value by adding web scraping to your skill set

Requirements

  • You should already be familiar with Java and Maven at a basic to medium level (the course will not show you how to setup Java, Maven or an IDE).
  • You should be familiar with HTML/CSS and know how to use your browser's developer tools.
  • You should know about CSS selectors, though we use them for scraping static web pages.
  • Prior knowledge of jQuery helps you getting started faster with Jsoup, though this is not required.
  • You should know what a web API and AJAX is (basic level is enough).
Learning
Course Introduction
Promo Videovideo
Promo Video Keytext
Introductionvideo
Scraping Static Web Pages
What is a Static Web Page - Professional Web Scraping with Javavideo • 1 min
Concept How to Scrape Static Web Pages - Professional Web Scraping with Javavideo • 2 mins
Jsoup - the jQuery for Java - Professional Web Scraping with Javavideo • 6 mins
Example - Scraping Google - Professional Web Scraping with Javavideo • 15 mins
Scraping Dynamic / AJAX Web Pages
What is a Dynamic Web Page - Professional Web Scraping with Javavideo • 2 mins
Concept How to Scrape Dynamic Web Pages - Professional Web Scraping with Javavideo • 3 mins
Unirest - Professional Web Scraping with Javavideo • 12 mins
Example - Scraping peoplescrapers - Professional Web Scraping with Javavideo • 15 mins
Exporting Your Data
Export as CSV - Professional Web Scraping with Javavideo • 3 mins
Export as JSON - Professional Web Scraping with Javavideo • 5 mins
Going Undercover
How to Stay Undetected - Professional Web Scraping with Javavideo • 4 mins
Conclusion
Conclusionvideo
Downloadable Supporting Filestext