Difference between Web Scraping and Web Crawling

1. Web Scraping :

Web Scraping is a technique used to extract a large amount of data from websites and then saving it to the local machine in the form of XML, excel or SQL. The tools used for web scraping are known as web scrapers. On the basis of the requirements given, they can extract the data from any website in a fraction of time. This automation of tasks is very helpful for developing data for machine learning and other purpose. They work in four steps: 

1. Sending the request to the target page.

2. Getting response from the target page.

3. Parsing and extracting the response.

4. Download the data. 

Some of the popular web scraping tools are ProWebScraper, Webscraper.io, etc. 

2. Web Crawling :

Web Crawling is analogous to a spider crawling but the place of crawling here is the web!. It basically visits a website and read web pages for the purpose of building entries for search engine index. The tools that are used for web crawling are known as web crawlers or spiders. A series of web pages are analyzed and links to the pages on them are then followed for even more links thus it does a deep search for extracting of information. Famous search engines such as Google, Yahoo and Bing do web crawling and use this information for indexing web pages. Examples are Scrapy and Apache nut. 

Difference between Web Scraping and Web Crawling: 

Web Scraping	Web Crawling
The tool used is Web Scraper.	The tool used Web Crawler or Spiders.
It is used for downloading information	It is used for indexing of Web pages
It need not visit all the pages of website for information.	It visits each and every page, until the last line for information.
A Web Scraper doesn’t obey robots.txt in most of cases.	Not all web crawlers obey robots.txt.
It is done on both small and large scale.	It is mostly employed in large scale.
Application areas include Retail Marketing, Equity search, and Machine learning.	Used in search engines to give search results to the user.
Data de-duplication is not necessarily a part of Web Scraping.	Data de-duplication is and integral part of Web Scraping.
This needs crawl agent and a parser for parsing the response.	This only needs only crawl agent.
ProWebScraper, Web Scraper.io are the examples	Google, Yahoo or Bing do Web Crawling

Source: geeksforgeeks.org