What is web scraping?
Web scraping also known as data scraping is the process of extracting valuable information from websites.
How does web scraping work?
When you copy and paste an email, a phone number or any piece of information from the internet to your document, you are basically doing the same process as a web scraper but manually.
In simple terms, web scrapers are basically copy and paste bots that can process data much faster than humans.
The crawler visits the webpage and retrieves all the information in its raw form. This is similar to when you open a web page in your browser but a bot can do this behind the scenes.
The parser takes the information from the crawler and extracts the text you want to save to a file or database.
Here in our documentation you can read all about the types of raw information Optimus Mine can parse.
What is web scraping used for and who uses data scraping?
The possibilities for webscraping are endless, because data can be aggregated and enriched in so many different ways, but here are some common use cases for webscraping.
A data connector for every source
Our integrations allow you to work with just about every database platform available, and if we don't have one that fits your needs, we'll build it for free!