Scraping data? Protect your IP with a proxy server

A quick look at implementing proxy settings in Selenium

The internet is a trove of (publicly accessible) data, and when it comes to Python, there is no shortage of tools you can use for treasure hunting.

A proxy server is an intermediary acting as a gateway that separates your machine from the website you are accessing.

Keep in mind if you’re working with data that should be encrypted, then you should opt for a VPN (Virtual Private Network). The use of the proxy in the context of this data scraping is simply to protect the IP of your machine.

Quick highlights:

The code in this post was written using Python 3.6.8 🐍.

You’ll need a proxy server. Preferably not one of the free ones you find off of a Google search. The IP of my machine was already whitelisted in the Smartproxy dashboard, so you will not see a username:password authentication in the code.

Related posts:

Before I started my business, I was a working professional. During an interview for the position of “Digital Marketing Head”, I easily cleared out the first three rounds with HR, marketing Manager…