520 status code - what is it and how to avoid it?

A 520 status code is related to Cloudflare. It is used by Cloudflare as a catch-all response for when the origin server sends something unexpected. It might be caused by some technical issues on the website. However, it can also be caused if your requests do not contain the required data that the website is expecting. So make sure that you are including all the required headers (CORS, Referrer, Auth) in your requests.

If there are no technical issues in the web server and you are also including all the required headers and data, then your requests might be getting recognized by the server as being automated. So make sure you are using undetected-chromedriver and pairing it up with proxies from a reliable proxy provider like ScrapingBee. This should help solve the issue.

Related Web scraping blocked web scraping questions: