I’m facing an issue when trying to access hotbit.io using Puppeteer. Each time I attempt to enter the site, I’m confronted with a message saying, “Checking your browser before accessing www.hotbit.io.” When I execute my script in non-headless mode, I can reach the page after about 5 seconds. However, my main issue is that I need to run it in headless mode. When switching to headless mode, it times out on the Cloudflare page, making it impossible to proceed.
I’ve made attempts using the “puppeteer-extra-plugin-stealth” and looked into “Cloudflare-scraper” but found the documentation lacking. From what I read, it seems that Cloudflare detects if I’m using headless mode.
Does anyone have advice on how to bypass the Cloudflare check when running in headless mode?
I’ve hit this same problem scraping Cloudflare-protected sites. They usually catch you on viewport settings and missing browser features that headless Chrome doesn’t have by default. Set a realistic viewport size and add JavaScript execution delays. Make sure you’re sending proper headers - Accept-Language and Accept-Encoding especially - so your requests look like they’re coming from a real browser. What fixed it for me was random delays between actions and making sure Chrome had all the standard browser features turned on. Also try adding a short delay after the page loads before you interact with anything - gives Cloudflare’s challenge time to finish.
yo, have you tried undetected-chromedriver? it works way better with cloudflare. also, check your user agent - it needs to match real browsers or you’ll get flagged. rotating proxies is key if you’re hitting the site too often.