Solve hCaptcha in Scrapy
When a response contains an hCaptcha widget, pause the crawl for one solve, then replay the request with the token in the form body.
Steps
- In the callback, look for data-sitekey. If it is absent, parse the page as usual.
- Call the API with that sitekey and response.url.
- Return a new FormRequest that includes h-captcha-response, or do the same inside downloader middleware so every spider gets it.
python
import os, re, scrapy, requests class LoginSpider(scrapy.Spider): name = "login" start_urls = ["https://example.com/login"] def parse(self, response): match = re.search(r'data-sitekey="([^"]+)"', response.text) if not match: return solved = requests.post( "https://api.vanishcaptcha.com/v1/solves", params={"wait": 90}, headers={"Authorization": f"Bearer {os.environ['VANISHCAPTCHA_KEY']}"}, json={"type": "hcaptcha", "sitekey": match.group(1), "url": response.url}, timeout=120, ).json() return scrapy.FormRequest.from_response( response, formdata={"h-captcha-response": solved["token"]}, callback=self.after_login, )
Enterprise rqdata
If the widget carries rqdata, send type: "hcaptcha_enterprise" and that value. The token that comes back is what the enterprise sitekey expects. Fields are in the API reference.
Questions
Does this solve reCAPTCHA or Turnstile?
No. These guides are hCaptcha only. reCAPTCHA and Turnstile are not live on VanishCaptcha.
How long is the token good for?
About two minutes, and it can be submitted once. Request it immediately before the form goes out.