I'm trying to scrap all the historical filings of a particular Central Index Key (CIK) using python but I get http error codes.
First, I got 403 error, I resolved by using requests session and timing the requests every 5 seconds.
Now, I get http 400 after processing 30-40 requests. Not sure how to resolve this.
I have set the user-agent field as well.