insane-search

When Claude Code hits a 403 and gives up, insane-search keeps going. It installs from a plugin marketplace URL with no API keys and no configuration, then takes over automatically whenever an ordinary fetch gets blocked. MIT licensed, free, and it pulls in its own dependencies like curl_cffi and yt-dlp the first time it needs them.

The mechanism is an escalation ladder. It starts with public endpoints and feeds, drops to mobile and .json and /rss variants of the URL, then impersonates a real browser's TLS fingerprint, and finally spins up an actual headless browser for JavaScript-rendered pages. Every response gets scanned for OGP and JSON-LD, so even a partial page yields a title and a summary. At a login wall or paywall it stops and reports authentication required instead of pretending.

Solo research and one-off lookups: it works, and it works on sites that routinely stonewall agents. Teams building something durable: do not put this in a pipeline you depend on.

The catch: everything past the syndication step is precisely what anti-bot systems exist to detect, and the project hands you the terms-of-service question directly. Reddit, X, and LinkedIn all prohibit this in writing. The usual outcome is not a lawsuit, it is your IP quietly getting nowhere.