Smart Web Crawler

This demo web crawler illustrates how token-based payments can be used to access and crawl protected websites.
There are two sample URLs provided:

• Unprotected Website: Accessible to the crawler without any token.
• Protected Website: Requires a kya+pay token to allow crawler access.