Sign in Subscribe

Crawler

Crawler

Let's build something to extract all the content from the web. Who said that we should not use all the information available. Let's crawl, crawl and crawl again. We'll stop when there is nothing left.

A Twitter Crawler and News Indexer

A Twitter Crawler and News Indexer

TrollHunter is a Twitter Crawler & News Website Indexer. It aims at finding Troll Farmers & Fake News on Twitter.

Get All URLs From A Website Using Python Script

Get All URLs From A Website Using Python Script

You can easily extract all the links on a web page using python script. Have you ever wanted to extract all the URLs of a website quickly? We'll tell you how! It is hundreds of times faster than crawling all the pages of a website to find all of its URLs.