Quick Pitch: 80legs is a web crawling and processing service that puts the power of 50,000 computers to use at a fraction of the cost.
Genius Idea: Web crawlers. Spiders. Indexing. They are all essential aspects of not only web searches, but of simply scouring the web for information. Without crawling, we wouldn't have updated information on our favorite websites or search engines at all.
Web crawling's not a simple business though, which is why Google spends millions per year on the task. Not a lot of small businesses can afford the price of crawling billions of web pages, although they could really use that type of information.
80Legs, which launched last week at the DEMO Conference, claims to democratize web crawling and make it effective and affordable to the small business and the entrepreneur, not just to the Googles and Bings of the world.
Using the interface is dead simple. Just pick where you want 80legs to crawl and set what you'd like it to find. Of course, there are a wide range of features for customizing your crawls as well, including changing the depth level, the number of URLs to crawl, the type of pages to analyze, whether to crawl outgoing links, and much more.
Once you've finished crawling, you'll get the data that overviews the results of your crawl.
