Class FreeGenerator

  • All Implemented Interfaces:
    Configurable, Tool

    public class FreeGenerator
    extends Configured
    implements Tool
    This tool generates fetchlists (segments to be fetched) from plain text files containing one URL per line. It's useful when arbitrary URL-s need to be fetched without adding them first to the CrawlDb, or during testing.