Package org.apache.nutch.util

Class AbstractChecker

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.nutch.util.AbstractChecker

All Implemented Interfaces:

Configurable, Tool

Direct Known Subclasses:

CrawlDbReader, IndexingFiltersChecker, LinkDbReader, ParserChecker, URLFilterChecker, URLNormalizerChecker

public abstract class AbstractChecker
extends Configured
implements Tool

Scaffolding class for the various Checker implementations. Can process cmdline input, stdin and TCP connections.

Author:

Jurian Broertjes

Field Summary

Fields

Modifier and Type	Field	Description
protected boolean	keepClientCnxOpen
protected boolean	stdin
protected int	tcpPort
protected String	usage

Constructor Summary

Constructors

Constructor	Description
AbstractChecker()

Method Summary

Modifier and Type	Method	Description
protected ProtocolOutput	getProtocolOutput(String url, CrawlDatum datum, boolean checkRobotsTxt)
protected int	parseArgs(String[] args, int i)
protected abstract int	process(String line, StringBuilder output)
protected int	processSingle(String input)
protected int	processStdin()
protected int	processTCP(int tcpPort)
protected int	run()

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.conf.Configurable

getConf, setConf

Methods inherited from interface org.apache.hadoop.util.Tool

run

Field Detail

keepClientCnxOpen

protected boolean keepClientCnxOpen

tcpPort

protected int tcpPort

stdin

protected boolean stdin

usage

protected String usage

Constructor Detail

AbstractChecker

public AbstractChecker()

Method Detail

process

protected abstract int process(String line,
                               StringBuilder output)
                        throws Exception

Throws:

Exception

parseArgs

protected int parseArgs(String[] args,
                        int i)

run

protected int run()
           throws Exception

Throws:

Exception

processSingle

protected int processSingle(String input)
                     throws Exception

Throws:

Exception

processStdin

protected int processStdin()
                    throws Exception

Throws:

Exception

processTCP

protected int processTCP(int tcpPort)
                  throws Exception

Throws:

Exception

getProtocolOutput

protected ProtocolOutput getProtocolOutput(String url,
                                           CrawlDatum datum,
                                           boolean checkRobotsTxt)
                                    throws Exception

Throws:

Exception