Package org.apache.nutch.util

Class NutchTool

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.nutch.util.NutchTool

All Implemented Interfaces:

Configurable

Direct Known Subclasses:

CommonCrawlDataDumper, CrawlDb, DeduplicationJob, Fetcher, Generator, IndexingJob, Injector, LinkDb, ParseSegment

public abstract class NutchTool
extends Configured

Field Summary

Fields

Modifier and Type	Field	Description
protected Job	currentJob
protected int	currentJobNum
protected int	numJobs
protected HashMap<String,Object>	results
protected Map<String,Object>	status

Constructor Summary

Constructors

Constructor	Description
NutchTool()
NutchTool(Configuration conf)

Method Summary

Modifier and Type	Method	Description
float	getProgress()	Get relative progress of the tool.
Map<String,Object>	getStatus()	Returns current status of the running tool
boolean	killJob()	Kill the job immediately.
abstract Map<String,Object>	run(Map<String,Object> args, String crawlId)	Runs the tool, using a map of arguments.
void	setConf(Configuration conf)
boolean	stopJob()	Stop the job with the possibility to resume.

Methods inherited from class org.apache.hadoop.conf.Configured

getConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

results

protected HashMap<String,Object> results

status

protected Map<String,Object> status

currentJob

protected Job currentJob

numJobs

protected int numJobs

currentJobNum

protected int currentJobNum

Constructor Detail

NutchTool

public NutchTool(Configuration conf)

NutchTool

public NutchTool()

Method Detail

run

public abstract Map<String,Object> run(Map<String,Object> args,
                                             String crawlId)
                                      throws Exception

Runs the tool, using a map of arguments. May return results, or null.

Parameters:

args - a Map of arguments to be run with the tool

crawlId - a crawl identifier to associate with the tool invocation

Returns:

Map results object if tool executes successfully otherwise null

Throws:

Exception - if there is an error during the tool execution

setConf

public void setConf(Configuration conf)

Specified by:

setConf in interface Configurable

Overrides:

setConf in class Configured

getProgress

public float getProgress()

Get relative progress of the tool. Progress is represented as a float in range [0,1] where 1 is complete.

Returns:

a float in range [0,1].

getStatus

public Map<String,Object> getStatus()

Returns current status of the running tool

Returns:

a populated Map, the fields of which can be accessed to obtain status.

stopJob

public boolean stopJob()
                throws Exception

Stop the job with the possibility to resume. Subclasses should override this, since by default it calls killJob().

Returns:

true if succeeded, false otherwise

Throws:

Exception - if there is an error stopping the current Job

killJob

public boolean killJob()
                throws Exception

Kill the job immediately. Clients should assume that any results that the job produced so far are in an inconsistent state or missing.

Returns:

true if succeeded, false otherwise.

Throws:

Exception - if there is an error stopping the current Job