Package org.apache.nutch.fetcher

Class FetchItemQueue

java.lang.Object

org.apache.nutch.fetcher.FetchItemQueue

public class FetchItemQueue
extends Object

This class handles FetchItems which come from the same host ID (be it a proto/hostname or proto/IP pair). It also keeps track of requests in progress and elapsed time between requests.

Constructor Summary

Constructors

Constructor	Description
FetchItemQueue(Configuration conf, int maxThreads, long crawlDelay, long minCrawlDelay)

Method Summary

Modifier and Type	Method	Description
void	addFetchItem(FetchItem it)
void	addInProgressFetchItem(FetchItem it)
void	dump()
int	emptyQueue()
void	finishFetchItem(FetchItem it, boolean asap)
Text	getCookie()
FetchItem	getFetchItem()
int	getInProgressSize()
int	getQueueSize()
int	incrementExceptionCounter()
void	setCookie(Text cookie)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FetchItemQueue

public FetchItemQueue(Configuration conf,
                      int maxThreads,
                      long crawlDelay,
                      long minCrawlDelay)

Method Detail

emptyQueue

public int emptyQueue()

getQueueSize

public int getQueueSize()

getInProgressSize

public int getInProgressSize()

incrementExceptionCounter

public int incrementExceptionCounter()

finishFetchItem

public void finishFetchItem(FetchItem it,
                            boolean asap)

addFetchItem

public void addFetchItem(FetchItem it)

addInProgressFetchItem

public void addInProgressFetchItem(FetchItem it)

getFetchItem

public FetchItem getFetchItem()

setCookie

public void setCookie(Text cookie)

getCookie

public Text getCookie()

dump

public void dump()