Package org.apache.nutch.protocol.okhttp

Class OkHttp

java.lang.Object

org.apache.nutch.protocol.http.api.HttpBase

org.apache.nutch.protocol.okhttp.OkHttp

All Implemented Interfaces:

Configurable, Pluggable, Protocol

public class OkHttp
extends HttpBase

Field Summary

Fields

Modifier and Type	Field	Description
protected static org.slf4j.Logger	LOG

Fields inherited from class org.apache.nutch.protocol.http.api.HttpBase

accept, acceptCharset, acceptLanguage, BUFFER_SIZE, COOKIE, enableCookieHeader, enableIfModifiedsinceHeader, maxContent, maxCrawlDelay, maxDuration, partialAsTruncated, proxyException, proxyHost, proxyPort, proxyType, RESPONSE_TIME, responseTime, storeHttpHeaders, storeHttpRequest, storeIPAddress, timeout, tlsCheckCertificate, tlsPreferredCipherSuites, tlsPreferredProtocols, useHttp11, useHttp2, useProxy, userAgent

Fields inherited from interface org.apache.nutch.protocol.Protocol

X_POINT_ID

Constructor Summary

Constructors

Constructor	Description
OkHttp()

Method Summary

Modifier and Type	Method	Description
protected okhttp3.OkHttpClient	getClient(URL url)	Distribute hosts over clients by host name
protected List<String[]>	getCustomRequestHeaders()
protected Response	getResponse(URL url, CrawlDatum datum, boolean redirect)
static void	main(String[] args)
void	setConf(Configuration conf)

Methods inherited from class org.apache.nutch.protocol.http.api.HttpBase

getAccept, getAcceptCharset, getAcceptLanguage, getConf, getCookie, getMaxContent, getMaxDuration, getProtocolOutput, getProxyHost, getProxyPort, getRobotRules, getTimeout, getTlsPreferredCipherSuites, getTlsPreferredProtocols, getUseHttp11, getUserAgent, isCookieEnabled, isIfModifiedSinceEnabled, isStoreHttpHeaders, isStoreHttpRequest, isStoreIPAddress, isStorePartialAsTruncated, isTlsCheckCertificates, logConf, main, processDeflateEncoded, processGzipEncoded, useProxy, useProxy, useProxy

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

protected static final org.slf4j.Logger LOG

Constructor Detail

OkHttp

public OkHttp()

Method Detail

setConf

public void setConf(Configuration conf)

Specified by:

setConf in interface Configurable

Overrides:

setConf in class HttpBase

getCustomRequestHeaders

protected List<String[]> getCustomRequestHeaders()

getClient

protected okhttp3.OkHttpClient getClient(URL url)

Distribute hosts over clients by host name

Parameters:

url - URL to fetch

Returns:

client responsible to fetch the given URL

getResponse

protected Response getResponse(URL url,
                               CrawlDatum datum,
                               boolean redirect)
                        throws ProtocolException,
                               IOException

Specified by:

getResponse in class HttpBase

Throws:

ProtocolException

IOException

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception