Package org.apache.nutch.protocol.selenium

Class HttpWebClient

java.lang.Object

org.apache.nutch.protocol.selenium.HttpWebClient

public class HttpWebClient
extends Object

Constructor Summary

Constructors

Constructor	Description
HttpWebClient()

Method Summary

Modifier and Type	Method	Description
static void	cleanUpDriver(org.openqa.selenium.WebDriver driver)
static org.openqa.selenium.remote.RemoteWebDriver	createChromeRemoteWebDriver(URL seleniumHubUrl, boolean enableHeadlessMode)
static org.openqa.selenium.WebDriver	createChromeWebDriver(String chromeDriverPath, boolean enableHeadlessMode)
static org.openqa.selenium.remote.RemoteWebDriver	createDefaultRemoteWebDriver(URL seleniumHubUrl, boolean enableHeadlessMode)
static org.openqa.selenium.remote.RemoteWebDriver	createFirefoxRemoteWebDriver(URL seleniumHubUrl, boolean enableHeadlessMode)
static org.openqa.selenium.WebDriver	createFirefoxWebDriver(String firefoxDriverPath, boolean enableHeadlessMode)
static org.openqa.selenium.remote.RemoteWebDriver	createRandomRemoteWebDriver(URL seleniumHubUrl, boolean enableHeadlessMode)
static org.openqa.selenium.WebDriver	getDriverForPage(String url, Configuration conf)
static String	getHtmlPage(String url)
static String	getHtmlPage(String url, Configuration conf)	Function for obtaining the HTML using the selected selenium webdriver There are a number of configuration properties within nutch-site.xml which determine whether to take screenshots of the rendered pages and persist them as timestamped .png's into HDFS.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HttpWebClient

public HttpWebClient()

Method Detail

getDriverForPage

public static org.openqa.selenium.WebDriver getDriverForPage(String url,
                                                             Configuration conf)

createFirefoxWebDriver

public static org.openqa.selenium.WebDriver createFirefoxWebDriver(String firefoxDriverPath,
                                                                   boolean enableHeadlessMode)

createChromeWebDriver

public static org.openqa.selenium.WebDriver createChromeWebDriver(String chromeDriverPath,
                                                                  boolean enableHeadlessMode)

createFirefoxRemoteWebDriver

public static org.openqa.selenium.remote.RemoteWebDriver createFirefoxRemoteWebDriver(URL seleniumHubUrl,
                                                                                      boolean enableHeadlessMode)

createChromeRemoteWebDriver

public static org.openqa.selenium.remote.RemoteWebDriver createChromeRemoteWebDriver(URL seleniumHubUrl,
                                                                                     boolean enableHeadlessMode)

createRandomRemoteWebDriver

public static org.openqa.selenium.remote.RemoteWebDriver createRandomRemoteWebDriver(URL seleniumHubUrl,
                                                                                     boolean enableHeadlessMode)

createDefaultRemoteWebDriver

public static org.openqa.selenium.remote.RemoteWebDriver createDefaultRemoteWebDriver(URL seleniumHubUrl,
                                                                                      boolean enableHeadlessMode)

cleanUpDriver

public static void cleanUpDriver(org.openqa.selenium.WebDriver driver)

getHtmlPage

public static String getHtmlPage(String url,
                                 Configuration conf)

Function for obtaining the HTML using the selected selenium webdriver There are a number of configuration properties within nutch-site.xml which determine whether to take screenshots of the rendered pages and persist them as timestamped .png's into HDFS.

Parameters:

url - the URL to fetch and render

conf - the Configuration

Returns:

the html page

getHtmlPage

public static String getHtmlPage(String url)