Package org.apache.nutch.protocol.htmlunit

Class HtmlUnitWebDriver

  • All Implemented Interfaces:
    org.openqa.selenium.HasCapabilities, org.openqa.selenium.interactions.Interactive, org.openqa.selenium.JavascriptExecutor, org.openqa.selenium.SearchContext, org.openqa.selenium.WebDriver
    public class HtmlUnitWebDriver
extends org.openqa.selenium.htmlunit.HtmlUnitDriver

    • Nested Class Summary

      • Nested classes/interfaces inherited from class org.openqa.selenium.htmlunit.HtmlUnitDriver

        org.openqa.selenium.htmlunit.HtmlUnitDriver.ElementsMap, org.openqa.selenium.htmlunit.HtmlUnitDriver.JavaScriptResultsCollection

      • Nested classes/interfaces inherited from interface org.openqa.selenium.WebDriver

        org.openqa.selenium.WebDriver.Navigation, org.openqa.selenium.WebDriver.Options, org.openqa.selenium.WebDriver.TargetLocator, org.openqa.selenium.WebDriver.Timeouts, org.openqa.selenium.WebDriver.Window

    • Field Summary

      • Fields inherited from class org.openqa.selenium.htmlunit.HtmlUnitDriver

        BROWSER_LANGUAGE_CAPABILITY, DOWNLOAD_IMAGES_CAPABILITY, JAVASCRIPT_ENABLED

    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      static void cleanUpDriver​(org.openqa.selenium.WebDriver driver)  
      static org.openqa.selenium.WebDriver getDriverForPage​(String url, Configuration conf)  
      static String getHTMLContent​(org.openqa.selenium.WebDriver driver, Configuration conf)  
      static String getHtmlPage​(String url, Configuration conf)
      Function for obtaining the HTML BODY using the selected selenium webdriver There are a number of configuration properties within nutch-site.xml which determine whether to take screenshots of the rendered pages and persist them as timestamped .png's into HDFS.
      protected com.gargoylesoftware.htmlunit.WebClient modifyWebClient​(com.gargoylesoftware.htmlunit.WebClient client)  

      • Methods inherited from class org.openqa.selenium.htmlunit.HtmlUnitDriver

        assertElementNotStale, click, close, doubleClick, executeAsyncScript, executeScript, findElement, findElement, findElements, findElements, get, get, getAlert, getBrowserVersion, getCapabilities, getCurrentUrl, getCurrentWindow, getElementsMap, getKeyboard, getMouse, getPageSource, getTitle, getWebClient, getWindowHandle, getWindowHandles, implicitlyWaitFor, isAcceptSslCertificates, isDownloadImages, isJavascriptEnabled, manage, mouseDown, mouseMove, mouseUp, navigate, newWebClient, openNewWindow, perform, quit, resetInputState, runAsync, sendKeys, setAcceptSslCertificates, setAutoProxy, setCurrentWindow, setDownloadImages, setExecutor, setHTTPProxy, setJavascriptEnabled, setProxy, setProxySettings, setSocksProxy, setSocksProxy, submit, switchTo, switchToDefaultContentOfWindow, toWebElement

      • Methods inherited from interface org.openqa.selenium.JavascriptExecutor

        executeScript, getPinnedScripts, pin, unpin

    • Constructor Detail

      • HtmlUnitWebDriver

        public HtmlUnitWebDriver()

    • Method Detail

      • modifyWebClient

        protected com.gargoylesoftware.htmlunit.WebClient modifyWebClient​(com.gargoylesoftware.htmlunit.WebClient client)
        Overrides:
        modifyWebClient in class org.openqa.selenium.htmlunit.HtmlUnitDriver

      • getDriverForPage

        public static org.openqa.selenium.WebDriver getDriverForPage​(String url,
                                                             Configuration conf)

      • getHTMLContent

        public static String getHTMLContent​(org.openqa.selenium.WebDriver driver,
                                    Configuration conf)

      • cleanUpDriver

        public static void cleanUpDriver​(org.openqa.selenium.WebDriver driver)

      • getHtmlPage

        public static String getHtmlPage​(String url,
                                 Configuration conf)
        Function for obtaining the HTML BODY using the selected selenium webdriver There are a number of configuration properties within nutch-site.xml which determine whether to take screenshots of the rendered pages and persist them as timestamped .png's into HDFS.
        Parameters:
        url - the URL to fetch and render
        conf - the Configuration
        Returns:
        the rendered inner HTML page