Package org.apache.nutch.util

Class TableUtil

  • public class TableUtil
extends Object

    • Field Detail

    • Constructor Detail

      • TableUtil

        public TableUtil()

    • Method Detail

      • reverseUrl

        public static String reverseUrl​(String urlString)
                         throws MalformedURLException
        Reverses a url's domain. This form is better for storing in hbase. Because scans within the same domain are faster.

        E.g. "http://bar.foo.com:8983/to/index.html?a=b" becomes "com.foo.bar:8983:http/to/index.html?a=b".

        Parameters:
        urlString - url to be reversed
        Returns:
        Reversed url
        Throws:
        MalformedURLException - if the input urlString is malformed

      • reverseUrl

        public static String reverseUrl​(URL url)
        Reverses a url's domain. This form is better for storing in hbase. Because scans within the same domain are faster.

        E.g. "http://bar.foo.com:8983/to/index.html?a=b" becomes "com.foo.bar:http:8983/to/index.html?a=b".

        Parameters:
        url - url to be reversed
        Returns:
        Reversed url

      • unreverseUrl

        public static String unreverseUrl​(String reversedUrl)

      • getReversedHost

        public static String getReversedHost​(String reversedUrl)
        Given a reversed url, returns the reversed host E.g "com.foo.bar:http:8983/to/index.html?a=b" -> "com.foo.bar"
        Parameters:
        reversedUrl - Reversed url
        Returns:
        Reversed host

      • reverseHost

        public static String reverseHost​(String hostName)

      • unreverseHost

        public static String unreverseHost​(String reversedHostName)

      • toString

        public static String toString​(CharSequence utf8)
        Convert given Utf8 instance to String and and cleans out any offending "�" from the String.
        Parameters:
        utf8 - Utf8 object
        Returns:
        string-ifed Utf8 object or null if Utf8 instance is null