Package org.apache.nutch.util

Class MimeUtil

java.lang.Object

org.apache.nutch.util.MimeUtil

public final class MimeUtil
extends Object

This is a facade class to insulate Nutch from its underlying Mime Type substrate library, Apache Tika. Any Mime handling code should be placed in this utility class, and hidden from the Nutch classes that rely on it.

Constructor Summary

Constructors

Constructor	Description
MimeUtil(Configuration conf)

Method Summary

Modifier and Type	Method	Description
String	autoResolveContentType(String typeName, String url, byte[] data)	A facade interface to trying all the possible mime type resolution strategies available within Tika.
static String	cleanMimeType(String origType)	Cleans a MimeType name by removing out the actual MimeType, from a string of the form:
String	forName(String name)	A facade interface to Tika's underlying MimeTypes.forName(String) method.
String	getMimeType(File f)	Facade interface to Tika's underlying MimeTypes.getMimeType(File) method.
String	getMimeType(String url)	Facade interface to Tika's underlying MimeTypes.getMimeType(String) method.
static void	setPoolSize(int poolSize)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MimeUtil

public MimeUtil(Configuration conf)

Method Detail

setPoolSize

public static void setPoolSize(int poolSize)

cleanMimeType

public static String cleanMimeType(String origType)

Cleans a MimeType name by removing out the actual MimeType, from a string of the form:

      <primary type>/<sub type> ; < optional params
 

Parameters:

origType - The original mime type string to be cleaned.

Returns:

The primary type, and subtype, concatenated, e.g., the actual mime type.

autoResolveContentType

public String autoResolveContentType(String typeName,
                                     String url,
                                     byte[] data)

A facade interface to trying all the possible mime type resolution strategies available within Tika. First, the mime type provided in typeName is cleaned, with cleanMimeType(String). Then the cleaned mime type is looked up in the underlying Tika MimeTypes registry, by its cleaned name. If the MimeType is found, then that mime type is used, otherwise URL resolution is used to try and determine the mime type. However, if mime.type.magic is enabled in NutchConfiguration, then mime type magic resolution is used to try and obtain a better-than-the-default approximation of the MimeType.

Parameters:

typeName - The original mime type, returned from a ProtocolOutput.

url - The given @see url, that Nutch was trying to crawl.

data - The byte data, returned from the crawl, if any.

Returns:

The correctly, automatically guessed MimeType name.

getMimeType

public String getMimeType(String url)

Facade interface to Tika's underlying MimeTypes.getMimeType(String) method.

Parameters:

url - A string representation of the document URL to sense the MimeType for.

Returns:

An appropriate MimeType, identified from the given Document url in string form.

forName

public String forName(String name)

A facade interface to Tika's underlying MimeTypes.forName(String) method.

Parameters:

name - The name of a valid MimeType in the Tika mime registry.

Returns:

The object representation of the MimeType, if it exists, or null otherwise.

getMimeType

public String getMimeType(File f)

Facade interface to Tika's underlying MimeTypes.getMimeType(File) method.

Parameters:

f - The File to sense the MimeType for.

Returns:

The MimeType of the given File, or null if it cannot be determined.