Package org.apache.nutch.scoring.similarity.util

Class LuceneAnalyzerUtil

java.lang.Object

org.apache.lucene.analysis.Analyzer

org.apache.nutch.scoring.similarity.util.LuceneAnalyzerUtil

All Implemented Interfaces:

Closeable, AutoCloseable

public class LuceneAnalyzerUtil
extends org.apache.lucene.analysis.Analyzer

Creates a custom analyzer based on user provided inputs

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	LuceneAnalyzerUtil.StemFilterType

Nested classes/interfaces inherited from class org.apache.lucene.analysis.Analyzer

org.apache.lucene.analysis.Analyzer.ReuseStrategy, org.apache.lucene.analysis.Analyzer.TokenStreamComponents

Field Summary

Fields inherited from class org.apache.lucene.analysis.Analyzer

GLOBAL_REUSE_STRATEGY, PER_FIELD_REUSE_STRATEGY

Constructor Summary

Constructors

Constructor	Description
LuceneAnalyzerUtil(LuceneAnalyzerUtil.StemFilterType stemFilterType, boolean useStopFilter)	Creates an analyzer instance based on Lucene default stopword set if the param useStopFilter is set to true
LuceneAnalyzerUtil(LuceneAnalyzerUtil.StemFilterType stemFilterType, List<String> stopWords, boolean addToDefault)	Creates an analyzer instance based on user provided stop words.

Method Summary

Modifier and Type	Method	Description
protected org.apache.lucene.analysis.Analyzer.TokenStreamComponents	createComponents(String fieldName)

Methods inherited from class org.apache.lucene.analysis.Analyzer

attributeFactory, close, getOffsetGap, getPositionIncrementGap, getReuseStrategy, getVersion, initReader, initReaderForNormalization, normalize, normalize, setVersion, tokenStream, tokenStream

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

LuceneAnalyzerUtil

public LuceneAnalyzerUtil(LuceneAnalyzerUtil.StemFilterType stemFilterType,
                          boolean useStopFilter)

Creates an analyzer instance based on Lucene default stopword set if the param useStopFilter is set to true

Parameters:

stemFilterType - a preferred LuceneAnalyzerUtil.StemFilterType to use. Can be one of LuceneAnalyzerUtil.StemFilterType.PORTERSTEM_FILTER, LuceneAnalyzerUtil.StemFilterType.ENGLISHMINIMALSTEM_FILTER, or LuceneAnalyzerUtil.StemFilterType.NONE

useStopFilter - if true use the default Lucene stopword set, false otherwise

LuceneAnalyzerUtil

public LuceneAnalyzerUtil(LuceneAnalyzerUtil.StemFilterType stemFilterType,
                          List<String> stopWords,
                          boolean addToDefault)

Creates an analyzer instance based on user provided stop words. If the param addToDefault is set to true, then user provided stop words will be added to the Lucene default stopset.

Parameters:

stemFilterType - a preferred LuceneAnalyzerUtil.StemFilterType to use. Can be one of LuceneAnalyzerUtil.StemFilterType.PORTERSTEM_FILTER, LuceneAnalyzerUtil.StemFilterType.ENGLISHMINIMALSTEM_FILTER, or LuceneAnalyzerUtil.StemFilterType.NONE

stopWords - a List of stop word Strings

addToDefault - if true the provided stop words will be added to the default Lucene stopword set, false otherwise

Method Detail

createComponents

protected org.apache.lucene.analysis.Analyzer.TokenStreamComponents createComponents(String fieldName)

Specified by:

createComponents in class org.apache.lucene.analysis.Analyzer