edu.berkeley.nlp.lm.values
Class KneserNeyCountValueContainer

java.lang.Object
  extended by edu.berkeley.nlp.lm.values.KneserNeyCountValueContainer
All Implemented Interfaces:
ValueContainer<KneserNeyCountValueContainer.KneserNeyCounts>, Serializable

public final class KneserNeyCountValueContainer
extends Object
implements ValueContainer<KneserNeyCountValueContainer.KneserNeyCounts>

Stored type and token counts necessary for estimating a Kneser-Ney language model

Author:
adampauls
See Also:
Serialized Form

Nested Class Summary
static class KneserNeyCountValueContainer.KneserNeyCounts
          Warning: type counts are stored internally as 32-bit ints.
 
Constructor Summary
KneserNeyCountValueContainer(int maxNgramOrder, int startIndex)
           
 
Method Summary
 boolean add(int[] ngram, int startPos, int endPos, int ngramOrder, long offset, long contextOffset, int word, KneserNeyCountValueContainer.KneserNeyCounts val, long suffixOffset, boolean ngramIsNew)
          Adds a new value at the specified offset.
 void clearStorageForOrder(int ngramOrder)
           
 KneserNeyCountValueContainer createFreshValues(long[] numNgramsForEachOrder)
          Creates a fresh value container for copying purposes.
 long getBigramTypeCounts()
           
 void getFromOffset(long offset, int ngramOrder, KneserNeyCountValueContainer.KneserNeyCounts outputVal)
          Gets the value living at a particular offset.
 int getNumOneCountNgrams(int ngramOrder)
           
 int getNumTwoCountNgrams(int ngramOrder)
           
 KneserNeyCountValueContainer.KneserNeyCounts getScratchValue()
          Creates a fresh value of object (useful for passing as an output parameter)
 int numValueBits(int ngramOrder)
           
 void setFromOtherValues(ValueContainer<KneserNeyCountValueContainer.KneserNeyCounts> other)
          Destructively sets internal storage from another object.
 void setMap(NgramMap<KneserNeyCountValueContainer.KneserNeyCounts> map)
          Initializes a value container with the map that contains it
 void setSizeAtLeast(long size, int ngramOrder)
          Sets internal storage for size for a particular n-gram order
 boolean storeSuffixoffsets()
           
 void trim()
          Final clean up of storage.
 void trimAfterNgram(int ngramOrder, long size)
          Clear storage after an n-gram order is complete
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

KneserNeyCountValueContainer

public KneserNeyCountValueContainer(int maxNgramOrder,
                                    int startIndex)
Method Detail

createFreshValues

public KneserNeyCountValueContainer createFreshValues(long[] numNgramsForEachOrder)
Description copied from interface: ValueContainer
Creates a fresh value container for copying purposes.

Specified by:
createFreshValues in interface ValueContainer<KneserNeyCountValueContainer.KneserNeyCounts>
Returns:

getFromOffset

public void getFromOffset(long offset,
                          int ngramOrder,
                          KneserNeyCountValueContainer.KneserNeyCounts outputVal)
Description copied from interface: ValueContainer
Gets the value living at a particular offset.

Specified by:
getFromOffset in interface ValueContainer<KneserNeyCountValueContainer.KneserNeyCounts>

trimAfterNgram

public void trimAfterNgram(int ngramOrder,
                           long size)
Description copied from interface: ValueContainer
Clear storage after an n-gram order is complete

Specified by:
trimAfterNgram in interface ValueContainer<KneserNeyCountValueContainer.KneserNeyCounts>

getScratchValue

public KneserNeyCountValueContainer.KneserNeyCounts getScratchValue()
Description copied from interface: ValueContainer
Creates a fresh value of object (useful for passing as an output parameter)

Specified by:
getScratchValue in interface ValueContainer<KneserNeyCountValueContainer.KneserNeyCounts>
Returns:

add

public boolean add(int[] ngram,
                   int startPos,
                   int endPos,
                   int ngramOrder,
                   long offset,
                   long contextOffset,
                   int word,
                   KneserNeyCountValueContainer.KneserNeyCounts val,
                   long suffixOffset,
                   boolean ngramIsNew)
Description copied from interface: ValueContainer
Adds a new value at the specified offset.

Specified by:
add in interface ValueContainer<KneserNeyCountValueContainer.KneserNeyCounts>
ngramOrder - As always, ngramOrder is 0-based (0=unigram)
Returns:
Whether or not the add was successful

setSizeAtLeast

public void setSizeAtLeast(long size,
                           int ngramOrder)
Description copied from interface: ValueContainer
Sets internal storage for size for a particular n-gram order

Specified by:
setSizeAtLeast in interface ValueContainer<KneserNeyCountValueContainer.KneserNeyCounts>

setFromOtherValues

public void setFromOtherValues(ValueContainer<KneserNeyCountValueContainer.KneserNeyCounts> other)
Description copied from interface: ValueContainer
Destructively sets internal storage from another object.

Specified by:
setFromOtherValues in interface ValueContainer<KneserNeyCountValueContainer.KneserNeyCounts>

trim

public void trim()
Description copied from interface: ValueContainer
Final clean up of storage.

Specified by:
trim in interface ValueContainer<KneserNeyCountValueContainer.KneserNeyCounts>

setMap

public void setMap(NgramMap<KneserNeyCountValueContainer.KneserNeyCounts> map)
Description copied from interface: ValueContainer
Initializes a value container with the map that contains it

Specified by:
setMap in interface ValueContainer<KneserNeyCountValueContainer.KneserNeyCounts>

clearStorageForOrder

public void clearStorageForOrder(int ngramOrder)
Specified by:
clearStorageForOrder in interface ValueContainer<KneserNeyCountValueContainer.KneserNeyCounts>

storeSuffixoffsets

public boolean storeSuffixoffsets()
Specified by:
storeSuffixoffsets in interface ValueContainer<KneserNeyCountValueContainer.KneserNeyCounts>

getBigramTypeCounts

public long getBigramTypeCounts()

getNumOneCountNgrams

public int getNumOneCountNgrams(int ngramOrder)

getNumTwoCountNgrams

public int getNumTwoCountNgrams(int ngramOrder)

numValueBits

public int numValueBits(int ngramOrder)
Specified by:
numValueBits in interface ValueContainer<KneserNeyCountValueContainer.KneserNeyCounts>