Package edu.berkeley.nlp.lm.io

Interface Summary
ArpaLmReaderCallback<V> Callback that is called for each n-gram in the collection
LmReader<V,C extends LmReaderCallback<V>>  
LmReaderCallback<V> Callback that is called for each n-gram in the collection
NgramOrderedLmReaderCallback<V> Callback that is called for each n-gram in the collection
 

Class Summary
ArpaLmReader<W> A parser for ARPA LM files.
ComputeLogProbabilityOfTextStream Computes the log probability of a list of files.
FirstPassCallback<V extends LongRepresentable<V>> Reader callback which adds n-grams to an NgramMap
GoogleLmReader<W> Reads in n-gram count collections in the format that the Google n-grams Web1T corpus comes in.
IOUtils Some IO utility functions.
KneserNeyFileWritingLmReaderCallback<W> Class for producing a Kneser-Ney language model in ARPA format from raw text.
KneserNeyLmReaderCallback<W> Class for producing a Kneser-Ney language model in ARPA format from raw text.
LmReaders This class contains a number of static methods for reading/writing/estimating n-gram language models.
MakeKneserNeyArpaFromText Estimates a Kneser-Ney language model from raw text, and writes the language model out in ARPA-format.
MakeLmBinaryFromArpa Given a language model in ARPA format, builds a binary representation of the language model and writes it to disk.
MakeLmBinaryFromGoogle Given a directory in Google n-grams format, builds a binary representation of a stupid-backoff language model language model and writes it to disk.
MakeNgramMapBinaryFromGoogle Like MakeLmBinaryFromGoogle, except it only writes the NgramMap portion of the LM, meaning the binary does not contain the vocabulary.
NgramMapAddingCallback<V> Reader callback which adds n-grams to an NgramMap
TextReader<W> Class for reading raw text files.