edu.berkeley.nlp.lm.io
Class GoogleLmReader<W>

java.lang.Object
  extended by edu.berkeley.nlp.lm.io.GoogleLmReader<W>
All Implemented Interfaces:
LmReader<LongRef,NgramOrderedLmReaderCallback<LongRef>>

public class GoogleLmReader<W>
extends Object
implements LmReader<LongRef,NgramOrderedLmReaderCallback<LongRef>>

Reads in n-gram count collections in the format that the Google n-grams Web1T corpus comes in.

Author:
adampauls

Constructor Summary
GoogleLmReader(String rootDir, WordIndexer<W> wordIndexer, ConfigOptions opts)
           
 
Method Summary
static
<W> void
addToIndexer(WordIndexer<W> wordIndexer, String sortedVocabPath)
           
 int getLmOrder()
           
 void parse(NgramOrderedLmReaderCallback<LongRef> callback)
           
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

GoogleLmReader

public GoogleLmReader(String rootDir,
                      WordIndexer<W> wordIndexer,
                      ConfigOptions opts)
Method Detail

getLmOrder

public int getLmOrder()

parse

public void parse(NgramOrderedLmReaderCallback<LongRef> callback)
Specified by:
parse in interface LmReader<LongRef,NgramOrderedLmReaderCallback<LongRef>>

addToIndexer

public static <W> void addToIndexer(WordIndexer<W> wordIndexer,
                                    String sortedVocabPath)
Parameters:
sortedVocabPath -