Authors: Njoyim Tchoubith Peguy Calusha

Affiliation: University of Fribourg, Switzerland

Email: pegpeg07@hotmail.com

Description: Here is a simple neural language model (NLM) that relies only on character-level inputs. This model employs a convolutional neural network (CNN) and a highway network over characters, whose output is given to a long short-term memory (LSTM) recurrent neural network language model (RNN-LM).

Unlike previous works that utilize subword information via morphemes, this model does not require morphological tagging as a pre-processing step. And, unlike the recent line of work which combines input word embeddings with features from a character-level model, this model does not utilize word embeddings at all in the input layer. Given that most of the parameters in NLMs are from the word embeddings, the proposed model has significantly fewer parameters than previous NLMs, making it attractive for applications where model size may be an issue (e.g. cell phones).

To adapt this model to the scanned receipts, the following modifications has been made:

- Unlike the original predictions made at word-level, the predictions was made at text-line level
- The two LSTM layers are bidirectional.
- A batch norm layer is added before the highway layer(s).
- The initialization of parameters is different for BiLSTM, and it is based on this paper: https://arxiv.org/pdf/1702.00071.pdf.

Using the website evaluation procedure, the OCR mismatches are removed and the discrepancies of total amount randomly prefixed by "RM" are fixed for fair comparison results with other participants.