Build multiword language models and analyze them with machine learning
An n-gram is a collection of n successive items in a text document that may include words, numbers, symbols, and punctuation. N-gram models are useful in many text analytics applications where sequences of words are relevant, such as in sentiment analysis, text classification, and text generation. N-gram modeling is one of the many techniques used to convert text from an unstructured format to a structured format. An alternative to n-gram is word embedding techniques, such as word2vec.
A language model incorporating n-grams can be created by counting the number of times each unique n-gram appears in a document. This is known as a bag-of-n-grams model. In MATLAB, a bag-of-n-grams model can be created using a “bagOfNgrams” function.
Once the language model is built, it can then be used with machine learning algorithms to build predictive models for text analytics applications. To learn more about n-grams and building models with text data, see Text Analytics Toolbox™, for use with MATLAB®.
Examples and How To
See also: natural language processing, sentiment analysis, word2vec, text mining with MATLAB, data science, deep learning, Deep Learning Toolbox™, Predictive Maintenance Toolbox™