[WIP] Create n-gram class for nlp.py - #362
lucasmoura wants to merge 1 commit into
Conversation
Sorry, something went wrong.
|
text.py already has an implementation of the n-gram text model. I'm not sure if it is complete though. |
Sorry, something went wrong.
|
@MrDupin Thanks for showing me that. I just looked at nlp.py. My mistake, I will close this MR and take a look on text.py |
Sorry, something went wrong.
|
I forgot to add, I have already written a notebook section for the two models, on #352. I'm pointing it out since you mentioned you want to write a notebook section on them. In the notebook, I have written how one may use them and what they do. You can expand on that once (and if) the PR gets merged. Sorry if I inconvenienced you. |
Sorry, something went wrong.
|
@MrDupin Yeah, I saw your MR. The idea would be to add an example on it, such as language detection. We would need some new files on aima-data, but I think would be a nice example to display how n-grams could be used on real problems. When your MR gets accepted, I will try to work on that. |
Sorry, something went wrong.
|
Sounds good. |
Sorry, something went wrong.
In the NLP chapter in the book, there is a whole section dedicated to the n-gram model, however this model in not implemented in the nlp.py module.
Although there is no pseudocode for the n-gram model. I believe that would be a good idea to include the model implementation and add how the model works in the nlp notebook, in order for students to better understand how the model works.
This is still a work in progress, because I still intend to add an example of the n-gram model to the nlp notebook. The first one I will use is the language detection problem. I have looked at some english text already in aima-data and I will find any similar text on a different language to perform the experiment.