Skip to content

DT-581 Better searches without stemming (core22)

This MR removes the stemming from the search process. The idea behind the smetting is to reduce words to their word stem, or root form. Unfortunately, it seems to not work fine for searching programs.

Also, now the graylist words are removed after the tokenization, instead of just searching and replacing them in the original string. This ensures that a word with a part that coincides with a graylist word won't be removed, only complete words.

Merge request reports