intficpy.tokenizer module

intficpy.tokenizer.cleanInput(input_string, record=True)

Used on player commands to remove punctuation and convert to lowercase Takes the raw user input (string) Returns a string

intficpy.tokenizer.removeArticles(tokens)

intficpy.tokenizer.tokenize(input_string)

Convert input to a list of tokens Takes a string as an argument, and returns a list of strings