Skip to main content

tokenize_sentences

use_token

Function tokenize_sentences 

Source 
pub fn tokenize_sentences(input: &str) -> Vec<Token>
Expand description

Extracts conservative sentence tokens.