Skip to main content
tokenize_chars
use_
text
0.2.0
In use_
text::
prelude
use_text
::
prelude
Function
tokenize_
chars
Copy item path
pub fn tokenize_chars(input: &
str
) ->
Vec
<
Token
>
Expand description
Splits input into Unicode scalar values.