Skip to content

Support for cjk. Please！and get an error on windows #35

New issue
New issue
Open
Open
Support for cjk. Please！and get an error on windows#35
@cgisky1980

Description

@cgisky1980
cgisky1980
opened on Sep 19, 2023

Tokenizer doesn't correctly handle asian writing (CJK, maybe others)

Support for cjk. Please
Or Just Chinese

))E4L)PC)N~6S7JIFV`DZ8

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions