-cJjitddlZ ddlZddlmZddlmZejdZ e jd ejd ej d d ejdej ejde d dejdejgdd d dejde dejde dd ededede dedef d Zy#e$redededwxYw)!Nz Please runz pip install clickzbefore you continue)Path) text2tokencNtjdtjy)z/ The shell entry point to sherpa-onnx. z?%(asctime)s %(levelname)s [%(filename)s:%(lineno)d] %(message)s)formatlevelN)logging basicConfigINFOY/Users/ahmed/devFolder/claude-voice/.venv/lib/python3.12/site-packages/sherpa_onnx/cli.pyclirs  Pllr r)nameinputTF)existsdir_okay)typeoutputz--tokenszThe path to tokens.txt.)rrequiredhelpz --tokens-type)cjkcharbpez cjkchar+bpefpinyinppinyinz phone+ppinyin)case_sensitivea3The type of modeling units, should be cjkchar, bpe, cjkchar+bpe, fpinyin, ppinyin or phone+ppinyin. fpinyin means full pinyin, each cjkchar has a pinyin(with tone). ppinyin means partial pinyin, it splits pinyin into initial and final, phone means English phonemes in CMU dictionary format. z --bpe-modelzLThe path to bpe.model. Only required when tokens-type is bpe or cjkchar+bpe.)rrz --lexiconzIThe path to lexicon.txt. Only required when tokens-type is phone+ppinyin.tokens tokens_type bpe_modellexiconcpg}g}t|dd5}|D]} g} g} | jj} | D]=} | ddk(s| ddk(s| ddk(r| j| -| j| ?|jdj | |j|  d d d t ||||| }t|d d5}t |D]0\}}|||z }|jdj |d z2 d d d y #1swYpxYw#1swYy xYw) u Encode the texts given by the INPUT to tokens and write the results to the OUTPUT. Each line in the texts contains the original phrase, it might also contain some extra items, for example, the boosting score (starting with :), the triggering threshold (starting with #, only used in keyword spotting task) and the original phrase (starting with @). Note: the extra items will be kept same in the output. example input 1 (tokens_type = ppinyin): 小爱同学 :2.0 #0.6 @小爱同学 你好问问 :3.5 @你好问问 小艺小艺 #0.6 @小艺小艺 example output 1: x iǎo ài t óng x ué :2.0 #0.6 @小爱同学 n ǐ h ǎo w èn w èn :3.5 @你好问问 x iǎo y ì x iǎo y ì #0.6 @小艺小艺 example input 2 (tokens_type = bpe): HELLO WORLD :1.5 #0.4 HI GOOGLE :2.0 #0.8 HEY SIRI #0.35 example output 2: ▁HE LL O ▁WORLD :1.5 #0.4 ▁HI ▁GO O G LE :2.0 #0.8 ▁HE Y ▁S I RI #0.35 rutf8)encodingr:#@ N)rrrrw )openstripsplitappendjoinr enumeratewrite)rrrrrrtexts extra_infoflineextratexttokstok encoded_textsitxts r encode_textr<s0\ EJ eS6 *aDED::<%%'Dq6S=CFcMSVs]LL%KK$  LL$ (   e $ +  M fcF +q .FAs :a= C GGCHHSMD( )/ , +) + *( , +sBD ?D, D),D5)rclick ImportErrorprintpathlibr sherpa_onnxrgrouprcommandargumentoptionstrChoicer<r r r rHs " ,jejjuEFzuzz|,   "      ( W  T B* B* B* B* B*  B*  B*  ) -G NB*y  ,      s DD7