Nj"dZddlZddlZddlZddlmZej eZddhZ dZ dejfdZ dZ d ejd eed ejfd Zd ejd eed dfd Zd ejd eed dfdZy)as Dynamic int8 quantization for pocket-tts. Uses torchao if available (torch 2.10+ with C++ extensions), otherwise falls back to torch.ao.quantization (deprecated but functional on torch 2.5-2.9). Quantizes attention (Q/K/V/output projections) and FFN (linear1/linear2) layers in the FlowLM transformer. The flow matching network and Mimi VAE decoder remain in float32. N attentionffnc ddl}|jjdyddl}t |ds t |ddsy y#t $rYywxYw)zDetect the best available quantization backend. Returns "torchao" if torchao is installed with working C++ extensions, otherwise returns "torch.ao". rNtorchaoztorch.ao_C_SKIPPED_CPP_EXTENSIONSF)importlib.utilutil find_specrhasattrgetattr ImportError) importlibrs h/Users/ahmed/devFolder/Ultron/claude-voice/.venv/lib/python3.12/site-packages/pocket_tts/quantization.py _get_backendrsa   >> # #I . 6 7D !:SUZ)[*\     sAA A Amodulec0ddlm}m}|||y)z.Apply int8 dynamic quantization using torchao.r)%Int8DynamicActivationInt8WeightConfig quantize_N)torchao.quantizationrr)rrrs r_quantize_module_torchaor-sU f;=>ctjdvr dtjj_ytjjj dk(r dtjj_yy)zKSet the quantization engine for torch.ao (QNNPACK for ARM, FBGEMM for x86).)arm64aarch64qnnpacknonefbgemmN)platformmachinetorchbackends quantizedenginerr_ensure_quantization_enginer&4sU11*3  '  ! ! ( (F 2*2  ' 3rflow_lmquantize_groupsreturnc|stjd|St}tjd||dk(rt|||St |||S)a Apply dynamic int8 quantization to the specified layer groups of a FlowLM model. Automatically selects the best available backend: - torchao (torch 2.10+): optimized C++ kernels, faster on both ARM and x86 - torch.ao (torch 2.5-2.9): deprecated but functional fallback Args: flow_lm: The FlowLM model (model.flow_lm) quantize_groups: Set of group keys to quantize. Valid keys: "attention", "ffn", "flow_net" Returns: The quantized model (modified in-place). zr&r2r6Linearr!qint8r3r4r5r7r8r9)r'r(r>r:s rr.r.ms6!_$))BII;ekkSWX$$++ / ) U__ryykVZ [ O #, emm,ryykEM- emm,ryykEM,r)__doc__loggingrr!torch.nnr6 getLogger__name__r+RECOMMENDED_CONFIGrModulerr&setstrr0r-r.r%rrrMs     8 $"5)*?RYY?3 CH>(BII(C(T($RYYSdr