7|h<'ddlZddlmZmZmZmZddlmZddlm Z ddl m Z dZ ejeZGdde Zy) N)AnyListMappingOptional)CallbackManagerForLLMRun)LLM) ConfigDictgpt2ceZdZUdZeZeed< dZe e ed< dZ e ed< dZ e ed< dZeed< ed Ze d dddd dede e d e ed ede ede defdZe d dddede e d e ede def dZe d!ded e ed ede edede e de e de fdZede dedede de de f dZedeee ffdZedefdZ d"dede eede ede def dZy)#IpexLLMzIpexLLM model. Example: .. code-block:: python from langchain_community.llms import IpexLLM llm = IpexLLM.from_model_id(model_id="THUDM/chatglm-6b") model_idN model_kwargsmodel tokenizerT streamingforbid)extra) tokenizer_id load_in_4bitload_in_low_bitrrrkwargsreturnc 2|j||d||||S)a Construct object from model_id Args: model_id: Path for the huggingface repo id to be downloaded or the huggingface checkpoint folder. tokenizer_id: Path for the huggingface repo id to be downloaded or the huggingface checkpoint folder which contains the tokenizer. load_in_4bit: "Whether to load model in 4bit. Unused if `load_in_low_bit` is not None. load_in_low_bit: Which low bit precisions to use when loading model. Example values: 'sym_int4', 'asym_int4', 'fp4', 'nf4', 'fp8', etc. Overrides `load_in_4bit` if specified. model_kwargs: Keyword arguments to pass to the model and tokenizer. kwargs: Extra arguments to pass to the model and tokenizer. Returns: An object of IpexLLM. Fr r low_bit_modelrrrr _load_model)clsr rrrrrs `/var/www/html/test/engine/venv/lib/python3.12/site-packages/langchain_community/llms/ipex_llm.py from_model_idzIpexLLM.from_model_id's0@%%+%  )rc 2|j||ddd||S)a Construct low_bit object from model_id Args: model_id: Path for the ipex-llm transformers low-bit model folder. tokenizer_id: Path for the huggingface repo id or local model folder which contains the tokenizer. model_kwargs: Keyword arguments to pass to the model and tokenizer. kwargs: Extra arguments to pass to the model and tokenizer. Returns: An object of IpexLLM. TFNrr)rr rrrs rfrom_model_id_low_bitzIpexLLM.from_model_id_low_bitQs/0% %  r!rc ddlm}m} ddlm} m} |xsi} |xsi}|xs|} d| vrd| d<| ddvrtd| dd | jd} | j| fi| }d | vr)| jDcic]\}}|d k7s ||} }}d d d }|s|d }||d<n d }||d<nd} |j| |||| }|j||d|||| d|S#t $r t dwxYw#t$r| j| fi| }YwxYwcc}}w#t$r|j||||| }YwxYw)Nr) AutoModelAutoModelForCausalLM) AutoTokenizerLlamaTokenizerzCould not import ipex-llm. Please install `ipex-llm` properly following installation guides: https://github.com/intel-analytics/ipex-llm?tab=readme-ov-file#install-ipex-llm.devicecpu)r*xpuzXIpexLLMBgeEmbeddings currently only supports device to be 'cpu' or 'xpu', but you have: .trust_remote_codeT) use_cacher-from_pretrainedrr load_low_bit)load_function_namer load_kwargsr)r rrr)ipex_llm.transformersr%r& transformersr'r( ImportError ValueErrorpopr/ Exceptionitems_load_model_generalto)rr rrrrrrr%r&r'r( _model_kwargs _tokenizer_idr)rkvr2r1rs rrzIpexLLM._load_modelss   C%* 2$0 = (&+M( #  ". 811>x1H0IL ""8, W5 55mU}UI - /!.!4!4!6A!?R:R1M !% *%6"1@ -.%6".: N+!/  ++$#5!'* ,E"  &     M c  0 W666}V VI W : ++#5!'* ,E s;C(D D"D"4D((C=DD(E  E  model_classr1r2c  t||}||fii||S#t$r2}tjd|jd|d|Yd}~yd}~wwxYw)z,General function to attempt to load a model.zFailed to load model using r,z: N)getattrr9loggererror__name__)rAr1r r2r load_functiones rr;zIpexLLM._load_model_generalss #K1CDM M-L -L|-LM M  LL-''(*<)=RsD   s A(AAc4|j|jdS)zGet the identifying parameters.r rrJselfs r_identifying_paramszIpexLLM._identifying_paramss  --  r!cy)Nzipex-llmr3rKs r _llm_typezIpexLLM._llm_typesr!promptstop run_managerc "|jrddlm}|jj |d}|j |j j}||jdd}|'ddlm }ddl m } || ||jg} nd} |j j|f|| d |} |jj| dd } | S|jj |d}|j |j j}|'ddlm }ddl m } || ||jg} nd} |j j|fd | i|} |jj| dd t|d} | S) Nr) TextStreamerpt)return_tensorsT) skip_promptskip_special_tokens)StoppingCriteriaList)StopSequenceCriteria)streamerstopping_criteria)rXr\)rr5rTrencoder<rr))transformers.generation.stopping_criteriarYtransformers.tools.agentsrZgeneratedecodelen) rLrPrQrRrrT input_idsr[rYrZr\outputtexts r_callz IpexLLM._calls >> 1--fT-JI! TZZ%6%67I#DdHK%9)$?@%!%)!(TZZ((!"3 F >>(((MDK--fT-JI! TZZ%6%67IK$8)$?@%!%)!(TZZ((->BHF>>(((MF DKr!)N)NFNFNN)NN) rF __module__ __qualname____doc__DEFAULT_MODEL_IDr str__annotations__rrdictrrrrboolr model_config classmethodrr r#r staticmethodr;propertyrrMrOrrrfr3r!rr r s%Hc$*#'L(4.'0E3Is&It8L(,' '+!)-' ' tn' sm '  ' "#' '  ' ' R(, '+   tn sm       B'+")-#'+!%] ] sm]  ] "# ]  ] tn] ]  ] ] ~    " WS#X%6  3 %):> 88tCy!867 8  8 8r!r )loggingtypingrrrrlangchain_core.callbacksr#langchain_core.language_models.llmsrpydanticr rj getLoggerrFrDr r3r!rrys?//=3   8 $[c[r!