7|h"ddlmZddlmZmZmZmZmZmZm Z ddlmZddl m Z ddlmZddl m Z mZddlmZddlmZddlmZddlmZddlmZddlmZGdd eZy ) )pre_init)Any AsyncIteratorDictIteratorListOptionalUnion)root_validator)AsyncCallbackManagerForLLMRunCallbackManagerForLLMRun)LLM)enforce_stop_tokens)GenerationChunkc eZdZUdZeed<eed< dZee eefed< dZ e dee fed< dZ e ed< ed e eeffd Zed efd Zed e d e fd Z ddedeeedeeded ef dZ ddedeeedeeded ef dZ ddedeeedeeded eef dZ ddedeeedeeded eef dZy) DeepSparseaHNeural Magic DeepSparse LLM interface. To use, you should have the ``deepsparse`` or ``deepsparse-nightly`` python package installed. See https://github.com/neuralmagic/deepsparse This interface let's you deploy optimized LLMs straight from the [SparseZoo](https://sparsezoo.neuralmagic.com/?useCase=text_generation) Example: .. code-block:: python from langchain_community.llms import DeepSparse llm = DeepSparse(model="zoo:nlg/text_generation/codegen_mono-350m/pytorch/huggingface/bigpython_bigquery_thepile/base_quant-none") pipelinemodelNmodel_configurationgeneration_configF streamingreturnc`|j|j|j|jdS)zGet the identifying parameters.)r model_configrr)rrrrselfs b/var/www/html/test/engine/venv/lib/python3.12/site-packages/langchain_community/llms/deepsparse.py_identifying_paramszDeepSparse._identifying_params1s.ZZ 44!%!7!7   cy)zReturn type of llm. deepsparsers r _llm_typezDeepSparse._llm_type;srvaluesc ddlm}|dxsi}|jd d|dd||d<|S#t$r tdwxYw) z2Validate that ``deepsparse`` package is installed.r)Pipelinez[Could not import `deepsparse` package. Please install it with `pip install deepsparse[llm]`rtext_generationr)task model_pathrr")r!r& ImportErrorcreate)clsr$r&rs rvalidate_environmentzDeepSparse.validate_environment@su  +34: ,X__ "g  z   G  s .Apromptstop run_managerkwargsc |jr/d}|jd|||d|D]}||jz }|}n5|jdd|i|jj dj}| t ||}|S)Generate text from a prompt. Args: prompt: The prompt to generate text from. stop: A list of strings to stop generation when encountered. Returns: The generated text. Example: .. code-block:: python from langchain_community.llms import DeepSparse llm = DeepSparse(model="zoo:nlg/text_generation/codegen_mono-350m/pytorch/huggingface/bigpython_bigquery_thepile/base_quant-none") llm.invoke("Tell me a joke.") r.r/r0 sequencesrr")r_streamtextrr generationsrrr.r/r0r1combined_outputchunkr8s r_callzDeepSparse._callTs& >> O%DkEK . 5::- .#D II$2H2HIQ    &tT2D rcK|jr1d}|jd|||d|23d{}||jz }|jdd|i|jj dj}| t ||}|S7Y6|}w)r3r4r5Nr6rr")r_astreamr8rrr9rr:s r_acallzDeepSparse._acallzs& >> O,t}} Dk EK  . .e 5::- II$2H2HIQ    &tT2D  . #Ds&%B BBBAB BB c+K|jd|dd|j}|D]G}t|jdj}|r|j |j|IywaYields results objects as they are generated in real time. It also calls the callback manager's on_llm_new_token event with similar parameters to the OpenAI LLM class method of the same name. Args: prompt: The prompt to pass into the model. stop: Optional list of stop words to use when generating. Returns: A generator representing the stream of tokens being generated. Yields: A dictionary like object containing a string token. Example: .. code-block:: python from langchain_community.llms import DeepSparse llm = DeepSparse( model="zoo:nlg/text_generation/codegen_mono-350m/pytorch/huggingface/bigpython_bigquery_thepile/base_quant-none", streaming=True ) for chunk in llm.stream("Tell me a joke", stop=["'"," "]): print(chunk, end='', flush=True) # noqa: T201 T)r6rr)r8)tokenNr"rrrr9r8on_llm_new_tokenrr.r/r0r1 inferencerCr<s rr7zDeepSparse._streamsy8"DMM  040F0F  E#):):1)=)B)BCE,,5::,>K  sA-A/c K|jd|dd|j}|D]P}t|jdj}|r$|j |jd{|Ry7 wrBrDrFs rr?zDeepSparse._astreams8"DMM  040F0F  E#):):1)=)B)BCE!222DDDK  EsA)A:+A8, A:)NN)__name__ __module__ __qualname____doc__r__annotations__strrr rrr rboolpropertyrr#rr-rr r=r r@rrr7rr?r"rrrrs M JV48$sCx.18E15uT3_-4) It8  T#s(^  3$4,%):> $$tCy!$67 $  $ $R%)?C $$tCy!$;< $  $ $R%):> $$tCy!$67 $  $ / " $R%)?C $$tCy!$;< $  $  ' $rrN)langchain_core.utilsrtypingrrrrrr r pydanticr langchain_core.callbacksr r #langchain_core.language_models.llmsrlangchain_community.llms.utilsrlangchain_core.outputsrrr"rrrXsA)LLL)#)*3)>)2WWr