7|h}jddlmZddlmZmZmZmZmZddlm Z m Z ddl m Z ddl mZGdde Zy) )partial)AnyDictListOptionalSequence)AsyncCallbackManagerForLLMRunCallbackManagerForLLMRun)LLM)pre_initc BeZdZUdZeed<eed< dZeeed< dZ eeed< dZ ee eefed< dZ eeed< e d e eeffd Ze d efd Zed e d e fd Z ddedeeedeeded ef dZ ddedeeedeeded ef dZy) CTransformersa_C Transformers LLM models. To use, you should have the ``ctransformers`` python package installed. See https://github.com/marella/ctransformers Example: .. code-block:: python from langchain_community.llms import CTransformers llm = CTransformers(model="/path/to/ggml-gpt-2.bin", model_type="gpt2") clientmodelN model_type model_fileconfiglibreturnc`|j|j|j|jdS)zGet the identifying parameters.rrrrrselfs e/var/www/html/test/engine/venv/lib/python3.12/site-packages/langchain_community/llms/ctransformers.py_identifying_paramsz!CTransformers._identifying_params-s*ZZ////kk   cy)zReturn type of llm. ctransformersrs r _llm_typezCTransformers._llm_type7srvaluesc ddlm}|dxsi}|j|df|d|d|dd ||d <|S#t$r tdwxYw) z5Validate that ``ctransformers`` package is installed.r)AutoModelForCausalLMz\Could not import `ctransformers` package. Please install it with `pip install ctransformers`rrrrr)rrrr)rr# ImportErrorfrom_pretrained)clsr!r#rs rvalidate_environmentz"CTransformers.validate_environment<s  :!'R?/?? 7O l+l+u    x  E  s 9Apromptstop run_managerkwargsc g}|xstj}|j||dD]0}|j||j ||j 2dj |S)aVGenerate text from a prompt. Args: prompt: The prompt to generate text from. stop: A list of sequences to stop generation when encountered. Returns: The generated text. Example: .. code-block:: python response = llm.invoke("Tell me a joke.") Tr)streamverbose)r get_noop_managerrappendon_llm_new_tokenr0join)rr(r)r*r+text _run_managerchunks r_callzCTransformers._callQsp*"Q&>&O&O&Q [[d4[@ GE KK   ) )% ) F Gwwt}rcKd}|r!t|j|j}d}|j||dD]}|r||d{||z }|S7 w)aAsynchronous Call out to CTransformers generate method. Very helpful when streaming (like with websockets!) Args: prompt: The prompt to pass into the model. stop: A list of strings to stop generation when encountered. Returns: The string generated by the model. Example: .. code-block:: python response = llm.invoke("Once upon a time, ") Nr/r1Tr-)rr4r0r)rr(r)r*r+ text_callbackr6tokens r_acallzCTransformers._acallmso* #K$@$@$,,WM[[d4[@ E#E*** EMD   +sA AAA)NN)__name__ __module__ __qualname____doc__r__annotations__strrrrrrrpropertyrr r r'rr r9rr r=rrrrr sa  K J!%J $ $J $:'+FHT#s(^ $+;C#H  T#s(^  3$4.)-:> x}%67    >%)?C tCy!;<    rrN) functoolsrtypingrrrrrlangchain_core.callbacksr r #langchain_core.language_models.llmsr langchain_core.utilsr rrrrrJs)664)@C@r