7|hf0ddlmZddlZddlmZddlmZmZmZm Z m Z m Z ddl m Z ddlmZddlmZddlmZmZdd lmZdd lmZmZej4eZGd d eZy) ) annotationsN)Path)AnyDictIteratorListOptionalUnion)CallbackManagerForLLMRun)LLM)GenerationChunk)get_pydantic_field_namespre_init)_build_model_kwargs)Fieldmodel_validatorceZdZUdZdZded<ded< dZded< dZded < ed d Z d ed < edd Z d ed< edd Z d ed< edd Z ded< edd Z ded< edd Zded< edd Zded< edd Zded< edd Zded< edd Zded< edZded< dZded< d Zd!ed"< d#Zd!ed$< edZded%< dZd&ed'< gZd(ed)< d*Zd!ed+< d,Zded-< d.Zded/< dZd&ed0< d1Zd2ed3< d4Zd2ed5< ee 6Z!d7ed8< dZ"ded9< dZ#d:ed;< dZ$d< e&dJd?Z'e(d@Ae)dKdBZ*e+dLdCZ,e+dLdDZ-e+dMdEZ.dNdOdFZ/ dP dQdGZ0 dP dRdHZ1dSdIZ2y)TLlamaCppallama.cpp model. To use, you should have the llama-cpp-python library installed, and provide the path to the Llama model as a named parameter to the constructor. Check out: https://github.com/abetlen/llama-cpp-python Example: .. code-block:: python from langchain_community.llms import LlamaCpp llm = LlamaCpp(model_path="/path/to/llama/model") Nrclientstr model_pathz Optional[str] lora_base lora_pathin_ctx)aliasintn_partsseedTf16_kvboolF logits_all vocab_only use_mlock n_threadsz Optional[int]n_batch n_gpu_layerssuffix max_tokensg?zOptional[float] temperaturegffffff?top_plogprobszOptional[bool]echoOptional[List[str]]stopg?repeat_penalty(top_k@last_n_tokens_sizeuse_mmapg?floatrope_freq_scaleg@rope_freq_base)default_factoryDict[str, Any] model_kwargs streamingzOptional[Union[str, Path]] grammar_pathzOptional[Union[str, Any]]grammarverbosec  ddlm}m}|d}gd}|Dcic]}||| }}|d|d|d<|j |d ||fi||d <|d r!|d r|d } |d } t d| d| dt|d tr|j|d |d <|S|d r|j|d |d <|S |S#t$r tdwxYwcc}w#t $r}t d |d |d}~wwxYw)z4Validate that llama-cpp-python library is installed.r)Llama LlamaGrammarzCould not import llama-cpp-python library. Please install the llama-cpp-python library to use this embedding model: pip install llama-cpp-pythonr)r9r:rrrrrr r"r#r$r%r'r7r6rAr(Nr=rz&Could not load Llama model from path: z. Received error r@r?zCCan only pass in one of grammar and grammar_path. Received grammar=z and grammar_path=.) llama_cpprCrD ImportErrorupdate Exception ValueError isinstancer from_string from_file) clsvaluesrCrDrmodel_param_namesk model_paramser@r?s `/var/www/html/test/engine/venv/lib/python3.12/site-packages/langchain_community/llms/llamacpp.pyvalidate_environmentzLlamaCpp.validate_environments  5L)  $/@@6!9 @ @ . ! -+1.+AL (F>23 $Z@<@F8  ) !7Y'G!.1L*/,3 y)3 / , 8 8 9J KF9   N # , 6 6vn7M NF9    m I  4A 8 E""#&  s(C C C$C$ D-C>>Dbefore)modec4t|}t||}|S)z>Build extra kwargs from additional params that were passed in.)rr)rNrOall_required_field_namess rTbuild_model_kwargszLlamaCpp.build_model_kwargss!$_streamrprjr) r^rmr1rnkwargscombined_text_outputchunkr_results rT_callzLlamaCpp._calls. >>$& %'  3 % 2$  3( '))$/F))&)F T[[99&9F)$Q'/ /r[c+*Ki|j||}|jd |dd|}|D]`}|ddjdd}t|dddd|i} |r(|j | j |j | | byw) a\Yields results objects as they are generated in real time. It also calls the callback manager's on_llm_new_token event with similar parameters to the OpenAI LLM class method of the same name. Args: prompt: The prompts to pass into the model. stop: Optional list of stop words to use when generating. Returns: A generator representing the stream of tokens being generated. Yields: A dictionary like objects containing a string token and metadata. See llama-cpp-python docs and below for more. Example: .. code-block:: python from langchain_community.llms import LlamaCpp llm = LlamaCpp( model_path="/path/to/local/model.bin", temperature = 0.5 ) for chunk in llm.stream("Ask 'Hi, how are you?' like a pirate:'", stop=["'"," "]): result = chunk["choices"][0] print(result["text"], end='', flush=True) # noqa: T201 T)rmstreamrorr.Nrp)rpgeneration_info)tokenrA log_probsrf)rjrgetr on_llm_new_tokenrprA) r^rmr1rnrrr_rupartr.rts rTrqzLlamaCpp._stream,sJ:D((.9&9BF4B6B DIq)--j$?H#)_Q'/!+X 6E,,**dllh-K sBBcl|jj|jd}t|S)Nzutf-8)rtokenizeencodelen)r^rptokenized_texts rTget_num_tokenszLlamaCpp.get_num_tokens_s*--dkk'.BC>""r[)rOrreturnr)rOr<rr)rr<)rr)N)r1r0rr<)NN) rmrr1r0rn"Optional[CallbackManagerForLLMRun]rrrrr) rmrr1r0rnrrrrrzIterator[GenerationChunk])rprrr)3__name__ __module__ __qualname____doc__r__annotations__rrrrrrr r"r#r$r%r'r(r)r+r,r-r.r/r1r2r4r6r7r9r:dictr=r>r?r@rArrUr classmethodrZpropertyr`rcrgrjrvrqrrfr[rTrrs FCO+#I}#0#I}#As'*E3*9-GS-?b'D#'-X.FD.1U,7J7@U,7J7/E5It5,$T=I}=C#1I6G]6/#(N"CL-CF!$KFM'S #J #3#&K&.!E?!.#DkHm)N D. % "D "@&)NO)2E=.(* *M#Hn#1 OU )#NE#+#(#>L.>?It8/3L,3 *.G &- GT) ::x(#$ "KK8%):> (0(0"(08 (0  (0 (0Z%):> 11"18 1  1 # 1f#r[r) __future__rloggingpathlibrtypingrrrrr r langchain_core.callbacksr #langchain_core.language_models.llmsr langchain_core.outputsr langchain_core.utilsrrlangchain_core.utils.utilsrpydanticrr getLoggerrloggerrrfr[rTrsF"===32C:+   8 $P#sP#r[