7|h'8ddlZddlZddlmZmZmZmZmZmZddl Z ddl m Z m Z ddl mZddlmZddlmZej&eZGddeZy) N)Any AsyncIteratorDictIteratorListOptional)AsyncCallbackManagerForLLMRunCallbackManagerForLLMRun)LLM)GenerationChunk)Fieldc eZdZUdZeed< dZeeed< dZee ed< e dd Z e ed< d Z eeed < d Zeeed < dZeeed< dZeeed< dZeeed< dZeeed< dZeeed< dZee ed< dZee ed< dZee ed< dZeeed< dZeeed< e dd Ze ed< e dd Ze ed< e dd Ze ed < d!Zee ed"< e dd# Ze ed#< e dd$ Ze ed$< gZ ee!eed%< dZ"e ed&< e#d'e$ee%ffd(Z&e#d'e$ee%ffd)Z'e#d'efd*Z(d4d+ee!ed'e$ee%ffd,Z) d5d-ed+ee!ed.ee*d/e%d'ef d0Z+ d5d-ed+ee!ed.ee,d/e%d'ef d1Z- d5d-ed+ee!ed.ee*d/e%d'e.e/f d2Z0 d5d-ed+ee!ed.ee,d/e%d'e1e/f d3Z2y)6TextGenazText generation models from WebUI. To use, you should have the text-generation-webui installed, a model loaded, and --api added as a command-line option. Suggested installation, use one-click installer for your OS: https://github.com/oobabooga/text-generation-webui#one-click-installers Parameters below taken from text-generation-webui api example: https://github.com/oobabooga/text-generation-webui/blob/main/api-examples/api-example.py Example: .. code-block:: python from langchain_community.llms import TextGen llm = TextGen(model_url="http://localhost:8500") model_urlNpresetmax_new_tokensT do_sample)aliasg? temperatureg?top_p typical_prepsilon_cutoff eta_cutoffgzG?repetition_penalty(top_k min_lengthno_repeat_ngram_size num_beams penalty_alphalength_penaltyFearly_stoppingseed add_bos_tokenitruncation_length ban_eos_tokenskip_special_tokensstopping_strings streamingreturncid|jd|jd|jd|jd|jd|j d|j d|jd |jd |jd |jd |jd |jd|jd|jd|jd|j |j"|j$|j&|j(dS)z/Get the default parameters for calling textgen.rrrrrrrrrrr r!r"r#r$r&r')r(r)r*r+)rrrrrrrrrrr r!r"r#r$r&r'r(r)r*r+selfs _/var/www/html/test/engine/venv/lib/python3.12/site-packages/langchain_community/llms/textgen.py_default_paramszTextGen._default_paramsvsU d11   4++  TZZ    d11   $//  !$"9"9  TZZ  $//  #D$=$=    T//  d11  d11 DII! " T//# $"&!7!7!//#'#;#; $ 5 5+  c:id|ji|jS)zGet the identifying parameters.r)rr2r/s r1_identifying_paramszTextGen._identifying_paramss$I;/H43G3GHHr3cy)zReturn type of llm.textgenr/s r1 _llm_typezTextGen._llm_typesr3stopc|jr | td|j |j}nd|ji}|jxs|xsg|d<|S)a Performs sanity check, preparing parameters in format needed by textgen. Args: stop (Optional[List[str]]): List of stop sequences for textgen. Returns: Dictionary containing the combined parameters. z2`stop` found in both the input and default params.rr+)r+ ValueErrorrr2)r0r:paramss r1_get_parameterszTextGen._get_parameterssc  T%5QR R ;; ))F ,F&*%:%:%Hd%Hb!" r3prompt run_managerkwargsc |jr0d}|jd |||d|D]}||jz }|}|S|jd}|j |} | j } || d<t j|| } | jdk(r| jddd }|Std | d}|S) Call the textgen web API and return the output. Args: prompt: The prompt to use for generation. stop: A list of strings to stop generation when encountered. Returns: The generated text. Example: .. code-block:: python from langchain_community.llms import TextGen llm = TextGen(model_url="http://localhost:5000") llm.invoke("Write a story about llamas.") r?r:r@/api/v1/generater?jsonresultsrtextERROR: response: r8) r,_streamrKrr>copyrequestspost status_coderHprint r0r?r:r@rAcombined_text_outputchunkresulturlr=requestresponses r1_callz TextGen._calls. >>#% %DkEK 3% 2$ 3*F ^^$$45C))$/FkkmG &GH }}Sw7H##s*!3A6v>  )(45 r3cK|jr1d}|jd |||d|23d{}||jz }|jd}|j |} | j } || d<t j|| } | jdk(r| jdd d }|Std | d}|S76|}|Sw) rCrDrENrFr?rGrIrJrrKrLr8) r,_astreamrKrr>rNrOrPrQrHrRrSs r1_acallzTextGen._acalls. >>#% ,t}} Dk EK  3 3e% 2$^^$$45C))$/FkkmG &GH }}Sw7H##s*!3A6v>  )(45 ' 3 *F s&%CC CC BCC  Cc+6K ddl}i|j||}|jd}|j }||d<|j } | j || jtj| | j} tj| } | ddk(r3t| dd } |r|j| j | n| dd k(r| jyz#t$r tdwxYww aYields results objects as they are generated in real time. It also calls the callback manager's on_llm_new_token event with similar parameters to the OpenAI LLM class method of the same name. Args: prompt: The prompts to pass into the model. stop: Optional list of stop words to use when generating. Returns: A generator representing the stream of tokens being generated. Yields: A dictionary like objects containing a string token and metadata. See text-generation-webui docs and below for more. Example: .. code-block:: python from langchain_community.llms import TextGen llm = TextGen( model_url = "ws://localhost:5005" streaming=True ) for chunk in llm.stream("Ask 'Hi, how are you?' like a pirate:'", stop=["'"," "]): print(chunk, end='', flush=True) # noqa: T201 rNz9The `websocket-client` package is required for streaming.z/api/v1/streamr?event text_streamrK)rKgeneration_info)token stream_end websocket ImportErrorr>rrN WebSocketconnectsendrHdumpsrecvloadsr on_llm_new_tokenrKclose r0r?r:r@rArfr=rWrXwebsocket_clientrVrUs r1rMzTextGen._streams#H   :D((.9&9 /++-"$..0  %djj12%**,FZZ'Fg-/'$(00uzz0B L0 &&(% K  sDDC9DDDc LK ddl}i|j||}|jd}|j }||d<|j } | j || jtj| | j} tj| } | ddk(rr rZr r]rr rMrr\r8r3r1rrs$NJ FHSM 1$'NHSM'3D 4It4#&K%&M!E8E? G"#Ix"8'(NHUO'"#J#*..F E8E??!"J !.*+(3-+< !Ix} %&M8E?&&'NHUO' .>?ND?b'D#'O:%):> ,,tCy!,67 ,  , ,b%)?C ,,tCy!,;< ,  , ,b%):> FFtCy!F67 F  F / " FV%)?C FFtCy!F;< F  F  ' Fr3r)rHloggingtypingrrrrrrrOlangchain_core.callbacksr r #langchain_core.language_models.llmsr langchain_core.outputsr pydanticr getLoggerrsloggerrr8r3r1rsD EE42   8 $NcNr3