7|h[zddlZddlmZmZmZmZmZmZmZddl Z ddl m Z ddl m Z ddlmZddlmZmZddlmZmZddl mZGd d e ZGd d e Zy) N)AnyDictIteratorListOptionalTupleUnion)CallbackManagerForLLMRun)LLM)GenerationChunk)convert_to_secret_strget_from_dict_or_env)Field SecretStr)Responsec eZdZUdZedZeed< eedZ eed< eddZ eed< eddZ eed < ed Z e ed < d Zeeeefed < GddZede fdZedeeeffdZedeeeffdZedefdZdedd ffd ZdedeeeffdZd#deeedeeeffdZ d$deeeefdeeed ee defdZ dedefdZ!dede"e#fdZ$ d%deeeefdeeed ee%dede"e#f d!Z& d%deeeefdeeed ee%dedef d"Z'xZ(S)& SambaStudiou SambaStudio large language models. Setup: To use, you should have the environment variables ``SAMBASTUDIO_URL`` set with your SambaStudio environment URL. ``SAMBASTUDIO_API_KEY`` set with your SambaStudio endpoint API key. https://sambanova.ai/products/enterprise-ai-platform-sambanova-suite read extra documentation in https://docs.sambanova.ai/sambastudio/latest/index.html Example: .. code-block:: python from langchain_community.llms.sambanova import SambaStudio SambaStudio( sambastudio_url="your-SambaStudio-environment-URL", sambastudio_api_key="your-SambaStudio-API-key, model_kwargs={ "model" : model or expert name (set for Bundle endpoints), "max_tokens" : max number of tokens to generate, "temperature" : model temperature, "top_p" : model top p, "top_k" : model top k, "do_sample" : wether to do sample "process_prompt": wether to process prompt (set for Bundle generic v1 and v2 endpoints) }, ) Key init args — completion params: model: str The name of the model to use, e.g., Meta-Llama-3-70B-Instruct-4096 (set for Bundle endpoints). streaming: bool Whether to use streaming handler when using non streaming methods model_kwargs: dict Extra Key word arguments to pass to the model: max_tokens: int max tokens to generate temperature: float model temperature top_p: float model top p top_k: int model top k do_sample: bool wether to do sample process_prompt: wether to process prompt (set for Bundle generic v1 and v2 endpoints) Key init args — client params: sambastudio_url: str SambaStudio endpoint Url sambastudio_api_key: str SambaStudio endpoint api key Instantiate: .. code-block:: python from langchain_community.llms import SambaStudio llm = SambaStudio=( sambastudio_url = set with your SambaStudio deployed endpoint URL, sambastudio_api_key = set with your SambaStudio deployed endpoint Key, model_kwargs = { "model" : model or expert name (set for Bundle endpoints), "max_tokens" : max number of tokens to generate, "temperature" : model temperature, "top_p" : model top p, "top_k" : model top k, "do_sample" : wether to do sample "process_prompt" : wether to process prompt (set for Bundle generic v1 and v2 endpoints) } ) Invoke: .. code-block:: python prompt = "tell me a joke" response = llm.invoke(prompt) Stream: .. code-block:: python for chunk in llm.stream(prompt): print(chunk, end="", flush=True) Async: .. code-block:: python response = llm.ainvoke(prompt) await response defaultsambastudio_urlsambastudio_api_keyT)rexcludebase_url streaming_urlF streamingN model_kwargsceZdZdZy)SambaStudio.ConfigTN__name__ __module__ __qualname__populate_by_namea/var/www/html/test/engine/venv/lib/python3.12/site-packages/langchain_community/llms/sambanova.pyConfigr|r&r(returncy)9Return whether this model can be serialized by Langchain.Tr%clss r'is_lc_serializablezSambaStudio.is_lc_serializablesr&c dddS)Nrr)rrr%selfs r' lc_secretszSambaStudio.lc_secretss 1#8  r&c:d|jid|jiS)Return a dictionary of identifying parameters. This information is used by the LangChain callback system, which is used for tracing purposes make it possible to monitor LLMs. rr)rrr1s r'_identifying_paramszSambaStudio._identifying_paramss#T^^S@Q@Q/RSSr&cy)zReturn type of llm.zsambastudio-llmr%r1s r' _llm_typezSambaStudio._llm_types!r&kwargsc t|dd|d<tt|dd|d<|j|d\|d<|d<t|di|y) 'init and validate environment variablesrSAMBASTUDIO_URLrSAMBASTUDIO_API_KEYrrNr%)rr _get_sambastudio_urlssuper__init__r2r9 __class__s r'r@zSambaStudio.__init__sx$8 %'8%  !)> )>@U V) $%7;6P6P $ %7 3zF?3 "6"r&urlcd|vr|}|}||fSd|vr|jdd}|}||fS|}d|vr$dj|jd}||fStd)a3 Get streaming and non streaming URLs from the given URL Args: url: string with sambastudio base or streaming endpoint url Returns: base_url: string with url to do non streaming calls streaming_url: string with url to do streaming calls chat/completionsstreamzstream/rgenericzgeneric/streamUnsupported URL)replacejoinsplit ValueError)r2rCr stream_urls r'r>z!SambaStudio._get_sambastudio_urlss  $HJ##3;;y"5 ## #!1!6!6syy7K!LJ##%%677r&stopc |g}|jxsi}|jdg|z}t|dkDr||d<d|jvrsd|j vr|j d|d<d|j vr|j d|d<d|j vr|j d|}|Sd |jvrPd|j vr|j d|d<d|j vr|j d|d<|}|Sd |jvrd|j vr|j d|d<d|j vr|j d|d<|j Dcic]'\}}|t|jt|d )}}}|Std |jd cc}}w)aD Get the tuning parameters to use when calling the LLM. Args: stop: Stop words to use when generating. Model output is cut off at the first occurrence of any of the stop substrings. Returns: The tuning parameters in the format required by api to use stop_sequencesrrE select_expertmodelmax_tokens_to_generate max_tokensprocess_promptapi/v2/predict/genericapi/predict/generic)typevaluerH9only openai, generic v1 and generic v2 APIs are supported) rgetlenrkeyspopitemsrXr!strrL)r2rN _model_kwargs_stop_sequences tuning_paramskvs r'_get_tuning_paramszSambaStudio._get_tuning_paramss& <D))/R (++,1B1B71K o.}1133:G:K:K ; 67*M*'#d&:&: :-,,..1>1B1B71K o.}1133:G:K:K ; 67 +002AqDG,,s1v>>M !$"6"6!7KL  s ,Gpromptct|tr|g}|j|}d|jvrXd|ddg}||d|}|j Dcic] \}}| || }}}d|j j dd } n;d |jvr|jd d r!tjd dd|ddgd}n|d}d|dg} |j Dcic] \}}| || }}}| |d}d|j j i} nd|jvr|jd d r>|d jddk(r!tjd dd|ddgd}n |d}n|d}|r||d}n|g|d}d|j j i} ntd|jdtj} |r | j|j| |d} n| j|j| |d } | j dk7r&t#d| j d| j$d| Scc}}wcc}}w) a Performs a post request to the LLM API. Args: prompt: The prompt to pass into the model stop: list of stop tokens streaming: wether to do a streaming call Returns: A request Response object rEuserrrolecontent)messagesrFNBearer application/json Authorizationz Content-TyperVrUFzsambaverse-conversation-id) message_idrkrl)conversation_idrmitem0)idrY)r_paramskeyrWrYTrue)instancerv) instancesrvrHrZTheadersjsonrF2Sambanova / complete call failed with status code .) isinstancer`rfrr_rget_secret_valuer[r}dumpsrLrequestsSessionpostrr status_code RuntimeErrortext) r2rgrNrrv messages_dictdatarwrYr|r_ http_sessionresponses r'_handle_requestzSambaStudio._handle_requests$ fc "XF((. !5!5 5&,CDM -MfMD15S:3ARCJSDS#*++<<>?"A 2G&)=)= =zz*E2+G+/FSTIV% #f56E39<<>WZS%UEVc5jWFW"f5Dd66GGIJG#d&:&: :zz*E2*+//8FB!ZZ/K37,2/5ay!")  F$AYF$*f=&,X@d66GGIJG!$"6"6!7KL   '') #((""G$t)H$(( wT%)H   3 &D''(==/$  ST*Xs I!I* I5Irc\ |j}d|jvr|dddd}|Sd |jvr|d dd d }|Sd |jvr |ddd }|St d|jd#t$r }td|d|jd}~wwxYw) Process a non streaming response from the api Args: response: A request Response object Returns completion: a string with model generation ;Sambanova /complete call failed couldn't get JSON response response: NrEchoicesrmessagerlrVr_rY completionrW predictionsrHrZ)r} ExceptionrrrrLr2r response_dicters r'_process_responsezSambaStudio._process_responsehs $MMOM !5!5 5&y1!4Y? JJ&)=)= =&w/27;LIJ#d&:&: :&}5a8FJ  !$"6"6!7KL ! MaSX]]O-  sB B+ B&&B+c#HK ddl}d|jvr7|j|}|j D]}|j dk(r&t d|jd|jd |jdk7rt|jtr tj|j}n&t d|jd|jd|jd r&t d|jd|jdt|d dkDr|d dd d }nd }t|}|yd|jvrM|j#D]9} tj| }|ddddd}t|}|;yd|jvrJ|j#D]6} tj| }|dddd}t|}|8yt%d|jd#t$r tdwxYw#t $r }t d|d|jd}~wwxYw#t $r}t d|d| d}~wwxYw#t $r}t d|d| d}~wwxYww) Process a streaming response from the api Args: response: An iterable request Response object Yields: GenerationChunk: a GenerationChunk with model partial generation rNTcould not import sseclient libraryPlease install it with `pip install sseclient-py`.rE error_event1Sambanova /complete call failed with status code r[DONE]errorrdeltarlrr3Error getting content chunk raw streamed response: data: rVresultr_rY stream_tokenzline: rW responsesrHrZ) sseclient ImportErrorr SSEClienteventseventrrrrr`r}loadsr[r\r r iter_linesrL) r2rrclientrrrlgenerated_chunkrlines r'_process_stream_responsez$SambaStudio._process_stream_responses   !5!5 5((2F# ;;-/&K#//0 ::,a) zzX-%ejj#6#'::ejj#9D"."S#+#7#7"8#(::,a!1#  88G,"."S#+#7#7"8#(::,a!1# tI/!3&*9oa&8&A)&LG&(G*9w*G--;# L&)=)= = ++-  ::d+D"8nW5a8A.QG&57&CO)) #d&:&: : ++-  ::d+D"8n[9!<^LG&57&CO)) !$"6"6!7KL [ E  P!&MaS  .!&MaS (!&MaS (sJ"HA)J"2CH2 %J"06I&$J" 3J>J"H//J"2 I;IIJ" I='I88I==J" J JJJ" run_managerc+K|j||d}|j|D]#}|r|j|j|%yw)a\Call out to Sambanova's complete endpoint. Args: prompt: The prompt to pass into the model. stop: a list of strings on which the model should stop generating. run_manager: A run manager with callbacks for the LLM. Yields: chunk: GenerationChunk with model partial generation TrNrron_llm_new_tokenrr2rgrNrr9rchunks r'_streamzSambaStudio._streamS '''E228< E,,UZZ8K A Ac |jr.d}|jd|||d|D]}||jz }|S|j||d}|j |}|S)a Call out to Sambanova's complete endpoint. Args: prompt: The prompt to pass into the model. stop: a list of strings on which the model should stop generating. Returns: result: string with model generation rrgrNrFrr%rrrrrr2rgrNrr9rrrs r'_callzSambaStudio._call >>J%DkEK )ejj(  )  '''F++H5 r&)NNFNN))r!r"r#__doc__rrr`__annotations__rrrrrboolrrrrr( classmethodr/propertyr3r6r8r@rr>rrfr rrrrr rr rr __classcell__rBs@r'rr sZx!,OS,%*9R=%AA"d3Hc3'r48M38#E*It*K-1L(4S>*12  4 DcN  TT#s(^TT!3!! # # #$$sCx$4?xS ':?d3PS8n?H%)$) dd3in%dtCy!dD> d  dL"("s"H]]h>W]D%):> d3in%tCy!67    / " 2%):> d3in%tCy!67    r&rc eZdZUdZedZeed< eedZ eed< edZ eed< edZ e ed < ed Z eed < ed Zeed < edZeeed< edZeeed< eddiZeed< GddZede fdZedeeeffdZedeeeffdZedefdZdeddffd Z d&de e!eefdee!ed ee de"fdZ#d e"defd!Z$d e"de%e&fd"Z' d'de e!eefdee!ed#ee(dedef d$Z) d'de e!eefdee!ed#ee(dede%e&f d%Z*xZ+S)(SambaNovaCloudu SambaNova Cloud large language models. Setup: To use, you should have the environment variables: ``SAMBANOVA_URL`` set with SambaNova Cloud URL. defaults to http://cloud.sambanova.ai/ ``SAMBANOVA_API_KEY`` set with your SambaNova Cloud API Key. Example: .. code-block:: python from langchain_community.llms.sambanova import SambaNovaCloud SambaNovaCloud( sambanova_api_key="your-SambaNovaCloud-API-key, model = model name, max_tokens = max number of tokens to generate, temperature = model temperature, top_p = model top p, top_k = model top k ) Key init args — completion params: model: str The name of the model to use, e.g., Meta-Llama-3-70B-Instruct-4096 (set for CoE endpoints). streaming: bool Whether to use streaming handler when using non streaming methods max_tokens: int max tokens to generate temperature: float model temperature top_p: float model top p top_k: int model top k Key init args — client params: sambanova_url: str SambaNovaCloud Url defaults to http://cloud.sambanova.ai/ sambanova_api_key: str SambaNovaCloud api key Instantiate: .. code-block:: python from langchain_community.llms.sambanova import SambaNovaCloud SambaNovaCloud( sambanova_api_key="your-SambaNovaCloud-API-key, model = model name, max_tokens = max number of tokens to generate, temperature = model temperature, top_p = model top p, top_k = model top k ) Invoke: .. code-block:: python prompt = "tell me a joke" response = llm.invoke(prompt) Stream: .. code-block:: python for chunk in llm.stream(prompt): print(chunk, end="", flush=True) Async: .. code-block:: python response = llm.ainvoke(prompt) await response rr sambanova_urlsambanova_api_keyzMeta-Llama-3.1-8B-InstructrRFrirTgffffff? temperatureNtop_ptop_k include_usageTstream_optionsceZdZdZy)SambaNovaCloud.ConfigTNr r%r&r'r(r{r)r&r(r*cy)r,Fr%r-s r'r/z!SambaNovaCloud.is_lc_serializable~sr&c ddiS)Nrr%r1s r'r3zSambaNovaCloud.lc_secretss#%899r&c|j|j|j|j|j|j |j dS)r5rRrrTrrrrrr1s r'r6z"SambaNovaCloud._identifying_paramssCZZ//++ZZZZ"11  r&cy)z7Get the type of language model used by this chat model.zsambanovacloud-llmr%r1s r'r8zSambaNovaCloud._llm_types$r&r9c zt|ddd|d<tt|dd|d<t| di|y) r;r SAMBANOVA_URLz,https://api.sambanova.ai/v1/chat/completionsrrSAMBANOVA_API_KEYNr%)rr r?r@rAs r'r@zSambaNovaCloud.__init__sP"6   B #  '< )<>Q R' "# "6"r&rgrNc dt|tr|g}d|ddg}|||j||j|j|j |j d}|jDcic] \}}| || }}}d|jjdd}tj} |r | j|j||d } n| j|j||d } | jd k7r&td | jd | j d | Scc}}w)z Performs a post request to the LLM API. Args: prompt: The prompt to pass into the model. stop: list of stop tokens Returns: A request Response object rirrj)rmrFrTrNrRrrrrnrorpTr{Fr~rr)rr`rTrRrrrr_rrrrrrrrr) r2rgrNrrrrwrYr|rrs r'rzSambaNovaCloud._handle_requestsO fc "XF"(VAY?@ %//ZZ++ZZZZ  .2ZZ\OzsEU=NU OO&t'='='N'N'P&QR.   '') #((""G$t)H$((""G$u)H   3 &D''(==/$  -Ps - D,8D,rc |j}|dddd}|S#t$r }td|d|jd}~wwxYw)rrrNrrrrl)r}rrrrs r'rz SambaNovaCloud._process_responsesi $MMOM#9-a0;IF  MaSX]]O-  s" A AA c# K ddl}|j|}|jD]}|jdk(r&t d|j d|jd |jdk7rt|jtr tj|j}n&t d|j d|jd|jdr&t d|j d|jdt|d dkDr|d dd d }nd }t| }|y#t$r tdwxYw#t$r }t d|d|jd}~wwxYww)rrNrrrrrrrrrlrrrr)rrrrrrrrrr`r}rr[r\r r) r2rrrrrrlrrs r'rz'SambaNovaCloud._process_stream_responses  $$X.]]_# E{{m+"G++,Azzl!%  ::)!%**c2#zz%**5*O'334A$zzl!- xx(*O'334A$zzl!- 4 ?+a/"&y/!"4W"=i"H"$&57&CO));#  E  L "I!UZZL* sAFD?AF#CE;F?EF F E;;FFrc |jr.d}|jd|||d|D]}||jz }|S|j||d}|j |}|S)Call out to SambaNovaCloud complete endpoint. Args: prompt: The prompt to pass into the model. stop: Optional list of stop words to use when generating. Returns: The string generated by the model. rrFrr%rrs r'rzSambaNovaCloud._call1rr&c+K|j||d}|j|D]#}|r|j|j|%yw)rTrNrrs r'rzSambaNovaCloud._streamNrrrr),r!r"r#rrrr`rrrrRrrrTintrfloatrrrrdictr(rr/rrr3rr6r8r@r rrrrrr rr rrrrs@r'rrs>@r*M3*#(2#?y?!; 4  4l(s077h>W7x%):> d3in%tCy!67    @%):> d3in%tCy!67    / " r&r)r}typingrrrrrrr r langchain_core.callbacks.managerr #langchain_core.language_models.llmsr langchain_core.outputsr langchain_core.utilsr rpydanticrrrrrr%r&r'rsC DDDE32L%O#OdCSCr&