7|h3ndZddlZddlZddlmZddlmZmZmZm Z m Z m Z m Z m Z mZddlmZddlmZddlmZddlmZdd lmZdd lmZe d eee ef Ze d eee e ee f ZGddZGddeeefZ Gdde eefZ!edddGddeZ"y)zSagemaker InvokeEndpoint API.N)abstractmethod) AnyDictGenericIteratorListMappingOptionalTypeVarUnion) deprecated)CallbackManagerForLLMRun)LLM)pre_init) ConfigDict)enforce_stop_tokens INPUT_TYPE)bound OUTPUT_TYPEc4eZdZdZdeddfdZddZdefdZy) LineIteratoraParse the byte stream input. The output of the model will be in the following format: b'{"outputs": [" a"]} ' b'{"outputs": [" challenging"]} ' b'{"outputs": [" problem"]} ' ... While usually each PayloadPart event from the event stream will contain a byte array with a full json, this is not guaranteed and some of the json objects may be split acrossPayloadPart events. For example: {'PayloadPart': {'Bytes': b'{"outputs": '}} {'PayloadPart': {'Bytes': b'[" problem"]} '}} This class accounts for this by concatenating bytes written via the 'write' function and then exposing a method which will return lines (ending with a ' ' character) within the buffer via the 'scan_lines' function. It maintains the position of the last read position to ensure that previous bytes are not exposed again. For more details see: https://aws.amazon.com/blogs/machine-learning/elevating-the-generative-ai-experience-introducing-streaming-support-in-amazon-sagemaker-hosting/ streamreturnNcdt||_tj|_d|_y)Nr)iter byte_iteratorioBytesIObufferread_pos)selfrs j/var/www/html/test/engine/venv/lib/python3.12/site-packages/langchain_community/llms/sagemaker_endpoint.py__init__zLineIterator.__init__2s"!&\jjl  c|S)Nr!s r"__iter__zLineIterator.__iter__7s r$c@ |jj|j|jj}|r4|dt dk(r#|xjt |z c_|ddS t |j}d|vr|jjdtj|jj|dd#t$r6|j|jjjkrYwxYw)N PayloadPartrBytes)rseekr readlineordlennextr StopIteration getbuffernbytesrSEEK_ENDwrite)r!linechunks r"__next__zLineIterator.__next__:s KK  T]] +;;'')DRCI- T* CRy  T//0 E) KK  Q , KK  eM27; rAr str__annotations__rBrrrbytesrFrrJr&r$r"r@r@Nsy"#/L(3-.<)GXc])C j     u   r$r@ceZdZdZy)LLMContentHandlerzContent handler for LLM class.N)r;r<r=r>r&r$r"rOrO|s(r$rOz0.3.16z1.0z$langchain_aws.llms.SagemakerEndpoint)sinceremovalalternative_importc :eZdZUdZ dZeed< dZeed< dZ eed< dZ e eed< e ed< d Z eed < dZe eed < dZe eed < ed ZededefdZedeeeffdZedefdZ ddede eede ededef dZy)SagemakerEndpointaSagemaker Inference Endpoint models. To use, you must supply the endpoint name from your deployed Sagemaker model & the region where it is deployed. To authenticate, the AWS client uses the following methods to automatically load credentials: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html If a specific credential profile should be used, you must pass the name of the profile from the ~/.aws/credentials file that is to be used. Make sure the credentials / roles used have the required policies to access the Sagemaker endpoint. See: https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies.html Nclient endpoint_name region_namecredentials_profile_namecontent_handlerF streamingrDendpoint_kwargsforbid)extravaluesrc,|jd|S ddl} |d|j|d}n|j}|jd|d|d<|S#t$r}t d |d}~wwxYw#t $r t d wxYw) z.Dont do anything if client provided externallyrUNrrY) profile_namezsagemaker-runtimerX)rXzCould not load credentials to authenticate with AWS client. Please check that credentials in the specified profile name are valid.zRCould not import boto3 python package. Please install it with `pip install boto3`.)getboto3SessionrU Exception ValueError ImportError)clsr_rcsessiones r"validate_environmentz&SagemakerEndpoint.validate_environments ::h  +MX   45A#mm%+,F%G,G $mmoG#*>>'VM5J$2$x    .   >  s)A>AA!! A;* A66A;;A>>BcJ|jxsi}id|jid|iS)zGet the identifying parameters.rWrD)rDrW)r! _model_kwargss r"_identifying_paramsz%SagemakerEndpoint._identifying_params's>))/R   2 23 }-  r$cy)zReturn type of llm.sagemaker_endpointr&r's r" _llm_typezSagemakerEndpoint._llm_type0s$r$rCstop run_managerkwargsc t|jxsi}i||}|jxsi}|jj||}|jj}|jj } |j r|r |jjd |j||jjd|} t| d} d} | D]O} tj| } | jdd}| t||}| |z } |j|Q| S |jj$d |j||| d|}|jj'|d}| t||}|S#t $r}t#d|d}~wwxYw#t $r}t#d |d}~wwxYw) afCall out to Sagemaker inference endpoint. Args: prompt: The prompt to pass into the model. stop: Optional list of stop words to use when generating. Returns: The string generated by the model. Example: .. code-block:: python response = se("Tell me a joke.") ) EndpointNameBody ContentTyperwrVoutputsrNz.Error raised by streaming inference endpoint: )rvrwrxAcceptz$Error raised by inference endpoint: r&)rDr\rZrFrArBr[rU$invoke_endpoint_with_response_streamrWrjsonloadsrbron_llm_new_tokenrerfinvoke_endpointrJ)r!rCrrrsrtrm_endpoint_kwargsbodyrArBrespiteratorcurrent_completionr8 resp_outputrjresponsetexts r"_callzSagemakerEndpoint._call5s*))/R 3=3F3 //52##33FMJ++88 &&.. >>k WGt{{GG!%!3!3 $ 4 4 A A'  (V 5*,"$>D::d+D"&((9"5a"8K'&9+t&L &+5&00=>*) M64;;66!%!3!3 ,"  ' ''88&9IJD+46K) W #QRSQT!UVV W M #Gs!KLL Ms1>B#E<#+F< FFF F7$F22F7)NN)r;r<r=r>rUrrLrWrKrXrYr rOr[boolrDrr\r model_configrrkpropertyr rnrqrrrr&r$r"rTrTsX "/`FC,M3,KP.2hsm2'& It(&$(L(4.'1&*OXd^* L"$"4""H WS#X%6  $3$$%):> DDtCy!D67 D  D Dr$rT)#r>rr|abcrtypingrrrrrr r r r langchain_core._api.deprecationr langchain_core.callbacksr#langchain_core.language_models.llmsrlangchain_core.utilsrpydanticrlangchain_community.llms.utilsrrKrfloatrrr@rOrTr&r$r"rs# XXX6=3)> \sDI~)> ? m5d4;6G1Q+RS 7=7=t+ [!89+ \)*384)  = tt  tr$