7|h(ddlmZddlZddlmZddlmZmZmZm Z m Z ddl Z ddl m Z ddlmZddlmZddlmZdd lmZGd d eZy) ) annotationsN)StringIO)AnyDictIteratorListOptional)CallbackManagerForLLMRun)LLM)GenerationChunk)get_pydantic_field_names) ConfigDictceZdZUdZdZded< dZded< dZd ed < d Zd ed < dZ ded< dZ d ed< dZ ded< dZ ded< d Z d ed< dZd ed< dZded< dZded< dZded< dZd ed < d!Zd ed"< d#Zded$< d#Zded%< dZd ed&< d'Zded(< d)Zded*< ed+,Zed4d-Zed5d.Zed6d/Z d7 d8d0Z d9 d:d1Z d9 d;d2Z!d>#F8  r4c z|jr[t5}|j|f||d|D]}|j|j|j }ddd|S|j dd|i|}d|i|} tj|jdddi| d|j } | jd | _| jd }|S#1swYSxYw#tjj$r.tjjd |jd wxYw)aRequest prompt completion from the llamafile server and return the output. Args: prompt: The prompt to use for generation. stop: A list of strings to stop generation when encountered. run_manager: **kwargs: Any additional options to pass as part of the generation request. Returns: The string generated by the model. )rJ run_managerNrJprompt /completion Content-Typeapplication/jsonFurlheadersjsonrKtimeoutTCould not connect to Llamafile server. Please make sure that a server is running at .zutf-8contentr0)rr_streamwritetextgetvaluerPrequestspostrr exceptionsConnectionErrorraise_for_statusencodingrZ) r1rSrJrRrNbuffchunkrarFpayloadresponses r2_callzLlamafile._calls], >> 't)T\\!%;BH+EJJuzz*+ }}  'K*T))>t>v>F262G #====/5&(:!  00  % % ' 'H ==?9-DKG 'K &&66 ))99337==/D sAC"3C/"C,/A D:c+K|jdd|i|}d|vrd|d<d|i|} tj|jdddi|d|j}d |_|jd D]A}|j|} t| } |r|j| j| Cy#tj j $r.tj j d |jd wxYww)a"Yields results objects as they are generated in real time. It also calls the callback manager's on_llm_new_token event with similar parameters to the OpenAI LLM class method of the same name. Args: prompt: The prompts to pass into the model. stop: Optional list of stop words to use when generating. run_manager: **kwargs: Any additional options to pass as part of the generation request. Returns: A generator representing the stream of tokens being generated. Yields: Dictionary-like objects each containing a token Example: .. code-block:: python from langchain_community.llms import Llamafile llm = Llamafile( temperature = 0.0 ) for chunk in llm.stream("Ask 'Hi, how are you?' like a pirate:'", stop=["'"," "]): result = chunk["choices"][0] print(result["text"], end='', flush=True) rJrKTrSrTrUrVrWr\r]utf8)decode_unicode)ra)tokenNr0) rPrcrdrrrerfrh iter_lines_get_chunk_contentr on_llm_new_tokenra) r1rSrJrRrNrFrkrl raw_chunkr^rjs r2r_zLlamafile._streams*L&%%:4:6: 6 !#F8 V.v. }}}}o[1"$6,,H#!,,D,A I--i8G#1E,,5::,>K  ""22 %%55//3}}oQ@  s$D3B8AD8A DDc~|jdr+|jd}tj|}|dS|S)zWhen streaming is turned on, llamafile server returns lines like: 'data: {"content":" They","multimodal":true,"slot_id":0,"stop":false}' Here, we convert this to a dict and return the value of the 'content' field zdata:zdata: r^) startswithlstriprZloads)r1rjcleaneddatas r2rszLlamafile._get_chunk_content1s=   G $ll8,G::g&D ? "Lr4)returnr)r|z List[str])r|Dict[str, Any]rD)rJOptional[List[str]]rNrr|r})NN) rSrrJr~rR"Optional[CallbackManagerForLLMRun]rNrr|r) rSrrJr~rRrrNrr|zIterator[GenerationChunk])rjrr|r)#__name__ __module__ __qualname____doc__r__annotations__rrrrrrrrr r!r"r#r%r&r'r(r)r*r+r model_configpropertyr3rBrHrPrmr_rsr0r4r2rrs0*,Hc+;%)O])%ItJ D#N#K#E3OE57E5@Is< FCO E5NIu  NEM3$K"e!A"u"AHcL%GL%FL.+/':= .%):> ::":8 :  : :~%):> DD"D8 D  D # DLr4r) __future__rrZiortypingrrrrr rc langchain_core.callbacks.managerr #langchain_core.language_models.llmsr langchain_core.outputsr langchain_core.utilsr pydanticrrr0r4r2rs4" 66E329ppr4