7|hx?dZddlZddlmZmZmZmZddlZddlZddl m Z ddl m Z m Z mZmZmZdefdZGdd eZGd d e Zy) aaUtility for using SearxNG meta search API. SearxNG is a privacy-friendly free metasearch engine that aggregates results from `multiple search engines `_ and databases and supports the `OpenSearch `_ specification. More details on the installation instructions `here. <../../integrations/searx.html>`_ For the search API refer to https://docs.searxng.org/dev/search_api.html Quick Start ----------- In order to use this utility you need to provide the searx host. This can be done by passing the named parameter :attr:`searx_host ` or exporting the environment variable SEARX_HOST. Note: this is the only required parameter. Then create a searx search instance like this: .. code-block:: python from langchain_community.utilities import SearxSearchWrapper # when the host starts with `http` SSL is disabled and the connection # is assumed to be on a private network searx_host='http://self.hosted' search = SearxSearchWrapper(searx_host=searx_host) You can now use the ``search`` instance to query the searx API. Searching --------- Use the :meth:`run() ` and :meth:`results() ` methods to query the searx API. Other methods are available for convenience. :class:`SearxResults` is a convenience wrapper around the raw json result. Example usage of the ``run`` method to make a search: .. code-block:: python s.run(query="what is the best search engine?") Engine Parameters ----------------- You can pass any `accepted searx search API `_ parameters to the :py:class:`SearxSearchWrapper` instance. In the following example we are using the :attr:`engines ` and the ``language`` parameters: .. code-block:: python # assuming the searx host is set as above or exported as an env variable s = SearxSearchWrapper(engines=['google', 'bing'], language='es') Search Tips ----------- Searx offers a special `search syntax `_ that can also be used instead of passing engine parameters. For example the following query: .. code-block:: python s = SearxSearchWrapper("langchain library", engines=['github']) # can also be written as: s = SearxSearchWrapper("langchain library !github") # or even: s = SearxSearchWrapper("langchain library !gh") In some situations you might want to pass an extra string to the search query. For example when the `run()` method is called by an agent. The search suffix can also be used as a way to pass extra parameters to searx or the underlying search engines. .. code-block:: python # select the github engine and pass the search suffix s = SearchWrapper("langchain library", query_suffix="!gh") s = SearchWrapper("langchain library") # select github the conventional google search syntax s.run("large language models", query_suffix="site:github.com") *NOTE*: A search suffix can be defined on both the instance and the method level. The resulting query will be the concatenation of the two with the former taking precedence. See `SearxNG Configured Engines `_ and `SearxNG Search Syntax `_ for more details. Notes ----- This wrapper is based on the SearxNG fork https://github.com/searxng/searxng which is better maintained than the original Searx project and offers more features. Public searxNG instances often use a rate limiter for API usage, so you might want to use a self hosted instance and disable the rate limiter. If you are self-hosting an instance you can customize the rate limiter for your own network as described `here `_. For a list of public SearxNG instances see https://searx.space/ N)AnyDictListOptional)get_from_dict_or_env) BaseModel ConfigDictField PrivateAttrmodel_validatorreturnc dddS)Nenjson)languageformatri/var/www/html/test/engine/venv/lib/python3.12/site-packages/langchain_community/utilities/searx_search.py_get_default_paramsrs  //rcpeZdZUdZdZeed<deffd ZdefdZe de fdZ e de fd Z xZ S) SearxResultsz,Dict like wrapper around search api results._datadatac\tj|}t| |||_y)zATake a raw result from Searx and make it into a dict like object.N)rloadssuper__init____dict__)selfr json_data __class__s rrzSearxResults.__init__s%JJt$  # rr c|jS)z$Text representation of searx result.)rr!s r__str__zSearxResults.__str__s zzrc$|jdS)zGSilence mypy for accessing this field. :meta private: resultsgetr%s rr(zSearxResults.resultss xx ""rc$|jdS)z#Helper accessor on the json result.answersr)r%s rr,zSearxResults.answerssxx ""r)__name__ __module__ __qualname____doc__rstr__annotations__rr&propertyrr(r, __classcell__)r#s@rrrs]6E3OS ######rrcLeZdZUdZeZeed<dZe ed<dZ e ed<e e Zeed<d Zeeed <gZeee ed <gZeee ed <dZee ed <dZeed<d Zeeed<edededefdZedZdedefdZ dedefdZ! d!de d eee d eee d ee dede f dZ" d"de d eee d ee dede f dZ# d!de ded eee d eee d ee dedeefdZ$ d"de ded eee d ee dedeef d Z%y )#SearxSearchWrapperaWrapper for Searx API. To use you need to provide the searx host by passing the named parameter ``searx_host`` or exporting the environment variable ``SEARX_HOST``. In some situations you might want to disable SSL verification, for example if you are running searx locally. You can do this by passing the named parameter ``unsecure``. You can also pass the host url scheme as ``http`` to disable SSL. Example: .. code-block:: python from langchain_community.utilities import SearxSearchWrapper searx = SearxSearchWrapper(searx_host="http://localhost:8888") Example with SSL disabled: .. code-block:: python from langchain_community.utilities import SearxSearchWrapper # note the unsecure parameter is not needed if you pass the url scheme as # http searx = SearxSearchWrapper(searx_host="http://localhost:8888", unsecure=True) _resultr searx_hostFunsecure)default_factoryparamsNheadersengines categories query_suffix k aiosessionbefore)modevaluesr c|jdi}t}i|||d<|jd}|rdj||dd<|jd}|rdj||dd<t|dd}|j dst d|d d |z}n|j d rd |d <||d<|S)z?Validate that custom searx params are merged with default ones.r;r=,r>r8 SEARX_HOSThttpzRWarning: missing the url scheme on host ! assuming secure https:// zhttps://zhttp://Tr9)r*rjoinr startswithprint)clsrE user_paramsdefaultr=r>r8s rvalidate_paramsz"SearxSearchWrapper.validate_paramssjj2. %'5g55x**Y' *-((7*;F8 Y 'ZZ - -0XXj-AF8 \ *)&, M $$V, ++5,a9 $j0J  " "9 -!%F: )| rforbid)extractj|j|j||j }|j st d|jt|j}||_ |S)zActual request to searx API.r<r;verifySearx API returned an error: ) requestsr*r8r<r9ok ValueErrortextrr7)r!r; raw_resultress r_searx_api_queryz#SearxSearchWrapper._searx_api_querys`\\ OOLL}}$  }}F: F>/ G:F<;GF"F  F" F FF F" G"F5(F+ )F50 G:F><G>GG G Gqueryrdc d|i}i|j||}|jr2t|jdkDr|dxxd|jzz cc<t|trt|dkDr|dxxd|zz cc<t|t r"t|dkDrdj ||d<t|t r"t|dkDrdj ||d<|j|}t|jdkDr|jd} | St|jdkDrHdj |jd|jD cgc]} | jd d c} } | Sd } | Scc} w) aoRun query through Searx API and parse results. You can pass any other params to the searx query API. Args: query: The query to search for. query_suffix: Extra suffix appended to the query. engines: List of engines to use for the query. categories: List of categories to use for the query. **kwargs: extra parameters to pass to the searx API. Returns: str: The result of the query. Raises: ValueError: If an error occurred with the query. Example: This will make a query to the qwant engine: .. code-block:: python from langchain_community.utilities import SearxSearchWrapper searx = SearxSearchWrapper(searx_host="http://my.searx.host") searx.run("what is the weather in France ?", engine="qwant") # the same result can be achieved using the `!` syntax of searx # to select the engine using `query_suffix` searx.run("what is the weather in France ?", query_suffix="!qwant") qrrJrGr=r> NcontentrNo good search result found) r;r?len isinstancer1listrKr^r,r(rAr*) r!rhr=r>r?rd_paramsr;r]toretrs rrunzSearxSearchWrapper.run%slP  6DKK575f5   T%6%6!7!!; 3K3!2!22 2K lC (S->-B 3K3- -K gt $W)9 # 1F9  j$ 'C Oa,?#&88J#7F< ##F+ s{{ a KKNE   !KKs{{8TVV?T U!y"!5 UVE 2E !VsFc Kd|i}i|j||}|jr2t|jdkDr|dxxd|jzz cc<t|trt|dkDr|dxxd|zz cc<t|t r"t|dkDrdj ||d<|j|d{}t|jdkDr|jd}|St|jdkDrHdj |jd|jD cgc]} | jdd c} }|Sd }|S7cc} ww) z Asynchronously version of `run`.rjrrJrGr=Nrkrlrrm) r;r?rnror1rprKrgr,r(rAr*) r!rhr=r?rdrqr;r]rrrss rarunzSearxSearchWrapper.arunjsO  6DKK575f5   T%6%6!7!!; 3K3!2!22 2K lC (S->-B 3K3- -K gt $W)9 # 1F9 **622 s{{ a KKNE   !KKs{{8TVV?T U!y"!5 UVE 2E 3!Vs%C E$ E A+E$7EE$E$ num_resultsc d|i}i|j||}|jr2t|jdkDr|dxxd|jzz cc<t|trt|dkDr|dxxd|zz cc<t|t r"t|dkDrdj ||d<t|t r"t|dkDrdj ||d<|j|jd|} t| dk(rdd igS| D cgc]&} | jd d | d | d | d| dd(c} Scc} w)a$Run query through Searx API and returns the results with metadata. Args: query: The query to search for. query_suffix: Extra suffix appended to the query. num_results: Limit the number of results to return. engines: List of engines to use for the query. categories: List of categories to use for the query. **kwargs: extra parameters to pass to the searx API. Returns: Dict with the following keys: { snippet: The description of the result. title: The title of the result. link: The link to the result. engines: The engines used for the result. category: Searx category of the result. } rjrrJrGr=r>NResultNo good Search Result was foundrlrtitleurlcategorysnippetr{linkr=r}) r;r?rnror1rprKr^r(r*) r!rhrwr=r>r?rdrqr;r(rfs rr(zSearxSearchWrapper.resultssa>  6DKK575f5   T%6%6!7!!; 3K3!2!22 2K lC (S->-B 3K3- -K gt $W)9 # 1F9  j$ 'C Oa,?#&88J#7F< ''/77 E w<1 @AB B"   "::i4u !),":.     s+E c VKd|i}i|j||}|jr2t|jdkDr|dxxd|jzz cc<t|trt|dkDr|dxxd|zz cc<t|t r"t|dkDrdj ||d<|j|d{jd|}t|dk(rddigS|D cgc]&} | jd d | d | d | d| d d(c} S7Vcc} ww)zdAsynchronously query with json results. Uses aiohttp. See `results` for more info. rjrrJrGr=Nryrzrlrr{r|r}r~) r;r?rnror1rprKrgr(r*) r!rhrwr=r?rdrqr;r(rfs raresultszSearxSearchWrapper.aresultssE  6DKK575f5   T%6%6!7!!; 3K3!2!22 2K lC (S->-B 3K3- -K gt $W)9 # 1F9 //77@@+N w<1 @AB B"   "::i4u !),":.    8  s$C D) D" (D)4+D$D)$D))NNr)Nr)&r-r.r/r0r r7rr2r8r1r9boolr rr;dictr<rr=rr>r?rAintrBrr classmethodrrQr model_configr^rgrtrvr(rrrrr6r6sn6(MG\)JHd)<=FD="GXd^"#%GXd3i %&(Jc#("$L(3-$ AsK $J $(#Tc$6L t  d|>(,*.&( CC$s)$CT#Y' C sm C  C CP(,&( !!$s)$!sm !  ! !N(,*.&( 7 7 7 $s)$ 7 T#Y' 7 sm 7 7  d7 z(,&( $ $ $ $s)$ $ sm $  $  d$ rr6)r0rtypingrrrrrarXlangchain_core.utilsrpydanticrr r r r rrrr6rrrrsPB ,,50T0#4#:x x r