ihJddlmZmZmZmZddlmZddlmZGddeZ y))DictIteratorListOptional) BaseLoader)DocumentceZdZdZ d deedeeddffd ZddZdeeefddfd Z de e fd Z de e fd Z dee fd ZxZS) NeedleLoadera NeedleLoader is a document loader for managing documents stored in a collection. Setup: Install the `needle-python` library and set your Needle API key. .. code-block:: bash pip install needle-python export NEEDLE_API_KEY="your-api-key" Key init args: - `needle_api_key` (Optional[str]): API key for authenticating with Needle. - `collection_id` (str): Needle collection to load documents from. Usage: .. code-block:: python from langchain_community.document_loaders.needle import NeedleLoader loader = NeedleLoader( needle_api_key="your-api-key", collection_id="your-collection-id" ) # Load documents documents = loader.load() for doc in documents: print(doc.metadata) # Lazy load documents for doc in loader.lazy_load(): print(doc.metadata) Nneedle_api_key collection_idreturnc ddlm}t|||_||_d|_|j r||j |_|j s tdy#t$r tdwxYw)a Initializes the NeedleLoader with API key and collection ID. Args: needle_api_key (Optional[str]): API key for authenticating with Needle. collection_id (Optional[str]): Identifier for the Needle collection. Raises: ImportError: If the `needle-python` library is not installed. ValueError: If the collection ID is not provided. r) NeedleClientzDPlease install with `pip install needle-python` to use NeedleLoader.N)api_keyCollection ID must be provided.) needle.v1r ImportErrorsuper__init__r r client ValueError)selfr r r __class__s i/var/www/html/dev/engine/venv/lib/python3.12/site-packages/langchain_community/document_loaders/needle.pyrzNeedleLoader.__init__+s   . ,*.2   &t/B/BCDK!!>? ?" V  s A((A=c`|j td|js tdy)z Ensures the Needle collection is set and the client is initialized. Raises: ValueError: If the Needle client is not initialized or if the collection ID is missing. Nz9NeedleClient is not initialized. Provide a valid API key.r)rrr rs r_get_collectionzNeedleLoader._get_collectionMs: ;; K !!>? ?"filesc^ ddlm}|j|jJd|j Dcgc]\}}|||}}}|jj jj|j|y#t$r tdwxYwcc}}w)an Adds files to the Needle collection. Args: files (Dict[str, str]): Dictionary where keys are file names and values are file URLs. Raises: ImportError: If the `needle-python` library is not installed. ValueError: If the collection is not properly initialized. r) FileToAddz=Please install with `pip install needle-python` to add files.N!NeedleClient must be initialized.)nameurl)r r) needle.v1.modelsr!rrritems collectionsraddr )rrr!r#r$ files_to_adds r add_fileszNeedleLoader.add_files\s  2 {{&K(KK&GL{{}U)$ t5U U %%)),,L *  O  VsBB)B&c `|j|jJd|jjjj |j }|Dcgc]A}|j dk(r0td|j|jt|dddC}}|Scc}w)a  Fetches metadata for documents from the Needle collection. Returns: List[Document]: A list of documents with metadata. Content is excluded. Raises: ValueError: If the collection is not properly initialized. Nr"indexedsize)sourcetitler.) page_contentmetadata) rrr'rlistr statusrr$r#getattr)rrfiledocss r_fetch_documentszNeedleLoader._fetch_documentsxs {{&K(KK& ''--2243E3EF  {{i' "hh!YY#D&$7        s!AB+c"|jS)z Loads all documents from the Needle collection. Returns: List[Document]: A list of documents from the collection. r8rs rloadzNeedleLoader.loads$$&&rc#@K|jEd{y7w)z Lazily loads documents from the Needle collection. Yields: Iterator[Document]: An iterator over the documents. Nr:rs r lazy_loadzNeedleLoader.lazy_loads((***s )NN)r N)__name__ __module__ __qualname____doc__rstrrrrr*rrr8r;rr= __classcell__)rs@rr r s!J)-'+ @   @ } @  @D @ tCH~ $ 8$x.8'd8n'+8H-+rr N) typingrrrr$langchain_core.document_loaders.baserlangchain_core.documentsrr rrrHs11;-]+:]+r