|hi ddlZddlZddlZddlmZddlmZmZddlm Z ddl m Z m Z m Z ddlZddlZddlmZddlmZmZmZmZddlmZmZdd lmZd e efd Zd e efd Z d(d e de de!de!de!de!f dZ"de de defdZ#de fdZ$dejJdejJfdZ&de e fdZ' d)de e e fde e e e fd e fd!Z(d"Z)d*d#e e e fd$ed%e!d&e!fd'Z*y)+N) defaultdict)ThreadPoolExecutor as_completed)Path)ListOptionalUnion)Image) DATASETS_DIRLOGGER NUM_THREADSTQDM)download zip_directory)increment_pathreturnc gdS)z Convert 91-index COCO class IDs to 80-index COCO class IDs. Returns: (List[int]): A list of 91 class IDs where the index represents the 80-index class ID and the value is the corresponding 91-index class ID. )[r N NNN !"#$%&'N()*+,-./0123456789:;N<NN=N>?@ABCDEFGHNIJKLMNONrcY/var/www/html/test/engine/venv/lib/python3.12/site-packages/ultralytics/data/converter.pycoco91_to_coco80_classrfs\ \rdc gdS)a Convert 80-index (val2014) to 91-index (paper). Returns: (List[int]): A list of 80 class IDs where each value is the corresponding 91-index class ID. References: https://tech.amikelive.com/node-718/what-object-categories-labels-are-in-coco-dataset/ Examples: >>> import numpy as np >>> a = np.loadtxt("data/coco.names", dtype="str", delimiter="\n") >>> b = np.loadtxt("data/coco_paper.names", dtype="str", delimiter="\n") Convert the darknet to COCO format >>> x1 = [list(a[i] == b).index(True) + 1 for i in range(80)] Convert the COCO to darknet format >>> x2 = [list(b[i] == a).index(True) if any(b[i] == a) else None for i in range(91)] )Prrrrrrrrrrrr r!r"r#r$r%r&r'r(r)r*r+r,r.r/r2r3r4r5r6r7r8r9r:r;r<r=r>r?rArBrCrDrErFrGrHrIrJrKrLrMrNrOrPrQrRrSrTrVrYr[r\r]r^r_r`rarbPQRTUVWXYZrcrcrdrecoco80_to_coco91_classrr{s*Q Qrd labels_dirsave_dir use_segments use_keypoints cls91to80lvisc  t|}|dz |dz fD]}|jddt}tt |j j dD]}|rdn|jjdd} t |dz | z } | jdd|r,| dz jdd| d z jddt|d 5} tj| } d d d  dD cic] } | d d|  }} tt}| dD]}||dj|g}t|j!d|D]\}}||d}|d|d}}|r&t#t |dj%dn|d} |r&|jt#t d| z g}g}g}|D]w}|j'ddrt)j*|dt(j,}|d dxxx|dd dz z ccc|ddgxx|zcc<|dd gxx|zcc<|ddks|d dkr|r ||d!dz n|d!dz }|g|j/z}||vs|j||r-|j'd"t1|d"dk(r|jgt1|d"dkDr[t3|d"}t)j4|d#t)j*||gz j7d$j/}nu|d"Dcgc] }|D]}| }}}t)j*|j7d$dt)j*||gz j7d$j/}|g|z}|j||s|j'd%|j|t)j*|d%j7d$d t)j*||dgz j7d$j/zzt| | z j9d&d'd 5}t;t1|D]^}|r g||} n g|rt1||dkDr||n||} |j=d(t1| zj?| zd)z` d d d |st ||j@jd*djd+d&z }!t|!d'd 5} | jCd,|Dd d d tEjF|rd-nd.d/|j y #1swYxYwcc} wcc}}w#1swYOxYw#1swYexYw)0a Convert COCO dataset annotations to a YOLO annotation format suitable for training YOLO models. Args: labels_dir (str, optional): Path to directory containing COCO dataset annotation files. save_dir (str, optional): Path to directory to save results to. use_segments (bool, optional): Whether to include segmentation masks in the output. use_keypoints (bool, optional): Whether to include keypoint annotations in the output. cls91to80 (bool, optional): Whether to map 91 COCO class IDs to the corresponding 80 COCO class IDs. lvis (bool, optional): Whether to convert data in lvis dataset way. Examples: >>> from ultralytics.data.converter import convert_coco Convert COCO annotations to YOLO format >>> convert_coco("coco/annotations/", use_segments=True, use_keypoints=False, cls91to80=False) Convert LVIS annotations to YOLO format >>> convert_coco("lvis/annotations/", use_segments=True, use_keypoints=False, cls91to80=False, lvis=True) labelsimagesTparentsexist_okz*.json instances_ train2017val2017utf-8encodingNidd annotationsimage_idz Annotations descheightwidthcoco_urlzhttp://images.cocodataset.org file_namez./imagesiscrowdFbbox)dtyperrrr category_id segmentationaxis keypoints.txta%g  lvis_v1_z.jsonc3&K|] }|d ywrNrc).0lines re zconvert_coco..Vs?TvR[?LVISCOCOz/ data converted successfully. Results saved to )$rmkdirrfsortedrresolveglobstemreplaceopenjsonloadrlistappendritemsstr relative_togetnparrayfloat64tolistlenmerge_multi_segment concatenatereshape with_suffixrangewriterstripname writelinesr info)"rsrtrurvrwrxpcoco80 json_filelnamefnfdataxr{rann image_txtimg_idannsimghwbboxessegmentsrboxclssijfilerfilenames" re convert_cocors:h'H  (X"5 5- t,-$ %FD,446;;HEFM@   6 6|R H (^h & . - +  $ $TD $ A )^ " "4$ " ? )g . !99Q23q8$OOB/$ ^!459 3C4G HA!#!:RXXq!f=M!M V VWY Z a a cA,/,? Kq KA K KA K!#!!4!4R!;bhh1v>N!N W WXZ [ b b dA EAI *$)=)I!((288C ,<#=#E#Eb!#LrxxYZ\]_`XaOb#b"k"kln"o"v"v"xx9 BrAv**62C'J Kds6{+KA$1)A,1 -9c(1+>NQR>RhqkX^_`Xa JJD 199;dBTIJK K KW3 Kj H~ (>(>z2(N(V(VW^`f(ggHhg6 @! ?Y?? @ @YM@^ KKT6v..^_g_o_o_q^rstM  =R!L K K @ @s1,VV,V1 A6V7WV) 7W W masks_dir output_dirclassesc t|Dcic]}|dz| }}t|jD]*}|jdvst j t |tj}|j\}}tjd|d|d|tj|} g} | D]} | dk(r |j| d} | dk(rtjd| d |d >t j|| k(j!tj"tj$tj&\} }| D]~}t)|d k\s|j+}| g}|D]D}|j-t/|d|z d |j-t/|d|z d F| j-|t||j0d z }t3|dd5}| D]5}dj5t7t |}|j9|dz7 dddtjd|d|d|-ycc}w#1swY0xYw)u Convert a dataset of segmentation mask images to the YOLO segmentation format. This function takes the directory containing the binary format mask images and converts them into YOLO segmentation format. The converted masks are saved in the specified output directory. Args: masks_dir (str): The path to the directory where all mask images (png, jpg) are stored. output_dir (str): The path to the directory where the converted YOLO segmentation masks will be stored. classes (int): Total classes in the dataset i.e. for COCO classes=80 Examples: >>> from ultralytics.data.converter import convert_segment_masks_to_yolo_seg The classes here is the total classes in the dataset, for COCO dataset we have 80 classes >>> convert_segment_masks_to_yolo_seg("path/to/masks_directory", "path/to/output/directory", classes=80) Notes: The expected directory structure for the masks is: - masks ├─ mask_image_01.png or mask_image_01.jpg ├─ mask_image_02.png or mask_image_02.jpg ├─ mask_image_03.png or mask_image_03.jpg └─ mask_image_04.png or mask_image_04.jpg After execution, the labels will be organized in the following structure: - output_dir ├─ mask_yolo_01.txt ├─ mask_yolo_02.txt ├─ mask_yolo_03.txt └─ mask_yolo_04.txt r>.jpg.png Processing z imgsz = z x rrzUnknown class for pixel value z in file z , skipping.rrrrrr rNzProcessed and stored at )rriterdirsuffixcv2imreadrIMREAD_GRAYSCALEshaper rruniquerwarning findContoursastypeuint8 RETR_EXTERNALCHAIN_APPROX_SIMPLErsqueezerroundrrjoinmapr)rrrrpixel_to_class_mapping mask_pathmask img_height img_width unique_valuesyolo_format_datavalue class_indexcontours_contour yolo_formatpoint output_pathritemrs re!convert_segment_masks_to_yolo_segr[sjF16g?1a!eQh??)_,,.%e   / /::c)nc.B.BCD$(JJ !J KK+i[ *S T UIIdOM! & =A:488C "$NN%CE7)T]S^^i#jk"..U]**2884c6G6GI`I` ! (=G7|q(")//"3'2m %,PE'..uU1X 5I1/MN'..uU1X 5JA/NOP)// <= =.z* /?t-DDKk39 ,T,,D88CTN3DJJtd{+, , KK2;-y TWXaWbc dK%e@D , ,s I%?;I**I3 dota_root_pathc N t|}iddddddddd d d d d ddddddddddddddddddd d!d"d#d$i d%td&td'td(td)tf fd* }d+D]}|d,z |z }|d-z |d.z }|d-z |z }|jd/d/0t |j }t |d1|d23D]Z}|jd4k7r|j}tjt|} | jd5d\} } ||| | ||\y5)6u Convert DOTA dataset annotations to YOLO OBB (Oriented Bounding Box) format. The function processes images in the 'train' and 'val' folders of the DOTA dataset. For each image, it reads the associated label from the original labels directory and writes new labels in YOLO OBB format to a new directory. Args: dota_root_path (str): The root directory path of the DOTA dataset. Examples: >>> from ultralytics.data.converter import convert_dota_to_yolo_obb >>> convert_dota_to_yolo_obb("path/to/DOTA") Notes: The directory structure assumed for the DOTA dataset: - DOTA ├─ images │ ├─ train │ └─ val └─ labels ├─ train_original └─ val_original After execution, the function will organize the labels into: - DOTA └─ labels ├─ train └─ val planershiprz storage-tankrzbaseball-diamondrz tennis-courtrzbasketball-courtrzground-track-fieldrharborrbridgerz large-vehiclerz small-vehicler helicopterr roundaboutrzsoccer-ball-fieldr z swimming-poolr!zcontainer-craner"airportr#helipadr$ image_name image_width image_heightorig_label_dirrtc z||dz }||dz }|jd5}|jd5}|j} | D]} | jj} t | dkr0| d} | } | ddDcgc] }t |}}t dDcgc]}|dzdk(r|||z n|||z }}|Dcgc]}|d }}|j| d d j|d  ddddddycc}wcc}wcc}w#1swY!xYw#1swYyxYw) zaConvert a single image's DOTA annotation to YOLO OBB format and save it to a specified directory.rrrrrNrrz.6grr) r readlinesstripsplitrfloatrrr)rrrrrtorig_label_path save_pathrglinesrparts class_name class_idxrcoordsrnormalized_coordscoordformatted_coords class_mappings re convert_labelz/convert_dota_to_yolo_obb..convert_labels|(j\+>>*T22  ! !# & G!Y^^C-@ GAKKME G **,u:>"1X )*5 ,1"1I6q%(66afghai%\]q1uzF1I +vayvaltrainr{rz _originalTr|rz imagesrrN) rrintrrrrrrrrr) rr'phase image_dirrrt image_paths image_pathimage_name_without_extrrrr&s @reconvert_dota_to_yolo_obbr1s@.)N  A    A  a ! !   b b R  2!" 2#$ 2%M*G#GCGsG\`GlpG("R"X-5 '(2wi5HH!H,u4td39,,./ {;ugW1MN RJ  F*%/__ "**S_-C99Ra=DAq 0!Q Q  RRrdarr1arr2c|dddddf|dddddfz dzjd}tjtj|d|jS)a Find a pair of indexes with the shortest distance between two arrays of 2D points. Args: arr1 (np.ndarray): A NumPy array of shape (N, 2) representing N 2D points. arr2 (np.ndarray): A NumPy array of shape (M, 2) representing M 2D points. Returns: idx1 (int): Index of the point in arr1 with the shortest distance. idx2 (int): Index of the point in arr2 with the shortest distance. Nrrr)sumr unravel_indexargminr)r2r3diss re min_indexr9sV D! tD!QJ/ /A 5 : :2 >C  BIIc5syy AArdrc.g}|Dcgc]'}tj|jdd)}}tt |Dcgc]}g}}tdt |D]E}t ||dz ||\}}||dz j |||j |GtdD]I}|dk(rt|D]\}}t |dk(r%|d|dkDr|ddd}||dddddf||<tj|||d d||<tj||||ddg||<|dt |dz hvr|j ||d|d|dz g}|j |||d|ddztt |dz ddD]E}|dt |dz hvs||}t|d|dz } |j ||| dGL|Scc}wcc}w)a Merge multiple segments into one list by connecting the coordinates with the minimum distance between each segment. This function connects these coordinates with a thin line to merge all segments into one. Args: segments (List[List]): Original segmentations in COCO's JSON file. Each element is a list of coordinates, like [segmentation1, segmentation2,...]. Returns: s (List[np.ndarray]): A list of connected segments represented as NumPy arrays. rrrrNr) rrrrrr9r enumeraterollrabs) rrrridx_listidx1idx2kidxnidxs rerrsD A4<=q ##B*=H=!#h-01q1H11c(m $!xA < dQt$ 4 ! 1X1 6#H- ?3s8q=SVc!f_dd)C"*1+ddAg"6HQK gghqkCF7C  nnhqk8A;r?-KL CMA-..HHXa[)c!fs1vo.CHHXa[Q#a&1*=> ? 3x=1,b"5 1QH  122"1+Cs1vA/DHHXa[/0  1'10 HE>1s ,H  Him_dir sam_modelc ddlm}ddlm}ddlm}||t ttd}t|jdddkDrtjd y tjd ||}t|jt|jd D]}|d\} } |d} t| dk(r| d d ddgfxx| zcc<| d d ddgfxx| zcc<tj|d} || || dd|} | dj j"|d<|r t%|nt%|j&dz }|j)dd|jD]}g}t%|dj+dj,}||z }|d}t/|dD]c\}}t|dk(rt1||g|j3d}|j5dt|zj7|zet9|dd5}|j;d |Dd d d tjd!|y #1swYxYw)"u Convert existing object detection dataset (bounding boxes) to segmentation dataset or oriented bounding box (OBB) in YOLO format. Generate segmentation data using SAM auto-annotator as needed. Args: im_dir (str | Path): Path to image directory to convert. save_dir (str | Path, optional): Path to save the generated labels, labels will be saved into `labels-segment` in the same directory level of `im_dir` if save_dir is None. sam_model (str): Segmentation model to use for intermediate segmentation data. device (int | str, optional): The specific device to run SAM models. Notes: The input directory structure assumed for dataset: - im_dir ├─ 001.jpg ├─ ... └─ NNN.jpg - labels ├─ 001.txt ├─ ... └─ NNN.txt r)SAM) YOLODataset) xywh2xyxyi)names)rrz;Segmentation labels detected, no need to generate new ones!NzBDetection labels detected, generating segment labels by SAM model!zGenerating segment labelstotalrrrrrrim_fileF)rverbosesavedevicezlabels-segmentTr|rrrrrrrc3&K|] }|dz ywrrc)rtexts rerz$yolo_bbox2segment..s77rz"Generated segment labels saved in ) ultralyticsrGultralytics.datarHultralytics.utils.opsrIdictrrrrzr rrrrmasksxynrparentrrrr;r+rrrrr)rDrtrErPrGrHrIdatasetlabelrrboxesim sam_resultstextslb_nametxt_filerrrrrs reyolo_bbox2segmentrbGsa4 ,/&t$uT{2C'DEG 7>>! Z ()A- QR KKTUIIgnnC,?Fab 5W~1h u:?  a!QiA a!QiA ZZi( )9U+;UQV_ef 'N0044j 5"*tH~tF|/B/BEU/UH NN4$N/ 8uY'(44V<AAg%EleJ/0 >DAq1v{AK0!))B-0D LL%#d)+335< =  > (C' 2 8a LL77 7 8 8 8 KK4XJ?@ 8 8s ?I;;J c dtfd}tdz }d}d}t||zg|jt j |dz dz d t t 5}d D]}|d z |z }|jd d ||dz }|jr~t|d5}|D cgc]} || jz } } ddd D cgc]} |j|| } } tt| t| d|D]} t!j"d|d|d dddt!j$dycc} w#1swYxYwcc} w#1swY5xYw)a  Create a synthetic COCO dataset with random images based on filenames from label lists. This function downloads COCO labels, reads image filenames from label list files, creates synthetic images for train2017 and val2017 subsets, and organizes them in the COCO dataset structure. It uses multithreading to generate images efficiently. Examples: >>> from ultralytics.data.converter import create_synthetic_coco_dataset >>> create_synthetic_coco_dataset() Notes: - Requires internet connection to download label files. - Generates random RGB images of varying sizes (480x480 to 640x640 pixels). - Existing test2017 directory is removed as it's not needed. - Reads image filenames from train2017.txt and val2017.txt files. image_filec J|jstjddtjddf}tjd|tjddtjddtjddfj |yy)zdGenerate synthetic images with random sizes and colors for dataset augmentation or testing purposes.iiRGBr)sizecolorN)existsrandomrandintr newrO)rdrhs recreate_synthetic_imagez=create_synthetic_coco_dataset..create_synthetic_images~  "NN3,fnnS#.FGD II~~a-v~~a/Ev~~VWY\G]^ d: #rdcocoz?https://github.com/ultralytics/assets/releases/download/v0.0.0/zcoco2017labels-segments.zip)dirrztest2017T) ignore_errors) max_workers>rrr{r|rrrNzGenerating images for rKz Labels file z- does not exist. Skipping image creation for .z,Synthetic COCO dataset created successfully.)rr rrYshutilrmtreerr rrjrrsubmitrrrr rr)rnrpurl label_zipexecutorsubset subset_dirlabel_list_filerr image_filesrdfuturesrs recreate_synthetic_coco_datasetrs&4  C KC-I cIo CJJ/ MM#.:-TB  4w. wFx&0J   TD  9"vhdO3O%%'/G<EBC"D$3#5"DK"DEbmmS]8??+A:Nmml733w>klrksstuv ww$ KK>?#EEEnwwsD AE*&E+E E E*E%,A E*EE"  E**E3path n_channelsrzipc ddlm}ddlm}t |}|j rot |ddhz Dcgc]}t|jd|!c}g}|D]!} t|||r|j#|r t|y y |jd } t!j"t!j$t'|t j(} t+j,gd } t+j.d d |} || j0| ddd}|| }t!j2t'| t+j4|ddj7t*j8j;dddtjd| y cc}w#t$r&} tjd|d| Yd } ~ {d } ~ wwxYw)aP Convert RGB images to multispectral images by interpolating across wavelength bands. This function takes RGB images and interpolates them to create multispectral images with a specified number of channels. It can process either a single image or a directory of images. Args: path (str | Path): Path to an image file or directory containing images to convert. n_channels (int): Number of spectral channels to generate in the output image. replace (bool): Whether to replace the original image file with the converted one. zip (bool): Whether to zip the converted images into a zip file. Examples: Convert a single image >>> convert_to_multispectral("path/to/image.jpg", n_channels=10) Convert a dataset >>> convert_to_multispectral("coco8", n_channels=10) r)interp1d) IMG_FORMATStiftiffz*.zError converting z: Nz.tiff)iiiiilinearF extrapolate)kind bounds_error fill_valuergrrz Converted )scipy.interpolaterultralytics.data.utilsrris_dirr5rrglobconvert_to_multispectralunlink Exceptionr rrrrcvtColorrr COLOR_BGR2RGBrrlinspaceT imwritemulticliprr transpose)rrrrrrextim_filesim_patherrrgb_wavelengthstarget_wavelengthsr multispectrals rerrs(+2 :D {{}PUW]A^`TZZ"SE 34`bde @G @(*=NN$  @  $  &&w/ ll3::c$i0#2C2CD((?3[[c:> _&&([h i,-  [)277=!S+I+P+PQSQYQY+Z+d+defhikl+mn j ./-a  @ /y1#>?? @s$F0(F55 G$>GG$)z../coco/annotations/zcoco_converted/FFTF)Nzsam_b.ptN)rFF)+rrkru collectionsrconcurrent.futuresrrpathlibrtypingrrr rnumpyrPILr ultralytics.utilsr r r rultralytics.utils.downloadsrrultralytics.utils.filesrr+rfrrrboolrrr1ndarrayr9rrbrrrcrdrers #?(( EE?2dS dNfS fT-% tutututu tu  tu  tunIeIe#IePSIeXZRSZRz BBJJ Bbjj B 0 $t*0 hpt?A #t) ?A(0sDy1A(B?AVY?AD7@t105d#31010TX10gk10rd