ujhS ddlZddlmZddlmZmZmZddlZddlm Z ddl m Z ddl m Z ddlmZddlmZd Zd Zd Zd gd gd dgdZdedeededeeeeffdZGdde Zy)N)Path)ListTupleUnion)Tensor)Dataset)download_url_to_file)_get_librispeech_metadata) _extract_tarlibrispeech_finetuningzIhttps://dl.fbaipublicfiles.com/librilight/data/librispeech_finetuning.tgz@5d1efdc777b548194d7e09ba89126e2188026df9fd57aa57eb14408d2b2342afz1h/0z1h/*9h)10min1h10hpathfolders _ext_audioreturnc jt|}g}|D]}|j|d|Dcgc]}|j|}}||Dcgc]@}t|jjjt|j fBc}z }|j d|Scc}wcc}w)aGet the file names and the corresponding file paths without `speaker_id` and `chapter_id` directories. The format of path is like: {root}/{_ARCHIVE_NAME}/1h/[0-5]/[clean, other] or {root}/{_ARCHIVE_NAME}/9h/[clean, other] Args: path (Path): Root path to the dataset. folders (List[str]): Folders that contain the desired audio files. _ext_audio (str): Extension of audio files. Returns: List[Tuple[str, str]]: List of tuples where the first element is the relative path to the audio file. The format of relative path is like: 1h/[0-5]/[clean, other] or 9h/[clean, other] The second element is the file name without audio extension. z/*/*/*/*c|d|dzS)Nr)xs d/var/www/html/dev/engine/venv/lib/python3.12/site-packages/torchaudio/datasets/librilight_limited.pyz$_get_fileids_paths..,s1Q4!A$;)key)rglob relative_tostrparentstemsort)rrr files_pathsfolderppathss r_get_fileids_pathsr)s( :DKS.2ii6((:,8W.XYt$YYERqQXX__334c!&&kBRR S./ ZRs B+ AB0c neZdZdZdZdZ d deeefdede ddfd Z d e de e e ee e e ffd Zde fd Zy)LibriLightLimitedaSubset of Libri-light :cite:`librilight` dataset, which was used in HuBERT :cite:`hsu2021hubert` for supervised fine-tuning. Args: root (str or Path): Path to the directory where the dataset is found or downloaded. subset (str, optional): The subset to use. Options: [``"10min"``, ``"1h"``, ``"10h"``] (Default: ``"10min"``). download (bool, optional): Whether to download the dataset if it is not found at root path. (default: ``False``). z .trans.txtz.flacrootsubsetdownloadrNcn|tvr#tdtjd|t|}tj|}tj j |t|_tj j |td}tj j|jsM|s tdtj j|stt|tt|t!|j||j"|_y)Nz`subset` must be one of z . Found: z.tgzz9Dataset not found. Please use `download=True` to download) hash_prefix) _SUBSET_MAP ValueErrorkeysosfspathrjoin _ARCHIVE_NAME_pathisdir RuntimeErrorisfiler _URL _CHECKSUMr r)r_fileids_paths)selfr,r-r.rarchives r__init__zLibriLightLimited.__init__?s  $7 8H8H8J7K9U[T\]^ ^f%yyWW\\$ 6 '',,t d%;<ww}}TZZ("#^__77>>'*$T7 J  !0WdooVrnc|j|\}}t||j||j|j}t j tjj|j|d\}}|f|ddzS)aLoad the n-th sample from the dataset. Args: n (int): The index of the sample to be loaded Returns: Tuple of the following items; Tensor: Waveform int: Sample rate str: Transcript int: Speaker ID int: Chapter ID int: Utterance ID rrN) r>r r8r_ext_txt torchaudioloadr4rr6)r?rB file_pathfileidmetadatawaveform_s r __getitem__zLibriLightLimited.__getitem__Tsv*!//2 6,VTZZDOO]a]j]jk oobggll4::x{&KL !{Xab\))rc,t|jS)N)lenr>)r?s r__len__zLibriLightLimited.__len__ns4&&''r)rF)__name__ __module__ __qualname____doc__rDrrr!rboolrAintrrrLrOrrrr+r+0s HJ  WCIWW W  W**S*U63S#s+J%K*4((rr+)r4pathlibrtypingrrrrEtorchrtorch.utils.datartorchaudio._internalr torchaudio.datasets.librispeechr torchaudio.datasets.utilsr r7r<r=r1r!r)r+rrrr]s %%$5E2) R N &$H TDI34PUVY[^V^P_K`:?(?(r