ujh! ddlZddlmZddlmZmZddlZddlmZddl m Z ddl m Z ddl mZdd d d d gd dddddd d gd ddddddd gd dddZGdde Zy)N)Path)TupleUnion)Tensor)Dataset)download_url_to_file) _extract_tarTEDLIUM_release1z:http://www.openslr.org/resources/7/TEDLIUM_release1.tar.gz@30301975fd8c5cac4040c261c0852f57cfa8adbbad2ce78e77e4986957445f27train)r testdevzTEDLIUM.150K.dic)folder_in_archiveurlchecksum data_pathsubsetsupported_subsetsdictTEDLIUM_release2z;http://www.openslr.org/resources/19/TEDLIUM_release2.tar.gz@93281b5fcaaae5c88671c9d000b443cb3c7ea3499ad12010b3934ca41a7b9c58zTEDLIUM.152k.diczTEDLIUM_release-3z9http://www.openslr.org/resources/51/TEDLIUM_release-3.tgz@ad1e454d14d1ad550bc2564c462d87c7a7ec83d4dc2b9210f22ab4973b9eccdbzdata/)release1release2release3ceZdZdZ ddeeefdededededdf d Zd ed e d ede e e ee e e ffd Z dd ede de de de e gf dZde de e e ee e e ffdZde fdZedZy)TEDLIUMa*Tedlium* :cite:`rousseau2012tedlium` dataset (releases 1,2 and 3). Args: root (str or Path): Path to the directory where the dataset is found or downloaded. release (str, optional): Release version. Allowed values are ``"release1"``, ``"release2"`` or ``"release3"``. (default: ``"release1"``). subset (str, optional): The subset of dataset to use. Valid options are ``"train"``, ``"dev"``, and ``"test"``. Defaults to ``"train"``. download (bool, optional): Whether to download the dataset if it is not found at root path. (default: ``False``). audio_ext (str, optional): extension for audio file (default: ``".sph"``) rootreleaserdownload audio_extreturnNc.||_|tjvr)t|d}t|d}|r|n t|d}n-tdj |tj|t|dvr%tdj |t|dt j |}t jj|}t jj||} |jdd}|d k(r^|d k(r1t jj||t|d |_ nYt jj||d ||_ n1t jj||t|d ||_ |rnt jj|jst jj| st|d } t|| | t| nBt jj!|jstd|jdg|_t jj|jd} t%t j&| D]j)dst jj|jd} t+| 5} t-| j/} j1dd|j"j3fdt5| Ddddt jj||t|d|_d|_y#1swYxYw)NrrrzFThe release {} does not match any of the supported tedlium releases{} rzDThe subset {} does not match any of the supported tedlium subsets{} .rrr rlegacyr) hash_prefixz The path zT doesn't exist. Please check the ``root`` path or set `download=True` to download itstm.stmr c3&K|]}|f yw)N).0linefiles Y/var/www/html/dev/engine/venv/lib/python3.12/site-packages/torchaudio/datasets/tedlium.py z#TEDLIUM.__init__..~s)L44,)Lsr) _ext_audio_RELEASE_CONFIGSkeys RuntimeErrorformatosfspathpathbasenamejoinsplit_pathisdirisfilerr exists _filelistsortedlistdirendswithopenlen readlinesreplaceextendrange _dict_path _phoneme_dict)selfrr rr!r"rrr9archiverstm_pathflr.s @r/__init__zTEDLIUM.__init__:s$ &++- - 0 9:M N "7+E2C%V+;G+DX+NFX__$))+  )'23FG GV]]$W-.AB yy77##C('',,tX.>>#&q) j  WW\\$0ACST[C\]hCij WW\\$0A8VT d,=?OPW?XYd?egmnDJ 77==,ww~~g./8DH(g8LW%77>>$**-" |,[[ 77<< E22::h/0 MD}}V$77<< E4@(^MqAKKM*A<<3DNN)))L58)LLMM M'',,t->@PQX@YZ`@ab! MMs 0AN  N fileidr-r8c~tjj|d|}t|dz5}|j |}|j dd\}}} } } } }dddtjj|d|} |j | |jz  \}}||  fS#1swYVxYw)aLoads a TEDLIUM dataset sample given a file name and corresponding sentence name. Args: fileid (str): File id to identify both text and audio files corresponding to the sample line (int): Line identifier for the sample inside the text file path (str): Dataset root path Returns: (Tensor, int, str, int, int, int): ``(waveform, sample_rate, transcript, talk_id, speaker_id, identifier)`` r(r) Nsph) start_timeend_time)r6r8r:rDrFr; _load_audior1)rLrRr-r8transcript_pathrO transcripttalk_id_ speaker_idrWrX identifier wave_pathwaveform sample_rates r/_load_tedlium_itemzTEDLIUM._load_tedlium_items'',,tUF; /F* + lqt,JS]ScScdgijSk PGQ J*j lGGLLuf5 $ 0 0T__1LYcnv 0 w++z7J SS l ls .B33B<rWrXrbctt||z}tt||z}|||z d}tj|fi|S)a"Default load function used in TEDLIUM dataset, you can overwrite this function to customize functionality and load individual sentences from a full ted audio talk file. Args: path (str): Path to audio file start_time (int): Time in seconds where the sample sentence stars end_time (int): Time in seconds where the sample sentence finishes sample_rate (float, optional): Sampling rate Returns: [Tensor, int]: Audio tensor representation and sample rate ) frame_offset num_frames)intfloat torchaudioload)rLr8rWrXrbkwargss r/rYzTEDLIUM._load_audiosNz*[89 uX45",Hz>""rlcZ|jszi|_t|jdd5}|jD]>}|j j }t |dd|j|d<@ ddd|jjS#1swY#xYw)zdict[str, tuple[str]]: Phonemes. Mapping from word to tuple of phonemes. Note that some words have empty phonemes. rzutf-8)encodingNr)rKrDrJrFstripr;tuplecopy)rLrOr-contents r/ phoneme_dictzTEDLIUM.phoneme_dicts !!!#D doosW= HKKMHD"jjl002G5:712;5GD&&wqz2H H!!&&((  H Hs AB!!B*)rr Fz.sph)i>)__name__ __module__ __qualname____doc__rstrrboolrQrgrrrcrhrYrorqpropertyrzr+rlr/rr+s "" G"CIG"G" G"  G"  G" G"RTTCTsTuVUXZ]_bdgilMlGmT,///%/VY/gmorfs/(ASAU63S#s+J%KA2## ) )rlr)r6pathlibrtypingrrritorchrtorch.utils.datartorchaudio._internalrtorchaudio.datasets.utilsr r2rr+rlr/rs $52 0KV5"0LV5"1JV5"'>o)go)rl