Skip to content

get-preprocessed-dataset-librispeech

Automatically generated README for this automation recipe: get-preprocessed-dataset-librispeech

Category: AI/ML datasets

License: Apache 2.0

  • CM meta description for this script: _cm.json
  • Output cached? True

Reuse this script in your project

Install MLCommons CM automation meta-framework

Pull CM repository with this automation recipe (CM script)

cm pull repo mlcommons@cm4mlops

cmr "get dataset speech-recognition librispeech preprocessed" --help

Run this script

Run this script via CLI
cm run script --tags=get,dataset,speech-recognition,librispeech,preprocessed[,variations] [--input_flags]
Run this script via CLI (alternative)
cmr "get dataset speech-recognition librispeech preprocessed [variations]" [--input_flags]
Run this script from Python
import cmind

r = cmind.access({'action':'run'
              'automation':'script',
              'tags':'get,dataset,speech-recognition,librispeech,preprocessed'
              'out':'con',
              ...
              (other input keys for this script)
              ...
             })

if r['return']>0:
    print (r['error'])
Run this script via Docker (beta)
cm docker script "get dataset speech-recognition librispeech preprocessed[variations]" [--input_flags]

Variations

  • Group "dataset-count"

    Click here to expand this section.

    • _1
      • ENV variables:
        • CM_DATASET_SIZE: 1
    • _5
      • ENV variables:
        • CM_DATASET_SIZE: 5
    • _50
      • ENV variables:
        • CM_DATASET_SIZE: 50
    • _500
      • ENV variables:
        • CM_DATASET_SIZE: 500
    • _full
      • ENV variables:
        • CM_DATASET_SIZE: ``

  • Group "dataset-precision"

    Click here to expand this section.

    • _fp32 (default)
      • ENV variables:
        • CM_DATASET_DTYPE: fp32
    • _int8
      • ENV variables:
        • CM_DATASET_DTYPE: int8

  • Group "dataset-type"

    Click here to expand this section.

    • _calibration
      • ENV variables:
        • CM_DATASET_PATH: <<<CM_CALIBRATION_DATASET_PATH>>>
    • _validation (default)
Default variations

_fp32,_validation

Script flags mapped to environment

  • --dir=value → CM_DATASET_PREPROCESSED_PATH=value
  • --threads=value → CM_NUM_PREPROCESS_THREADS=value

Default environment

These keys can be updated via --env.KEY=VALUE or env dictionary in @input.json or using script flags.

  • CM_DATASET: kits19
  • CM_DATASET_DTYPE: fp32

Native script being run

No run file exists for Windows

Script output

cmr "get dataset speech-recognition librispeech preprocessed [variations]" [--input_flags] -j