Skip to content

run-mlperf-inference-submission-checker

Automatically generated README for this automation recipe: run-mlperf-inference-submission-checker

Category: MLPerf benchmark support

License: Apache 2.0

  • Notes from the authors, contributors and users: README-extra

  • CM meta description for this script: _cm.json

  • Output cached? False

Reuse this script in your project

Install MLCommons CM automation meta-framework

Pull CM repository with this automation recipe (CM script)

cm pull repo mlcommons@cm4mlops

cmr "run mlc mlcommons mlperf inference mlperf-inference submission checker submission-checker mlc-submission-checker" --help

Run this script

Run this script via CLI
cm run script --tags=run,mlc,mlcommons,mlperf,inference,mlperf-inference,submission,checker,submission-checker,mlc-submission-checker[,variations] [--input_flags]
Run this script via CLI (alternative)
cmr "run mlc mlcommons mlperf inference mlperf-inference submission checker submission-checker mlc-submission-checker [variations]" [--input_flags]
Run this script from Python
import cmind

r = cmind.access({'action':'run'
              'automation':'script',
              'tags':'run,mlc,mlcommons,mlperf,inference,mlperf-inference,submission,checker,submission-checker,mlc-submission-checker'
              'out':'con',
              ...
              (other input keys for this script)
              ...
             })

if r['return']>0:
    print (r['error'])
Run this script via Docker (beta)
cm docker script "run mlc mlcommons mlperf inference mlperf-inference submission checker submission-checker mlc-submission-checker[variations]" [--input_flags]

Variations

  • No group (any combination of variations can be selected)

    Click here to expand this section.

    • _short-run
      • ENV variables:
        • CM_MLPERF_SHORT_RUN: yes

Script flags mapped to environment

  • --extra_args=value → CM_MLPERF_SUBMISSION_CHECKER_EXTRA_ARGS=value
  • --extra_model_benchmark_map=value → CM_MLPERF_EXTRA_MODEL_MAPPING=value
  • --input=value → CM_MLPERF_INFERENCE_SUBMISSION_DIR=value
  • --power=value → CM_MLPERF_POWER=value
  • --push_to_github=value → CM_MLPERF_RESULT_PUSH_TO_GITHUB=value
  • --skip_compliance=value → CM_MLPERF_SKIP_COMPLIANCE=value
  • --skip_power_check=value → CM_MLPERF_SKIP_POWER_CHECK=value
  • --src_version=value → CM_MLPERF_SUBMISSION_CHECKER_VERSION=value
  • --submission_dir=value → CM_MLPERF_INFERENCE_SUBMISSION_DIR=value
  • --submitter=value → CM_MLPERF_SUBMITTER=value
  • --tar=value → CM_TAR_SUBMISSION_DIR=value

Default environment

These keys can be updated via --env.KEY=VALUE or env dictionary in @input.json or using script flags.

  • CM_MLPERF_SHORT_RUN: no

Versions

Default version: master

  • master
  • r3.0
  • r3.1
  • r4.0

Native script being run

Script output

cmr "run mlc mlcommons mlperf inference mlperf-inference submission checker submission-checker mlc-submission-checker [variations]" [--input_flags] -j