Skip to content

compile-model-for.qaic

Automatically generated README for this automation recipe: compile-model-for.qaic

Category: AI/ML optimization

License: Apache 2.0

  • CM meta description for this script: _cm.json
  • Output cached? True

Reuse this script in your project

Install MLCommons CM automation meta-framework

Pull CM repository with this automation recipe (CM script)

cm pull repo mlcommons@cm4mlops

cmr "qaic compile model model-compile qaic-compile" --help

Run this script

Run this script via CLI
cm run script --tags=qaic,compile,model,model-compile,qaic-compile[,variations] [--input_flags]
Run this script via CLI (alternative)
cmr "qaic compile model model-compile qaic-compile [variations]" [--input_flags]
Run this script from Python
import cmind

r = cmind.access({'action':'run'
              'automation':'script',
              'tags':'qaic,compile,model,model-compile,qaic-compile'
              'out':'con',
              ...
              (other input keys for this script)
              ...
             })

if r['return']>0:
    print (r['error'])
Run this script via Docker (beta)
cm docker script "qaic compile model model-compile qaic-compile[variations]" [--input_flags]

Variations

  • No group (any combination of variations can be selected)

    Click here to expand this section.

    • _bert-99
      • ENV variables:
        • CM_COMPILE_BERT: on
        • CM_QAIC_MODEL_TO_CONVERT: calibrate_bert_mlperf
        • CM_QAIC_MODEL_COMPILER_PARAMS_BASE: -aic-hw -aic-hw-version=2.0 -execute-nodes-in-fp16=Add,Div,Erf,Softmax -quantization-schema=symmetric_with_uint8 -quantization-precision=Int8 -quantization-precision-bias=Int32 -vvv -compile-only -onnx-define-symbol=batch_size,1 -onnx-define-symbol=seg_length,384 -multicast-weights -combine-inputs=false -combine-outputs=false
        • CM_QAIC_MODEL_COMPILER_ARGS: ``
    • _bert-99.9
      • ENV variables:
        • CM_COMPILE_BERT: on
        • CM_QAIC_MODEL_TO_CONVERT: bert_mlperf
        • CM_QAIC_MODEL_COMPILER_PARAMS_BASE: -aic-hw -aic-hw-version=2.0 -convert-to-fp16 -vvv -compile-only -onnx-define-symbol=batch_size,1 -onnx-define-symbol=seg_length,384 -combine-inputs=false -combine-outputs=false
        • CM_QAIC_MODEL_COMPILER_ARGS: ``
    • _resnet50
      • ENV variables:
        • CM_COMPILE_RESNET: on
        • CM_QAIC_MODEL_TO_CONVERT: compile_resnet50_tf
        • CM_QAIC_MODEL_COMPILER_PARAMS_BASE: -aic-hw -aic-hw-version=2.0 -quantization-schema=symmetric_with_uint8 -quantization-precision=Int8 -output-node-name=ArgMax -vvv -compile-only -use-producer-dma=1
    • _retinanet
      • ENV variables:
        • CM_COMPILE_RETINANET: on
        • CM_QAIC_MODEL_TO_CONVERT: calibrate_retinanet_no_nms_mlperf
        • CM_QAIC_MODEL_COMPILER_ARGS: -aic-enable-depth-first
        • CM_QAIC_MODEL_COMPILER_PARAMS_BASE: -aic-hw -aic-hw-version=2.0 -compile-only -enable-channelwise -onnx-define-symbol=batch_size,1 -node-precision-info=<<<CM_ML_MODEL_RETINANET_QAIC_NODE_PRECISION_INFO_FILE_PATH>>> -quantization-schema-constants=symmetric_with_uint8 -quantization-schema-activations=asymmetric -quantization-calibration=None

  • Group "batch-size"

    Click here to expand this section.

    • _bs.#
      • ENV variables:
        • CM_QAIC_MODEL_BATCH_SIZE: #
    • _bs.1
      • ENV variables:
        • CM_QAIC_MODEL_BATCH_SIZE: 1

  • Group "calib-dataset-filter-size"

    Click here to expand this section.

    • _filter-size.#

  • Group "mlperf-scenario"

    Click here to expand this section.

    • _multistream
    • _offline
    • _server
    • _singlestream (default)

  • Group "model-framework"

    Click here to expand this section.

    • _tf

  • Group "nsp"

    Click here to expand this section.

    • _nsp.14
    • _nsp.16
    • _nsp.8
    • _nsp.9

  • Group "percentile-calibration"

    Click here to expand this section.

    • _pc.#
      • ENV variables:
        • CM_QAIC_MODEL_COMPILER_PERCENTILE_CALIBRATION_VALUE: #
        • CM_QAIC_MODEL_COMPILER_QUANTIZATION_PARAMS: -quantization-calibration=Percentile -percentile-calibration-value=<<<CM_QAIC_MODEL_COMPILER_PERCENTILE_CALIBRATION_VALUE>>>

  • Group "quantization"

    Click here to expand this section.

    • _no-quantized
      • ENV variables:
        • CM_QAIC_MODEL_QUANTIZATION: no
    • _quantized (default)
      • ENV variables:
        • CM_QAIC_MODEL_QUANTIZATION: yes
Default variations

_quantized,_singlestream

Script flags mapped to environment

  • --register=value → CM_REGISTER_CACHE=value

Native script being run

No run file exists for Windows

Script output

cmr "qaic compile model model-compile qaic-compile [variations]" [--input_flags] -j