Inference model

class tract.inference_model.InferenceModel(ptr)

ONNX models are loaded as InferenceModel instances instead of Model instances: many ONNX models come with partial shape and element type information, while tract’s Model assumes full shape and element type knowledge. In this case, it is generally sufficient to inform tract about the input shape and type, then let tract infer the rest of the missing shape information before converting the InferenceModel to a regular Model.

# load the model as an InferenceModel
model = tract.onnx().load("./mobilenetv2-7.onnx")

# set the shape and type of its first and only input
model.set_input_fact(0, "1,3,224,224,f32")

# get ready to run the model
model = model.into_model().into_runnable()

into_model() → Model

Convert an InferenceModel to a regular typed Model.

This will leave the opportunity to run more transformation on the intermediary form of the model, before optimising it all the way.

input_count() → int

Return the number of inputs of the model

output_count() → int

Return the number of outputs of the model

input_name(input_id: int) → str

Return the name of the input_id-th input.

input_fact(input_id: int) → InferenceFact

Extract the InferenceFact of the input_id-th input.

set_input_fact(input_id: int, fact: InferenceFact | str | None) → None

Change the InferenceFact of the input_id-th input.

set_output_names(names: List[str])

Change the output nodes of the model

output_name(output_id: int) → str

Return the name of the output_id-th output.

output_fact(output_id: int) → InferenceFact

Extract the InferenceFact of the output_id-th output.

set_output_fact(output_id: int, fact: InferenceFact | str | None) → None

Change the InferenceFact of the output_id-th output.

fact(spec: str) → InferenceFact

Parse a fact specification as an InferenceFact.

Typical InferenceFact specification is in the form “1,224,224,3,f32”. Comma-separated list of dimension, one for each axis, plus an mnemonic for the element type. f32 is single precision “float”, i16 is a 16-bit signed integer, and u8 a 8-bit unsigned integer.

analyse() → None

Perform shape and element type inference on the model.

into_analysed() → InferenceModel

Perform shape and element type inference on the model.