Skip to content
torch_to_nnef

Welcome to the documentation

Goals & Scope

torch_to_nnef Python package is used to export any model formulated with vanilla PyTorch, whatever the internal tensor types (including quantized models), into NNEF format.

tract, the neural network inference engine developed openly by SONOS, is the primary supported target, and best compatibility with it is ensured. To use it, the TractNNEF inference_target must be specified. This allows extended NNEF operators and specificities to be expressed:

  • Transformer blocks
  • recurrent layers (LSTM, GRU, …)
  • dynamic streamable input dimensions
  • data type casting (since NNEF spec is too vague in this regard)

Minimal dependencies are kept in this package (to allow easy integration in other projects).

Support

PyTorch >= 1.10.0 with the last 2 major releases of tract (>= 0.20.22 to date) over Linux and MacOS systems is officially supported, and the package is maintained/tested for Python versions that are not end of life, nor pre-release. Only pre-compiled PyTorch wheels and dependencies available on pypi are used in CI, so this support evolves over time. Latest package versions ensure better opset coverage and unlock all features.

Install

Today, the project is packaged in PyPi. Installation can be performed depending on the package manager:

pip install torch_to_nnef

uv add torch_to_nnef

poetry add torch_to_nnef

Note

The project scope is broad and contributions are welcome, if any bug is encountered, the Bug report instructions should be followed.