torch_to_nnef.tensor.quant.qtract

Classes:

Name	Description
QTensorTract	All QTensorTract implementations.
QTensorTractScaleOnly	Tract data format it serializes to: Q4_0.

Functions:

Name	Description
fp_to_tract_q4_0_with_min_max_calibration	Min-Max method to quantize float tensor to tract supported Q4_0.

QTensorTract

QTensorTract(fp_tensor: Tensor, qscheme: QScheme, dequant_to_dtype=torch.float32, u8_compressors: Optional[List[U8Compressor]] = None)

Bases: QTensor

All QTensorTract implementations.

QTensorTractScaleOnly

QTensorTractScaleOnly(*args, specific_machine: Optional[str] = None, **kwargs)

Bases: QTensorTract, SupportsOffloadState

Tract data format it serializes to: Q4_0.

Methods:

Name	Description
decompress	Tract dequantization depends on hardware.

decompress

decompress()

Tract dequantization depends on hardware.

Typically dequantization happen with ops in f16 on ARM and f32 (scale directly casted) on others so we overwrite the function to be consistant with tract.

fp_to_tract_q4_0_with_min_max_calibration

fp_to_tract_q4_0_with_min_max_calibration(fp_tensor, percentile: float = 1.0) -> QTensorTractScaleOnly

Min-Max method to quantize float tensor to tract supported Q4_0.