qtract

torch_to_nnef.tensor.quant.qtract

QTensorTract

QTensorTract(fp_tensor: torch.Tensor, qscheme: QScheme, dequant_to_dtype=torch.float32, u8_compressors: T.Optional[T.List[U8Compressor]] = None)

Bases: QTensor

All QTensorTract implementations.

QTensorTractScaleOnly

QTensorTractScaleOnly(*args, specific_machine: T.Optional[str] = None, **kwargs)

Bases: QTensorTract

Tract data format it serializes to: Q4_0.

decompress

decompress()

Tract dequantization depends on hardware.

Typically dequantization happen with ops in f16 on ARM and f32 (scale directly casted) on others so we overwrite the function to be consistant with tract.

fp_to_tract_q4_0_with_min_max_calibration

fp_to_tract_q4_0_with_min_max_calibration(fp_tensor, percentile: float = 1.0) -> QTensorTractScaleOnly

Min-Max method to quantize float tensor to tract supported Q4_0.