How to run post-training quantization for TFLite

Edge deployment often starts after a model has already been trained and exported, when the remaining constraint is the runtime artifact rather than the training loop. Post-training quantization converts that finished TensorFlow model into a smaller integer TFLite flatbuffer that LiteRT and TensorFlow Lite runtimes can load on mobile, embedded, or accelerator-backed targets.

Full-int8 quantization uses tf.lite.TFLiteConverter with tf.lite.Optimize.DEFAULT and a representative dataset to calibrate activation ranges before the flatbuffer is written. Restricting supported operations to tf.lite.OpsSet.TFLITE_BUILTINS_INT8 makes the conversion fail when an operation cannot stay integer-only, which is safer than discovering a float fallback inside a deployment runtime that expects int8 tensors.

Start from a SavedModel whose serving signature, input feature order, and preprocessing scale are already accepted. The support-score sample uses four numeric features so the conversion path is readable, but production calibration rows should come from normal training, validation, or logged inference data rather than invented values.

Steps to run post-training quantization with TFLiteConverter:

Open a terminal in the Python environment that can import TensorFlow and read the exported SavedModel directory.
```
$ python3 -c "import tensorflow as tf; print(tf.__version__)"
2.21.0
```
The sample output was verified with TensorFlow 2.21.0 in the campaign CPU image. Use the same environment that exported or smoke-tested the SavedModel so converter behavior matches the model package set.
Related: How to create a virtual environment for TensorFlow
Related: How to install TensorFlow with pip

Inspect the default serving signature before choosing representative rows.

$ saved_model_cli show --dir support_score_savedmodel --tag_set serve --signature_def serving_default
The given SavedModel SignatureDef contains the following input(s):
  inputs['keras_tensor'] tensor_info:
      dtype: DT_FLOAT
      shape: (-1, 4)
      name: serving_default_keras_tensor:0
The given SavedModel SignatureDef contains the following output(s):
  outputs['output_0'] tensor_info:
      dtype: DT_FLOAT
      shape: (-1, 1)
      name: StatefulPartitionedCall_1:0
Method name is: tensorflow/serving/predict

The representative rows and smoke-test sample must match this input shape, dtype, feature order, and preprocessing scale.

Save representative calibration rows beside the SavedModel directory.
support_score_representative.csv
```
feature_a,feature_b,feature_c,feature_d
0.10,0.70,0.20,0.30
0.18,0.62,0.25,0.35
0.24,0.58,0.32,0.40
0.36,0.46,0.42,0.45
0.64,0.30,0.68,0.72
0.72,0.24,0.76,0.82
0.82,0.16,0.84,0.90
0.92,0.08,0.90,0.96
```
The short CSV keeps the demonstration compact. For a production model, use enough typical rows to cover the normal input range without adding outliers that the model should not accept.

Do not reorder, rename, or rescale feature columns unless the exported model was trained with that exact input contract.

Save the quantization workflow as quantize_support_score.py.

quantize_support_score.py

import csv
import os
import pathlib
import warnings
 
os.environ["TF_CPP_MIN_LOG_LEVEL"] = "3"
warnings.filterwarnings("ignore", message=".*tf.lite.Interpreter is deprecated.*")
 
import numpy as np
import tensorflow as tf
 
tf.get_logger().setLevel("ERROR")
 
SAVEDMODEL_PATH = pathlib.Path("support_score_savedmodel")
REPRESENTATIVE_PATH = pathlib.Path("support_score_representative.csv")
FLOAT_TFLITE_PATH = pathlib.Path("support_score_float.tflite")
INT8_TFLITE_PATH = pathlib.Path("support_score_int8.tflite")
SAMPLE = np.array([[0.72, 0.24, 0.76, 0.82]], dtype=np.float32)
 
 
def load_representative_rows():
    rows = []
 
    with REPRESENTATIVE_PATH.open("r", newline="", encoding="utf-8") as handle:
        reader = csv.DictReader(handle)
        for row in reader:
            rows.append(
                [
                    float(row["feature_a"]),
                    float(row["feature_b"]),
                    float(row["feature_c"]),
                    float(row["feature_d"]),
                ]
            )
 
    return np.asarray(rows, dtype=np.float32)
 
 
representative_rows = load_representative_rows()
 
float_converter = tf.lite.TFLiteConverter.from_saved_model(str(SAVEDMODEL_PATH))
FLOAT_TFLITE_PATH.write_bytes(float_converter.convert())
 
 
def representative_dataset():
    for row in representative_rows:
        yield [row.reshape(1, 4)]
 
 
int8_converter = tf.lite.TFLiteConverter.from_saved_model(str(SAVEDMODEL_PATH))
int8_converter.optimizations = [tf.lite.Optimize.DEFAULT]
int8_converter.representative_dataset = representative_dataset
int8_converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
int8_converter.inference_input_type = tf.int8
int8_converter.inference_output_type = tf.int8
INT8_TFLITE_PATH.write_bytes(int8_converter.convert())
 
interpreter = tf.lite.Interpreter(model_path=str(INT8_TFLITE_PATH))
interpreter.allocate_tensors()
 
input_details = interpreter.get_input_details()[0]
output_details = interpreter.get_output_details()[0]
input_scale, input_zero_point = input_details["quantization"]
output_scale, output_zero_point = output_details["quantization"]
 
quantized_sample = np.round(SAMPLE / input_scale + input_zero_point).astype(np.int8)
interpreter.set_tensor(input_details["index"], quantized_sample)
interpreter.invoke()
raw_output = interpreter.get_tensor(output_details["index"])
int8_score = (raw_output.astype(np.float32) - output_zero_point) * output_scale
 
float_interpreter = tf.lite.Interpreter(model_path=str(FLOAT_TFLITE_PATH))
float_interpreter.allocate_tensors()
float_input = float_interpreter.get_input_details()[0]
float_output = float_interpreter.get_output_details()[0]
float_interpreter.set_tensor(float_input["index"], SAMPLE)
float_interpreter.invoke()
float_score = float_interpreter.get_tensor(float_output["index"])
 
print(f"TensorFlow {tf.__version__}")
print(f"saved_model_dir={SAVEDMODEL_PATH.name}")
print(f"representative_rows={len(representative_rows)}")
print(f"float_model_file={FLOAT_TFLITE_PATH.name}")
print(f"int8_model_file={INT8_TFLITE_PATH.name}")
print(f"float_model_bytes={FLOAT_TFLITE_PATH.stat().st_size}")
print(f"int8_model_bytes={INT8_TFLITE_PATH.stat().st_size}")
print(f"input_dtype={np.dtype(input_details['dtype']).name}")
print(f"output_dtype={np.dtype(output_details['dtype']).name}")
print(f"input_quantization=scale:{input_scale:.8f},zero_point:{input_zero_point}")
print(f"output_quantization=scale:{output_scale:.8f},zero_point:{output_zero_point}")
print(f"float_sample_output={float_score[0][0]:.6f}")
print(f"int8_sample_output={int8_score[0][0]:.6f}")
print(f"absolute_difference={abs(float_score[0][0] - int8_score[0][0]):.6f}")

The script writes a float flatbuffer first so the same sample row can be compared against the int8 output after conversion.

Run the quantization script.

$ python3 quantize_support_score.py
##### snipped #####
TensorFlow 2.21.0
saved_model_dir=support_score_savedmodel
representative_rows=8
float_model_file=support_score_float.tflite
int8_model_file=support_score_int8.tflite
float_model_bytes=5024
int8_model_bytes=4552
input_dtype=int8
output_dtype=int8
input_quantization=scale:0.00376471,zero_point:-128
output_quantization=scale:0.00390625,zero_point:-128
float_sample_output=0.563334
int8_sample_output=0.562500
absolute_difference=0.000834

The int8_model_file line confirms the artifact was written, the input_dtype and output_dtype lines confirm the requested integer tensor contract, and the sample comparison confirms the converted model can run through the TFLite interpreter.

Check the int8 flatbuffer before handing it to the deployment build.
```
$ ls -lh support_score_int8.tflite
-rw-r--r-- 1 user user 4.5K Jun 29 04:45 support_score_int8.tflite
```
A successful conversion does not guarantee acceptable model quality. Evaluate the int8 flatbuffer against the same acceptance data used for the float model before shipping it, refresh the representative dataset when accuracy drops, and move to quantization-aware training only when post-training quantization no longer meets the deployment target.