Seldon Core 2 Model Zoo 实战：七类框架模型工件的部署、推理与源码级解读

• 人工智能
• 大模型
• MLOps
• LLMOps
• 模型推理服务
• 云原生
• 后端

本文以 Seldon Core 2 仓库中的 Model Zoo 示例文档为主体，完整覆盖 SKLearn、TensorFlow、XGBoost、ONNX、LightGBM、MLFlow、PyTorch 七类模型工件从ModelCRD 定义、部署加载、推理调用到卸载删除的全流程；并结合samples/下的训练脚本、operator/apis/mlops/v1alpha1/的 CRD 类型定义与scheduler/的调度过滤逻辑，讲解storageUri、requirements、memory等字段的底层含义，帮助你把任意框架训练的模型工件接入 Seldon Core 2 数据平面。

1. Model Zoo 是什么：七类模型工件一览

Model Zoo（docs-gb/examples/model-zoo.md）收录了在不同机器学习框架下训练、并以不同工件格式运行的模型示例，对应 Seldon Core 2 中两种推理服务器：MLServer（sklearn、tensorflow、xgboost、lightgbm、mlflow）与Triton（onnx、pytorch、tensorrt 等 C++ 系后端）。文档列出的完整模型清单为：

• SKLearn
• Tensorflow
• XGBoost
• ONNX
• Lightgbm
• MLFlow
• PyTorch

每个示例遵循同一套模式：

1. 模型工件（joblib / savedmodel / .onnx / .pt / mlflow run artifacts）被上传到对象存储（示例使用gs://seldon-models/scv2/samples/...）；
2. 编写一个kind: Model的 CRD YAML，声明storageUri（工件位置）和requirements（运行该工件所需的 Python 依赖）；
3. 用kubectl apply或seldon model load提交，Scheduler 根据requirements把模型调度到能加载它的服务器上；
4. 通过 Model Gateway 的POST /v2/models/{name}/infer（KServe V2 协议）发起推理；
5. 用kubectl delete或seldon model unload卸载。

示例客户端所需的 Python 依赖在 samples/model-zoo-requirements.txt 中列出：

alibi_detect matplotlib numpy requests tensorflow torch torchvision

所有示例 YAML 位于 samples/models/，训练脚本位于 samples/scripts/models/。

2. Model CRD 字段解剖：storageUri、requirements 与 memory

以 SKLearn Iris 模型为例，samples/models/sklearn-iris-gs.yaml 的完整内容：

apiVersion: mlops.seldon.io/v1alpha1 kind: Model metadata: name: iris spec: storageUri: "gs://seldon-models/scv2/samples/mlserver_1.3.5/iris-sklearn" requirements: - sklearn memory: 100Ki

这三个字段在 operator/apis/mlops/v1alpha1/model_types.go 中有明确的类型定义：

从源码结构看，Model资源会被转换为 Scheduler 的 proto 消息：AsSchedulerModel()（model_types.go）将StorageURI映射为ModelSpec.Uri、Requirements原样透传、Memory转换为字节数写入MemoryBytes。而"requirements 决定调度到哪类服务器"这一关键逻辑，由 scheduler/pkg/scheduler/filters/serverrequirements.go 中的服务器需求过滤器实现——服务器（如 mlserver、triton）在注册时声明自己支持的能力，过滤器据此把requirements: [sklearn]的模型只调度到 MLServer 实例，把requirements: [onnx]的模型只调度到 Triton 实例。

版本说明：仓库当前示例 YAML 中的storageUri已随 MLServer 版本升级为mlserver_1.6.0（如 samples/models/sklearn-iris-gs.yaml 中为gs://seldon-models/scv2/samples/mlserver_1.6.0/iris-sklearn），而原文档快照为mlserver_1.3.5。实际使用时应以你环境安装的服务器版本为准，二者不可混用。

2.1 模型训练与工件上传：Makefile 全链路

所有模型的训练和上传都在 samples/scripts/models/Makefile 中编排，其中 MLServer 版本从scheduler/Makefile动态读取：

VERSION := $(shell grep seldonio/mlserver ../../../scheduler/Makefile | cut -d':' -f2) MLSERVER_VERSION ?= ${VERSION} MLSERVER_FOLDER = mlserver_${MLSERVER_VERSION} TRITON_VERSION=$(shell grep 'nvidia/tritonserver' ../../../scheduler/Makefile | cut -d':' -f2 | cut -d'-' -f1 | tr '.' '-') TRITON_FOLDER=triton_${TRITON_VERSION}

关键 target 与对应工件：

其中mnist-pytorchtarget 有一个值得注意的细节：由于 Triton 无法为 PyTorch 工件自动生成配置（见 Makefile 中注释，引用了 PyTorch issue #38273），脚本会先sed改写官方main.py，把torch.save(model.state_dict(), ...)替换为torch.jit.script(model)导出 TorchScript，并同时上传手写的config.pbtxt。

SKLearn Iris 的训练代码 samples/scripts/models/iris/train.py 非常简短——LogisticRegression装入Pipeline后joblib.dump为model.joblib：

clf = LogisticRegression(solver="liblinear", multi_class='ovr') p = Pipeline([("clf", clf)]) p.fit(X, y) joblib.dump(p, "model.joblib")

3. SKLearn Iris 分类模型

部署（kubectl 方式）：

kubectl apply -f ./models/sklearn-iris-gs.yaml -n ${NAMESPACE} # model.mlops.seldon.io/iris created kubectl wait --for condition=ready --timeout=300s model iris -n ${NAMESPACE} # model.mlops.seldon.io/iris condition met

发起 KServe V2 推理请求：

curl --location 'http://${SELDON_INFER_HOST}/v2/models/iris/infer' \ --header 'Content-Type: application/json' \ --data '{"inputs": [{"name": "predict", "shape": [1, 4], "datatype": "FP32", "data": [[1, 2, 3, 4]]}]}'

响应（outputs[0].data为类别索引 2，即 virginica）：

{ "model_name": "iris_1", "model_version": "1", "id": "09263298-ca66-49c5-acb9-0ca75b06f825", "parameters": {}, "outputs": [ { "name": "predict", "shape": [1, 1], "datatype": "INT64", "data": [2] } ] }

seldon-cli 方式等价操作：

seldon model load -f ./models/sklearn-iris-gs.yaml seldon model status iris -w ModelAvailable | jq -M . seldon model infer iris \ '{"inputs": [{"name": "predict", "shape": [1, 4], "datatype": "FP32", "data": [[1, 2, 3, 4]]}]}' seldon model unload iris

seldon model status的-w ModelAvailable参数等待的就 CRD 状态条件，对应 operator/apis/mlops/v1alpha1/model_types.go 中 kubebuilder 打印列声明的ModelReady条件族；CLI 内部将其映射为ModelAvailable等待条件（见 operator/pkg/cli/generic.go）。

清理：

kubectl delete -f ./models/sklearn-iris-gs.yaml -n ${NAMESPACE}

4. TensorFlow CIFAR10 图像分类模型

samples/models/cifar10-no-config.yaml：

apiVersion: mlops.seldon.io/v1alpha1 kind: Model metadata: name: cifar10 spec: storageUri: "gs://seldon-models/scv2/samples/tensorflow/cifar10" requirements: - tensorflow

注意文件名中的 "no-config"：该 SavedModel 未附带额外配置文件，requirements只需tensorflow。部署与等待：

kubectl apply -f ./models/cifar10-no-config.yaml -n ${NAMESPACE} # model.mlops.seldon.io/cifar10 created kubectl wait --for condition=ready --timeout=300s model cifar10 -n ${NAMESPACE} # model.mlops.seldon.io/cifar10 condition met

或 CLI：

seldon model load -f ./models/cifar10-no-config.yaml seldon model status cifar10 -w ModelAvailable | jq -M .

推理客户端（文档中的 Python 片段）：从tf.keras.datasets.cifar10加载 50000×32×32×3 的 float32 归一化训练图，把单张图片 flatten 后放入inputs[0].data，shape设为[1, 32, 32, 3]，并带seldon-model请求头：

import requests import json import numpy as np import tensorflow as tf import matplotlib.pyplot as plt tf.keras.backend.clear_session() train, test = tf.keras.datasets.cifar10.load_data() X_train, y_train = train X_test, y_test = test X_train = X_train.astype('float32') / 255 X_test = X_test.astype('float32') / 255 classes = ("plane", "car", "bird", "cat", "deer", "dog", "frog", "horse", "ship", "truck") reqJson = json.loads('{"inputs":[{"name":"input_1","data":[],"datatype":"FP32","shape":[]}]}') url = "http://0.0.0.0:9000/v2/models/model/infer" def infer(resourceName: str, idx: int): rows = X_train[idx:idx+1] show(rows[0]) reqJson["inputs"][0]["data"] = rows.flatten().tolist() reqJson["inputs"][0]["shape"] = [1, 32, 32, 3] headers = {"Content-Type": "application/json", "seldon-model": resourceName} response_raw = requests.post(url, json=reqJson, headers=headers) probs = np.array(response_raw.json()["outputs"][0]["data"]) print(classes[probs.argmax(axis=0)]) def show(X): plt.imshow(X.reshape(32, 32, 3)) plt.axis("off") plt.show() infer("cifar10", 4) # 输出一张训练集图片并打印分类结果（示例中为 "car"）

卸载：

kubectl delete -f ./models/cifar10-no-config.yaml -n ${NAMESPACE} # 或 seldon model unload cifar10

5. XGBoost 收入预测模型

训练代码位于 samples/scripts/models/income-xgb。samples/models/income-xgb.yaml：

apiVersion: mlops.seldon.io/v1alpha1 kind: Model metadata: name: income-xgb spec: storageUri: "gs://seldon-models/scv2/samples/mlserver_1.3.5/income-xgb" requirements: - xgboost

部署：

kubectl apply -f ./models/income-xgb.yaml -n ${NAMESPACE} # model.mlops.seldon.io/income-xgb is created kubectl wait --for condition=ready --timeout=300s model income-xgb -n ${NAMESPACE} # model.mlops.seldon.io/income-xgb condition met

推理请求的关键点是parameters.content_type = "pd"（pandas DataFrame 格式），每个输入字段（Age、Workclass、Education……Country）作为独立inputs元素，全部为 INT64：

curl --location 'http://${SELDON_INFER_HOST}/v2/models/income-xgb/infer' \ --header 'Content-Type: application/json' \ --data '{ "parameters": {"content_type": "pd"}, "inputs": [{"name": "Age", "shape": [1, 1], "datatype": "INT64", "data": [47]},{"name": "Workclass", "shape": [1, 1], "datatype": "INT64", "data": [4]},{"name": "Education", "shape": [1, 1], "datatype": "INT64", "data": [1]},{"name": "Marital Status", "shape": [1, 1], "datatype": "INT64", "data": [1]},{"name": "Occupation", "shape": [1, 1], "datatype": "INT64", "data": [1]},{"name": "Relationship", "shape": [1, 1], "datatype": "INT64", "data": [3]},{"name": "Race", "shape": [1, 1], "datatype": "INT64", "data": [4]},{"name": "Sex", "shape": [1, 1], "datatype": "INT64", "data": [1]},{"name": "Capital Gain", "shape": [1, 1], "datatype": "INT64", "data": [0]},{"name": "Capital Loss", "shape": [1, 1], "datatype": "INT64", "data": [0]},{"name": "Hours per week", "shape": [1, 1], "datatype": "INT64", "data": [40]},{"name": "Country", "shape": [1, 1], "datatype": "INT64", "data": [9]}]}'

CLI 等价流程：

seldon model load -f ./models/income-xgb.yaml seldon model status income-xgb -w ModelAvailable | jq -M . seldon model infer income-xgb \ '{ "parameters": {"content_type": "pd"}, "inputs": [ ... 同上 ... ]}' seldon model unload income-xgb # kubectl 方式清理：kubectl delete -f ./models/income-xgb.yaml -n ${NAMESPACE}

响应输出单值predict（文档快照中 kubectl 路径返回 FP64 概率0.0628...，CLI 路径返回 FP32 分数-1.838...，取决于模型版本与是否走概率/分数输出）。

6. ONNX MNIST 模型

ONNX 模型工件由 Makefile 的download-mnist-onnxtarget 直接下载（ONNX 官方验证模型mnist-12.onnx），samples/models/mnist-onnx.yaml：

apiVersion: mlops.seldon.io/v1alpha1 kind: Model metadata: name: mnist-onnx spec: storageUri: "gs://seldon-models/scv2/samples/triton_23-03/mnist-onnx" requirements: - onnx

注意storageUri前缀为triton_23-03而非mlserver_*——这正是该工件运行在 Triton 服务器上的体现。部署：

kubectl apply -f ./models/mnist-onnx.yaml -n ${NAMESPACE} # model.mlops.seldon.io/mnist-onnx created kubectl wait --for condition=ready --timeout=300s model mnist-onnx -n ${NAMESPACE} # model.mlops.seldon.io/mnist-onnx condition met # 或：seldon model load -f ./models/mnist-onnx.yaml && seldon model status mnist-onnx -w ModelAvailable

推理客户端（与 PyTorch 版本几乎一致，差别在输入张量名Input3与请求头seldon-model: mnist-onnx）：

import matplotlib.pyplot as plt import json import requests from torchvision.datasets import MNIST from torchvision import transforms from torch.utils.data import DataLoader import numpy as np training_data = MNIST(root=".", download=True, train=False, transform=transforms.Compose([transforms.ToTensor()])) reqJson = json.loads('{"inputs":[{"name":"Input3","data":[],"datatype":"FP32","shape":[]}]}') url = "http://0.0.0.0:9000/v2/models/model/infer" dl = DataLoader(training_data, batch_size=1, shuffle=False) dlIter = iter(dl) def infer_mnist(): x, y = next(dlIter) data = x.cpu().numpy() reqJson["inputs"][0]["data"] = data.flatten().tolist() reqJson["inputs"][0]["shape"] = [1, 1, 28, 28] headers = {"Content-Type": "application/json", "seldon-model": "mnist-onnx"} response_raw = requests.post(url, json=reqJson, headers=headers) show_mnist(x) probs = np.array(response_raw.json()["outputs"][0]["data"]) print(probs.argmax(axis=0)) def show_mnist(X): plt.imshow(X.reshape(28, 28)) plt.axis("off") plt.show()

文档示例中该请求对第一张测试集图片输出预测类别7。卸载：kubectl delete -f ./models/mnist-onnx.yaml -n ${NAMESPACE}或seldon model unload mnist-onnx。

7. LightGBM 模型

训练代码位于 samples/scripts/models/income-lgb。samples/models/income-lgb.yaml：

apiVersion: mlops.seldon.io/v1alpha1 kind: Model metadata: name: income-lgb spec: storageUri: "gs://seldon-models/scv2/samples/mlserver_1.3.5/income-lgb" requirements: - lightgbm

部署与推理流程和 XGBoost 完全一致（kubectl apply→kubectl wait→curl /v2/models/income-lgb/infer，content_type同为pd，输入为同样的 12 个 INT64 字段）：

kubectl apply -f ./models/income-lgb.yaml -n ${NAMESPACE} kubectl wait --for condition=ready --timeout=300s model income-lgb -n ${NAMESPACE} seldon model load -f ./models/income-lgb.yaml seldon model status income-lgb -w ModelAvailable | jq -M . seldon model infer income-lgb '{ ... 与 income-xgb 相同的 pd 请求体 ... }' seldon model unload income-lgb # 或 kubectl delete -f ./models/income-lgb.yaml -n ${NAMESPACE}

一个重要的运维提示来自 samples/scripts/models/Makefile：note that income-lgb fails with new version of lightgbm——新版 lightgbm 会使该训练失败，所以 Makefile 的train-all默认不包含income-lgb，工件沿用旧版上传的mlserver_1.3.5路径。

8. MLFlow Wine 回归模型

训练代码位于 samples/scripts/models/wine-mlflow。samples/models/wine-mlflow.yaml：

apiVersion: mlops.seldon.io/v1alpha1 kind: Model metadata: name: wine spec: storageUri: "gs://seldon-models/scv2/samples/mlserver_1.3.5/wine-mlflow" requirements: - mlflow

Makefile 上传的是最近一次 MLFlow run 的artifacts/model目录（Makefile）：

MLFLOW_FOLDER=$(shell ls -td ./wine-mlflow/mlruns/0/* | head -1) upload-wine-mlflow: gsutil cp -R ${MLFLOW_FOLDER}/artifacts/model gs://seldon-models/scv2/samples/${MLSERVER_FOLDER}/wine-mlflow

部署：

kubectl apply -f ./models/wine-mlflow.yaml -n ${NAMESPACE} # model.mlops.seldon.io/wine created kubectl wait --for condition=ready --timeout=300s model wine -n ${NAMESPACE} # 或 seldon model load -f ./models/wine-mlflow.yaml && seldon model status wine -w ModelAvailable

推理请求（11 个 FP32 单值输入，seldon-model: wine头）：

import requests url = "http://0.0.0.0:9000/v2/models/model/infer" inference_request = { "inputs": [ {"name": "fixed acidity", "shape": [1], "datatype": "FP32", "data": [7.4]}, {"name": "volatile acidity", "shape": [1], "datatype": "FP32", "data": [0.7000]}, {"name": "citric acid", "shape": [1], "datatype": "FP32", "data": [0]}, {"name": "residual sugar", "shape": [1], "datatype": "FP32", "data": [1.9]}, {"name": "chlorides", "shape": [1], "datatype": "FP32", "data": [0.076]}, {"name": "free sulfur dioxide", "shape": [1], "datatype": "FP32", "data": [11]}, {"name": "total sulfur dioxide", "shape": [1], "datatype": "FP32", "data": [34]}, {"name": "density", "shape": [1], "datatype": "FP32", "data": [0.9978]}, {"name": "pH", "shape": [1], "datatype": "FP32", "data": [3.51]}, {"name": "sulphates", "shape": [1], "datatype": "FP32", "data": [0.56]}, {"name": "alcohol", "shape": [1], "datatype": "FP32", "data": [9.4]}, ] } headers = {"Content-Type": "application/json", "seldon-model": "wine"} response_raw = requests.post(url, json=inference_request, headers=headers) print(response_raw.json())

响应为单值回归输出：

{'model_name': 'wine_1', 'model_version': '1', 'id': '0d7e44f8-b46c-4438-b8af-a749e6aa6039', 'parameters': {}, 'outputs': [{'name': 'output-1', 'shape': [1, 1], 'datatype': 'FP64', 'data': [5.576883936610762]}]}

卸载：kubectl delete model wine -n ${NAMESPACE}或seldon model unload wine。

9. PyTorch MNIST 模型

samples/models/mnist-pytorch.yaml：

apiVersion: mlops.seldon.io/v1alpha1 kind: Model metadata: name: mnist-pytorch spec: storageUri: "gs://seldon-models/scv2/samples/triton_23-03/mnist-pytorch" requirements: - pytorch

训练工件由 Makefile targetmnist-pytorch生成：拉取 pytorch 官方 mnist 示例，sed改写为 TorchScript 导出（torch.jit.script(model)→model.pt），并额外上传 Triton 所需的config.pbtxt（因为 Triton 不能为 PyTorch 工件自动生成配置）。

部署与等待：

kubectl apply -f ./models/mnist-pytorch.yaml -n ${NAMESPACE} # model.mlops.seldon.io/mnist-pytorch created kubectl wait --for condition=ready --timeout=300s model mnist-pytorch -n ${NAMESPACE} # model.mlops.seldon.io/mnist-pytorch condition met # 或 seldon model load -f ./models/mnist-pytorch.yaml

推理客户端（输入张量名为x__0，请求头seldon-model: mnist-pytorch）：

reqJson = json.loads('{"inputs":[{"name":"x__0","data":[],"datatype":"FP32","shape":[]}]}') url = "http://0.0.0.0:9000/v2/models/model/infer" dl = DataLoader(training_data, batch_size=1, shuffle=False) dlIter = iter(dl) def infer_mnist(): x, y = next(dlIter) data = x.cpu().numpy() reqJson["inputs"][0]["data"] = data.flatten().tolist() reqJson["inputs"][0]["shape"] = [1, 1, 28, 28] headers = {"Content-Type": "application/json", "seldon-model": "mnist-pytorch"} response_raw = requests.post(url, json=reqJson, headers=headers) show_mnist(x) probs = np.array(response_raw.json()["outputs"][0]["data"]) print(probs.argmax(axis=0)) # 示例输出 7

卸载：

kubectl delete -f ./models/mnist-pytorch.yaml -n ${NAMESPACE} # model.mlops.seldon.io "mnist-pytorch" deleted # 或 seldon model unload mnist-pytorch

10. 总结与延伸阅读

Model Zoo 示例浓缩了 Seldon Core 2 加载模型的三个核心概念，均可在仓库源码中找到直接证据：

1. 工件与位置解耦：spec.storageUri（InferenceArtifactSpec）指向对象存储中的任意工件格式，MLServer 与 Triton 各自支持的后端由requirements声明；
2. 按需求调度：AsSchedulerModel()把 CRD 转换为 scheduler proto，[scheduler/pkg/scheduler/filters/serverrequirements.go](https://link.gitcode.com/i/9cf26e0a5ac2d8888aa8bbfca7631671)过滤器保证模型只落到能力匹配的服务器上；
3. 统一的 KServe V2 入口：无论底层框架如何，推理请求都是POST /v2/models/{name}/infer，DataFrame 类模型通过parameters.content_type = "pd"指定输入形态。

仓库内相关路径速查：示例配置 samples/models/、训练与上传脚本 samples/scripts/models/Makefile、CRD 类型定义 operator/apis/mlops/v1alpha1/model_types.go、CLI 实现 operator/pkg/cli/、客户端依赖 samples/model-zoo-requirements.txt。

• 人工智能
• 大模型
• MLOps
• LLMOps
• 模型推理服务
• 云原生
• 后端