Unstract Adapter JSON Schema 指南:用静态 JSON Schema 生成 Adapter 配置 UI
【免费下载链接】unstractLLM-Driven Extraction of Unstructured Data — Built for API Deployments & ETL Pipeline Workflows项目地址: https://gitcode.com/GitHub_Trending/un/unstract
本文以 Unstract 仓库中的 Adapter JSON Schema 参考文档为核心,系统讲解如何用一份 JSON Schema 文件自动生成 LLM/Embedding Adapter 的配置表单:包括 Schema 顶层结构、8 种字段类型的写法、adapter_name必填约定、allOf+if/then条件字段,以及 9 个可直接复用的完整示例。读完之后,你可以直接为unstract/sdk1新增或修改一个 Provider Adapter,并让后端平台无需改动前端代码就渲染出对应配置 UI。
Schema 的落地位置:从静态 JSON 文件到配置表单
在动手写 Schema 之前,先明确它在 Unstract 中的完整生命周期。按 adapter-ops 技能说明 中的文件布局约定,每个 Provider Adapter 由两个文件构成:
unstract/sdk1/src/unstract/sdk1/adapters/ ├── base1.py # 参数类(参数类定义) ├── llm1/ # LLM adapters │ ├── {provider}.py # Adapter 实现 │ └── static/{provider}.json # UI schema(本文主角) └── embedding1/ # Embedding adapters ├── {provider}.py └── static/{provider}.json以llm1/static目录为例,仓库中已存在的真实 Schema 文件包括openai.json、anthropic.json、azure.json、bedrock.json、ollama.json、vertex_ai.json、gemini.json、mistral.json、openrouter.json、anyscale.json、minimax.json、nvidia_build.json、azure_ai.json、custom_openai.json等,全部位于 unstract/sdk1/src/unstract/sdk1/adapters/llm1/static/。
Schema 文件的读取入口在 base1.py,get_json_schema()类方法按“适配器类型 + provider”拼出路径:
@classmethod def get_json_schema(cls) -> str: schema_path = ( f"{os.path.dirname(__file__)}/" f"{cls.get_adapter_type().name.lower()}1/static/" f"{cls.get_provider()}.json" ) with open(schema_path) as f: return f.read()从这段源码可以看到两个关键约定:Schema 文件名必须与get_provider()返回值完全一致(大小写敏感的open()),且路径由get_adapter_type()决定(llm1/或embedding1/)。这也是 SKILL.md 验证清单中 "get_provider()matches the static JSON filename" 一条的来源。
平台后端在 backend/adapter_processor_v2/adapter_processor.py 的AdapterProcessor.get_json_schema()中,用 adapter_id 从 SDK 注册表中取回该 JSON 并以json_schema键返回给前端渲染表单。用户提交表单后,同一处理器中的test_adapter()(adapter_processor.py)会真正实例化 Adapter 并调用test_connection()做连通性校验——因此 Schema 中字段名必须与参数类(base1.py中的 Pydantic 模型)字段名一一对应。值得注意的一点是:update_adapter_metadata()(adapter_processor.py)会用 Fernet 对包含密钥的adapter_metadata整体加密后再入库,所以 Schema 中声明format: "password"的字段,其值最终会以密文形式存储。
Schema 顶层结构
一个 Adapter 配置 Schema 的标准骨架如下(直接继承自参考文档):
{ "title": "Provider Name Type", "type": "object", "required": ["field1", "field2"], "properties": { ... }, "allOf": [ ... ] }各键的职责:
| 键 | 作用 |
|---|---|
title | 顶层表单标题,约定为 "Provider Name Type"(如 "Ollama AI LLM") |
type | 固定为"object",表示整个表单是一个对象 |
required | 必填字段数组,表单提交前会强制校验 |
properties | 字段定义区,每个字段对应一个控件 |
allOf | 条件逻辑区,用if/then实现字段显隐与条件必填 |
仓库中的真实 Schema 还会使用若干 JSON Forms 风格的扩展键,例如 bedrock.json 顶部的"ui:order": [...](含通配符"*")用于控制字段渲染顺序,以及enumNames为枚举值提供人类可读的显示名。这些扩展不影响校验逻辑,只影响表单呈现。
字段类型详解
以下 8 种字段类型覆盖了 Adapter 配置的全部控件需求。每种类型均给出可直接复制的写法。
字符串字段(String Field)
{ "field_name": { "type": "string", "title": "Display Label", "default": "default value", "description": "Help text shown to user" } }title会成为表单标签,description作为帮助文案展示给用户。
密码字段(Password Field)
{ "api_key": { "type": "string", "title": "API Key", "format": "password", "description": "Your secret API key" } }format: "password"让控件以掩码方式显示输入,适用于 API Key、Access Key、Bearer Token 等一切机密值。
URL 字段(URL Field)
{ "endpoint": { "type": "string", "title": "Endpoint URL", "format": "uri", "default": "https://api.example.com/v1" } }用于 Endpoint、Base URL 等地址类输入。注意仓库中的用法并不统一:参考文档 与 ollama.json 中的base_url不使用format,而 llm_schema.json.template 中api_base用的是"format": "url"。两者都能工作,建议在同一 Schema 内保持一致。
数字字段(Number Field)
{ "timeout": { "type": "number", "title": "Timeout", "default": 300, "minimum": 0, "maximum": 3600, "multipleOf": 1, "description": "Timeout in seconds" } }minimum/maximum给出取值范围,multipleOf: 1可把浮点输入约束为整数步长(timeout、max_tokens 这类"逻辑上是整数"的参数在仓库模板中普遍如此标注)。
整数字段(Integer Field)
{ "max_retries": { "type": "integer", "title": "Max Retries", "default": 3, "minimum": 0, "maximum": 10 } }布尔字段(Boolean Field)
{ "enable_feature": { "type": "boolean", "title": "Enable Feature", "default": false, "description": "Toggle to enable this feature" } }布尔开关是条件字段的典型触发源,配合allOf可实现"勾选后才出现相关配置项"的效果。
下拉枚举字段(Dropdown / Enum Field)
{ "model": { "type": "string", "title": "Model", "enum": ["model-a", "model-b", "model-c"], "default": "model-a", "description": "Select the model to use" } }当可选值有限(区域、认证方式、模型清单)时使用enum。仓库中 bedrock.json 的auth_type字段还搭配了enumNames:
"enumNames": [ "Access Keys", "IAM Role / Instance Profile (on-prem AWS only)", "Bedrock API Key (Bearer Token)" ]这样下拉框显示友好文案,而提交值仍是access_keys/iam_role/bearer_token这类程序可用的键。
多行文本(Multi-line Text)
{ "json_credentials": { "type": "string", "title": "JSON Credentials", "format": "textarea", "description": "Paste your JSON credentials here" } }适用于需要用户整段粘贴 JSON(如 GCP Service Account 凭证)的场景。
必填字段:adapter_name约定
所有 Adapter Schema 都应把adapter_name列为必填,作为该 Adapter 实例在组织内的唯一名称:
{ "required": ["adapter_name", "api_key"], "properties": { "adapter_name": { "type": "string", "title": "Name", "default": "", "description": "Provide a unique name for this adapter instance" } } }这个约定在后端有明确消费方:AdapterProcessor.test_adapter()在连接测试失败时,会用adapter_metadata[AdapterKeys.ADAPTER_NAME]填充错误信息(见 adapter_processor.py 中TestAdapterError的构造);同时AdapterInstance模型也以adapter_name + adapter_type作为查询唯一标识(get_adapter_by_name_and_type,adapter_processor.py)。SKILL.md 的验证清单也把 "JSON schema hasadapter_nameas required field" 列为硬性检查项。
条件字段:allOf + if/then
当某些字段只在特定取值下才需要填写时,用allOf数组包裹若干if/then分支,实现字段显隐与条件必填。
基础条件(按枚举值分支)
{ "properties": { "auth_type": { "type": "string", "enum": ["api_key", "oauth"], "default": "api_key" } }, "allOf": [ { "if": { "properties": { "auth_type": { "const": "api_key" } } }, "then": { "properties": { "api_key": { "type": "string", "format": "password", "title": "API Key" } }, "required": ["api_key"] } }, { "if": { "properties": { "auth_type": { "const": "oauth" } } }, "then": { "properties": { "client_id": { "type": "string", "title": "Client ID" }, "client_secret": { "type": "string", "format": "password" } }, "required": ["client_id", "client_secret"] } } ] }模式要点:if中用const精确匹配触发值;then里声明该分支下的字段并在其required中列为必填。这样用户选择 oauth 时看不到api_key,反之亦然,表单噪音被最小化。
布尔开关条件(Boolean Toggle)
{ "properties": { "enable_reasoning": { "type": "boolean", "default": false, "title": "Enable Reasoning" } }, "allOf": [ { "if": { "properties": { "enable_reasoning": { "const": true } } }, "then": { "properties": { "reasoning_effort": { "type": "string", "enum": ["low", "medium", "high"], "default": "medium", "title": "Reasoning Effort" } }, "required": ["reasoning_effort"] } }, { "if": { "properties": { "enable_reasoning": { "const": false } } }, "then": { "properties": {} } } ] }注意第二个const: false的"空分支":显式声明未开启时没有任何附加字段,避免表单渲染器对未覆盖分支的处理出现歧义。
仓库实证:Bedrock Schema 的条件组合
真实的 bedrock.json 是条件字段的完整范本,它同时使用了三种机制:
dependencies(旧版 draft-04 风格)按auth_type三个取值切换认证字段(bedrock.json):access_keys分支要求aws_access_key_id与aws_secret_access_key(均为format: "password"),iam_role分支不附加任何字段(依赖云实例的 ambient 凭证),bearer_token分支要求aws_bearer_token(minLength: 1)。allOf开关条件:enable_thinking: true时出现budget_tokens(minimum: 1024,仅对 Claude 生效);enable_thinking: false时为空分支(bedrock.json)。- 字段联动必填:当
guardrail_identifier存在且非空(minLength: 1)时,then分支把guardrail_version追加为必填——这是"一个字段触发另一个字段必填"的写法,比显隐控制更细。
写条件逻辑时可优先使用allOf+if/then(现代 JSON Schema 标准写法),dependencies写法在仓库里属于既有实现,新 Schema 不必刻意模仿。
完整示例合集
以下 9 个示例全部继承自参考文档,覆盖 LLM、Embedding、云厂商、自托管与推理控制等主要场景,可作为新 Provider 的起点。
1. 简单 LLM Adapter Schema
{ "title": "Simple Provider LLM", "type": "object", "required": ["adapter_name", "api_key"], "properties": { "adapter_name": { "type": "string", "title": "Name", "default": "", "description": "Unique name for this adapter" }, "api_key": { "type": "string", "title": "API Key", "format": "password" }, "model": { "type": "string", "title": "Model", "default": "default-model" }, "max_tokens": { "type": "number", "minimum": 0, "title": "Max Tokens" }, "timeout": { "type": "number", "minimum": 0, "default": 900, "title": "Timeout (seconds)" } } }2. 带区域选择的云厂商(Cloud Provider with Regions)
{ "title": "Cloud Provider LLM", "type": "object", "required": ["adapter_name", "api_key", "region"], "properties": { "adapter_name": { "type": "string", "title": "Name" }, "api_key": { "type": "string", "format": "password", "title": "API Key" }, "region": { "type": "string", "title": "Region", "enum": ["us-east-1", "us-west-2", "eu-west-1", "ap-southeast-1"], "default": "us-east-1" }, "model": { "type": "string", "title": "Model", "enum": ["model-small", "model-medium", "model-large"], "default": "model-medium" } } }3. Azure 风格(Endpoint + Deployment)
{ "title": "Azure-Style Provider", "type": "object", "required": ["adapter_name", "api_key", "azure_endpoint", "deployment_name"], "properties": { "adapter_name": { "type": "string", "title": "Name" }, "azure_endpoint": { "type": "string", "format": "uri", "title": "Endpoint", "description": "Your Azure endpoint URL" }, "api_key": { "type": "string", "format": "password", "title": "API Key" }, "deployment_name": { "type": "string", "title": "Deployment Name", "description": "Name of your model deployment" }, "api_version": { "type": "string", "title": "API Version", "default": "2024-02-01" } } }4. 自托管(Ollama 风格)
{ "title": "Self-Hosted LLM", "type": "object", "required": ["adapter_name", "base_url"], "properties": { "adapter_name": { "type": "string", "title": "Name" }, "base_url": { "type": "string", "format": "uri", "title": "Server URL", "default": "http://localhost:11434", "description": "URL of your local server" }, "model": { "type": "string", "title": "Model", "default": "llama2", "description": "Model name (must be pulled on server)" } } }对照仓库中的 ollama.json 可以看到实战版比示例更丰富:base_url的description给出了http://host.docker.internal:11434这类 Docker 网络下的实际取值,并额外提供temperature(minimum: 0, maximum: 2,对应参数类BaseChatCompletionParameters中temperature的ge=0, le=2约束)、context_window(默认 3900)、max_retries(默认 3)、request_timeout(默认 900 秒)以及json_mode布尔开关。
5. Embedding Adapter Schema
{ "title": "Provider Embedding", "type": "object", "required": ["adapter_name", "api_key"], "properties": { "adapter_name": { "type": "string", "title": "Name" }, "model": { "type": "string", "title": "Model", "default": "text-embedding-model" }, "api_key": { "type": "string", "format": "password", "title": "API Key" }, "api_base": { "type": "string", "format": "uri", "title": "API Base URL" }, "embed_batch_size": { "type": "number", "minimum": 1, "default": 10, "title": "Batch Size" }, "timeout": { "type": "number", "minimum": 0, "default": 240, "title": "Timeout (seconds)" } } }其中embed_batch_size默认 10 与 base1.py 中BaseEmbeddingParameters的参数默认约定一致;Embedding 的连通性测试走EmbeddingCompat(adapter_id, adapter_metadata).test_connection()(adapter_processor.py)。
6. 带维度参数的 Embedding(OpenAI 风格)
{ "title": "OpenAI Embedding", "type": "object", "required": ["adapter_name", "api_key"], "properties": { "adapter_name": { "type": "string", "title": "Name" }, "model": { "type": "string", "title": "Model", "default": "text-embedding-3-small", "description": "text-embedding-3-small/large support custom dimensions" }, "api_key": { "type": "string", "format": "password", "title": "API Key" }, "dimensions": { "type": "number", "minimum": 1, "multipleOf": 1, "title": "Dimensions", "description": "Output dimensions (only for text-embedding-3-* models)" } } }dimensions刻意不设default且multipleOf: 1:留空时走模型默认维度,填写时必须是正整数。
7. 推理能力开关(Mistral Magistral、OpenAI o1/o3)
{ "properties": { "enable_reasoning": { "type": "boolean", "title": "Enable Reasoning", "default": false, "description": "Enable reasoning for Magistral models" } }, "allOf": [ { "if": { "properties": { "enable_reasoning": { "const": true } } }, "then": { "properties": { "reasoning_effort": { "type": "string", "enum": ["low", "medium", "high"], "default": "medium", "title": "Reasoning Effort" } }, "required": ["reasoning_effort"] } } ] }SKILL.md 特别提醒:推理参数的"形状"是 Provider 特定的——Anthropic 用带 token 预算的thinking块,多数推理系模型用reasoning_effort,发错字段会被 LiteLLM 静默丢弃(silent no-op)。Schema 只是让用户选对,参数类validate()里必须按模型家族分派,可参考base1.py中AnthropicLLMParameters与AWSBedrockLLMParameters._apply_bedrock_reasoning_config的实现。
8. 可选凭证(AWS Bedrock)
{ "title": "Bedrock LLM", "type": "object", "required": ["adapter_name", "region_name", "model"], "properties": { "adapter_name": { "type": "string", "title": "Name" }, "model": { "type": "string", "title": "Model" }, "region_name": { "type": "string", "title": "AWS Region" }, "aws_access_key_id": { "type": "string", "format": "password", "title": "AWS Access Key ID", "description": "Leave empty if using AWS Profile or IAM role." }, "aws_secret_access_key": { "type": "string", "format": "password", "title": "AWS Secret Access Key", "description": "Leave empty if using AWS Profile or IAM role." }, "aws_profile_name": { "type": "string", "title": "AWS Profile Name", "description": "AWS SSO profile name for authentication." } } }这里体现了"存在多种认证方式时,凭证字段不列入required"的原则:只有region_name、model、adapter_name必填,密钥/Profile 均可选,description中明确告诉用户何时可以留空。
9. JSON Mode 开关(Ollama)
{ "properties": { "json_mode": { "type": "boolean", "title": "JSON Mode", "default": false, "description": "Constrain output to valid JSON" } } }对结构化抽取场景(Unstract 的核心业务之一),json_mode约束输出为合法 JSON,与仓库中 ollama.json 的实际字段一致。
Schema 与参数类的衔接:表单提交之后
Schema 只负责"让用户填对",提交后的校验由 base1.py 中的参数类完成。BaseChatCompletionParameters定义了所有 LLM Provider 的公共参数基线:
class BaseChatCompletionParameters(BaseModel): model: str temperature: float | None = Field(default=0.1, ge=0, le=2) n: int | None = 1 timeout: float | int | None = 600 max_tokens: int | None = None max_retries: int | None = None两个推论值得写入 Schema 时参考:temperature的服务端约束是 0–2(默认 0.1),所以表单侧的minimum/maximum应对齐;timeout服务端默认 600 秒,而 LLM Schema 模板普遍写default: 900,即以表单默认值覆盖基类默认值。
每个参数类还需实现两个静态方法:validate()(对adapter_metadata做转换并整体校验)和validate_model()(补全模型名前缀)。SKILL.md 给出 LiteLLM 要求的前缀约定:
| Provider | 前缀 | 示例 |
|---|---|---|
| OpenAI | openai/ | openai/gpt-4 |
| Azure | azure/ | azure/gpt-4-deployment |
| Anthropic | anthropic/ | anthropic/claude-3-opus |
| Bedrock (Converse/Invoke) | bedrock/ | bedrock/anthropic.claude-v2 |
| Bedrock Mantle (OpenAI-compatible) | bedrock_mantle/ | bedrock_mantle/openai.gpt-5.6-terra |
| VertexAI | vertex_ai/ | vertex_ai/gemini-pro |
| Ollama | ollama_chat/ | ollama_chat/llama2 |
| Mistral | mistral/ | mistral/mistral-large |
| Anyscale | anyscale/ | anyscale/meta-llama/Llama-2-70b |
前缀不是装饰,而是成本核算的命门:成本查表使用validate_model()产出的最终模型串(调用点在unstract/sdk1/src/unstract/sdk1/audit.py的 LLM 路径与unstract/sdk1/src/unstract/sdk1/usage_handler.py的 Embedding 路径,均为litellm.cost_per_token(model=...))。若该串在 LiteLLM 成本映射表中不存在,异常会被捕获并回退为0.0——不会报错、不会失败,唯一症状是使用量记录为零成本。因此 SKILL.md 要求validate_model()对前缀做幂等处理(不重复拼接),并建议用 sdk1 的 venv 直接跑cost_per_token验证。这也是"Schema 里model字段填什么、default写什么"需要与参数类逻辑联动检查的原因。
最佳实践与提交前检查
参考文档 总结的 10 条最佳实践:
- 始终包含
adapter_name作为必填字段; - 机密与 API Key 一律使用
format: "password"; - 为可选字段提供合理默认值;
- 为不直观的字段添加
description; - 可选值有限时使用
enum; title保持简短——它会直接成为表单标签;- 按重要性/使用频率排列
properties(也可用ui:order显式控制,见 bedrock.json); - 用条件字段减少表单噪音;
- 部署前用 JSON Schema validator 校验一遍;
- 存在多种认证方式时,让凭证字段可选。
叠加 SKILL.md 中与 Schema 直接相关的提交前检查项:
get_provider()返回值与静态 JSON 文件名一致(static/{get_provider()}.json,大小写敏感)- JSON Schema 中
adapter_name为必填字段 validate()为模型串加正确前缀validate_model()幂等(不会双重加前缀)- 模型串能通过 LiteLLM 成本映射解析(避免静默 $0 记账)
- Adapter 类同时继承参数类与
BaseAdapter,get_adapter_type()返回AdapterTypes.LLM或AdapterTypes.EMBEDDING
配套模板与辅助脚本速查
写 Schema 不必从零开始。.claude/skills/adapter-ops/assets/templates/ 下提供即用的模板文件:
| 文件 | 用途 |
|---|---|
| llm_schema.json.template | LLM Schema 骨架,${PROVIDER_NAME}/${PROVIDER_ID}占位符,含adapter_name、api_key(password)、model、api_base(format url)、max_tokens、max_retries(默认 5)、timeout(默认 900) |
| embedding_schema.json.template | Embedding Schema 骨架 |
| llm_adapter.py.template | LLM Adapter 类骨架 |
| llm_parameters.py.template | 参数类骨架 |
| embedding_adapter.py.template / embedding_parameters.py.template | Embedding 侧骨架 |
配合以下脚本完成全生命周期操作(均位于 .claude/skills/adapter-ops/scripts/):
# 1. 初始化新 LLM Adapter(生成 {provider}.py + static/{provider}.json 骨架) python .claude/skills/adapter-ops/scripts/init_llm_adapter.py \ --provider newprovider \ --name "New Provider" \ --description "New Provider LLM adapter" \ --auto-logo # 2. 初始化 Embedding Adapter python .claude/skills/adapter-ops/scripts/init_embedding_adapter.py \ --provider newprovider --name "New Provider" # 3. 修改既有 Adapter 的模型清单(向 enum 增删模型) python .claude/skills/adapter-ops/scripts/manage_models.py \ --adapter llm \ --provider openai \ --action add-enum \ --models "gpt-4-turbo,gpt-4o-mini" # 4. 对照 LiteLLM 特性数据库检查各 Adapter 是否缺参数/过时的默认值 python .claude/skills/adapter-ops/scripts/check_adapter_updates.py \ --adapter llm --provider openai模型清单的修改方式分两档:模型少时用enum下拉(manage_models.py --action add-enum),模型多且变化快时用自由输入 +description列出可用模型:
{ "properties": { "model": { "type": "string", "title": "Model", "default": "new-default-model", "description": "Available models: model-1, model-2, model-3" } } }验证改动是否生效,按 SKILL.md 的推荐流程执行:
from unstract.sdk1.adapters.adapterkit import Adapterkit kit = Adapterkit() adapters = kit.get_adapters_list() # 确认新 adapter 出现在列表中,且 get_json_schema() 可读取小结
Unstract 的 Adapter 配置 UI 采用"一份静态 JSON Schema 即一个表单"的约定:文件名等于get_provider(),路径由适配器类型决定,adapter_name必填,机密字段用format: "password",有限选项用enum(可配enumNames),动态字段用allOf+if/then控制显隐与条件必填。表单之外的另一半功夫在参数类的validate()/validate_model()——尤其要保证模型前缀既正确又幂等,否则成本核算会静默归零。按本文的字段写法、9 个完整示例与检查清单操作,即可为任意 OpenAI 兼容、云厂商或自托管模型接入 Unstract 平台。
【免费下载链接】unstractLLM-Driven Extraction of Unstructured Data — Built for API Deployments & ETL Pipeline Workflows项目地址: https://gitcode.com/GitHub_Trending/un/unstract
创作声明:本文部分内容由AI辅助生成(AIGC),仅供参考