---
name: bench-603c5dbfa570
description: 测试 API 模型性能 - TTFB 和吐字速度
---

快速测试第三方 API 的模型性能。两个脚本，按 API 格式选择。

## 用法

用户提供：API Key + Base URL

**默认 → OpenAI 格式**（`/v1/chat/completions`）：
```bash
cd .claude/skills/bench && uv run python test_models.py "<API_KEY>" "<BASE_URL>"
```

**用户说"Anthropic 格式"→ Anthropic 原生**（`/v1/messages`）：
```bash
cd .claude/skills/bench && uv run python test_anthropic.py "<API_KEY>" "<BASE_URL>"
```

## 判断规则

- 默认走 `test_models.py`（OpenAI 格式）
- 仅当用户明确说"Anthropic 格式/原生 API"时，走 `test_anthropic.py`
- 不要自动猜测，由用户指定

## 测试指标

- **TTFB**: 首次响应延迟（10秒超时）
- **Tok/s**: 吐字速度
- 总超时：30秒

## OpenAI 脚本特性（test_models.py）

- 自动从 `/models` 端点获取可用模型并智能筛选
- 筛选最新主流文本模型（Claude 4.x, GPT 5.x, Gemini 3.x, Qwen 3.x, GLM 4.7+, Kimi k2.5+）
- 排除 DeepSeek、多模态模型、过时版本
- Base URL 必须带 `/v1` 后缀

## Anthropic 脚本特性（test_anthropic.py）

- 使用 `x-api-key` + `anthropic-version` 认证
- 默认测试：opus-4-6, sonnet-4-6, sonnet-4-5, haiku-4-5
- Base URL 带不带 `/v1` 都行（自动处理）

## 常见问题

- **依赖**：必须用 `uv run`，不能直接 `python`
- **OpenAI Base URL**：必须带 `/v1`，否则 `/models` 返回 HTML
- **Anthropic `/models` 被拦截**：正常，Cloudflare 保护，脚本用默认模型列表


---
Source attribution: OpenDCAI
Original project: https://github.com/OpenDCAI/leonai/blob/9a39b22e2359358837721bf7dc6dad72c185180c/.claude/skills/bench/SKILL.md
License: MIT
License information: https://opensource.org/license/mit
Original upstream instructions; downloaded from Dave AI.

## Upstream license notice

MIT License

Copyright (c) 2026 nmhjklnm

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.



# Dave AI resource metadata

Creator / publisher: OpenDCAI
Original project: https://github.com/OpenDCAI/leonai/blob/9a39b22e2359358837721bf7dc6dad72c185180c/.claude/skills/bench/SKILL.md
Attribution basis: Source repository record

Fields of work: Data & Analytics, Development & Engineering
Roles: Backend Developer, Data Analyst, ML / AI Engineer
Specialties: API, Testing
Tools: Anthropic API, OpenAI API
Country applicability: Country not specified
Classification basis: Source catalog labels
Instruction SHA-256: 0771588e55c2a565943cf045db1cf1dd2b55de5b1da5d793ea5c212d229347d9