这一课收尾 Part 7,也收尾整条「开发者工作流」弧线。Playground(游乐场)是你交互式试 prompt 的地方——敲几句消息、选个模型、点运行、看回复,满意了再去版本化(第 37 课)、跑实验(第 36 课)。但要让任何服务端 LLM 调用成真,背后得先有两样东西:一个安全存好的 LLM 连接(你的 provider 凭证),以及一批定义好的结构化输出 schema 与工具。 这一课讲它们,重点落在一个绕不开的安全话题:provider 的 API key 是能花你真金白银的「皇冠明珠」,Langfuse 怎么把它加密着存、又在该用的那一刻安全地取出来用。你还会看到一个贯穿全书的收束:Playground、评估(第 29 课)、实验(第 36 课)共用同一套 LLM 调用引擎。
fetchLLMCompletion),所以模拟里跑顺了,上路也跑得顺。
可发动引擎得用钥匙(provider 的 API key)。这把钥匙太值钱了——谁拿到都能开你的车、花你的油钱,所以它被锁进保险柜(AES 加密存储)。平时柜子只在表面露出钥匙串末尾几位(displaySecretKey)供你辨认是哪把,真要发动的那一刻才把真钥匙取出来(解密)插进引擎。钥匙从不在外面裸放,只在点火的瞬间现身。
Playground 看着像个独立的小工具,底下却复用了和评估、实验完全相同的 LLM 调用核心——fetchLLMCompletion。它的 chatCompletionHandler 把你在界面上敲的消息、选的模型、配的 tools 和 structuredOutputSchema 一并交给这个核心。于是第 29 课的裁判、第 36 课的实验、和这里的 Playground,本质上是同一台引擎的三个消费者。
import { fetchLLMCompletion } from "@langfuse/shared/src/server"; // 和评估/实验同一个 const fetchLLMCompletionParams = { messages, modelParams, /* 你在界面配的 */ tools, // 来自 LlmTool(函数调用定义) structuredOutputSchema // 来自 LlmSchema(结构化输出,呼应第29课) }; if (structuredOutputSchema) { const result = await fetchLLMCompletion({ ...fetchLLMCompletionParams, structuredOutputSchema }); } // Playground 不过是这台引擎的「交互式前台」——同一引擎,换个用法
| 消费者 | 它把什么交给引擎 | 典型用途 |
|---|---|---|
| Playground(本课) | 你界面上敲的 messages + 选的模型 + tools/schema | 交互试 prompt,发布前手动验证 |
| LLM 裁判(第29课) | 编译后的评分 prompt + 结构化输出 schema | 自动给 trace 评分(source=EVAL) |
| prompt 实验(第36课) | 填好变量的 prompt + 被考的 provider/model | 逐题跑出一场 run 供对比 |
三者要解决的业务问题各不相同,但「怎么调一个 LLM」这件苦活——选 adapter、解密凭证、拼请求、收流式、要结构化——都压进同一个 fetchLLMCompletion。这就是好抽象的标志:上层千变万化,底座岿然不动。
要调任何 provider,都得带上它的 API key。这把 key 是能直接花你钱、动你账号的最高敏感数据,绝不能明文躺在数据库里。Langfuse 的 LlmApiKeys 表把它加密存进 secretKey 字段,加密用的是 AES-256-GCM——一种认证加密:既保密、又能用一个 authTag 检测密文有没有被篡改。同时另存一个脱敏的 displaySecretKey(只露末尾几位)专供 UI 展示,让你认得出是哪把钥匙,却看不到全貌。
encrypt()(encryption.ts:18-35)用 AES-256-GCM,每次生成随机 12 字节 IV,输出 iv:encrypted:authTag。secretKey 存密文、displaySecretKey 存脱敏串(schema.prisma:299 LlmApiKeys)。ENCRYPTION_KEY 是 256 位、独立于库保管。export function encrypt(plainText: string): string { const iv = crypto.randomBytes(12); // 每次随机 IV → 同明文每次密文都不同 const cipher = crypto.createCipheriv("aes-256-gcm", Buffer.from(ENCRYPTION_KEY, "hex"), iv); // 256 位密钥,独立于数据库保管 let encrypted = cipher.update(plainText, "utf8", "hex") + cipher.final("hex"); const authTag = cipher.getAuthTag(); // GCM 认证标签 → 解密时校验,篡改即报错 return iv.toString("hex") + ":" + encrypted + ":" + authTag.toString("hex"); } // 存进 LlmApiKeys.secretKey;调 provider 前一刻才 decrypt(fetchLLMCompletion 内)
| 在哪 | key 以什么形态出现 | 谁能看到 |
|---|---|---|
数据库(secretKey) | 密文 iv:密文:authTag | 脱库者只拿到乱码(无 ENCRYPTION_KEY 解不开) |
UI(displaySecretKey) | 脱敏串 sk-…xyz | 你能认出是哪把,但看不到全貌 |
| 调 provider 前一刻 | 明文(瞬时) | 仅内存里短暂存在,用完即弃 |
一把 key、三种形态,各就各位:静态存储只见密文、人看只见脱敏、明文只在点火的一瞬现身。把「明文的暴露面」压到几乎为零,正是凭证管理的核心纪律。
除了凭证,现代 LLM 调用还常带两样东西:结构化输出 schema(要求模型按固定 JSON 形状返回,第 29 课裁判正靠它)和工具/函数定义(让模型能「调用」外部能力)。Langfuse 把它们也建成可复用的实体——LlmSchema 和 LlmTool,于是你在 Playground 里定义一次,评估、实验里都能直接拿来用。
定义模型必须遵守的 JSON 形状(如 {score, reasoning})。这正是第 29 课裁判能稳定返回「分数+理由」的底层依赖——同一套 schema 机制,被裁判与 Playground 共用。
定义模型可调用的工具签名(名字、参数 schema)。让模型在 Playground 里就能演示「函数调用」式的 agent 行为,定义沉淀下来供别处复用。
加密保管的 provider 凭证 + adapter(抽象 openai/anthropic 等接口)+ 可选 baseURL(接自建/代理端点)。三处 LLM 调用的共同底座。
这三块积木——连接、schema、工具——加上统一的 fetchLLMCompletion 引擎,构成了 Langfuse 里一切服务端 LLM 调用的公共底座。到这里,Part 7(乃至前七部分的「开发者工作流」主线)合拢成一个闭环:在 Playground 里交互地试 prompt → 满意了版本化(第 37 课)→ 拉进数据集跑实验对比(第 36 课)→ 用 label 安全发布(第 37 课)→ 上线后被观测(第一~四部分)、评估(第五部分)、必要时告警(第 33 课)。一个完整的、有数据支撑、可安全迭代的 LLM 工程生命周期,至此严丝合缝。
displaySecretKey 则解决一个很实际的体验问题:你在 UI 上得能认出「这是哪把 key」,但又不该为了显示就把整把 key 解密暴露。于是单存一个脱敏串(只露末尾几位),展示用脱敏版、调用才解密真版——安全与可用,两头都顾上了。这套「明文只在存入和点火的瞬间存在、其余时间只见密文与脱敏」的纪律,正是处理一切敏感凭证的范式。
chatCompletionHandler 复用和评估(第29课)、实验(第36课)完全相同的 fetchLLMCompletion 核心——一台引擎,三个消费者,凭证解密/provider 适配只写一遍。LlmApiKeys.secretKey 用 AES-256-GCM(认证加密)存密文,displaySecretKey 存脱敏串供 UI,adapter 抽象 provider 接口、baseURL 接自建端点。authTag 让解密时能检出密文是否被动过——保密之外再加「完整性保证」,对凭证这种数据是底线。LlmSchema(结构化输出,呼应第29课)、LlmTool(函数调用)一次定义处处复用。Playground→版本化→实验→label 发布→观测/评估/告警,开发者工作流至此合拢成完整闭环。This lesson caps Part 7 and the whole "developer workflow" arc. The Playground is where you interactively test a prompt—type a few messages, pick a model, hit run, see the response, and once satisfied, go version it (Lesson 37) and run an experiment (Lesson 36). But for any server-side LLM call to happen, two things must exist behind the scenes: a safely stored LLM connection (your provider credentials), and a set of defined structured-output schemas and tools. This lesson covers them, focusing on an unavoidable security topic: a provider's API key is a "crown jewel" that can spend your real money, so how does Langfuse store it encrypted and safely retrieve it at the moment of use. You'll also see a guide-wide convergence: the Playground, evaluation (Lesson 29), and experiments (Lesson 36) share one LLM-call engine.
fetchLLMCompletion as evals and experiments), so what runs smoothly in the sim runs smoothly on the road.
But starting the engine needs a key (the provider's API key). This key is too valuable—anyone holding it can drive your car and burn your fuel—so it's locked in a safe (AES-encrypted storage). The safe normally exposes only the key ring's last few digits (displaySecretKey) so you can tell which key it is, and only at the moment of ignition is the real key taken out (decrypted) and put into the engine. The key is never left bare outside—it appears only in the instant of ignition.
The Playground looks like a standalone little tool, but underneath it reuses the exact same LLM-call core as evals and experiments—fetchLLMCompletion. Its chatCompletionHandler hands the messages you typed, the model you picked, and the tools and structuredOutputSchema you configured to this core. So Lesson 29's judge, Lesson 36's experiment, and the Playground here are essentially three consumers of one engine.
web/src/features/playground/server/chatCompletionHandler.ts:19,69-86 directly calls fetchLLMCompletion, passing tools and structuredOutputSchema. This shares the same engine with Lesson 29's judge callLLM and Lesson 36's experiment model call—credential decryption and provider adaptation written once.LlmApiKeys model; values illustrative): the connection on the left comes from LlmApiKeys — provider/adapter pick the interface, secretKey is stored AES-256-GCM-encrypted and decrypted only at call time, while the UI only ever sees the masked displaySecretKey; the request params on the right are model/temperature/maxTokens plus messages. Both flow into fetchLLMCompletion() — the same engine reused by Lesson 29's judge and Lesson 36's experiments.import { fetchLLMCompletion } from "@langfuse/shared/src/server"; // same as evals/experiments const fetchLLMCompletionParams = { messages, modelParams, /* configured in the UI */ tools, // from LlmTool (function-call definitions) structuredOutputSchema // from LlmSchema (structured output, echoing Lesson 29) }; if (structuredOutputSchema) { const result = await fetchLLMCompletion({ ...fetchLLMCompletionParams, structuredOutputSchema }); } // the Playground is just this engine's "interactive front desk"—same engine, different use
| consumer | what it hands the engine | typical use |
|---|---|---|
| Playground (this lesson) | messages you typed + model picked + tools/schema | interactive prompt testing, manual pre-release validation |
| LLM judge (Lesson 29) | the compiled scoring prompt + structured-output schema | auto-scoring traces (source=EVAL) |
| prompt experiment (Lesson 36) | the variable-filled prompt + the provider/model under test | run a run per question for comparison |
The three solve different business problems, but the grind of "how to call an LLM"—picking the adapter, decrypting credentials, assembling the request, receiving streams, demanding structure—is all pressed into one fetchLLMCompletion. That's the mark of a good abstraction: the upper layers vary endlessly, the foundation stands firm.
To call any provider you must carry its API key. This key is the most sensitive data—it can directly spend your money and act on your account—and must never sit in plaintext in the database. Langfuse's LlmApiKeys table stores it encrypted in the secretKey field, using AES-256-GCM—an authenticated encryption: both confidential, and able to detect tampering via an authTag. It separately stores a masked displaySecretKey (only the last few digits) for the UI, so you can tell which key it is without seeing the whole thing.
encrypt() (encryption.ts:18-35) uses AES-256-GCM, generating a random 12-byte IV each time, outputting iv:encrypted:authTag. secretKey stores ciphertext, displaySecretKey a masked string (schema.prisma:299 LlmApiKeys). ENCRYPTION_KEY is 256-bit, kept separately from the DB.export function encrypt(plainText: string): string { const iv = crypto.randomBytes(12); // random IV each time → same plaintext, different ciphertext const cipher = crypto.createCipheriv("aes-256-gcm", Buffer.from(ENCRYPTION_KEY, "hex"), iv); // 256-bit key, kept separately from the DB let encrypted = cipher.update(plainText, "utf8", "hex") + cipher.final("hex"); const authTag = cipher.getAuthTag(); // GCM auth tag → verified on decrypt, errors on tampering return iv.toString("hex") + ":" + encrypted + ":" + authTag.toString("hex"); } // stored in LlmApiKeys.secretKey; decrypted only just before calling the provider (inside fetchLLMCompletion)
| where | what form the key takes | who can see it |
|---|---|---|
database (secretKey) | ciphertext iv:ciphertext:authTag | a breach yields gibberish (can't decrypt without ENCRYPTION_KEY) |
UI (displaySecretKey) | masked string sk-…xyz | you can recognize which key, but not see the whole |
| just before calling the provider | plaintext (momentary) | exists briefly in memory only, discarded after |
One key, three forms, each in its place: at rest only ciphertext, to a human only a mask, plaintext appearing only in the instant of ignition. Squeezing "the exposure surface of plaintext" to nearly zero is the core discipline of credential management.
Besides credentials, modern LLM calls often carry two things: a structured-output schema (requiring the model to return a fixed JSON shape, which Lesson 29's judge relies on) and tool/function definitions (letting the model "call" external capabilities). Langfuse models these as reusable entities too—LlmSchema and LlmTool—so you define them once in the Playground and reuse them in evals and experiments.
Defines the JSON shape the model must obey (e.g. {score, reasoning}). This is the underlying dependency that lets Lesson 29's judge reliably return "score + reasoning"—the same schema mechanism shared by the judge and the Playground.
Defines tool signatures the model can call (name, parameter schema). It lets the model demonstrate "function-calling" agent behavior right in the Playground, with the definition saved for reuse elsewhere.
Encrypted provider credentials + adapter (abstracting the openai/anthropic/etc. interface) + optional baseURL (for self-hosted/proxy endpoints). The common base for all three LLM-call sites.
These three blocks—connection, schema, tool—plus the unified fetchLLMCompletion engine, form the common foundation for every server-side LLM call in Langfuse. Here Part 7 (and indeed the first seven parts' "developer workflow" through-line) closes into a loop: interactively test a prompt in the Playground → version it (Lesson 37) once satisfied → pull it into a dataset to run experiments and compare (Lesson 36) → release safely via a label (Lesson 37) → after shipping, get observed (Parts 1–4), evaluated (Part 5), and alerted when needed (Lesson 33). A complete, data-backed, safely-iterable LLM engineering lifecycle, sealed tight.
displaySecretKey solves a very practical UX problem: in the UI you must recognize "which key this is", yet you shouldn't decrypt and expose the whole key just to display it. So a masked string (last few digits only) is stored separately—display the masked version, decrypt only to call—covering both security and usability. This discipline—"plaintext exists only in the instants of storing and igniting, the rest of the time only ciphertext and masks are seen"—is the paradigm for handling any sensitive credential.
chatCompletionHandler reuses the exact fetchLLMCompletion core as evals (L29) and experiments (L36)—one engine, three consumers, credential decryption/provider adaptation written once.LlmApiKeys.secretKey stores ciphertext via AES-256-GCM (authenticated encryption), displaySecretKey a masked string for the UI, adapter abstracts the provider interface, baseURL hits self-hosted endpoints.authTag lets decryption detect whether the ciphertext was altered—integrity on top of confidentiality, the floor for credential data.LlmSchema (structured output, echoing L29), LlmTool (function calls) are defined once and reused everywhere. Playground→versioning→experiments→label release→observe/evaluate/alert—the developer workflow closes into a complete loop.