🪢 Langfuse 图解教程Langfuse Visual Guide 第七部分 · Prompt 与 PlaygroundPart 7 · Prompts & Playground 39 / 55
第七部分 · Prompt 与 PlaygroundPart 7 · Prompts & Playground

Playground 与 LLM 连接Playground & LLM connections

🧭 读前Before you start
先读 第37课 的 prompt 管理,本课讲 Playground 与 LLM 连接如何配置。
Read Lesson 37 on prompt management; this lesson covers the Playground and LLM connection config.

这一课收尾 Part 7,也收尾整条「开发者工作流」弧线。Playground(游乐场)是你交互式试 prompt 的地方——敲几句消息、选个模型、点运行、看回复,满意了再去版本化(第 37 课)、跑实验(第 36 课)。但要让任何服务端 LLM 调用成真,背后得先有两样东西:一个安全存好的 LLM 连接(你的 provider 凭证),以及一批定义好的结构化输出 schema 与工具。 这一课讲它们,重点落在一个绕不开的安全话题:provider 的 API key 是能花你真金白银的「皇冠明珠」,Langfuse 怎么把它加密着存、又在该用的那一刻安全地取出来用。你还会看到一个贯穿全书的收束:Playground、评估(第 29 课)、实验(第 36 课)共用同一套 LLM 调用引擎。

📋 生活类比
Playground 像一台「驾驶模拟器 + 钥匙保险柜」。模拟器(playground)让你在正式上路(版本化、上线)之前反复试驾(试 prompt)——而且用的就是真车的同一台引擎(和评估、实验同一套 fetchLLMCompletion),所以模拟里跑顺了,上路也跑得顺。 可发动引擎得用钥匙(provider 的 API key)。这把钥匙太值钱了——谁拿到都能开你的车、花你的油钱,所以它被锁进保险柜(AES 加密存储)。平时柜子只在表面露出钥匙串末尾几位(displaySecretKey)供你辨认是哪把,真要发动的那一刻才把真钥匙取出来(解密)插进引擎。钥匙从不在外面裸放,只在点火的瞬间现身。

一台引擎,三个消费者

Playground 看着像个独立的小工具,底下却复用了和评估、实验完全相同的 LLM 调用核心——fetchLLMCompletion。它的 chatCompletionHandler 把你在界面上敲的消息、选的模型、配的 tools 和 structuredOutputSchema 一并交给这个核心。于是第 29 课的裁判、第 36 课的实验、和这里的 Playground,本质上是同一台引擎的三个消费者。

同一套 fetchLLMCompletion,被三处共用 Playground(本课)交互试 prompt LLM 裁判(第29课)结构化输出评分 prompt 实验(第36课)逐题跑 prompt fetchLLMCompletion统一的 LLM 调用核心 OpenAI Anthropic …(按 adapter) 一台引擎统一封装:凭证解密、provider 适配、结构化输出、工具调用、流式——三处复用,不重复造轮子
一个核心,三处复用:web/src/features/playground/server/chatCompletionHandler.ts:19,69-86 直接调 fetchLLMCompletion,传入 tools 与 structuredOutputSchema。这与第 29 课裁判的 callLLM、第 36 课实验的模型调用,底层是同一台引擎——凭证解密、provider 适配都只写一遍。
示例:一次 Playground 请求 LLM connection · LlmApiKeys provideropenai adapteropenai displaySecretKeysk-…a1b2 baseURL(默认) 🔒 secretKey: AES-256-GCM 加密存储,调用前一刻才解密 请求参数 modelgpt-4o temperature0.2 maxTokens512 messages: [{role:"system", …}, {role:"user", content:"退款?"}] fetchLLMCompletion() 同一引擎也被第29课裁判、第36课实验复用——连接定义一次,处处可用 → { content:"30 days…", usage:{ in:42,out:18 } }
Playground 就是把「连接 + 参数」喂给统一引擎(字段取自 LlmApiKeys 模型;值为示例):左边的连接来自 LlmApiKeys——provider/adapter 选定接口,secretKey 以 AES-256-GCM 加密落库、只在调用前一刻解密,UI 只看得到掩码的 displaySecretKey;右边是请求参数(model/temperature/maxTokens + 消息)。两者汇入 fetchLLMCompletion()——同一个引擎也被第 29 课的裁判、第 36 课的实验复用。
web/src/features/playground/server/chatCompletionHandler.tsPlayground 调同一核心
import { fetchLLMCompletion } from "@langfuse/shared/src/server";   // 和评估/实验同一个

const fetchLLMCompletionParams = { messages, modelParams, /* 你在界面配的 */
  tools,                  // 来自 LlmTool(函数调用定义)
  structuredOutputSchema  // 来自 LlmSchema(结构化输出,呼应第29课)
};
if (structuredOutputSchema) {
  const result = await fetchLLMCompletion({ ...fetchLLMCompletionParams, structuredOutputSchema });
}
// Playground 不过是这台引擎的「交互式前台」——同一引擎,换个用法
消费者它把什么交给引擎典型用途
Playground(本课)你界面上敲的 messages + 选的模型 + tools/schema交互试 prompt,发布前手动验证
LLM 裁判(第29课)编译后的评分 prompt + 结构化输出 schema自动给 trace 评分(source=EVAL)
prompt 实验(第36课)填好变量的 prompt + 被考的 provider/model逐题跑出一场 run 供对比

三者要解决的业务问题各不相同,但「怎么调一个 LLM」这件苦活——选 adapter、解密凭证、拼请求、收流式、要结构化——都压进同一个 fetchLLMCompletion。这就是好抽象的标志:上层千变万化,底座岿然不动。

皇冠明珠:API key 的加密存储

要调任何 provider,都得带上它的 API key。这把 key 是能直接花你钱、动你账号的最高敏感数据,绝不能明文躺在数据库里。Langfuse 的 LlmApiKeys 表把它加密存进 secretKey 字段,加密用的是 AES-256-GCM——一种认证加密:既保密、又能用一个 authTag 检测密文有没有被篡改。同时另存一个脱敏的 displaySecretKey(只露末尾几位)专供 UI 展示,让你认得出是哪把钥匙,却看不到全貌。

明文只在「存」和「用」的瞬间存在,库里永远是密文 明文 API keysk-abc...xyz AES-256-GCM 加密随机 IV(每次不同)+ authTag 防篡改 secretKey(库里只存这个)iv : 密文 : authTag(全 hex)看不懂、改不了、对不上就拒绝 displaySecretKey脱敏:sk-…xyz(供 UI 认) 用时才 decrypt调 provider 前一刻还原明文,用完即弃 数据库被脱库也只拿到密文;没有 ENCRYPTION_KEY(独立保管),密文就是一堆乱码
认证加密 + 脱敏展示 + 即用即解:encrypt()(encryption.ts:18-35)用 AES-256-GCM,每次生成随机 12 字节 IV,输出 iv:encrypted:authTag。secretKey 存密文、displaySecretKey 存脱敏串(schema.prisma:299 LlmApiKeys)。ENCRYPTION_KEY 是 256 位、独立于库保管。
packages/shared/src/encryption/encryption.ts:18-35AES-256-GCM 加密
export function encrypt(plainText: string): string {
  const iv = crypto.randomBytes(12);                       // 每次随机 IV → 同明文每次密文都不同
  const cipher = crypto.createCipheriv("aes-256-gcm",
    Buffer.from(ENCRYPTION_KEY, "hex"), iv);              // 256 位密钥,独立于数据库保管
  let encrypted = cipher.update(plainText, "utf8", "hex") + cipher.final("hex");
  const authTag = cipher.getAuthTag();                     // GCM 认证标签 → 解密时校验,篡改即报错
  return iv.toString("hex") + ":" + encrypted + ":" + authTag.toString("hex");
}
// 存进 LlmApiKeys.secretKey;调 provider 前一刻才 decrypt(fetchLLMCompletion 内)
在哪key 以什么形态出现谁能看到
数据库(secretKey)密文 iv:密文:authTag脱库者只拿到乱码(无 ENCRYPTION_KEY 解不开)
UI(displaySecretKey)脱敏串 sk-…xyz你能认出是哪把,但看不到全貌
调 provider 前一刻明文(瞬时)仅内存里短暂存在,用完即弃

一把 key、三种形态,各就各位:静态存储只见密文、人看只见脱敏、明文只在点火的一瞬现身。把「明文的暴露面」压到几乎为零,正是凭证管理的核心纪律。

可复用的积木:结构化 schema 与工具

除了凭证,现代 LLM 调用还常带两样东西:结构化输出 schema(要求模型按固定 JSON 形状返回,第 29 课裁判正靠它)和工具/函数定义(让模型能「调用」外部能力)。Langfuse 把它们也建成可复用的实体——LlmSchema 和 LlmTool,于是你在 Playground 里定义一次,评估、实验里都能直接拿来用。

三块可复用积木 → 一台引擎 → 三个消费者 LlmApiKeys · 连接 凭证 · adapter · baseURL LlmSchema · 结构化输出 {score, reasoning} LlmTool · 工具/函数 名字 + 参数 schema fetchLLMCompletion 统一引擎 Playground 交互试 prompt 评估(L29 裁判) 靠 schema 稳定返回 实验(L36 对比) 逐题跑应用

LlmSchema · 结构化输出

定义模型必须遵守的 JSON 形状(如 {score, reasoning})。这正是第 29 课裁判能稳定返回「分数+理由」的底层依赖——同一套 schema 机制,被裁判与 Playground 共用。

LlmTool · 函数/工具

定义模型可调用的工具签名(名字、参数 schema)。让模型在 Playground 里就能演示「函数调用」式的 agent 行为,定义沉淀下来供别处复用。

LlmApiKeys · 连接

加密保管的 provider 凭证 + adapter(抽象 openai/anthropic 等接口)+ 可选 baseURL(接自建/代理端点)。三处 LLM 调用的共同底座。

这三块积木——连接、schema、工具——加上统一的 fetchLLMCompletion 引擎,构成了 Langfuse 里一切服务端 LLM 调用的公共底座。到这里,Part 7(乃至前七部分的「开发者工作流」主线)合拢成一个闭环:在 Playground 里交互地试 prompt → 满意了版本化(第 37 课)→ 拉进数据集跑实验对比(第 36 课)→ 用 label 安全发布(第 37 课)→ 上线后被观测(第一~四部分)、评估(第五部分)、必要时告警(第 33 课)。一个完整的、有数据支撑、可安全迭代的 LLM 工程生命周期,至此严丝合缝。

🎯 设计取舍
为什么用 AES-256-GCM 这种「认证加密」,而不是普通的对称加密? 因为对 API key 这种数据,光保密还不够,还要防篡改。普通加密(如 AES-CBC)能保证「别人看不懂密文」,但挡不住有人偷偷改几个字节——改完你解密出来可能是另一串看似合法的乱码,悄无声息地出错。GCM 模式在加密的同时算一个 authTag(认证标签),解密时一并校验:密文哪怕被动一个比特,校验就失败、直接拒绝。于是你得到的不只是「保密」,还有「完整性保证」——拿到的明文要么和当初存的一模一样、要么干脆报错,绝不会是被悄悄篡改过的版本。对凭证这种「错一点就酿大祸」的数据,认证加密是底线。

为什么每次加密都用一个随机 IV,还要单存一个脱敏的 displaySecretKey? 随机 IV(初始向量)解决的是「同样的明文,每次加密出来的密文都不同」——否则两个一样的 key 会有一样的密文,攻击者一眼就能看出「这俩用了同一把钥匙」,泄露信息。每次换随机 IV,让密文彻底无规律可循。而 displaySecretKey 则解决一个很实际的体验问题:你在 UI 上得能认出「这是哪把 key」,但又不该为了显示就把整把 key 解密暴露。于是单存一个脱敏串(只露末尾几位),展示用脱敏版、调用才解密真版——安全与可用,两头都顾上了。这套「明文只在存入和点火的瞬间存在、其余时间只见密文与脱敏」的纪律,正是处理一切敏感凭证的范式。
🎯 本课要点 · 第七部分收官
  • Playground = 同一引擎的交互式前台:chatCompletionHandler 复用和评估(第29课)、实验(第36课)完全相同的 fetchLLMCompletion 核心——一台引擎,三个消费者,凭证解密/provider 适配只写一遍。
  • API key 加密存储:LlmApiKeys.secretKey 用 AES-256-GCM(认证加密)存密文,displaySecretKey 存脱敏串供 UI,adapter 抽象 provider 接口、baseURL 接自建端点。
  • 认证加密防篡改:GCM 的 authTag 让解密时能检出密文是否被动过——保密之外再加「完整性保证」,对凭证这种数据是底线。
  • 随机 IV + 即用即解:每次随机 IV 让同明文密文各异、不泄露信息;明文只在「存入」与「调用前一刻」短暂存在,库里和 UI 上永远只见密文 / 脱敏串。
  • 可复用积木 + 闭环:LlmSchema(结构化输出,呼应第29课)、LlmTool(函数调用)一次定义处处复用。Playground→版本化→实验→label 发布→观测/评估/告警,开发者工作流至此合拢成完整闭环。

🧪 自测 · 想一想为什么这么设计

1. Langfuse 的 Playground、LLM 裁判(第29课)、prompt 实验(第36课)三者在底层有什么关系?
  1. 三者各有独立的 LLM 调用实现
  2. 三者共用同一个 fetchLLMCompletion 核心——一台引擎、三个消费者;凭证解密、provider 适配、结构化输出、工具调用都只写一遍
  3. Playground 只能调 OpenAI
  4. 实验不调真实 LLM
看答案与解析 点击展开
答案:B。playground 的 chatCompletionHandler 直接 import 并调用 fetchLLMCompletion,传 tools 和 structuredOutputSchema——和第29课裁判的 callLLM、第36课实验的模型调用是同一台引擎。Playground 不过是这台引擎的「交互式前台」。一处封装、三处复用,避免重复造轮子,也保证三处行为一致。
2. Langfuse 用 AES-256-GCM(而非普通对称加密)来加密 provider 的 API key。GCM 模式的 authTag 在这里起什么独特作用?
  1. 防篡改/完整性保证:解密时校验 authTag,密文哪怕被改一个比特都会校验失败、直接拒绝——保证拿到的明文要么和当初存的一模一样、要么报错,绝不是被悄悄改过的版本
  2. 允许多人共享密钥
  3. 压缩密文体积
  4. 让加密更快
看答案与解析 点击展开
答案:A。对 API key 这种数据,光保密不够、还要防篡改。普通加密(如 AES-CBC)挡不住有人偷改几字节、解出另一串看似合法的乱码而悄悄出错。GCM 在加密时算 authTag,解密一并校验:密文动一个比特就失败。于是你得到「完整性保证」——明文要么和存入时一模一样、要么直接报错。对凭证这种错一点就酿大祸的数据,认证加密是底线。
3. LlmApiKeys 表既存加密的 secretKey,又单独存一个脱敏的 displaySecretKey(如 sk-…xyz)。为什么要多存这个脱敏串?
  1. 脱敏串是加密的备份
  2. 为了在 UI 上能认出「这是哪把 key」,又不必为了显示就解密暴露整把 key——展示用脱敏版、调用才解密真版,安全与可用两头兼顾
  3. 脱敏串才是真正调用用的
  4. 用来加速解密
看答案与解析 点击展开
答案:B。这是「明文只在存入和点火瞬间存在」纪律下的实用设计:你在 UI 上得能认出是哪把 key,但不该为了显示就把整把 key 解密暴露。于是单存一个只露末尾几位的脱敏串供展示,真正调 provider 前一刻才 decrypt 出 secretKey 的明文、用完即弃。展示脱敏、调用解密——安全与体验都照顾到。这套纪律是处理一切敏感凭证的范式。
💭 发散思考(没有标准答案,动手或动脑想想)
  • Part 7 到此,整条「开发者工作流」闭环合拢:Playground 试 → 版本化 → 实验对比 → label 发布 → 观测/评估/告警。回顾这条闭环,它和你熟悉的「软件开发生命周期」(写代码→测试→CI→发布→监控)有哪些惊人相似、又有哪些因为 LLM 的「不确定性」而独有的环节?如果让你给团队推广这套 prompt 工程流程,最难落地的是哪一步,为什么?

This lesson caps Part 7 and the whole "developer workflow" arc. The Playground is where you interactively test a prompt—type a few messages, pick a model, hit run, see the response, and once satisfied, go version it (Lesson 37) and run an experiment (Lesson 36). But for any server-side LLM call to happen, two things must exist behind the scenes: a safely stored LLM connection (your provider credentials), and a set of defined structured-output schemas and tools. This lesson covers them, focusing on an unavoidable security topic: a provider's API key is a "crown jewel" that can spend your real money, so how does Langfuse store it encrypted and safely retrieve it at the moment of use. You'll also see a guide-wide convergence: the Playground, evaluation (Lesson 29), and experiments (Lesson 36) share one LLM-call engine.

📋 Analogy
The Playground is like a "driving simulator + key safe". The simulator (playground) lets you test-drive (test a prompt) repeatedly before going on the road (versioning, shipping)—and it uses the real car's same engine (the same fetchLLMCompletion as evals and experiments), so what runs smoothly in the sim runs smoothly on the road. But starting the engine needs a key (the provider's API key). This key is too valuable—anyone holding it can drive your car and burn your fuel—so it's locked in a safe (AES-encrypted storage). The safe normally exposes only the key ring's last few digits (displaySecretKey) so you can tell which key it is, and only at the moment of ignition is the real key taken out (decrypted) and put into the engine. The key is never left bare outside—it appears only in the instant of ignition.

One engine, three consumers

The Playground looks like a standalone little tool, but underneath it reuses the exact same LLM-call core as evals and experiments—fetchLLMCompletion. Its chatCompletionHandler hands the messages you typed, the model you picked, and the tools and structuredOutputSchema you configured to this core. So Lesson 29's judge, Lesson 36's experiment, and the Playground here are essentially three consumers of one engine.

one fetchLLMCompletion, shared by three Playground (this lesson)interactive prompt testing LLM judge (Lesson 29)structured-output scoring prompt experiment (L36)run a prompt per question fetchLLMCompletionthe unified LLM-call core OpenAI Anthropic … (by adapter) one engine wraps it all: credential decryption, provider adaptation, structured output, tool calls, streaming—reused by three, no reinventing
one core, three reuses: web/src/features/playground/server/chatCompletionHandler.ts:19,69-86 directly calls fetchLLMCompletion, passing tools and structuredOutputSchema. This shares the same engine with Lesson 29's judge callLLM and Lesson 36's experiment model call—credential decryption and provider adaptation written once.
Example: one Playground request LLM connection · LlmApiKeys provideropenai adapteropenai displaySecretKeysk-…a1b2 baseURL(default) 🔒 secretKey: AES-256-GCM encrypted, decrypted at call time request params modelgpt-4o temperature0.2 maxTokens512 messages: [{role:"system", …}, {role:"user", content:"Refund?"}] fetchLLMCompletion() the same engine is reused by L29 judge and L36 experiments — define a connection once, reuse everywhere → { content:"30 days…", usage:{ in:42,out:18 } }
The Playground just feeds "connection + params" into one shared engine (fields from the LlmApiKeys model; values illustrative): the connection on the left comes from LlmApiKeys — provider/adapter pick the interface, secretKey is stored AES-256-GCM-encrypted and decrypted only at call time, while the UI only ever sees the masked displaySecretKey; the request params on the right are model/temperature/maxTokens plus messages. Both flow into fetchLLMCompletion() — the same engine reused by Lesson 29's judge and Lesson 36's experiments.
web/src/features/playground/server/chatCompletionHandler.tsPlayground calls the same core
import { fetchLLMCompletion } from "@langfuse/shared/src/server";   // same as evals/experiments

const fetchLLMCompletionParams = { messages, modelParams, /* configured in the UI */
  tools,                  // from LlmTool (function-call definitions)
  structuredOutputSchema  // from LlmSchema (structured output, echoing Lesson 29)
};
if (structuredOutputSchema) {
  const result = await fetchLLMCompletion({ ...fetchLLMCompletionParams, structuredOutputSchema });
}
// the Playground is just this engine's "interactive front desk"—same engine, different use
consumerwhat it hands the enginetypical use
Playground (this lesson)messages you typed + model picked + tools/schemainteractive prompt testing, manual pre-release validation
LLM judge (Lesson 29)the compiled scoring prompt + structured-output schemaauto-scoring traces (source=EVAL)
prompt experiment (Lesson 36)the variable-filled prompt + the provider/model under testrun a run per question for comparison

The three solve different business problems, but the grind of "how to call an LLM"—picking the adapter, decrypting credentials, assembling the request, receiving streams, demanding structure—is all pressed into one fetchLLMCompletion. That's the mark of a good abstraction: the upper layers vary endlessly, the foundation stands firm.

The crown jewel: encrypted storage of the API key

To call any provider you must carry its API key. This key is the most sensitive data—it can directly spend your money and act on your account—and must never sit in plaintext in the database. Langfuse's LlmApiKeys table stores it encrypted in the secretKey field, using AES-256-GCM—an authenticated encryption: both confidential, and able to detect tampering via an authTag. It separately stores a masked displaySecretKey (only the last few digits) for the UI, so you can tell which key it is without seeing the whole thing.

plaintext exists only at "store" and "use"; the DB always holds ciphertext plaintext API keysk-abc...xyz AES-256-GCM encryptrandom IV (different each time)+ authTag anti-tamper secretKey (only this in the DB)iv : ciphertext : authTag (all hex)unreadable, unchangeable, rejected if mismatched displaySecretKeymasked: sk-…xyz (for the UI) decrypt only when usedrestore plaintext just before calling the provider, discard after a DB breach yields only ciphertext; without ENCRYPTION_KEY (kept separately), the ciphertext is gibberish
authenticated encryption + masked display + decrypt-on-use: encrypt() (encryption.ts:18-35) uses AES-256-GCM, generating a random 12-byte IV each time, outputting iv:encrypted:authTag. secretKey stores ciphertext, displaySecretKey a masked string (schema.prisma:299 LlmApiKeys). ENCRYPTION_KEY is 256-bit, kept separately from the DB.
packages/shared/src/encryption/encryption.ts:18-35AES-256-GCM encrypt
export function encrypt(plainText: string): string {
  const iv = crypto.randomBytes(12);                       // random IV each time → same plaintext, different ciphertext
  const cipher = crypto.createCipheriv("aes-256-gcm",
    Buffer.from(ENCRYPTION_KEY, "hex"), iv);              // 256-bit key, kept separately from the DB
  let encrypted = cipher.update(plainText, "utf8", "hex") + cipher.final("hex");
  const authTag = cipher.getAuthTag();                     // GCM auth tag → verified on decrypt, errors on tampering
  return iv.toString("hex") + ":" + encrypted + ":" + authTag.toString("hex");
}
// stored in LlmApiKeys.secretKey; decrypted only just before calling the provider (inside fetchLLMCompletion)
wherewhat form the key takeswho can see it
database (secretKey)ciphertext iv:ciphertext:authTaga breach yields gibberish (can't decrypt without ENCRYPTION_KEY)
UI (displaySecretKey)masked string sk-…xyzyou can recognize which key, but not see the whole
just before calling the providerplaintext (momentary)exists briefly in memory only, discarded after

One key, three forms, each in its place: at rest only ciphertext, to a human only a mask, plaintext appearing only in the instant of ignition. Squeezing "the exposure surface of plaintext" to nearly zero is the core discipline of credential management.

Reusable building blocks: structured schemas and tools

Besides credentials, modern LLM calls often carry two things: a structured-output schema (requiring the model to return a fixed JSON shape, which Lesson 29's judge relies on) and tool/function definitions (letting the model "call" external capabilities). Langfuse models these as reusable entities too—LlmSchema and LlmTool—so you define them once in the Playground and reuse them in evals and experiments.

three reusable blocks → one engine → three consumers LlmApiKeys · connection creds · adapter · baseURL LlmSchema · structured {score, reasoning} LlmTool · tools/fns name + param schema fetchLLMCompletion one engine Playground test a prompt eval (L29 judge) schema → stable JSON experiments (L36) app per item

LlmSchema · structured output

Defines the JSON shape the model must obey (e.g. {score, reasoning}). This is the underlying dependency that lets Lesson 29's judge reliably return "score + reasoning"—the same schema mechanism shared by the judge and the Playground.

LlmTool · function/tool

Defines tool signatures the model can call (name, parameter schema). It lets the model demonstrate "function-calling" agent behavior right in the Playground, with the definition saved for reuse elsewhere.

LlmApiKeys · connection

Encrypted provider credentials + adapter (abstracting the openai/anthropic/etc. interface) + optional baseURL (for self-hosted/proxy endpoints). The common base for all three LLM-call sites.

These three blocks—connection, schema, tool—plus the unified fetchLLMCompletion engine, form the common foundation for every server-side LLM call in Langfuse. Here Part 7 (and indeed the first seven parts' "developer workflow" through-line) closes into a loop: interactively test a prompt in the Playground → version it (Lesson 37) once satisfied → pull it into a dataset to run experiments and compare (Lesson 36) → release safely via a label (Lesson 37) → after shipping, get observed (Parts 1–4), evaluated (Part 5), and alerted when needed (Lesson 33). A complete, data-backed, safely-iterable LLM engineering lifecycle, sealed tight.

🎯 Design trade-off
Why use "authenticated encryption" like AES-256-GCM rather than plain symmetric encryption? Because for data like an API key, confidentiality alone isn't enough—you also need anti-tampering. Plain encryption (e.g. AES-CBC) guarantees "others can't read the ciphertext", but can't stop someone quietly altering a few bytes—after which you might decrypt to another seemingly-valid garbage string, failing silently. GCM mode computes an authTag (authentication tag) alongside encryption, verified on decrypt: flip even one bit of the ciphertext and verification fails, rejecting it outright. So you get not just "confidentiality" but "integrity guarantee"—the plaintext you get is either exactly what was stored, or an outright error, never a silently-tampered version. For credentials, where "a small error brews a big disaster", authenticated encryption is the floor.

Why a random IV per encryption, and why also store a masked displaySecretKey? The random IV (initialization vector) solves "the same plaintext encrypts to different ciphertext each time"—otherwise two identical keys would have identical ciphertext, and an attacker could instantly see "these two use the same key", leaking information. A fresh random IV makes ciphertext patternless. The displaySecretKey solves a very practical UX problem: in the UI you must recognize "which key this is", yet you shouldn't decrypt and expose the whole key just to display it. So a masked string (last few digits only) is stored separately—display the masked version, decrypt only to call—covering both security and usability. This discipline—"plaintext exists only in the instants of storing and igniting, the rest of the time only ciphertext and masks are seen"—is the paradigm for handling any sensitive credential.
🎯 Key points · Part 7 wrap-up
  • Playground = the same engine's interactive front desk: chatCompletionHandler reuses the exact fetchLLMCompletion core as evals (L29) and experiments (L36)—one engine, three consumers, credential decryption/provider adaptation written once.
  • API key encrypted storage: LlmApiKeys.secretKey stores ciphertext via AES-256-GCM (authenticated encryption), displaySecretKey a masked string for the UI, adapter abstracts the provider interface, baseURL hits self-hosted endpoints.
  • Authenticated encryption is anti-tamper: GCM's authTag lets decryption detect whether the ciphertext was altered—integrity on top of confidentiality, the floor for credential data.
  • Random IV + decrypt-on-use: a random IV per encryption makes identical plaintext differ in ciphertext, leaking nothing; plaintext exists only briefly at "store" and "just before call", with the DB and UI only ever seeing ciphertext / masks.
  • Reusable blocks + the loop: LlmSchema (structured output, echoing L29), LlmTool (function calls) are defined once and reused everywhere. Playground→versioning→experiments→label release→observe/evaluate/alert—the developer workflow closes into a complete loop.

🧪 Self-test - think about the design

1. What's the underlying relationship among Langfuse's Playground, LLM judge (Lesson 29), and prompt experiment (Lesson 36)?
  1. each has its own independent LLM-call implementation
  2. all three share the same fetchLLMCompletion core—one engine, three consumers; credential decryption, provider adaptation, structured output, tool calls all written once
  3. the Playground can only call OpenAI
  4. experiments don't call a real LLM
Show answer & explanation click to expand
Answer: B. The playground's chatCompletionHandler directly imports and calls fetchLLMCompletion, passing tools and structuredOutputSchema—the same engine as Lesson 29's judge callLLM and Lesson 36's experiment model call. The Playground is just this engine's 'interactive front desk'. One encapsulation, three reuses, avoiding reinvention and ensuring consistent behavior across all three.
2. Langfuse uses AES-256-GCM (not plain symmetric encryption) to encrypt a provider's API key. What unique role does GCM mode's authTag play here?
  1. anti-tamper / integrity guarantee: decryption verifies the authTag, so flipping even one bit of the ciphertext fails verification and is rejected—ensuring the plaintext is either exactly what was stored or an error, never a silently-altered version
  2. lets multiple people share the key
  3. compresses the ciphertext
  4. makes encryption faster
Show answer & explanation click to expand
Answer: A. For data like an API key, confidentiality alone isn't enough—you need anti-tampering. Plain encryption (e.g. AES-CBC) can't stop someone altering a few bytes and decrypting to another seemingly-valid garbage, failing silently. GCM computes an authTag during encryption, verified on decrypt: flip one bit and it fails. So you get an 'integrity guarantee'—plaintext is either exactly as stored or an outright error. For credentials where a small error brews disaster, authenticated encryption is the floor.
3. The LlmApiKeys table stores both the encrypted secretKey and a separate masked displaySecretKey (like sk-…xyz). Why store this masked string too?
  1. the masked string is a backup of the encryption
  2. so the UI can recognize 'which key this is' without decrypting and exposing the whole key just to display it—show the masked version, decrypt only to call, covering both security and usability
  3. the masked string is what's actually used to call
  4. to speed up decryption
Show answer & explanation click to expand
Answer: B. It's a practical design under the discipline 'plaintext exists only at storing and ignition': in the UI you must recognize which key it is, but shouldn't decrypt and expose the whole key just to display. So a masked string showing only the last few digits is stored for display, with secretKey's plaintext decrypted only just before calling the provider and discarded after. Show masked, decrypt to call—security and UX both covered. This discipline is the paradigm for any sensitive credential.
💭 Open questions (no single right answer - just think or try)
  • By the end of Part 7, the whole 'developer workflow' loop closes: try in the Playground → version → experiment-compare → label release → observe/evaluate/alert. Reviewing this loop, how does it strikingly resemble the familiar 'software development lifecycle' (write code→test→CI→release→monitor), and which links are unique due to LLMs' 'non-determinism'? If you rolled out this prompt-engineering process to a team, which step would be hardest to adopt, and why?