Change the reward. Question the learning.改变奖励, 学习会怎样改变?
A series of poker-AI experiments examining reward scale, reward shaping and learning rate, with results, methods and limitations published together.在扑克 AI 环境里,逐项考察奖励缩放、奖励塑形和学习率,并一起公开结果、方法和限制。
Select a condition to inspect its published mean.点选一种条件,查看已公开的平均值。
Terminal终局奖励AUC 1.187
OPEN THE LAB NOTEBOOK翻开实验记录
More reward is not a result.奖励的数字变大, 学习不一定更好。
Changing a reward can also change the effective optimization problem. This study compares reward conditions in a bounded poker environment and follows up with mechanism and learning-rate screens.改变奖励,也可能影响实际优化过程。研究先在有限的扑克环境里比较奖励条件,再通过机制与学习率实验继续检查。
HARRY XIN / Python · RLCard · Leduc poker
Published data, not a playable poker game.这是已公开的研究数据,不是可玩的扑克游戏。
RL / MECHANISM STUDYn = 8 / ARM
K♠Q♥J♣REWARD DESIGN
EXACT EXPLOITABILITY · LOWER IS BETTER精确可利用度 · 越低越好
Terminal终局奖励1.187
Scaled缩放奖励1.330
Scaled + PBRS缩放 + PBRS1.345
Unscaled + PBRS未缩放 + PBRS1.166
00.751.50
Published means · 32 runs · exploratory, not confirmatory已公开的平均值 · 32 次运行 · 探索性,非验证性
♣
Compare the results. Keep the caveats beside them.把结果放在一起, 也把限制放在旁边。
The public mechanism report contains four conditions, eight seeds per condition and 32 completed runs. The chart above reads its published means directly; lower exploitability is better.公开机制报告包含 4 种条件、每种 8 个随机种子,共 32 次完成运行。上方图表直接读取报告中的平均值;可利用度越低越好。
Compare conditions within the same environment and retain per-seed results instead of reporting only a favorable run.在同一环境中比较不同条件,保留各随机种子的结果,避免只报告一次有利运行。
§2
Follow the mechanism继续检查机制
Separate terminal reward, scaled reward and potential-based shaping conditions. Read them alongside the learning-rate follow-up.区分终局奖励、缩放奖励与势函数塑形条件,并结合后续学习率实验阅读。
§3
Publish the boundary公开结论的边界
Use exact exploitability in Leduc poker and identify the mechanism screen as post-outcome exploratory analysis.使用 Leduc poker 的精确可利用度指标,并明确机制实验属于结果之后开展的探索性分析。
A result worth reading. A method worth questioning.结果值得阅读, 方法也值得追问。