Skip to content跳至正文
Harry Xin
Projects项目索引09
RLCard07 / 09
← All work← 全部项目REINFORCEMENT LEARNING强化学习研究

LEDUC POKER / REWARD STUDYLEDUC 扑克 / 奖励研究

RLCard

Change the reward.
Question the learning.
改变奖励,
学习会怎样改变?

A series of poker-AI experiments examining reward scale, reward shaping and learning rate, with results, methods and limitations published together.在扑克 AI 环境里,逐项考察奖励缩放、奖励塑形和学习率,并一起公开结果、方法和限制。

Read the full research report阅读完整研究报告Author’s GitHub作者 GitHubExplore ↓往下看 ↓
4 CONDITIONS / 8 SEEDS EACH

Same table.
Different rewards.
同一张桌,
不同的奖励。

Select a condition to inspect its published mean.点选一种条件,查看已公开的平均值。

Terminal终局奖励AUC 1.187

OPEN THE LAB NOTEBOOK翻开实验记录

More reward
is not a result.
奖励的数字变大,
学习不一定更好。

Changing a reward can also change the effective optimization problem. This study compares reward conditions in a bounded poker environment and follows up with mechanism and learning-rate screens.改变奖励,也可能影响实际优化过程。研究先在有限的扑克环境里比较奖励条件,再通过机制与学习率实验继续检查。

Published data, not a playable poker game.这是已公开的研究数据,不是可玩的扑克游戏。

RL / MECHANISM STUDYn = 8 / ARM

EXACT EXPLOITABILITY · LOWER IS BETTER精确可利用度 · 越低越好

Terminal终局奖励1.187
Scaled缩放奖励1.330
Scaled + PBRS缩放 + PBRS1.345
Unscaled + PBRS未缩放 + PBRS1.166

Published means · 32 runs · exploratory, not confirmatory已公开的平均值 · 32 次运行 · 探索性,非验证性

Compare the results.
Keep the caveats beside them.
把结果放在一起,
也把限制放在旁边。

The public mechanism report contains four conditions, eight seeds per condition and 32 completed runs. The chart above reads its published means directly; lower exploitability is better.公开机制报告包含 4 种条件、每种 8 个随机种子,共 32 次完成运行。上方图表直接读取报告中的平均值;可利用度越低越好。

04conditions种条件08seeds each个种子 / 条件32completed runs次完成运行
  1. §1

    Hold the comparison still建立可比条件

    Compare conditions within the same environment and retain per-seed results instead of reporting only a favorable run.在同一环境中比较不同条件,保留各随机种子的结果,避免只报告一次有利运行。

  2. §2

    Follow the mechanism继续检查机制

    Separate terminal reward, scaled reward and potential-based shaping conditions. Read them alongside the learning-rate follow-up.区分终局奖励、缩放奖励与势函数塑形条件,并结合后续学习率实验阅读。

  3. §3

    Publish the boundary公开结论的边界

    Use exact exploitability in Leduc poker and identify the mechanism screen as post-outcome exploratory analysis.使用 Leduc poker 的精确可利用度指标,并明确机制实验属于结果之后开展的探索性分析。

A result worth reading.
A method worth questioning.
结果值得阅读,
方法也值得追问。

Next project下一个项目MOCI