In the updated Gymnasium environment interface, the distinction between “terminated” and “truncated” provides more clarity on why an episode ended, which is useful for more nuanced reinforcement learning […]

先来回忆一下几个变量的定义，Policy Gradient的关键是通过Gradient来更新Policy $$\theta_{k+1} = \theta_{k} + a \nabla _{\theta}J(\pi_{\theta})|_{\theta_k}$$ 其中$\pi_{\theta}$是参数话的policy，$\theta$是它的系数，$J(\pi_{\theta})$用来衡量当前policy $\pi_{\theta}$的性能，咱们这里用$\pi_{\theta}$的期望收益$E_{\tau \sim \pi_{\theta}}[R(\tau)]$作为policy的性能，$R(\tau)$表示一局游戏的收益,$\tau \sim \pi_{\theta}$表示是在当前policy $\pi_{\theta}$下。 $\nabla _{\theta}J(\pi_{\theta})$等于下面这一串 $$\nabla _{\theta}J(\pi_{\theta}) = E_{\tau \sim \pi_{\theta}} \left [ \sum_{t=0}^{T} \nabla_{\theta} […]