Skip to content

fix: correct EpisodeReplayBuffer default max_ep_len from 24 to 48#6

Open
idealiu555 wants to merge 1 commit into
alimama-tech:mainfrom
idealiu555:fix_correct_max_ep_len
Open

fix: correct EpisodeReplayBuffer default max_ep_len from 24 to 48#6
idealiu555 wants to merge 1 commit into
alimama-tech:mainfrom
idealiu555:fix_correct_max_ep_len

Conversation

@idealiu555

@idealiu555 idealiu555 commented Jun 15, 2026

Copy link
Copy Markdown

Problem

strategy_train_env/bidding_train_env/baseline/dt/utils.py 文件中EpisodeReplayBuffer.__init__max_ep_len 默认值设为 24,与数据生成器以及提供数据集的 episode 长度不符。
训练数据生成器(train_data_generator.py:93)硬编码 timeStepIndexNum = 48,每条轨迹共 48 个时间步。
由于 utils.py 中存在截断逻辑:

timesteps[timesteps >= self.max_ep_len] = self.max_ep_len - 1

默认值 24 会将第 24–47 步的时间步全部截断为 23,导致后半段 episode 的 timestep embedding 全部相同,模型丧失对后半程时间进度的感知能力。

Fix

strategy_train_env/run/run_decision_transformer.py 文件中创建 EpisodeReplayBuffer 对象的 max_ep_len 从默认值 24 修正为 48,与数据生成逻辑以及提供的数据集对齐。

Added max_ep_len parameter to EpisodeReplayBuffer.
@idealiu555 idealiu555 changed the title Update replay buffer initialization with max_ep_len fix: correct EpisodeReplayBuffer default max_ep_len from 24 to 48 Jun 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant