Skip to content

Commit 7eeea95

Browse files
authored
Update learning rate for #116 (#117)
1 parent bd8707f commit 7eeea95

File tree

1 file changed

+4
-2
lines changed

1 file changed

+4
-2
lines changed

machine/jobs/settings.yaml

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,15 +8,17 @@ default:
88
train_params:
99
do_train: true
1010
optim: adamw_torch
11-
warmup_steps: 4000
11+
warmup_steps: 1000
1212
per_device_train_batch_size: 16
1313
gradient_accumulation_steps: 4
1414
label_smoothing_factor: 0.2
1515
group_by_length: true
1616
gradient_checkpointing: true
17+
lr_scheduler_type: cosine
18+
learning_rate: 0.0002
1719
fp16: true
1820
save_strategy: no
19-
max_steps: 20000
21+
max_steps: 5000
2022
generate_params:
2123
device: 0
2224
num_beams: 2

0 commit comments

Comments
 (0)