FazBrowse GitHub Viewer
|
Trending
|
URL:
|
Home
Tools:
[Download Repo ZIP]
[View Raw Code]
[Original HTTPS Page]
Medusa/deepspeed.json at main · FasterDecoding/Medusa · GitHub
Uh oh!
There was an error while loading.
Please reload this page
.
FasterDecoding
/
Medusa
Public
Notifications
You must be signed in to change notification settings
Fork
205
Star
2.8k
Code
Issues
51
Pull requests
6
Discussions
Actions
Projects
Security and quality
0
Insights
Additional navigation options
Code
Issues
Pull requests
Discussions
Actions
Projects
Security and quality
Insights
Expand file tree
Breadcrumbs
Medusa
/
deepspeed.json
Copy path
More file actions
More file actions
Latest commit
History
History
History
24 lines (22 loc) · 672 Bytes
Breadcrumbs
Medusa
/
deepspeed.json
Copy path
File metadata and controls
24 lines (22 loc) · 672 Bytes
Raw
Copy raw file
Download raw file
Open symbols panel
Edit and raw actions
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
{
"bf16"
: {
"enabled"
:
"
auto
"
},
"zero_optimization"
: {
"stage"
:
3
,
"overlap_comm"
:
true
,
"contiguous_gradients"
:
true
,
"sub_group_size"
:
1e9
,
"reduce_bucket_size"
:
"
auto
"
,
"stage3_prefetch_bucket_size"
:
"
auto
"
,
"stage3_param_persistence_threshold"
:
"
auto
"
,
"stage3_max_live_parameters"
:
1e9
,
"stage3_max_reuse_distance"
:
1e9
,
"stage3_gather_16bit_weights_on_model_save"
:
true
},
"gradient_accumulation_steps"
:
"
auto
"
,
"steps_per_print"
:
2000
,
"train_batch_size"
:
"
auto
"
,
"train_micro_batch_size_per_gpu"
:
"
auto
"
,
"wall_clock_breakdown"
:
false
}
Back
|
FazBrowse Home
|
New Git URL