PKU-DAIR
/
Hetu
Not watched
1
0
Fork 0
Code
Releases 0 Wiki evaluate Activity
Issues 0 Pull Requests 0 Datasets Model Cloudbrain HPC
You can not select more than 25 topics Topics must start with a chinese character,a letter or number, can include dashes ('-') and can be up to 35 characters long.
24 Commits
1 Branch
116 MB
0 Download
 
 
 
 
 
 
Tree: e0fd6f3fc7
Branches Tags
${ item.name }
Create branch ${ searchTerm }
from 'e0fd6f3fc7'
${ noResults }
Hetu/examples/runner
Hsword 7fd3de80ce initial commit 4 years ago
..
models initial commit 4 years ago
parallel initial commit 4 years ago
README.md initial commit 4 years ago
local_allreduce.yml initial commit 4 years ago
local_ps.yml initial commit 4 years ago
remote_allreduce.yml initial commit 4 years ago
remote_ps.yml initial commit 4 years ago
run_mlp.py initial commit 4 years ago
run_wdl.py initial commit 4 years ago

README.md

Usage

This directory contains examples using heturun command.

  • Data Parallel (MLP model and WDL model):
# Local Data Parallel Using AllReduce
heturun -c local_allreduce.yml python run_mlp.py --config lar

# Local Data Parallel Using AllReduce for Dense Parameters and PS for Sparse(Embedding) Parameters
heturun -c local_ps.yml python run_wdl.py --config lhy

# Local Data Parallel Using PS
heturun -c local_ps.yml python run_mlp.py --config lps
heturun -c local_ps.yml python run_wdl.py --config lps

# Distributed Data Parallel Using AllReduce
heturun -c remote_allreduce.yml python run_mlp.py --config rar

# Distributed Data Parallel Using AllReduce for Dense Parameters and PS for Sparse(Embedding) Parameters
heturun -c remote_ps.yml python run_wdl.py --config rhy

# Distributed Data Parallel Using PS
heturun -c remote_ps.yml python run_mlp.py --config rps
heturun -c remote_ps.yml python run_wdl.py --config rps
  • For other parallel schemes, please refer to parallel directory.
Delete task

Are you sure you want to delete this task? Once this task is deleted, it cannot be recovered.

cancel
confirm

分布式深度学习系统

Python C++ Cuda C CMake other