wangshuide2020
7877f33b70
Use multiple processes to calc events.
1. To accelerate summary file parsing, multiple processes are used. As the first step to mindinsight parsing performance optimization, we only made changes to _load_single_file function.
2. This PR will imporve summary parsing throughput dramatically (about cpu_count times)
3. Changes are mainly about _load_single_file function
In the future, a more global concurrent computing framework is needed for mindinsight. See the gitee wiki doc for details.
5 years ago
luopengting
9e6f852c5d
delete lineage cache when parse failed
5 years ago
liangyongxiong
4913543f31
add robust check for summary watcher in case of FileNotFound exception
5 years ago
liangyongxiong
0eae537da0
fix bugs of cache mechanism
5 years ago
liangyongxiong
34648a48d2
update cache status on loading
5 years ago
liangyongxiong
8893236417
compare scalars within multiple train jobs
5 years ago
mindspore-ci-bot
6f599aa191
!134 modify lineage parsing for multi lineage files
Merge pull request !134 from luopengting/lineage_parsing
5 years ago
luopengting
665f2d680a
modify lineage parsing for multi lineage files, modify ut/st
5 years ago
wenkai
3cb0fbbbef
fix detail_cache.has_content works not properly (always return the wrong value false)
5 years ago
mindspore-ci-bot
2f28388be4
!120 establish a caching mechanism for lineage
Merge pull request !120 from luopengting/lineage_cache_datamgr
5 years ago
luopengting
6963af9374
add lineage cache, add update method and put api, modify ut/st
5 years ago
wenkai
c63a32b0bb
log exception raised by new thread to help debugging
5 years ago
wenkai
7e17d6ffae
refactor data manager and unify cache and data access/reload
5 years ago
ougongchang
a807c45a4a
add some exceptions, such as TrainJobNotExistError, GraphNotExistError and so on
5 years ago
gaocongli
e7a0496e87
initial version
5 years ago