yuze.zyz
acba1786b0
[to #42322933 ] Fix bug in UT daily
1. Fix bugs in daily test
2. Fix a bug that the updating of lr is before the first time of updating of optimizer
TODO this will still cause warnings when GA is above 1
3. Remove the judgement of mode in text-classification's preprocessor to fit the base trainer(Bug)
Update some regression bins to fit the preprocessor
4. Update the regression tool to let outer code modify atol and rtol
5. Add the default metric for text-classification task
6. Remove the useless ckpt conversion method in bert to avoid the requirement of tf when loading modeling_bert
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10430764
2 years ago
zhangzhicheng.zzc
d721fabb34
[to #42322933 ]bert with sequence classification / token classification/ fill mask refactor
1.新增支持原始bert模型(非easynlp的 backbone prefix版本)
2.支持bert的在sequence classification/fill mask /token classification上的backbone head形式
3.统一了sequence classification几个任务的pipeline到一个类
4.fill mask 支持backbone head形式
5.token classification的几个子任务(ner,word seg, part of speech)的preprocessor 统一到了一起TokenClassificationPreprocessor
6. sequence classification的几个子任务(single classification, pair classification)的preprocessor 统一到了一起SequenceClassificationPreprocessor
7. 改动register中 cls的group_key 赋值位置,之前的group_key在多个decorators的情况下,会被覆盖,obj_cls的group_key信息不正确
8. 基于backbone head形式将 原本group_key和 module同名的情况尝试做调整,如下在modelscope/pipelines/nlp/sequence_classification_pipeline.py 中
原本
@PIPELINES.register_module(
Tasks.sentiment_classification, module_name=Pipelines.sentiment_classification)
改成
@PIPELINES.register_module(
Tasks.text_classification, module_name=Pipelines.sentiment_classification)
相应的configuration.json也有改动,这样的改动更符合任务和pipline(子任务)的关系。
8. 其他相应改动为支持上述功能
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10041463
3 years ago
wenmeng.zwm
6808e9a301
[to #44902099 ] add license for framework files
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10189613
3 years ago
shuying.shu
a9deb3895c
[to #42322933 ] movie scene segmentation模型接入
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9872869
3 years ago
feiwu.yfw
2b64cf2bb6
[to #42322933 ]支持从dataset json文件中获取参数
* dataset json file add args
3 years ago
feiwu.yfw
39485426e7
[to #42322933 ]:fix msdataset
* 修复了zip文件不同打包模式下返回路径错误问题。
* 修复了替换了数据集文件重新下载时校验失败问题。
* 修复dataset oss文件在 REUSE 模式下重复下载的问题。
* 修复了csv数据集的meta json文件中某个split的meta和file字段都为''时加载所有split失败的问题。
* 修复了不同版本datasets路径不一致的问题。
3 years ago
xingjun.wxj
44033290d4
[to #42322933 ]MsDataset 支持上传数据集压缩包和meta
1. MsDataset支持upload数据文件(压缩包)
2. MsDataset支持clone和upload meta data
3. 使用MsDataset.load()下载数据集,支持web端显示数据集下载计数
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9831232
3 years ago
feiwu.yfw
35548bd492
[to #43875101 ]
msdataset add coco dataset
unify taskdataset and ms dataset
fix hf datasets
3 years ago
feiwu.yfw
743e876981
[to #43660556 ] msdataset数据集加载
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9552632
* load csv dataset from modelscoop
3 years ago
feiwu.yfw
2c3875c0e1
[to #43299989 ] Fix msdataset
* fix msdataset
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9436292
* fix msdataset
3 years ago
feiwu.yfw
5da470fd5d
[to #42791465 , #42779255 , #42777959 , #42757844 , #42756050 , #42746916 , #42743595 , #42791863 ] fix: fix msdataset
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9174075
* fix msdataset
3 years ago
yingda.chen
6702b29e21
[to #42794773 ]rename pydataset to msdataset
Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9165402
3 years ago