Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10814720

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10572162

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10549680

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10533214
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10533214

Features:
1. Refactor the directory structure of nlp models. All model files are placed into either the model folder or the task_model folder
2. Refactor all the comments to google style
3. Add detail comments to important tasks and nlp models, to list the description of the model, and its preprocessor&trainer
4. Model Exporting now supports a direct all to TorchModelExporter(no need to derive from it)
5. Refactor model save_pretrained method to support direct running(independent from trainer)
6. Remove the judgement of Model in the pipeline base class, to support outer register models running in our pipelines
7. Nlp trainer now has a NLPTrainingArguments class , user can pass arguments into the dataclass, and use it as a normal cfg_modify_fn, to simplify the operation of modify cfg.
8. Merge the BACKBONES and the MODELS, so user can get a backbone with the Model.from_pretrained call
9. Model.from_pretrained now support a task argument, so user can use a backbone and load it with a specific task class.
10. Support Preprocessor.from_pretrained method
11. Add standard return classes to important nlp tasks, so some of the pipelines and the models are independent now, the return values of the models will always be tensors, and the pipelines will take care of the conversion to numpy and the following stuffs.
12. Split the file of the nlp preprocessors, to make the dir structure more clear.

Bugs Fixing:
1. Fix a bug that lr_scheduler can be called earlier than the optimizer's step
2. Fix a bug that the direct call of Pipelines (not from pipeline(xxx)) throws error
3. Fix a bug that the trainer will not call the correct TaskDataset class
4. Fix a bug that the internal loading of dataset will throws error in the trainer class
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10490585

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10463992

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10434107
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10434107

    * [Update] update finetune_result_upload

* [Update] rename finetune_result_upload to model_dir_upload

* Merge branch 'master' into feat/upload_ckpt

* Merge branch 'master' into feat/upload_ckpt

* [Fix] fix import error

* [Fix] fix import error

* Merge branch 'master' into feat/upload_ckpt

* [Update] changes name to upload_folder and using tempfile to save repo

* Merge branch 'master' into feat/upload_ckpt

* [Fix] fix commit

* Merge branch 'master' into feat/upload_ckpt

* [Fix] fix format

* Merge branch 'master' into feat/upload_ckpt

* [Fix] add uuid after model created from upload ut

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10386689

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10189613

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/10106602

    * [to #44857956]fix: disable git command username/password prompt

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9736416

    * [to #44031139]fix: fix hub ci case delete test project failure

move access token to environment variable.
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9681287

    * [to #43909247]fix: access token in code, move to environment

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9672696

    * [to #43887377]fix: sdk api concurrent call snapshort download file  will conflict

添加list model查询api
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9446917

ci 测试迁移新的机器，并且在容器中运行，减小互相干扰的可能
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9427096

    * add docker ci script

未登录warn提示信息重构，只有遇到请求失败，exception时，如果cookie为空，提示用户login，单元测试用户修改成单独
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9283579

    * [to #42322933] add space dialog-state tracking pipeline 
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9227018

    * init

* token to ids

* add model

* model forward ready

* add intent

* intent preprocessor ready

* intent success

* merge master

* test with model hub

* add flake8

* update

* update

* update

* Merge branch 'master' into nlp/space/gen

* delete file about gen

* init

* fix flake8 bug

* [to #42322933] init

* bug fix

* [to #42322933] init

* update pipeline registry info

* Merge remote-tracking branch 'origin/master' into feat/nli

* [to #42322933] init

* [to #42322933] init

* modify forward

* [to #42322933] init

* generation ready

* init

* Merge branch 'master' into feat/zero_shot_classification

# Conflicts:
#	modelscope/preprocessors/__init__.py

* [to #42322933] bugfix

* [to #42322933] pre commit fix

* fill mask

* registry multi models on model and pipeline

* add tests

* test level >= 0

* local gen ready

* merge with master

* dialog modeling ready

* fix comments: rename and refactor AliceMindMLM; adjust pipeline

* space intent and modeling(generation) are ready

* bug fix

* add dep

* add dep

* support dst data processor

* merge with nlp/space/dst

* merge with master

* Merge remote-tracking branch 'origin' into feat/fill_mask

Conflicts:
	modelscope/models/nlp/__init__.py
	modelscope/pipelines/builder.py
	modelscope/pipelines/outputs.py
	modelscope/preprocessors/nlp.py
	requirements/nlp.txt

* merge with master

* merge with master 2/2

* fix comments

* fix isort for pre-commit check

* allow params pass to pipeline's __call__ method

* Merge remote-tracking branch 'origin/master' into feat/zero_shot_classification

* merge with nli task

* merge with sentiment_classification

* merge with zero_shot_classfication

* merge with fill_mask

* merge with space

* merge with master head

* Merge remote-tracking branch 'origin' into feat/fill_mask

Conflicts:
	modelscope/utils/constant.py

* fix: pipeline module_name from model_type to 'fill_mask' & fix merge bug

* unfiinished change

* fix bug

* unfinished

* unfinished

* revise modelhub dependency

* Merge branch 'feat/nlp_refactor' of http://gitlab.alibaba-inc.com/Ali-MaaS/MaaS-lib into feat/nlp_refactor

* add eval() to pipeline call

* add test level

* ut run passed

* add default args

* tmp

* merge master

* all ut passed

* remove an useless enum

* revert a mis modification

* revert a mis modification

* Merge commit 'ace8af92465f7d772f035aebe98967726655f12c' into feat/nlp

* commit 'ace8af92465f7d772f035aebe98967726655f12c':
  [to #42322933] Add cv-action-recongnition-pipeline to maas lib
  [to #42463204]  support Pil.Image for image_captioning_pipeline
  [to #42670107] restore pydataset test
  [to #42322933] add create if not exist and add(back) create model example         Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9130661
  [to #41474818]fix: fix errors in task name definition

# Conflicts:
#	modelscope/pipelines/builder.py
#	modelscope/utils/constant.py

* Merge branch 'feat/nlp' into feat/nlp_refactor

* feat/nlp:
  [to #42322933] Add cv-action-recongnition-pipeline to maas lib
  [to #42463204]  support Pil.Image for image_captioning_pipeline
  [to #42670107] restore pydataset test
  [to #42322933] add create if not exist and add(back) create model example         Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9130661
  [to #41474818]fix: fix errors in task name definition

# Conflicts:
#	modelscope/pipelines/builder.py

* fix compile bug

* refactor space

* Merge branch 'feat/nlp_refactor' of http://gitlab.alibaba-inc.com/Ali-MaaS/MaaS-lib into feat/nlp_refactor

* Merge remote-tracking branch 'origin' into feat/fill_mask

* fix

* pre-commit lint

* lint file

* lint file

* lint file

* update modelhub dependency

* lint file

* ignore dst_processor temporary

* solve comment: 1. change MaskedLMModelBase to MaskedLanguageModelBase 2. remove a useless import

* recommit

* remove MaskedLanguageModel from __all__

* Merge commit '1a0d4af55a2eee69d89633874890f50eda8f8700' into feat/nlp_refactor

* commit '1a0d4af55a2eee69d89633874890f50eda8f8700':
  [to #42322933] test level check         Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9143809
  [to #42322933] update nlp models name in metainfo         Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9134657

# Conflicts:
#	modelscope/metainfo.py

* update

* revert pipeline params update

* remove zeroshot

* update sequence classfication outpus

* merge with fill mask

* Merge remote-tracking branch 'origin' into feat/fill_mask

* fix

* init dialog state tracking

* fix flake8 warning of dst

* Merge remote-tracking branch 'origin/feat/fill_mask' into feat/nlp

* merge with master

* remove useless test.py

* add init

* merge nlp

* Merge remote-tracking branch 'origin/master' into feat/nlp

* remove unformatted space trainer

* Merge branch 'feat/nlp' into nlp/space/dst

* revise based on comment except chinease comment

* skip ci blocking

* change Chinese notes of space3.0 into English

* translate chinese comment to english

* add space to metainfo

* space dst pipeline is ready, but model's result is wrong

* merge feat/nlp

* merge with master

* change processor

* change example

* test case ready

* dst loacl ready

* update dst conf

* merge feat/nlp

* inform revise

* inherit bug fix

* init

* add 2 complete examples

* fix bug

* add test case

* merge with master

* modify model name

* add missing setting

* add outputs

* modify test level

* modify chinese comment

* remove useless doc

* merge feat/nlp

* Merge remote-tracking branch 'origin' into nlp/space/dst

* Merge branch 'feat/nlp' into nlp/space/dst

* dst test ready

* merge feat nlp

* space outputs normalization

* update dst

* merge feat nlp

* Merge branch 'master' into nlp/space/dst

* update requirement

* merge with master

* Merge remote-tracking branch 'origin/master' into nlp/space/dst

* formating output

* update requirements/nlp

* merge with master

* add test cases

* Merge remote-tracking branch 'origin/master' into nlp/space/dst

* merge with master

* login warn refactor, error message when exception, ut case to new user

* login warn refactor, error message when exception, ut case to new user

Repository在clone后执行 git lfs install初始化lfs支持，下载技术添加到header里
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9204291

    * [to #42879757 #42887254]fix: Repository initialize lfs after clone, add download times counter

修复测试bug
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9186775

    * [to #42849800 #42822853 #42822836 #42822791 #42822717 #42820011]fix: test bugs

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9159678

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9130661

合并model hub 代码
        Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9096493