Browse Source

[to #43115513] update quick_start doc and add pai-easynlp back

Link: https://code.alibaba-inc.com/Ali-MaaS/MaaS-lib/codereview/9655518

* update quick_start doc and add pai-easynlp back

* update doc with sndfile
master
wenmeng.zwm 3 years ago
parent
commit
638cdc632e
2 changed files with 75 additions and 29 deletions
  1. +74
    -26
      docs/source/quick_start.md
  2. +1
    -3
      requirements/nlp.txt

+ 74
- 26
docs/source/quick_start.md View File

@@ -1,70 +1,118 @@
# 快速开始
ModelScope Library目前支持tensorflow,pytorch深度学习框架进行模型训练、推理, 在Python 3.7+, Pytorch 1.8+, Tensorflow1.15,Tensorflow 2.x上测试可运行。 ModelScope Library目前支持tensorflow,pytorch深度学习框架进行模型训练、推理, 在Python 3.7+, Pytorch 1.8+, Tensorflow1.15,Tensorflow 2.x上测试可运行。


注: `语音相关`的功能仅支持 python3.7,tensorflow1.15的`linux`环境使用。 其他功能可以在windows、mac上安装使用。
**注: **`**语音相关**`**的功能仅支持 python3.7,tensorflow1.15的**`**linux**`**环境使用。 其他功能可以在windows、mac上安装使用。**


## python环境配置 ## python环境配置
首先,参考[文档](https://docs.anaconda.com/anaconda/install/) 安装配置Anaconda环境


首先,参考[文档](https://docs.anaconda.com/anaconda/install/) 安装配置Anaconda环境。
安装完成后,执行如下命令为modelscope library创建对应的python环境。 安装完成后,执行如下命令为modelscope library创建对应的python环境。

```shell ```shell
conda create -n modelscope python=3.7 conda create -n modelscope python=3.7
conda activate modelscope conda activate modelscope
``` ```

## 安装深度学习框架 ## 安装深度学习框架
* 安装pytorch[参考链接](https://pytorch.org/get-started/locally/)

- 安装pytorch[参考链接](https://pytorch.org/get-started/locally/)。

```shell ```shell
pip install torch torchvision
pip3 install torch torchvision torchaudio
``` ```
* 安装Tensorflow[参考链接](https://www.tensorflow.org/install/pip)

- 安装Tensorflow[参考链接](https://www.tensorflow.org/install/pip)。

```shell ```shell
pip install --upgrade tensorflow pip install --upgrade tensorflow
``` ```

## ModelScope library 安装 ## ModelScope library 安装


注: 如果在安装过程中遇到错误,请前往[常见问题](faq.md)查找解决方案。 注: 如果在安装过程中遇到错误,请前往[常见问题](faq.md)查找解决方案。


### pip安装 ### pip安装
执行如下命令:
执行如下命令可以安装所有领域依赖
```shell ```shell
pip install "modelscope[all]" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html
pip install "modelscope[cv,nlp,audio,multi-modal]" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html
``` ```


如需体验`语音功能`,请`额外`执行如下命令:
需体验`语音功能`,请执行如下命令:
```shell ```shell
pip install "modelscope[audio]" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html pip install "modelscope[audio]" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html
``` ```

如仅需体验CV功能,可执行如下命令安装依赖:
```shell
pip install "modelscope[cv]" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html
```

如仅需体验NLP功能,可执行如下命令安装依赖:
```shell
pip install "modelscope[nlp]" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html
```

如仅需体验多模态功能,可执行如下命令安装依赖:
```shell
pip install "modelscope[multi-modal]" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html
```
**注**:

1. `**语音相关**`**的功能仅支持 python3.7,tensorflow1.15的**`**linux**`**环境使用。 其他功能可以在windows、mac上安装使用。**

2. 语音领域中一部分模型使用了三方库SoundFile进行wav文件处理,**在Linux系统上用户需要手动安装SoundFile的底层依赖库libsndfile**,在Windows和MacOS上会自动安装不需要用户操作。详细信息可参考[SoundFile官网](https://github.com/bastibe/python-soundfile#installation)。以Ubuntu系统为>例,用户需要执行如下命令:

```shell
sudo apt-get update
sudo apt-get install libsndfile1
```

3. **CV功能使用需要安装mmcv-full, 请参考mmcv**[**安装手册**](https://github.com/open-mmlab/mmcv#installation)**进行安装**

### 使用源码安装 ### 使用源码安装
适合本地开发调试使用,修改源码后可以直接执行
下载源码可以直接clone代码到本地

适合本地开发调试使用,修改源码后可以直接执行。
ModelScope的源码可以直接clone到本地:

```shell ```shell
git clone git@gitlab.alibaba-inc.com:Ali-MaaS/MaaS-lib.git modelscope git clone git@gitlab.alibaba-inc.com:Ali-MaaS/MaaS-lib.git modelscope
cd modelscope
git fetch origin master git fetch origin master
git checkout master git checkout master
cd modelscope
``` ```
安装依赖并设置PYTHONPATH


安装依赖
如需安装所有依赖,请执行如下命令
```shell ```shell
pip install -e ".[all]" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html
export PYTHONPATH=`pwd`
pip install -e ".[cv,nlp,audio,multi-modal]" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html
``` ```
注: 6.30版本需要把cv、nlp、multi-modal领域依赖都装上,7.30号各个领域依赖会作为选装,用户需要使用哪个领域安装对应领域依赖即可。


如需使用语音功能,请执行如下命令安装语音功能所需依赖


如需体验`语音功能`,请单独执行如下命令:
```shell ```shell
pip install -e ".[audio]" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo
pip install -e ".[audio]" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html
``` ```


### 安装验证
安装成功后,可以执行如下命令进行验证安装是否正确
如仅需体验CV功能,可执行如下命令安装依赖:
```shell ```shell
python -c "from modelscope.pipelines import pipeline;print(pipeline('word-segmentation')('今天天气不错,适合 出去游玩'))"
{'output': '今天 天气 不错 , 适合 出去 游玩'}
pip install -e ".[cv]" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html
```
如仅需体验NLP功能,可执行如下命令安装依赖:
```shell
pip install -e ".[nlp]" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html
``` ```
## 推理


pipeline函数提供了简洁的推理接口,相关介绍和示例请参考[pipeline使用教程](tutorials/pipeline.md)
如仅需体验多模态功能,可执行如下命令安装依赖:
```shell
pip install -e ".[multi-modal]" -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html
```
###
### 安装验证


## 训练 & 评估
安装成功后,可以执行如下命令进行验证安装是否正确:


Trainer类提供了简洁的Finetuning和评估接口,相关介绍和示例请参考[Trainer使用教程](tutorials/trainer.md)
```shell
python -c "from modelscope.pipelines import pipeline;print(pipeline('word-segmentation')('今天天气不错,适合 出去游玩'))"
```

+ 1
- 3
requirements/nlp.txt View File

@@ -1,8 +1,6 @@
en_core_web_sm>=2.3.5 en_core_web_sm>=2.3.5
fairseq>=0.10.2 fairseq>=0.10.2
# temporarily remove pai-easynl due to its hard dependency scipy==1.5.4
# will be added back
# pai-easynlp
pai-easynlp
# rough-score was just recently updated from 0.0.4 to 0.0.7 # rough-score was just recently updated from 0.0.4 to 0.0.7
# which introduced compatability issues that are being investigated # which introduced compatability issues that are being investigated
rouge_score<=0.0.4 rouge_score<=0.0.4


Loading…
Cancel
Save