diff --git a/README.md b/README.md
index bb62fc38..f4a8f2a6 100644
--- a/README.md
+++ b/README.md
@@ -6,94 +6,108 @@
![Hex.pm](https://img.shields.io/hexpm/l/plug.svg)
[![Documentation Status](https://readthedocs.org/projects/fastnlp/badge/?version=latest)](http://fastnlp.readthedocs.io/?badge=latest)
-FastNLP is a modular Natural Language Processing system based on PyTorch, built for fast development of NLP models.
+fastNLP 是一款轻量级的 NLP 处理套件。你既可以使用它快速地完成一个命名实体识别(NER)、中文分词或文本分类任务; 也可以使用他构建许多复杂的网络模型,进行科研。它具有如下的特性:
+
+- 统一的Tabular式数据容器,让数据预处理过程简洁明了。内置多种数据集的DataSet Loader,省去预处理代码。
+- 各种方便的NLP工具,例如预处理embedding加载; 中间数据cache等;
+- 详尽的中文文档以供查阅;
+- 提供诸多高级模块,例如Variational LSTM, Transformer, CRF等;
+- 封装CNNText,Biaffine等模型可供直接使用;
+- 便捷且具有扩展性的训练器; 提供多种内置callback函数,方便实验记录、异常捕获等。
+
+
+## 安装指南
+
+fastNLP 依赖如下包:
+
++ numpy
++ torch>=0.4.0
++ tqdm
++ nltk
+
+其中torch的安装可能与操作系统及 CUDA 的版本相关,请参见 PyTorch 官网 。
+在依赖包安装完成的情况,您可以在命令行执行如下指令完成安装
+
+```shell
+pip install fastNLP
+```
+
+
+## 内置组件
+
+大部分用于的 NLP 任务神经网络都可以看做由编码(encoder)、聚合(aggregator)、解码(decoder)三种模块组成。
+
+
+![](./docs/source/figures/text_classification.png)
+
+fastNLP 在 modules 模块中内置了三种模块的诸多组件,可以帮助用户快速搭建自己所需的网络。 三种模块的功能和常见组件如下:
-A deep learning NLP model is the composition of three types of modules:
- module type |
- functionality |
- example |
+ 类型 |
+ 功能 |
+ 例子 |
encoder |
- encode the input into some abstract representation |
+ 将输入编码为具有具 有表示能力的向量 |
embedding, RNN, CNN, transformer
|
aggregator |
- aggregate and reduce information |
+ 从多个向量中聚合信息 |
self-attention, max-pooling |
decoder |
- decode the representation into the output |
+ 将具有某种表示意义的 向量解码为需要的输出 形式 |
MLP, CRF |
-For example:
-
-![](docs/source/figures/text_classification.png)
-
-## Requirements
-
-- Python>=3.6
-- numpy>=1.14.2
-- torch>=0.4.0
-- tensorboardX
-- tqdm>=4.28.1
+## 完整模型
+fastNLP 为不同的 NLP 任务实现了许多完整的模型,它们都经过了训练和测试。
-## Resources
+你可以在以下两个地方查看相关信息
+- [介绍](reproduction/)
+- [源码](fastNLP/models/)
-- [Tutorials](https://github.com/fastnlp/fastNLP/tree/master/tutorials)
-- [Documentation](https://fastnlp.readthedocs.io/en/latest/)
-- [Source Code](https://github.com/fastnlp/fastNLP)
+## 项目结构
+![](./docs/source/figures/workflow.png)
-## Installation
-Run the following commands to install fastNLP package.
-```shell
-pip install fastNLP
-```
-
-## Models
-fastNLP implements different models for variant NLP tasks.
-Each model has been trained and tested carefully.
-
-Check out models' performance, usage and source code here.
-- [Documentation](reproduction/)
-- [Source Code](fastNLP/models/)
-
-## Project Structure
+fastNLP的大致工作流程如上图所示,而项目结构如下:
fastNLP |
- an open-source NLP library |
-
-
- fastNLP.api |
- APIs for end-to-end prediction |
+ 开源的自然语言处理库 |
fastNLP.core |
- data representation & train/test procedure |
+ 实现了核心功能,包括数据处理组件、训练器、测速器等 |
fastNLP.models |
- a collection of NLP models |
+ 实现了一些完整的神经网络模型 |
fastNLP.modules |
- a collection of PyTorch sub-models/components/wheels |
+ 实现了用于搭建神经网络模型的诸多组件 |
fastNLP.io |
- readers & savers |
+ 实现了读写功能,包括数据读入,模型读写等 |
+## 参考资源
+
+- [教程](https://github.com/fastnlp/fastNLP/tree/master/tutorials)
+- [文档](https://fastnlp.readthedocs.io/en/latest/)
+- [源码](https://github.com/fastnlp/fastNLP)
+
+
*In memory of @FengZiYjun. May his soul rest in peace. We will miss you very very much!*
\ No newline at end of file
diff --git a/fastNLP/io/dataset_loader.py b/fastNLP/io/dataset_loader.py
index b820af44..8273d2f8 100644
--- a/fastNLP/io/dataset_loader.py
+++ b/fastNLP/io/dataset_loader.py
@@ -214,27 +214,27 @@ class ConllLoader(DataSetLoader):
11:N Predicate Arguments
N Coreference
- :param headers: 每一列数据的名称,需为List or Tuple of str。``header`` 与 ``indexs`` 一一对应
- :param indexs: 需要保留的数据列下标,从0开始。若为 ``None`` ,则所有列都保留。Default: ``None``
+ :param headers: 每一列数据的名称,需为List or Tuple of str。``header`` 与 ``indexes`` 一一对应
+ :param indexes: 需要保留的数据列下标,从0开始。若为 ``None`` ,则所有列都保留。Default: ``None``
:param dropna: 是否忽略非法数据,若 ``False`` ,遇到非法数据时抛出 ``ValueError`` 。Default: ``False``
"""
- def __init__(self, headers, indexs=None, dropna=False):
+ def __init__(self, headers, indexes=None, dropna=False):
super(ConllLoader, self).__init__()
if not isinstance(headers, (list, tuple)):
raise TypeError('invalid headers: {}, should be list of strings'.format(headers))
self.headers = headers
self.dropna = dropna
- if indexs is None:
- self.indexs = list(range(len(self.headers)))
+ if indexes is None:
+ self.indexes = list(range(len(self.headers)))
else:
- if len(indexs) != len(headers):
+ if len(indexes) != len(headers):
raise ValueError
- self.indexs = indexs
+ self.indexes = indexes
def load(self, path):
ds = DataSet()
- for idx, data in _read_conll(path, indexes=self.indexs, dropna=self.dropna):
+ for idx, data in _read_conll(path, indexes=self.indexes, dropna=self.dropna):
ins = {h: data[i] for i, h in enumerate(self.headers)}
ds.append(Instance(**ins))
return ds
diff --git a/fastNLP/modules/encoder/bert.py b/fastNLP/modules/encoder/bert.py
index e87f6f5d..e123fda6 100644
--- a/fastNLP/modules/encoder/bert.py
+++ b/fastNLP/modules/encoder/bert.py
@@ -224,9 +224,9 @@ class BertPooler(nn.Module):
class BertModel(nn.Module):
- """Bidirectional Embedding Representations from Transformers.
+ """BERT(Bidirectional Embedding Representations from Transformers).
- If you want to use pre-trained weights, please download from the following sources provided by pytorch-pretrained-BERT.
+ 如果你想使用预训练好的权重矩阵,请在以下网址下载.
sources::
'bert-base-uncased': "https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased.tar.gz",
@@ -238,13 +238,28 @@ class BertModel(nn.Module):
'bert-base-chinese': "https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-chinese.tar.gz",
- Construct a BERT model with pre-trained weights::
+ 用预训练权重矩阵来建立BERT模型::
model = BertModel.from_pretrained("path/to/weights/directory")
+ 用随机初始化权重矩阵来建立BERT模型::
+
+ model = BertModel()
+
+ :param int vocab_size: 词表大小,默认值为30522,为BERT English uncase版本的词表大小
+ :param int hidden_size: 隐层大小,默认值为768,为BERT base的版本
+ :param int num_hidden_layers: 隐藏层数,默认值为12,为BERT base的版本
+ :param int num_attention_heads: 多头注意力头数,默认值为12,为BERT base的版本
+ :param int intermediate_size: FFN隐藏层大小,默认值是3072,为BERT base的版本
+ :param str hidden_act: FFN隐藏层激活函数,默认值为``gelu``
+ :param float hidden_dropout_prob: FFN隐藏层dropout,默认值为0.1
+ :param float attention_probs_dropout_prob: Attention层的dropout,默认值为0.1
+ :param int max_position_embeddings: 最大的序列长度,默认值为512,
+ :param int type_vocab_size: 最大segment数量,默认值为2
+ :param int initializer_range: 初始化权重范围,默认值为0.02
"""
- def __init__(self, vocab_size,
+ def __init__(self, vocab_size=30522,
hidden_size=768,
num_hidden_layers=12,
num_attention_heads=12,
diff --git a/legacy/api/api.py b/legacy/api/api.py
index 0b5d3cd3..d5d1df6b 100644
--- a/legacy/api/api.py
+++ b/legacy/api/api.py
@@ -101,7 +101,7 @@ class ConllxDataLoader(ConllLoader):
indexs = [
1, 3, 6, 7,
]
- super(ConllxDataLoader, self).__init__(headers=headers, indexs=indexs)
+ super(ConllxDataLoader, self).__init__(headers=headers, indexes=indexs)
class API:
diff --git a/tutorials/README.md b/tutorials/README.md
index 1de342e6..83df2bb9 100644
--- a/tutorials/README.md
+++ b/tutorials/README.md
@@ -1,12 +1,7 @@
# fastNLP 教程
### 上手教程 Quick Start
-- 一分钟上手:`fastnlp_1min_tutorial.ipynb` [Click Here](https://github.com/fastnlp/fastNLP/tree/master/tutorials/fastnlp_1min_tutorial.ipynb)
-- 十分钟上手:`fastnlp_10min_tutorial.ipynb` [Click Here](https://github.com/fastnlp/fastNLP/tree/master/tutorials/fastnlp_10min_tutorial.ipynb)
+`quickstart.ipynb` [Click Here](https://github.com/fastnlp/fastNLP/tree/master/tutorials/quickstart.ipynb)
-### 进阶教程 Advanced Tutorial
-- `fastnlp_advanced_tutorial/advance_tutorial.ipynb` [Click Here](https://github.com/fastnlp/fastNLP/tree/master/tutorials/fastnlp_advanced_tutorial/advance_tutorial.ipynb)
-
-
-### 开发者指南 Developer Guide
-- `tutorial_for_developer.md` [Click Here](https://github.com/fastnlp/fastNLP/tree/master/tutorials/tutorial_for_developer.md)
+### 详细教程 Tutorial 1
+十分钟上手:`tutorial_1.ipynb` [Click Here](https://github.com/fastnlp/fastNLP/tree/master/tutorials/tutorial_1.ipynb)
diff --git a/tutorials/fastNLP_padding_tutorial.ipynb b/tutorials/fastNLP_padding_tutorial.ipynb
deleted file mode 100644
index 7dc50206..00000000
--- a/tutorials/fastNLP_padding_tutorial.ipynb
+++ /dev/null
@@ -1,370 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "code",
- "execution_count": 1,
- "metadata": {},
- "outputs": [
- {
- "name": "stderr",
- "output_type": "stream",
- "text": [
- "/Users/yh/miniconda2/envs/python3/lib/python3.6/site-packages/tqdm/autonotebook/__init__.py:14: TqdmExperimentalWarning: Using `tqdm.autonotebook.tqdm` in notebook mode. Use `tqdm.tqdm` instead to force console mode (e.g. in jupyter console)\n",
- " \" (e.g. in jupyter console)\", TqdmExperimentalWarning)\n"
- ]
- },
- {
- "data": {
- "text/plain": [
- "DataSet({'raw_sent': this is a bad idea . type=str,\n",
- "'label': 0 type=int,\n",
- "'word_str_lst': ['this', 'is', 'a', 'bad', 'idea', '.'] type=list,\n",
- "'words': [4, 2, 5, 6, 7, 3] type=list},\n",
- "{'raw_sent': it is great . type=str,\n",
- "'label': 1 type=int,\n",
- "'word_str_lst': ['it', 'is', 'great', '.'] type=list,\n",
- "'words': [8, 2, 9, 3] type=list})"
- ]
- },
- "execution_count": 1,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
- "source": [
- "# 假设有以下的DataSet, 这里只是为了举例所以只选择了两个sample\n",
- "import sys\n",
- "import os\n",
- "sys.path.append('/Users/yh/Desktop/fastNLP/fastNLP')\n",
- "\n",
- "from fastNLP import DataSet\n",
- "from fastNLP import Instance\n",
- "from fastNLP import Vocabulary\n",
- "\n",
- "dataset = DataSet()\n",
- "dataset.append(Instance(raw_sent='This is a bad idea .', label=0))\n",
- "dataset.append(Instance(raw_sent='It is great .', label=1))\n",
- "\n",
- "# 按照fastNLP_10min_tutorial.ipynb的步骤,对数据进行一些处理。这里为了演示padding操作,把field的名称做了一些改变\n",
- "dataset.apply(lambda x:x['raw_sent'].lower(), new_field_name='raw_sent')\n",
- "dataset.apply(lambda x:x['raw_sent'].split(), new_field_name='word_str_lst')\n",
- "\n",
- "# 建立Vocabulary\n",
- "word_vocab = Vocabulary()\n",
- "dataset.apply(lambda x:word_vocab.update(x['word_str_lst']))\n",
- "dataset.apply(lambda x:[word_vocab.to_index(word) for word in x['word_str_lst']], new_field_name='words')\n",
- "\n",
- "# 检查以下是否得到我们想要的结果了\n",
- "dataset[:2]"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 2,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "batch_x has: {'word_str_lst': array([list(['this', 'is', 'a', 'bad', 'idea', '.']),\n",
- " list(['it', 'is', 'great', '.'])], dtype=object), 'words': tensor([[4, 2, 5, 6, 7, 3],\n",
- " [8, 2, 9, 3, 0, 0]])}\n",
- "batch_y has: {'label': tensor([0, 1])}\n"
- ]
- },
- {
- "data": {
- "text/plain": [
- "'\"\\n结果中\\n Batch会对元素类型(元素即最内层的数据,raw_sent为str,word_str_lst为str,words为int, label为int)为int或者float的数据进行默认\\n padding,而非int或float的则不进行padding。但若每个Instance中该field为二维数据,也不进行padding。因为二维数据的padding涉及到\\n 两个维度的padding,不容易自动判断padding的形式。\\n'"
- ]
- },
- "execution_count": 2,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
- "source": [
- "# 将field设置为input或者target\n",
- "dataset.set_input('word_str_lst')\n",
- "dataset.set_input('words')\n",
- "dataset.set_target('label')\n",
- "\n",
- "# 使用Batch取出batch数据\n",
- "from fastNLP.core.batch import Batch\n",
- "from fastNLP.core.sampler import RandomSampler\n",
- "\n",
- "batch_iterator = Batch(dataset=dataset, batch_size=2, sampler=RandomSampler())\n",
- "for batch_x, batch_y in batch_iterator:\n",
- " print(\"batch_x has: \", batch_x)\n",
- " print(\"batch_y has: \", batch_y)\n",
- "\"\"\"\"\n",
- "结果中\n",
- " Batch会对元素类型(元素即最内层的数据,raw_sent为str,word_str_lst为str,words为int, label为int)为int或者float的数据进行默认\n",
- " padding,而非int或float的则不进行padding。但若每个Instance中该field为二维数据,也不进行padding。因为二维数据的padding涉及到\n",
- " 两个维度的padding,不容易自动判断padding的形式。\n",
- "\"\"\""
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 3,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "batch_x has: {'word_str_lst': array([list(['it', 'is', 'great', '.']),\n",
- " list(['this', 'is', 'a', 'bad', 'idea', '.'])], dtype=object), 'words': tensor([[ 8, 2, 9, 3, -100, -100],\n",
- " [ 4, 2, 5, 6, 7, 3]])}\n",
- "batch_y has: {'label': tensor([1, 0])}\n"
- ]
- }
- ],
- "source": [
- "# 所有的pad_val都默认为0,如果需要修改某一个field的默认pad值,可以通过DataSet.set_pad_val(field_name, pad_val)进行修改\n",
- "# 若需要将word的padding修改为-100\n",
- "dataset.set_pad_val('words', pad_val=-100)\n",
- "batch_iterator = Batch(dataset=dataset, batch_size=2, sampler=RandomSampler())\n",
- "for batch_x, batch_y in batch_iterator:\n",
- " print(\"batch_x has: \", batch_x)\n",
- " print(\"batch_y has: \", batch_y)\n",
- "# pad的值修改为-100了"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 4,
- "metadata": {},
- "outputs": [
- {
- "data": {
- "text/plain": [
- "DataSet({'raw_sent': this is a bad idea . type=str,\n",
- "'label': 0 type=int,\n",
- "'word_str_lst': ['this', 'is', 'a', 'bad', 'idea', '.'] type=list,\n",
- "'words': [4, 2, 5, 6, 7, 3] type=list,\n",
- "'char_str_lst': [['t', 'h', 'i', 's'], ['i', 's'], ['a'], ['b', 'a', 'd'], ['i', 'd', 'e', 'a'], ['.']] type=list,\n",
- "'chars': [[4, 9, 2, 5], [2, 5], [3], [10, 3, 6], [2, 6, 7, 3], [8]] type=list},\n",
- "{'raw_sent': it is great . type=str,\n",
- "'label': 1 type=int,\n",
- "'word_str_lst': ['it', 'is', 'great', '.'] type=list,\n",
- "'words': [8, 2, 9, 3] type=list,\n",
- "'char_str_lst': [['i', 't'], ['i', 's'], ['g', 'r', 'e', 'a', 't'], ['.']] type=list,\n",
- "'chars': [[2, 4], [2, 5], [11, 12, 7, 3, 4], [8]] type=list})"
- ]
- },
- "execution_count": 4,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
- "source": [
- "# 若需要使用二维padding或指定padding方式,可以通过设置该field的padder实现,下面以英文的character padding为例。在某些场景下,可能想要\n",
- "# 使用英文word的character作为特征,character的padding为二维padding,fastNLP默认只会进行一维padding。\n",
- "\n",
- "dataset.apply(lambda x: [[c for c in word] for word in x['word_str_lst']], new_field_name='char_str_lst')\n",
- "char_vocab = Vocabulary()\n",
- "dataset.apply(lambda x:[char_vocab.update(chars) for chars in x['char_str_lst']])\n",
- "dataset.apply(lambda x:[[char_vocab.to_index(c) for c in chars] for chars in x['char_str_lst']],new_field_name='chars')\n",
- "dataset[:2]"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 5,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "batch_x has: {'word_str_lst': array([list(['this', 'is', 'a', 'bad', 'idea', '.']),\n",
- " list(['it', 'is', 'great', '.'])], dtype=object), 'words': tensor([[ 4, 2, 5, 6, 7, 3],\n",
- " [ 8, 2, 9, 3, -100, -100]]), 'chars': array([list([[4, 9, 2, 5], [2, 5], [3], [10, 3, 6], [2, 6, 7, 3], [8]]),\n",
- " list([[2, 4], [2, 5], [11, 12, 7, 3, 4], [8]])], dtype=object)}\n",
- "batch_y has: {'label': tensor([0, 1])}\n"
- ]
- },
- {
- "data": {
- "text/plain": [
- "'\\n 其它field与之前的是相同的。chars因为存在两个维度需要padding,不能自动决定padding方式,所以直接输出了原始形式。\\n'"
- ]
- },
- "execution_count": 5,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
- "source": [
- "# 如果不针对二维的character指定padding方法\n",
- "dataset.set_input('chars')\n",
- "batch_iterator = Batch(dataset=dataset, batch_size=2, sampler=RandomSampler())\n",
- "for batch_x, batch_y in batch_iterator:\n",
- " print(\"batch_x has: \", batch_x)\n",
- " print(\"batch_y has: \", batch_y)\n",
- " \n",
- "\"\"\"\n",
- " 其它field与之前的是相同的。chars因为存在两个维度需要padding,不能自动决定padding方式,所以直接输出了原始形式。\n",
- "\"\"\""
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 6,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "batch_x has: {'word_str_lst': array([list(['this', 'is', 'a', 'bad', 'idea', '.']),\n",
- " list(['it', 'is', 'great', '.'])], dtype=object), 'words': tensor([[ 4, 2, 5, 6, 7, 3],\n",
- " [ 8, 2, 9, 3, -100, -100]]), 'chars': tensor([[[ 4, 9, 2, 5],\n",
- " [ 2, 5, 0, 0],\n",
- " [ 3, 0, 0, 0],\n",
- " [10, 3, 6, 0],\n",
- " [ 2, 6, 7, 3],\n",
- " [ 8, 0, 0, 0]],\n",
- "\n",
- " [[ 2, 4, 0, 0],\n",
- " [ 2, 5, 0, 0],\n",
- " [11, 12, 7, 3],\n",
- " [ 8, 0, 0, 0],\n",
- " [ 0, 0, 0, 0],\n",
- " [ 0, 0, 0, 0]]])}\n",
- "batch_y has: {'label': tensor([0, 1])}\n"
- ]
- },
- {
- "data": {
- "text/plain": [
- "'\\n chars被正确padding了\\n'"
- ]
- },
- "execution_count": 6,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
- "source": [
- "# 若要使用二维padding,需要手动设置padding方式\n",
- "from fastNLP.core.fieldarray import EngChar2DPadder\n",
- "dataset.set_padder('chars', EngChar2DPadder())\n",
- "batch_iterator = Batch(dataset=dataset, batch_size=2, sampler=RandomSampler())\n",
- "for batch_x, batch_y in batch_iterator:\n",
- " print(\"batch_x has: \", batch_x)\n",
- " print(\"batch_y has: \", batch_y)\n",
- " \n",
- "\"\"\"\n",
- " chars被正确padding了\n",
- "\"\"\""
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 7,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "batch_x has: {'raw_sent': ['this is a bad idea .', 'it is great . '], 'word_str_lst': array([list(['this', 'is', 'a', 'bad', 'idea', '.']),\n",
- " list(['it', 'is', 'great', '.'])], dtype=object), 'words': tensor([[ 4, 2, 5, 6, 7, 3],\n",
- " [ 8, 2, 9, 3, -100, -100]]), 'chars': tensor([[[ 4, 9, 2, 5],\n",
- " [ 2, 5, 0, 0],\n",
- " [ 3, 0, 0, 0],\n",
- " [10, 3, 6, 0],\n",
- " [ 2, 6, 7, 3],\n",
- " [ 8, 0, 0, 0]],\n",
- "\n",
- " [[ 2, 4, 0, 0],\n",
- " [ 2, 5, 0, 0],\n",
- " [11, 12, 7, 3],\n",
- " [ 8, 0, 0, 0],\n",
- " [ 0, 0, 0, 0],\n",
- " [ 0, 0, 0, 0]]])}\n",
- "batch_y has: {'label': tensor([0, 1])}\n"
- ]
- },
- {
- "data": {
- "text/plain": [
- "'\\n raw_sent正确输出,对应内容也进行了pad。\\n'"
- ]
- },
- "execution_count": 7,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
- "source": [
- "# 如果AutoPad与EngChar2DPadder不能满足需要,可以自己实现Padder对象。这里举一个例子,比如需要把raw_sentence pad到一样长\n",
- "from fastNLP.core.fieldarray import PadderBase\n",
- "\n",
- "class PadStr(PadderBase):\n",
- " def __init__(self, pad_val=' '):\n",
- " super().__init__(pad_val=pad_val) #让父类管理pad_val的值,这样可以通过DataSet.set_pad_val()修改到该值\n",
- " \n",
- " def __call__(self, contents, field_name, field_ele_dtype):\n",
- " \"\"\"\n",
- " 如果以上面的例子举例,在raw_sent这个field进行pad时,传入的\n",
- " contents:\n",
- " [\n",
- " 'This is a bad idea .',\n",
- " 'It is great .'\n",
- " ]\n",
- " field_name: 'raw_sent',当前field的名称,主要用于帮助debug。\n",
- " field_ele_dtype: np.str. 这个参数基本都用不上,是该field中内部元素的类型\n",
- " \"\"\"\n",
- " max_len = max([len(str_) for str_ in contents])\n",
- " pad_strs = []\n",
- " for content in contents:\n",
- " pad_strs.append(content + (max_len-len(content))*self.pad_val)\n",
- " return pad_strs\n",
- "\n",
- "dataset.set_input('raw_sent')\n",
- "dataset.set_padder('raw_sent', PadStr())\n",
- "batch_iterator = Batch(dataset=dataset, batch_size=2, sampler=RandomSampler())\n",
- "for batch_x, batch_y in batch_iterator:\n",
- " print(\"batch_x has: \", batch_x)\n",
- " print(\"batch_y has: \", batch_y)\n",
- "\n",
- "\"\"\"\n",
- " raw_sent正确输出,对应内容也进行了pad。\n",
- "\"\"\""
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": []
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": "Python 3",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.6.7"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}
diff --git a/tutorials/fastnlp_10min_tutorial.ipynb b/tutorials/fastnlp_10min_tutorial.ipynb
deleted file mode 100644
index 526fd49f..00000000
--- a/tutorials/fastnlp_10min_tutorial.ipynb
+++ /dev/null
@@ -1,751 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "fastNLP10 分钟上手教程\n",
- "-------\n",
- "\n",
- "fastNLP提供方便的数据预处理,训练和测试模型的功能"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "如果您还没有通过pip安装fastNLP,可以执行下面的操作加载当前模块"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 4,
- "metadata": {},
- "outputs": [],
- "source": [
- "import sys\n",
- "sys.path.append(\"../\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "DataSet & Instance\n",
- "------\n",
- "\n",
- "fastNLP用DataSet和Instance保存和处理数据。每个DataSet表示一个数据集,每个Instance表示一个数据样本。一个DataSet存有多个Instance,每个Instance可以自定义存哪些内容。\n",
- "\n",
- "有一些read_*方法,可以轻松从文件读取数据,存成DataSet。"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 1,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "77\n"
- ]
- }
- ],
- "source": [
- "from fastNLP import DataSet\n",
- "from fastNLP import Instance\n",
- "\n",
- "# 从csv读取数据到DataSet\n",
- "dataset = DataSet.read_csv('sample_data/tutorial_sample_dataset.csv', headers=('raw_sentence', 'label'), sep='\\t')\n",
- "print(len(dataset))"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 2,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "{'raw_sentence': A series of escapades demonstrating the adage that what is good for the goose is also good for the gander , some of which occasionally amuses but none of which amounts to much of a story . type=str,\n",
- "'label': 1 type=str}\n",
- "{'raw_sentence': The plot is romantic comedy boilerplate from start to finish . type=str,\n",
- "'label': 2 type=str}\n"
- ]
- }
- ],
- "source": [
- "# 使用数字索引[k],获取第k个样本\n",
- "print(dataset[0])\n",
- "\n",
- "# 索引也可以是负数\n",
- "print(dataset[-3])"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Instance\n",
- "Instance表示一个样本,由一个或多个field(域,属性,特征)组成,每个field有名字和值。\n",
- "\n",
- "在初始化Instance时即可定义它包含的域,使用 \"field_name=field_value\"的写法。"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 3,
- "metadata": {},
- "outputs": [
- {
- "data": {
- "text/plain": [
- "{'raw_sentence': fake data type=str,\n",
- "'label': 0 type=str}"
- ]
- },
- "execution_count": 3,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
- "source": [
- "# DataSet.append(Instance)加入新数据\n",
- "dataset.append(Instance(raw_sentence='fake data', label='0'))\n",
- "dataset[-1]"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## DataSet.apply方法\n",
- "数据预处理利器"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 4,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "{'raw_sentence': a series of escapades demonstrating the adage that what is good for the goose is also good for the gander , some of which occasionally amuses but none of which amounts to much of a story . type=str,\n",
- "'label': 1 type=str}\n"
- ]
- }
- ],
- "source": [
- "# 将所有数字转为小写\n",
- "dataset.apply(lambda x: x['raw_sentence'].lower(), new_field_name='raw_sentence')\n",
- "print(dataset[0])"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 5,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "{'raw_sentence': a series of escapades demonstrating the adage that what is good for the goose is also good for the gander , some of which occasionally amuses but none of which amounts to much of a story . type=str,\n",
- "'label': 1 type=int}\n"
- ]
- }
- ],
- "source": [
- "# label转int\n",
- "dataset.apply(lambda x: int(x['label']), new_field_name='label')\n",
- "print(dataset[0])"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 6,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "{'raw_sentence': a series of escapades demonstrating the adage that what is good for the goose is also good for the gander , some of which occasionally amuses but none of which amounts to much of a story . type=str,\n",
- "'label': 1 type=int,\n",
- "'words': ['a', 'series', 'of', 'escapades', 'demonstrating', 'the', 'adage', 'that', 'what', 'is', 'good', 'for', 'the', 'goose', 'is', 'also', 'good', 'for', 'the', 'gander', ',', 'some', 'of', 'which', 'occasionally', 'amuses', 'but', 'none', 'of', 'which', 'amounts', 'to', 'much', 'of', 'a', 'story', '.'] type=list}\n"
- ]
- }
- ],
- "source": [
- "# 使用空格分割句子\n",
- "def split_sent(ins):\n",
- " return ins['raw_sentence'].split()\n",
- "dataset.apply(split_sent, new_field_name='words')\n",
- "print(dataset[0])"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 7,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "{'raw_sentence': a series of escapades demonstrating the adage that what is good for the goose is also good for the gander , some of which occasionally amuses but none of which amounts to much of a story . type=str,\n",
- "'label': 1 type=int,\n",
- "'words': ['a', 'series', 'of', 'escapades', 'demonstrating', 'the', 'adage', 'that', 'what', 'is', 'good', 'for', 'the', 'goose', 'is', 'also', 'good', 'for', 'the', 'gander', ',', 'some', 'of', 'which', 'occasionally', 'amuses', 'but', 'none', 'of', 'which', 'amounts', 'to', 'much', 'of', 'a', 'story', '.'] type=list,\n",
- "'seq_len': 37 type=int}\n"
- ]
- }
- ],
- "source": [
- "# 增加长度信息\n",
- "dataset.apply(lambda x: len(x['words']), new_field_name='seq_len')\n",
- "print(dataset[0])"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## DataSet.drop\n",
- "筛选数据"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 8,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "77\n"
- ]
- }
- ],
- "source": [
- "# 删除低于某个长度的词语\n",
- "dataset.drop(lambda x: x['seq_len'] <= 3)\n",
- "print(len(dataset))"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## 配置DataSet\n",
- "1. 哪些域是特征,哪些域是标签\n",
- "2. 切分训练集/验证集"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 9,
- "metadata": {},
- "outputs": [],
- "source": [
- "# 设置DataSet中,哪些field要转为tensor\n",
- "\n",
- "# set target,loss或evaluate中的golden,计算loss,模型评估时使用\n",
- "dataset.set_target(\"label\")\n",
- "# set input,模型forward时使用\n",
- "dataset.set_input(\"words\")"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 10,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "54\n",
- "23\n"
- ]
- }
- ],
- "source": [
- "# 分出测试集、训练集\n",
- "\n",
- "test_data, train_data = dataset.split(0.3)\n",
- "print(len(test_data))\n",
- "print(len(train_data))"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Vocabulary\n",
- "------\n",
- "\n",
- "fastNLP中的Vocabulary轻松构建词表,将词转成数字"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 11,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "{'raw_sentence': the performances are an absolute joy . type=str,\n",
- "'label': 4 type=int,\n",
- "'words': [3, 1, 1, 26, 1, 1, 2] type=list,\n",
- "'seq_len': 7 type=int}\n"
- ]
- }
- ],
- "source": [
- "from fastNLP import Vocabulary\n",
- "\n",
- "# 构建词表, Vocabulary.add(word)\n",
- "vocab = Vocabulary(min_freq=2)\n",
- "train_data.apply(lambda x: [vocab.add(word) for word in x['words']])\n",
- "vocab.build_vocab()\n",
- "\n",
- "# index句子, Vocabulary.to_index(word)\n",
- "train_data.apply(lambda x: [vocab.to_index(word) for word in x['words']], new_field_name='words')\n",
- "test_data.apply(lambda x: [vocab.to_index(word) for word in x['words']], new_field_name='words')\n",
- "\n",
- "\n",
- "print(test_data[0])"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 12,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "batch_x has: {'words': tensor([[ 15, 72, 15, 73, 74, 7, 3, 75, 6, 3, 16, 16,\n",
- " 76, 2],\n",
- " [ 15, 72, 15, 73, 74, 7, 3, 75, 6, 3, 16, 16,\n",
- " 76, 2]])}\n",
- "batch_y has: {'label': tensor([ 1, 1])}\n"
- ]
- }
- ],
- "source": [
- "# 如果你们需要做强化学习或者GAN之类的项目,你们也可以使用这些数据预处理的工具\n",
- "from fastNLP.core.batch import Batch\n",
- "from fastNLP.core.sampler import RandomSampler\n",
- "\n",
- "batch_iterator = Batch(dataset=train_data, batch_size=2, sampler=RandomSampler())\n",
- "for batch_x, batch_y in batch_iterator:\n",
- " print(\"batch_x has: \", batch_x)\n",
- " print(\"batch_y has: \", batch_y)\n",
- " break"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# Model\n",
- "定义一个PyTorch模型"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 15,
- "metadata": {},
- "outputs": [
- {
- "data": {
- "text/plain": [
- "CNNText(\n",
- " (embed): Embedding(\n",
- " 77, 50\n",
- " (dropout): Dropout(p=0.0)\n",
- " )\n",
- " (conv_pool): ConvMaxpool(\n",
- " (convs): ModuleList(\n",
- " (0): Conv1d(50, 3, kernel_size=(3,), stride=(1,), padding=(2,))\n",
- " (1): Conv1d(50, 4, kernel_size=(4,), stride=(1,), padding=(2,))\n",
- " (2): Conv1d(50, 5, kernel_size=(5,), stride=(1,), padding=(2,))\n",
- " )\n",
- " )\n",
- " (dropout): Dropout(p=0.1)\n",
- " (fc): Linear(\n",
- " (linear): Linear(in_features=12, out_features=5, bias=True)\n",
- " )\n",
- ")"
- ]
- },
- "execution_count": 15,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
- "source": [
- "from fastNLP.models import CNNText\n",
- "model = CNNText((len(vocab), 50), num_classes=5, padding=2, dropout=0.1)\n",
- "model"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "这是上述模型的forward方法。如果你不知道什么是forward方法,请参考我们的PyTorch教程。\n",
- "\n",
- "注意两点:\n",
- "1. forward参数名字叫**word_seq**,请记住。\n",
- "2. forward的返回值是一个**dict**,其中有个key的名字叫**output**。\n",
- "\n",
- "```Python\n",
- " def forward(self, word_seq):\n",
- " \"\"\"\n",
- "\n",
- " :param word_seq: torch.LongTensor, [batch_size, seq_len]\n",
- " :return output: dict of torch.LongTensor, [batch_size, num_classes]\n",
- " \"\"\"\n",
- " x = self.embed(word_seq) # [N,L] -> [N,L,C]\n",
- " x = self.conv_pool(x) # [N,L,C] -> [N,C]\n",
- " x = self.dropout(x)\n",
- " x = self.fc(x) # [N,C] -> [N, N_class]\n",
- " return {'output': x}\n",
- "```"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "这是上述模型的predict方法,是用来直接输出该任务的预测结果,与forward目的不同。\n",
- "\n",
- "注意两点:\n",
- "1. predict参数名也叫**word_seq**。\n",
- "2. predict的返回值是也一个**dict**,其中有个key的名字叫**predict**。\n",
- "\n",
- "```\n",
- " def predict(self, word_seq):\n",
- " \"\"\"\n",
- "\n",
- " :param word_seq: torch.LongTensor, [batch_size, seq_len]\n",
- " :return predict: dict of torch.LongTensor, [batch_size, seq_len]\n",
- " \"\"\"\n",
- " output = self(word_seq)\n",
- " _, predict = output['output'].max(dim=1)\n",
- " return {'predict': predict}\n",
- "```"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Trainer & Tester\n",
- "------\n",
- "\n",
- "使用fastNLP的Trainer训练模型"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 16,
- "metadata": {},
- "outputs": [],
- "source": [
- "from fastNLP import Trainer\n",
- "from copy import deepcopy\n",
- "from fastNLP.core.losses import CrossEntropyLoss\n",
- "from fastNLP.core.metrics import AccuracyMetric\n",
- "\n",
- "\n",
- "# 更改DataSet中对应field的名称,与模型的forward的参数名一致\n",
- "# 因为forward的参数叫word_seq, 所以要把原本叫words的field改名为word_seq\n",
- "# 这里的演示是让你了解这种**命名规则**\n",
- "train_data.rename_field('words', 'word_seq')\n",
- "test_data.rename_field('words', 'word_seq')\n",
- "\n",
- "# 顺便把label换名为label_seq\n",
- "train_data.rename_field('label', 'label_seq')\n",
- "test_data.rename_field('label', 'label_seq')"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### loss\n",
- "训练模型需要提供一个损失函数\n",
- "\n",
- "下面提供了一个在分类问题中常用的交叉熵损失。注意它的**初始化参数**。\n",
- "\n",
- "pred参数对应的是模型的forward返回的dict的一个key的名字,这里是\"output\"。\n",
- "\n",
- "target参数对应的是dataset作为标签的field的名字,这里是\"label_seq\"。"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 17,
- "metadata": {},
- "outputs": [],
- "source": [
- "loss = CrossEntropyLoss(pred=\"output\", target=\"label_seq\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Metric\n",
- "定义评价指标\n",
- "\n",
- "这里使用准确率。参数的“命名规则”跟上面类似。\n",
- "\n",
- "pred参数对应的是模型的predict方法返回的dict的一个key的名字,这里是\"predict\"。\n",
- "\n",
- "target参数对应的是dataset作为标签的field的名字,这里是\"label_seq\"。"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 18,
- "metadata": {},
- "outputs": [],
- "source": [
- "metric = AccuracyMetric(pred=\"predict\", target=\"label_seq\")"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 19,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "input fields after batch(if batch size is 2):\n",
- "\tword_seq: (1)type:torch.Tensor (2)dtype:torch.int64, (3)shape:torch.Size([2, 11]) \n",
- "target fields after batch(if batch size is 2):\n",
- "\tlabel_seq: (1)type:torch.Tensor (2)dtype:torch.int64, (3)shape:torch.Size([2]) \n",
- "\n"
- ]
- },
- {
- "ename": "NameError",
- "evalue": "\nProblems occurred when calling CNNText.forward(self, words, seq_len=None)\n\tmissing param: ['words']\n\tunused field: ['word_seq']\n\tSuggestion: You need to provide ['words'] in DataSet and set it as input. ",
- "output_type": "error",
- "traceback": [
- "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
- "\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)",
- "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m()\u001b[0m\n\u001b[1;32m 7\u001b[0m \u001b[0msave_path\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 8\u001b[0m \u001b[0mbatch_size\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m32\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 9\u001b[0;31m n_epochs=5)\n\u001b[0m\u001b[1;32m 10\u001b[0m \u001b[0moverfit_trainer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtrain\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
- "\u001b[0;32m/Users/fdujyn/anaconda3/lib/python3.6/site-packages/fastNLP/core/trainer.py\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, train_data, model, optimizer, loss, batch_size, sampler, update_every, n_epochs, print_every, dev_data, metrics, metric_key, validate_every, save_path, prefetch, use_tqdm, device, callbacks, check_code_level)\u001b[0m\n\u001b[1;32m 447\u001b[0m _check_code(dataset=train_data, model=model, losser=losser, metrics=metrics, dev_data=dev_data,\n\u001b[1;32m 448\u001b[0m \u001b[0mmetric_key\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mmetric_key\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcheck_level\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcheck_code_level\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 449\u001b[0;31m batch_size=min(batch_size, DEFAULT_CHECK_BATCH_SIZE))\n\u001b[0m\u001b[1;32m 450\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 451\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtrain_data\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtrain_data\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
- "\u001b[0;32m/Users/fdujyn/anaconda3/lib/python3.6/site-packages/fastNLP/core/trainer.py\u001b[0m in \u001b[0;36m_check_code\u001b[0;34m(dataset, model, losser, metrics, batch_size, dev_data, metric_key, check_level)\u001b[0m\n\u001b[1;32m 808\u001b[0m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minfo_str\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 809\u001b[0m _check_forward_error(forward_func=model.forward, dataset=dataset,\n\u001b[0;32m--> 810\u001b[0;31m batch_x=batch_x, check_level=check_level)\n\u001b[0m\u001b[1;32m 811\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 812\u001b[0m \u001b[0mrefined_batch_x\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0m_build_args\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmodel\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mforward\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mbatch_x\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
- "\u001b[0;32m/Users/fdujyn/anaconda3/lib/python3.6/site-packages/fastNLP/core/utils.py\u001b[0m in \u001b[0;36m_check_forward_error\u001b[0;34m(forward_func, batch_x, dataset, check_level)\u001b[0m\n\u001b[1;32m 594\u001b[0m \u001b[0msugg_str\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0msuggestions\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 595\u001b[0m \u001b[0merr_str\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m'\\n'\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;34m'\\n'\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mjoin\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0merrs\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;34m'\\n\\tSuggestion: '\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0msugg_str\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 596\u001b[0;31m \u001b[0;32mraise\u001b[0m \u001b[0mNameError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0merr_str\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 597\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0m_unused\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 598\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mcheck_level\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0mWARNING_CHECK_LEVEL\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
- "\u001b[0;31mNameError\u001b[0m: \nProblems occurred when calling CNNText.forward(self, words, seq_len=None)\n\tmissing param: ['words']\n\tunused field: ['word_seq']\n\tSuggestion: You need to provide ['words'] in DataSet and set it as input. "
- ]
- }
- ],
- "source": [
- "# 实例化Trainer,传入模型和数据,进行训练\n",
- "# 先在test_data拟合(确保模型的实现是正确的)\n",
- "copy_model = deepcopy(model)\n",
- "overfit_trainer = Trainer(model=copy_model, train_data=test_data, dev_data=test_data,\n",
- " loss=loss,\n",
- " metrics=metric,\n",
- " save_path=None,\n",
- " batch_size=32,\n",
- " n_epochs=5)\n",
- "overfit_trainer.train()"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 22,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "input fields after batch(if batch size is 2):\n",
- "\tword_seq: (1)type:torch.Tensor (2)dtype:torch.int64, (3)shape:torch.Size([2, 20]) \n",
- "target fields after batch(if batch size is 2):\n",
- "\tlabel_seq: (1)type:torch.Tensor (2)dtype:torch.int64, (3)shape:torch.Size([2]) \n",
- "\n",
- "training epochs started 2019-01-12 17-09-05\n"
- ]
- },
- {
- "data": {
- "text/plain": [
- "HBox(children=(IntProgress(value=0, layout=Layout(flex='2'), max=5), HTML(value='')), layout=Layout(display='i…"
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- },
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Evaluation at Epoch 1/5. Step:1/5. AccuracyMetric: acc=0.37037\n",
- "Evaluation at Epoch 2/5. Step:2/5. AccuracyMetric: acc=0.37037\n",
- "Evaluation at Epoch 3/5. Step:3/5. AccuracyMetric: acc=0.462963\n",
- "Evaluation at Epoch 4/5. Step:4/5. AccuracyMetric: acc=0.425926\n",
- "Evaluation at Epoch 5/5. Step:5/5. AccuracyMetric: acc=0.481481\n",
- "\n",
- "In Epoch:5/Step:5, got best dev performance:AccuracyMetric: acc=0.481481\n",
- "Reloaded the best model.\n",
- "Train finished!\n"
- ]
- }
- ],
- "source": [
- "# 用train_data训练,在test_data验证\n",
- "trainer = Trainer(model=model, train_data=train_data, dev_data=test_data,\n",
- " loss=CrossEntropyLoss(pred=\"output\", target=\"label_seq\"),\n",
- " metrics=AccuracyMetric(pred=\"predict\", target=\"label_seq\"),\n",
- " save_path=None,\n",
- " batch_size=32,\n",
- " n_epochs=5)\n",
- "trainer.train()\n",
- "print('Train finished!')"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 23,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "[tester] \n",
- "AccuracyMetric: acc=0.481481\n",
- "{'AccuracyMetric': {'acc': 0.481481}}\n"
- ]
- }
- ],
- "source": [
- "# 调用Tester在test_data上评价效果\n",
- "from fastNLP import Tester\n",
- "\n",
- "tester = Tester(data=test_data, model=model, metrics=AccuracyMetric(pred=\"predict\", target=\"label_seq\"),\n",
- " batch_size=4)\n",
- "acc = tester.test()\n",
- "print(acc)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# In summary\n",
- "\n",
- "## fastNLP Trainer的伪代码逻辑\n",
- "### 1. 准备DataSet,假设DataSet中共有如下的fields\n",
- " ['raw_sentence', 'word_seq1', 'word_seq2', 'raw_label','label']\n",
- " 通过\n",
- " DataSet.set_input('word_seq1', word_seq2', flag=True)将'word_seq1', 'word_seq2'设置为input\n",
- " 通过\n",
- " DataSet.set_target('label', flag=True)将'label'设置为target\n",
- "### 2. 初始化模型\n",
- " class Model(nn.Module):\n",
- " def __init__(self):\n",
- " xxx\n",
- " def forward(self, word_seq1, word_seq2):\n",
- " # (1) 这里使用的形参名必须和DataSet中的input field的名称对应。因为我们是通过形参名, 进行赋值的\n",
- " # (2) input field的数量可以多于这里的形参数量。但是不能少于。\n",
- " xxxx\n",
- " # 输出必须是一个dict\n",
- "### 3. Trainer的训练过程\n",
- " (1) 从DataSet中按照batch_size取出一个batch,调用Model.forward\n",
- " (2) 将 Model.forward的结果 与 标记为target的field 传入Losser当中。\n",
- " 由于每个人写的Model.forward的output的dict可能key并不一样,比如有人是{'pred':xxx}, {'output': xxx}; \n",
- " 另外每个人将target可能也会设置为不同的名称, 比如有人是label, 有人设置为target;\n",
- " 为了解决以上的问题,我们的loss提供映射机制\n",
- " 比如CrossEntropyLosser的需要的输入是(prediction, target)。但是forward的output是{'output': xxx}; 'label'是target\n",
- " 那么初始化losser的时候写为CrossEntropyLosser(prediction='output', target='label')即可\n",
- " (3) 对于Metric是同理的\n",
- " Metric计算也是从 forward的结果中取值 与 设置target的field中取值。 也是可以通过映射找到对应的值 \n",
- " \n",
- " \n",
- "\n",
- "## 一些问题.\n",
- "### 1. DataSet中为什么需要设置input和target\n",
- " 只有被设置为input或者target的数据才会在train的过程中被取出来\n",
- " (1.1) 我们只会在设置为input的field中寻找传递给Model.forward的参数。\n",
- " (1.2) 我们在传递值给losser或者metric的时候会使用来自: \n",
- " (a)Model.forward的output\n",
- " (b)被设置为target的field\n",
- " \n",
- "\n",
- "### 2. 我们是通过forwad中的形参名将DataSet中的field赋值给对应的参数\n",
- " (1.1) 构建模型过程中,\n",
- " 例如:\n",
- " DataSet中x,seq_lens是input,那么forward就应该是\n",
- " def forward(self, x, seq_lens):\n",
- " pass\n",
- " 我们是通过形参名称进行匹配的field的\n",
- " \n",
- "\n",
- "\n",
- "### 1. 加载数据到DataSet\n",
- "### 2. 使用apply操作对DataSet进行预处理\n",
- " (2.1) 处理过程中将某些field设置为input,某些field设置为target\n",
- "### 3. 构建模型\n",
- " (3.1) 构建模型过程中,需要注意forward函数的形参名需要和DataSet中设置为input的field名称是一致的。\n",
- " 例如:\n",
- " DataSet中x,seq_lens是input,那么forward就应该是\n",
- " def forward(self, x, seq_lens):\n",
- " pass\n",
- " 我们是通过形参名称进行匹配的field的\n",
- " (3.2) 模型的forward的output需要是dict类型的。\n",
- " 建议将输出设置为{\"pred\": xx}.\n",
- " \n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": []
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": "Python 3",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.6.7"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}
diff --git a/tutorials/fastnlp_test_tutorial.ipynb b/tutorials/fastnlp_test_tutorial.ipynb
deleted file mode 100644
index fb87606e..00000000
--- a/tutorials/fastnlp_test_tutorial.ipynb
+++ /dev/null
@@ -1,97 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## fastNLP测试说明\n",
- "### 测试环境\n",
- "fastNLP使用pytest对代码进行单元测试,测试代码在test文件夹下,测试所需数据在test/data_for_tests文件夹下\n",
- "测试的步骤主要分为准备数据,执行测试,比对结果,清除环境四步\n",
- "测试代码以test_xxx.py命名,以DataSet的测试代码为例,测试代码文件名为test_dataset.py"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "import os\n",
- "import unittest # 单元测试需要用到unittest\n",
- "\n",
- "from fastNLP.core.dataset import DataSet\n",
- "from fastNLP.core.fieldarray import FieldArray\n",
- "from fastNLP.core.instance import Instance\n",
- "# 在这个单元测试文件中,需要测试DataSet、FieldArray、以及Instance"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "class TestDataSet(unittest.TestCase): # 类名字以Test打头,继承unittest.TestCase\n",
- "\n",
- " def test_init_v1(self): # 测试样例1, 函数名称以test_打头\n",
- " # 该测试样例测试的是DataSet的初始化\n",
- " ins = Instance(x=[1, 2, 3, 4], y=[5, 6]) # 准备数据\n",
- " ds = DataSet([ins] * 40) # 执行测试(调用DataSet的初始化函数)\n",
- " self.assertTrue(\"x\" in ds.field_arrays and \"y\" in ds.field_arrays) # 比对结果:'x'跟'y'都是ds的field\n",
- " self.assertEqual(ds.field_arrays[\"x\"].content, [[1, 2, 3, 4], ] * 40) # 比对结果: field 'x'的内容正确\n",
- " self.assertEqual(ds.field_arrays[\"y\"].content, [[5, 6], ] * 40) # 比对结果: field 'y'的内容正确\n",
- " \n",
- " def test_init_v2(self): # 测试样例2,该样例测试DataSet的另一种初始化方式\n",
- " ds = DataSet({\"x\": [[1, 2, 3, 4]] * 40, \"y\": [[5, 6]] * 40})\n",
- " self.assertTrue(\"x\" in ds.field_arrays and \"y\" in ds.field_arrays)\n",
- " self.assertEqual(ds.field_arrays[\"x\"].content, [[1, 2, 3, 4], ] * 40)\n",
- " self.assertEqual(ds.field_arrays[\"y\"].content, [[5, 6], ] * 40)\n",
- " \n",
- " def test_init_assert(self): # 测试样例3,该样例测试不规范初始化DataSet时是否会报正确错误\n",
- " with self.assertRaises(AssertionError):\n",
- " _ = DataSet({\"x\": [[1, 2, 3, 4]] * 40, \"y\": [[5, 6]] * 100})\n",
- " with self.assertRaises(AssertionError):\n",
- " _ = DataSet([[1, 2, 3, 4]] * 10)\n",
- " with self.assertRaises(ValueError):\n",
- " _ = DataSet(0.00001)\n",
- " \n",
- " def test_contains(self): # 测试样例4,该样例测试DataSet的contains函数,是功能测试\n",
- " ds = DataSet({\"x\": [[1, 2, 3, 4]] * 40, \"y\": [[5, 6]] * 40})\n",
- " self.assertTrue(\"x\" in ds)\n",
- " self.assertTrue(\"y\" in ds)\n",
- " self.assertFalse(\"z\" in ds)\n",
- " \n",
- " # 更多测试样例见test/core/test_dataset.py"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": []
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": "Python 3",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.6.7"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}
diff --git a/tutorials/tutorial_one.ipynb b/tutorials/tutorial_1.ipynb
similarity index 100%
rename from tutorials/tutorial_one.ipynb
rename to tutorials/tutorial_1.ipynb
diff --git a/tutorials/tutorial_for_developer.md b/tutorials/tutorial_for_developer.md
deleted file mode 100644
index b2ec0b98..00000000
--- a/tutorials/tutorial_for_developer.md
+++ /dev/null
@@ -1,283 +0,0 @@
-# fastNLP开发者指南
-#### 本教程涉及以下类:
-- DataSet
-- Sampler
-- Batch
-- Model
-- Loss
-- Metric
-- Trainer
-- Tester
-
-#### DataSet: 用于承载数据。
-1. DataSet里面每个元素只能是以下的三类`np.float64`, `np.int64`, `np.str`。如果传入的数据是`int`则被转换为`np.int64`, `float`被转为`np.float64`。
-2. DataSet可以将field设置为input或者target。其中被设置为input的field会被传递给Model.forward, 这个过程中我们是通过键匹配完成传递的。举例来说,假设DataSet中有'x1', 'x2', 'x3'被设置为了input,而
- - 函数是Model.forward(self, x1, x3), 那么DataSet中'x1', 'x3'会被传递给forward函数。多余的'x2'会被忽略
- - 函数是Model.forward(self, x1, x4), 这里多需要了一个'x4', 但是DataSet的input field中没有这个field,会报错。
- - 函数是Model.forward(self, x1, **kwargs), 会把'x1', 'x2', 'x3'都传入。但如果是Model.forward(self, x4, **kwargs)就会发生报错,因为没有'x4'。
-3. 对于设置为target的field的名称,我们建议取名为'target'(如果只有一个需要predict的值),但是不强制。后面会讲为什么target可以不强制。
-DataSet应该是不需要单独再开发的,如果有不能满足的场景,请在开发群提出或者github提交issue。
-
-#### Sampler: 给定一个DataSet,返回一个序号的list,Batch按照这个list输出数据。
-Sampler需要继承fastNLP.core.sampler.BaseSampler
-```python
-class BaseSampler(object):
- """The base class of all samplers.
-
- Sub-classes must implement the __call__ method.
- __call__ takes a DataSet object and returns a list of int - the sampling indices.
- """
-def __call__(self, *args, **kwargs):
- raise NotImplementedError
-
-# 子类需要复写__call__方法。这个函数只能有一个必选参数, 且必须是DataSet类别, 否则Trainer没法调
-class SonSampler(BaseSampler):
- def __init__(self, xxx):
- # 可以实现init也不可以不实现。
- pass
- def __call__(self, data_set):
- pass
-```
-
-#### Batch: 将DataSet中设置为input和target的field取出来构成batch_x, batch_y
-并且根据情况(主要根据数据类型能不能转为Tensor)将数据转换为pytorch的Tensor。batch中sample的取出顺序是由Sampler决定的。
-Sampler是传入一个DataSet,返回一个与DataSet等长的序号list,Batch一次会取出batch_size个sample(最后一个batch可能数量不足batch_size个)。
-举例:
-1. SequentialSampler是顺序采样
-
- 假设传入的DataSet长度是100, SequentialSampler返回的序号list就是[0, 1, ...,98, 99]. batch_size如果被设置为4,那么第一个batch所获取的instance就是[0, 1, 2, 3]这四个instance. 第二个batch所获取instace就是[4, 5, 6, 7], ...直到采完所有的sample。
-2. RandomSampler是随机采样
-
- 假设传入的DataSet长度是100, RandomSampler返回的序号list可能是[0, 99, 20, 5, 3, 1, ...]. 依次按照batch_size的大小取出sample。
-
-Batch应该不需要继承与开发,如果你有特殊需求请在开发群里提出。
-
-#### Model:用户自定的Model
-必须是nn.Module的子类
-1. 必须实现forward方法,并且forward方法不能出现*arg这种参数. 例如
- ```python
- def forward(self, word_seq, *args): #这是不允许的.
- # ...
- pass
- ```
- 返回值必须是dict的
- ```python
- def forward(self, word_seq, seq_lens):
- xxx = "xxx"
- return {'pred': xxx} #return的值必须是dict的。里面的预测的key推荐使用pred,但是不做强制限制。输出元素数目不限。
- ```
-2. 如果实现了predict方法,在做evaluation的时候将调用predict方法而不是forward。如果没有predict方法,则在evaluation时调用forward方法。predict方法也不能使用*args这种参数形式,同时结果也必须返回一个dict,同样推荐key为'pred'。
-
-#### Loss: 根据model.forward()返回的prediction(是一个dict)和batch_y计算相应的loss
-1. 先介绍"键映射"。 如在DataSet, Model一节所看见的那样,fastNLP并不限制Model.forward()的返回值,也不限制DataSet中target field的key。计算的loss的时候,怎么才能知道从哪里取值呢?
-这里以CrossEntropyLoss为例,一般情况下, 计算CrossEntropy需要prediction和target两个值。而在CrossEntropyLoss初始化时可以传入两个参数(pred=None, target=None), 这两个参数接受的类型是str,假设(pred='output', target='label'),那么CrossEntropyLoss会使用'output'这个key在forward的output与batch_y中寻找值;'label'也是在forward的output与batch_y中寻找值。注意这里pred或target的来源并不一定非要来自于model.forward与batch_y,也可以只来自于forward的结果。
-2. 如何创建一个自己的loss
- - 使用fastNLP.LossInForward, 在model.forward()的结果中包含一个为loss的key。
- - trainer中使用loss(假设loss=CrossEntropyLoss())的时候其实是
- los = loss(prediction, batch_y),即直接调用的是`loss.__call__()`方法,但是CrossEntropyLoss里面并没有自己实现`__call__`方法,这是因为`__call__`在LossBase中实现了。所有的loss必须继承fastNLP.core.loss.LossBase, 下面先说一下LossBase的几个方法,见下一节。
-3. 尽量不要复写`__call__()`, `_init_param_map()`方法。
-
-```python
-class LossBase():
- def __init__(self):
- self.param_map = {} # 一般情况下也不需要自己创建。调用_init_param_map()更好
- self._checked = False # 这个参数可以忽略
-
- def _init_param_map(self, key_map=None, **kwargs):
- # 这个函数是用于注册Loss的“键映射”,有两种传值方法,
- # 第一种是通过key_map传入dict,取值是用value到forward和batch_y取
- # key_map = {'pred': 'output', 'target': 'label'}
- # 第二种是自己写
- # _init_param_map(pred='output', target='label')
- # 为什么会提供这么一个方法?通过调用这个方法会自动注册param_map,并会做一些检查,防止出现传入的key其实并不是get_loss
- # 的一个参数。注意传入这个方法的参数必须都是需要做键映射的内容,其它loss参数不要传入。如果传入(pred=None, target=None)
- # 则__call__()会到pred_dict与target_dict去寻找key为'pred'和'target'的值。
- # 但这个参数不是必须要调用的。
-
- def __call__(self, pred_dict, target_dict, check=False): # check=False忽略这个参数,之后应该会被删除的
- # 这个函数主要会做一些check的工作,比如pred_dict与target_dict中是否包含了计算loss所必须的key等。检查通过,则调用get_loss
- # 方法。
- fast_param = self._fast_param_map(predict_dict, target_dict):
- if fast_param:
- return self.get_loss(**fast_param)
- # 如果没有fast_param则通过匹配参数然后调用get_loss完成
- xxxx
- return loss # 返回为Tensor的loss
- def _fast_param_map(self, pred_dict, target_dict):
- # 这是一种快速计算loss的机制,因为在很多情况下其实都不需要通过"键映射",比如计算loss时,pred_dict只有一个元素,
- # target_dict也只有一个元素,那么无歧义地就可以把预测值与实际值用于计算loss, 基类判断了这种情况(可能还有其它无歧义的情况)。
- # 即_fast_param_map成功的话,就不需要使用键映射,这样即使在没有传递或者传递错误"键映射"的情况也可以直接计算loss。
- # 返回值是一个dict, 如果匹配成功,应该返回类似{'pred':value, 'target': value}的结果;如果dict为空则说明匹配失败,
- # __call__方法会继续执行。
-
- def get_loss(self, *args, **kwargs):
- # 这个是一定需要实现的,计算loss的地方。
- # (1) get_loss中一定不能包含*arg这种参数形式。
- # (2) 如果包含**kwargs这种参数,这会将pred_dict与target_dict中所有参数传入。但是建议不要用这个参数
- raise NotImplementedError
-
-# 下面使用L1Loss举例
-class L1Loss(LossBase): # 继承LossBase
- # 初始化需要映射的值,这里需要映射的值'pred', 'target'必须与get_loss需要参数名是对应的
- def __init__(self, pred=None, target=None):
- super(L1Loss, self).__init__()
- # 这里传入_init_param_map以使得pred和target被正确注册,但这一步不是必须的, 建议调用。传入_init_param_map的是用于
- # “键映射"的键值对。假设初始化__init__(pred=None, target=None, threshold=0.1)中threshold是用于控制loss计算的,则
- # 不要将threshold传入_init_param_map.
- self._init_param_map(pred=pred, target=target)
-
- def get_loss(self, pred, target):
- # 这里'pred', 'target'必须和初始化的映射是一致的。
- return F.l1_loss(input=pred, target=target) #直接返回一个loss即可
-```
-
-### Metric: 根据Model.forward()或者Model.predict()的结果计算metric
-metric的设计和loss的设计类似。都是传入pred_dict与target_dict进行计算。但是metric的pred_dict来源可能是Model.forward的返回值, 也可能是Model.predict(如果Model具有predict方法则会调用predict方法)的返回值,下面统一用pred_dict代替。
-1. 这里的"键映射"与loss的"键映射"是类似的。举例来说,若Metric(pred='output', target='label'),则使用'output'到pred_dict和target_dict中寻找pred, 用'label'寻找target。
-2. 如何创建一个自己的Metric方法
-Metric与loss的计算不同在于,Metric的计算有两个步骤。
- - **每个batch的输出**都会调用Metric的``__call__(pred_dict, target_dict)``方法,而``__call__``方法会调用evaluate()(需要实现)方法。
- - 在所有batch传入之后,调用Metric的get_metric()方法得到最终的metric值。
- - 所以Metric在调用evaluate方法时,根据拿到的数据: pred_dict与batch_y, 改变自己的状态(比如累加正确的次数,总的sample数等)。在调用get_metric()的时候给出一个最终计算结果。
- 所有的Metric必须继承自fastNLP.core.metrics.MetricBase. 例子见下一个cell
-3. 尽量不要复写``__call__()``,``_init_param_map()``方法。
-
-```python
-class MetricBase:
- def __init__(self):
- self.param_map = {} # 一般情况下也不需要自己创建。调用_init_param_map()更好
- self._checked = False # 这个参数可以忽略
-
- def _init_param_map(self, key_map=None, **kwargs):
- # 这个函数是用于注册Metric的“键映射”,有两种传值方法,
- # 第一种是通过key_map传入dict,取值是用value到forward和batch_y取
- # key_map = {'pred': 'output', 'target': 'label'}
- # 第二种是自己写(建议使用改种方式)
- # _init_param_map(pred='output', target='label')
- # 为什么会提供这么一个方法?通过调用这个方法会自动注册param_map,并会做一些检查,防止出现传入的key其实并不是evaluate()
- # 的一个参数。注意传入这个方法的参数必须都是需要做键映射的内容,其它evaluate参数不要传入。如果传入(pred=None, target=None)
- # 则__call__()会到pred_dict与target_dict去寻找key为'pred'和'target'的值。
- # 但这个参数不是必须要调用的。
- pass
-
- def __call__(self, pred_dict, target_dict, check=False): # check=False忽略这个参数,之后应该会被删除的
- # 这个函数主要会做一些check的工作,比如pred_dict与target_dict中是否包含了计算evaluate所必须的key等。检查通过,则调用
- # evaluate方法。
- fast_param = self._fast_param_map(predict_dict, target_dict):
- if fast_param:
- return self.evaluate(**fast_param)
- # 如果没有fast_param则通过匹配参数然后调用get_loss完成
- # xxxx
-
- def _fast_param_map(self, pred_dict, target_dict):
- # 这是一种快速计算loss的机制,因为在很多情况下其实都不需要通过"键映射",比如evaluate时,pred_dict只有一个元素,
- # target_dict也只有一个元素,那么无歧义地就可以把预测值与实际值用于计算metric, 基类判断了这种情况(可能还有其它无歧义的
- # 情况)。即_fast_param_map成功的话,就不需要使用键映射,这样即使在没有传递或者传递错误"键映射"的情况也可以直接计算metric。
- # 返回值是一个dict, 如果匹配成功,应该返回类似{'pred':value, 'target': value}的结果;如果dict为空则说明匹配失败,
- # __call__方法会继续尝试匹配。
- pass
-
- def evaluate(self, *args, **kwargs):
- # 这个是一定需要实现的,累加metric状态
- # (1) evaluate()中一定不能包含*arg这种参数形式。
- # (2) 如果包含**kwargs这种参数,这会将pred_dict与target_dict中所有参数传入。但是建议不要用这个参数
- raise NotImplementedError
-
- def get_metric(self, reset=True):
- # 这是一定需要实现的,获取最终的metric。返回值必须是一个dict。会在所有batch传入之后调用
- raise NotImplementedError
-
-# 下面使用AccuracyMetric举例
-class AccuracyMetric(MetricBase): # MetricBase
- # 初始化需要映射的值,这里需要映射的值'pred', 'target'必须与evaluate()需要参数名是对应的
- def __init__(self, pred=None, target=None):
- super(AccuracyMetric, self).__init__()
- # 这里传入_init_param_map以使得pred和target被正确注册,但这一步不是必须的, 建议调用。传入_init_param_map的是用于
- # “键映射"的键值对。假设初始化__init__(pred=None, target=None, threshold=0.1)中threshold是用于控制loss计算的,则
- # 不要将threshold传入_init_param_map.
- self._init_param_map(pred=pred, target=target)
-
- self.total = 0 # 用于累加一共有多少sample
- self.corr = 0 # 用于累加一共有多少正确的sample
-
- def evaluate(self, pred, target):
- # 对pred和target做一些基本的判断或者预处理等
- if pred.size()==target.size() and len(pred.size())=1: #如果pred已经做了argmax
- pass
- elif len(pred.size())==2 and len(target.size())==1: # pred还没有进行argmax
- pred = pred.argmax(dim=1)
- else:
- raise ValueError("The shape of pred and target should be ((B, n_classes), (B, )) or ("
- "(B,),(B,)).")
- assert pred.size(0)==target.size(0), "Mismatch batch size."
- # 进行相应的累加
- self.total += pred.size(0)
- self.corr += torch.sum(torch.eq(pred, target).float()).item()
-
- def get_metric(self, reset=True):
- # reset用于指示是否清空累加信息。默认为True
- # 这个函数需要返回dict,可以包含多个metric。
- metric = {}
- metric['acc'] = self.corr/self.total
- if reset:
- self.total = 0
- self.corr = 0
- return metric
-```
-
-#### Tester: 用于做evaluation,应该不需要更改
-重要的初始化参数有data, model, metric;比较重要的function是test()。
-
-test中的运行过程
-```
-predict_func = 如果有model.predict则为model.predict, 否则是model.forward
-for batch_x, batch_y in batch:
-# (1) 同步数据与model
-# (2) 根据predict_func的参数从batch_x中取出数据传入到predict_func中,得到结果pred_dict
-# (3) 调用metric(pred_dict, batch_y
-# (4) 当所有batch都运行完毕,会调用metric的get_metric方法,并且以返回的值作为evaluation的结果
-metric.get_metric()
-```
-
-#### Trainer: 对训练过程的封装。
-里面比较重要的function是train()
-train()中的运行过程
-```
-(1) 创建batch
- batch = Batch(dataset, batch_size, sampler=sampler)
- for batch_x, batch_y in batch:
- # ...
- batch_x,batch_y都是dict。batch_x是DataSet中被设置为input的field;batch_y是DataSet中被设置为target的field。
- 两个dict中的key就是DataSet中的key,value会根据情况做好padding的tensor。
-(2)会将batch_x, batch_y中tensor移动到model所在的device
-(3)根据model.forward的参数列表, 从batch_x中取出需要传递给forward的数据。
-(4)获取model.forward的输出结果pred_dict,并与batch_y一起传递给loss函数, 求得loss
-(5)对loss进行反向梯度并更新参数
-(6) 如果有验证集,则需要做验证
- tester = Tester(model, dev_data,metric)
- eval_results = tester.test()
-(7) 如果eval_results是当前的最佳结果,则保存模型。
-```
-
-#### 其他
-Trainer中还提供了"预跑"的功能。该功能通过check_code_level管理,如果check_code_level为-1,则不进行"预跑"。
-
-check_code_level=0,1,2代表不同的提醒级别。
-目前不同提醒级别对应的是对DataSet中设置为input或target但又没有使用的field的提醒级别。
-0是忽略(默认);1是会warning发生了未使用field的情况;2是出现了unused会直接报错并退出运行
-
-"预跑"的主要目的有两个:
-- 防止train完了之后进行evaluation的时候出现错误。之前的train就白费了
-- 由于存在"键映射",直接运行导致的报错可能不太容易debug,通过"预跑"过程的报错会有一些debug提示
-
-"预跑"会进行以下的操作:
-- 使用很小的batch_size, 检查batch_x中是否包含Model.forward所需要的参数。只会运行两个循环。
-- 将Model.foward的输出pred_dict与batch_y输入到loss中, 并尝试backward. 不会更新参数,而且grad会被清零
- 如果传入了dev_data,还将进行metric的测试
-- 创建Tester,并传入少量数据,检测是否可以正常运行
-
-"预跑"操作是在Trainer初始化的时候执行的。
-
-正常情况下,应该不需要改动"预跑"的代码。但如果你遇到bug或者有什么好的建议,欢迎在开发群或者github提交issue。
-
-