@@ -18,13 +18,43 @@ IndexType = Union[str, slice, int, list, LongTypeTensor, BoolTypeTensor, np.ndar | |||
# https://github.com/open-mmlab/mmdetection/blob/master/mmdet/core/data_structures/instance_data.py # noqa | |||
class ListData(BaseDataElement): | |||
""" | |||
Data structure for example-level data. | |||
Subclass of :class:`BaseDataElement`. All value in `data_fields` | |||
should have the same length. This design refer to | |||
https://github.com/facebookresearch/detectron2/blob/master/detectron2/structures/instances.py | |||
ListData supports `index` and `slice` for data field. The type of value in data field can be either `None` or `list` of base data structures such as `torch.Tensor`, `numpy.ndarray`, `list`, `str` and `tuple`. | |||
Abstract Data Interface used throughout the ABL-Package. | |||
`ListData` is the underlying data structure used in the ABL-Package, | |||
designed to manage diverse forms of data dynamically generated throughout the | |||
Abductive Learning (ABL) framework. This includes handling raw data, predicted | |||
pseudo-labels, abduced pseudo-labels, pseudo-label indices, etc. | |||
As a fundamental data structure in ABL, `ListData` is essential for the smooth | |||
transfer and manipulation of data across various components of the ABL framework, | |||
such as prediction, abductive reasoning, and training phases. It provides a | |||
unified data format across these stages, ensuring compatibility and flexibility | |||
in handling diverse data forms in the ABL framework. | |||
The attributes in ``ListData`` are divided into two parts, | |||
the ``metainfo`` and the ``data`` respectively. | |||
- ``metainfo``: Usually used to store basic information about data examples, | |||
such as symbol number, image size, etc. The attributes can be accessed or | |||
modified by dict-like or object-like operations, such as ``.`` (for data | |||
access and modification), ``in``, ``del``, ``pop(str)``, ``get(str)``, | |||
``metainfo_keys()``, ``metainfo_values()``, ``metainfo_items()``, | |||
``set_metainfo()`` (for set or change key-value pairs in metainfo). | |||
- ``data``: raw data, labels, predictions, and abduced results are stored. | |||
The attributes can be accessed or modified by dict-like or object-like operations, | |||
such as ``.``, ``in``, ``del``, ``pop(str)``, ``get(str)``, ``keys()``, | |||
``values()``, ``items()``. Users can also apply tensor-like | |||
methods to all :obj:`torch.Tensor` in the ``data_fields``, such as ``.cuda()``, | |||
``.cpu()``, ``.numpy()``, ``.to()``, ``to_tensor()``, ``.detach()``. | |||
ListData supports `index` and `slice` for data field. The type of value in | |||
data field can be either `None` or `list` of base data structures such as | |||
`torch.Tensor`, `numpy.ndarray`, `list`, `str` and `tuple`. | |||
This design is inspired by and extends the functionalities of the `BaseDataElement` | |||
class implemented in MMEngine. | |||
https://github.com/open-mmlab/mmengine/blob/main/mmengine/structures/base_data_element.py # noqa E501 | |||
Examples: | |||
>>> from abl.data.structures import ListData | |||
@@ -22,8 +22,7 @@ AI: data, models, and knowledge. | |||
.. image:: ../img/ABL-Package.png | |||
**Data** part manages the storage, operation, and evaluation of data. | |||
It first features class ``ListData`` (derived from base class | |||
``BaseDataElement``), which defines the data structures used in | |||
It first features class ``ListData``, which defines the data structures used in | |||
Abductive Learning, and comprises common data operations like insertion, | |||
deletion, retrieval, slicing, etc. Additionally, a series of Evaluation | |||
Metrics, including class ``SymbolAccuracy`` and ``ReasoningMetric`` (both | |||
@@ -53,11 +53,11 @@ As an illustration, in the MNIST Addition example, the data used for training ar | |||
Data Structure | |||
-------------- | |||
Besides the user-provided dataset, various forms of data are utilized and dynamicly generate throughout the training and testing process of Abductive Learning framework. Examples include raw data, predicted pseudo-label, abduced pseudo-label, pseudo-label indices, and so on. To manage this diversity and ensure a stable, versatile interface, ABL-Package employs `abstract data interfaces <../API/abl.data.html>`_ to encapsulate different forms of data that will be used in the total learning process. | |||
Besides the user-provided dataset, various forms of data are utilized and dynamicly generated throughout the training and testing process of Abductive Learning framework. Examples include raw data, predicted pseudo-label, abduced pseudo-label, pseudo-label indices, and so on. To manage this diversity and ensure a stable, versatile interface, ABL-Package employs `abstract data interfaces <../API/abl.data.html#data-structure>`_ to encapsulate different forms of data that will be used in the total learning process. | |||
``BaseDataElement`` is the base class for all abstract data interfaces. Inherited from ``BaseDataElement``, ``ListData`` is the most commonly used abstract data interface in ABL-Package. As the fundamental data structure, ``ListData`` implements commonly used data manipulation methods and is responsible for transferring data between various components of ABL, ensuring that stages such as prediction, training, and abductive reasoning can utilize ``ListData`` as a unified input format. | |||
``ListData`` is the underlying abstract data interface utilized in ABL-Package. As the fundamental data structure, ``ListData`` implements commonly used data manipulation methods and is responsible for transferring data between various components of ABL, ensuring that stages such as prediction, abductive reasoning, and training can utilize ``ListData`` as a unified input format. | |||
Before proceeding to other stages, user-provided datasets are firstly converted into ``ListData``. For flexibility, ABL-Package also allows user to directly supply data in ``ListData`` format, which similarly requires the inclusion of three attributes: ``X``, ``gt_pseudo_label``, and ``Y``. The following code shows the basic usage of ``ListData``. More information can be found in the `API documentation <../API/abl.data.html>`_. | |||
Before proceeding to other stages, user-provided datasets are firstly converted into ``ListData``. For flexibility, ABL-Package also allows user to directly supply data in ``ListData`` format, which similarly requires the inclusion of three attributes: ``X``, ``gt_pseudo_label``, and ``Y``. The following code shows the basic usage of ``ListData``. More information can be found in the `API documentation <../API/abl.data.html#data-structure>`_. | |||
.. code-block:: python | |||