| @@ -66,7 +66,7 @@ Continuous Integration (CI) tools help you stick to the quality standards by run | |||||
| ``pre-commit`` Config | ``pre-commit`` Config | ||||
| ======================== | ======================== | ||||
| The ``Learnware`` Package support config ``pre-commit``. Run the following command to install ``pre-commit``: | |||||
| The ``learnware`` package support config ``pre-commit``. Run the following command to install ``pre-commit``: | |||||
| .. code-block:: bash | .. code-block:: bash | ||||
| @@ -82,7 +82,7 @@ Run the following command in the root directory of ``Learnware`` Project to enab | |||||
| ``isort`` Config | ``isort`` Config | ||||
| =================== | =================== | ||||
| The codes in the ``Learnware`` Package will be processed by ``isort`` (``examples`` and ``tests`` are excluded). Run the following command to install ``isort``: | |||||
| The codes in the ``learnware`` package will be processed by ``isort`` (``examples`` and ``tests`` are excluded). Run the following command to install ``isort``: | |||||
| .. code-block:: bash | .. code-block:: bash | ||||
| @@ -55,7 +55,7 @@ All Reuse Methods | |||||
| =========================== | =========================== | ||||
| In addition to applying ``Learnware``, ``FeatureAlignLearnware`` or ``HeteroMapAlignLearnware`` objects directly by calling their ``predict`` interface, | In addition to applying ``Learnware``, ``FeatureAlignLearnware`` or ``HeteroMapAlignLearnware`` objects directly by calling their ``predict`` interface, | ||||
| the ``Learnware`` package also provides a set of ``Reuse Methods`` for users to further customize a single or multiple learnwares, with the hope of enabling learnwares to be | |||||
| the ``learnware`` package also provides a set of ``Reuse Methods`` for users to further customize a single or multiple learnwares, with the hope of enabling learnwares to be | |||||
| helpful beyond their original purposes, and eliminating the need for users to build models from scratch. | helpful beyond their original purposes, and eliminating the need for users to build models from scratch. | ||||
| There are two main categories of ``Reuse Methods``: (1) direct reuse and (2) reuse based on a small amount of labeled data. | There are two main categories of ``Reuse Methods``: (1) direct reuse and (2) reuse based on a small amount of labeled data. | ||||
| @@ -107,7 +107,7 @@ specifies the ensemble method(default is set to ``mean``). | |||||
| Reuse Learnware with Labeled Data | Reuse Learnware with Labeled Data | ||||
| ---------------------------------- | ---------------------------------- | ||||
| When users have a small amount of labeled data available, ``Learnware`` package provides two methods: ``EnsemblePruningReuser`` and ``FeatureAugmentReuser`` to help reuse learnwares. | |||||
| When users have a small amount of labeled data available, the ``learnware`` package provides two methods: ``EnsemblePruningReuser`` and ``FeatureAugmentReuser`` to help reuse learnwares. | |||||
| They are both initialized with a list of ``Learnware`` objects ``learnware_list``, and have different implementations of ``fit`` and ``predict`` methods. | They are both initialized with a list of ``Learnware`` objects ``learnware_list``, and have different implementations of ``fit`` and ``predict`` methods. | ||||
| EnsemblePruningReuser | EnsemblePruningReuser | ||||
| @@ -48,7 +48,7 @@ This ``checker`` checks the statistical specification and functionality of a lea | |||||
| Current Markets | Current Markets | ||||
| ====================================== | ====================================== | ||||
| The ``Learnware`` package provide two different implementation of ``market``, i.e. ``Easy Market`` and ``Hetero Market``. They have different implementation of ``organizer`` and ``searcher``. | |||||
| The ``learnware`` package provide two different implementation of ``market``, i.e. ``Easy Market`` and ``Hetero Market``. They have different implementation of ``organizer`` and ``searcher``. | |||||
| Easy Market | Easy Market | ||||
| ------------- | ------------- | ||||
| @@ -5,7 +5,7 @@ Specification | |||||
| Learnware specification is the core component of the learnware paradigm, linking all processes about learnwares, including uploading, organizing, searching, deploying and reusing. | Learnware specification is the core component of the learnware paradigm, linking all processes about learnwares, including uploading, organizing, searching, deploying and reusing. | ||||
| In this section, we will introduce the concept and design of learnware specification in the ``Learnware`` package. | |||||
| In this section, we will introduce the concept and design of learnware specification in the ``learnware`` package. | |||||
| We will then explore ``regular specification``\ s tailored for different data types such as tables, images and texts. | We will then explore ``regular specification``\ s tailored for different data types such as tables, images and texts. | ||||
| Lastly, we cover a ``system specification`` specifically assigned to table learnwares by the learnware market, aimed at accommodating all available table learnwares into a unified "specification world" despite their heterogeneity. | Lastly, we cover a ``system specification`` specifically assigned to table learnwares by the learnware market, aimed at accommodating all available table learnwares into a unified "specification world" despite their heterogeneity. | ||||
| @@ -13,7 +13,7 @@ Concepts & Types | |||||
| ================== | ================== | ||||
| The learnware specification describes the model's specialty and utility in a certain format, allowing the model to be identified and reused by future users who may have no prior knowledge of the learnware. | The learnware specification describes the model's specialty and utility in a certain format, allowing the model to be identified and reused by future users who may have no prior knowledge of the learnware. | ||||
| The ``Learnware`` package employs a highly extensible specification design, which consists of two parts: | |||||
| The ``learnware`` package employs a highly extensible specification design, which consists of two parts: | |||||
| - **Semantic specification** describes the model's type and functionality through a set of descriptions and tags. Learnwares with similar semantic specifications reside in the same specification island | - **Semantic specification** describes the model's type and functionality through a set of descriptions and tags. Learnwares with similar semantic specifications reside in the same specification island | ||||
| - **Statistical specification** characterizes the statistical information contained in the model using various machine learning techniques. It plays a crucial role in locating the appropriate place for the model within the specification island. | - **Statistical specification** characterizes the statistical information contained in the model using various machine learning techniques. It plays a crucial role in locating the appropriate place for the model within the specification island. | ||||
| @@ -28,7 +28,7 @@ We employ the ``Reduced Kernel Mean Embedding (RKME) Specification`` as the foun | |||||
| with adjustments made according to the characteristics of each data type. | with adjustments made according to the characteristics of each data type. | ||||
| The RKME specification is a recent development in learnware specification design, which represents the distribution of a model's training data in a privacy-preserving manner. | The RKME specification is a recent development in learnware specification design, which represents the distribution of a model's training data in a privacy-preserving manner. | ||||
| Within the ``Learnware`` package, you'll find two types of statistical specifications: ``regular specification`` and ``system specification``. The former is generated locally | |||||
| Within the ``learnware`` package, you'll find two types of statistical specifications: ``regular specification`` and ``system specification``. The former is generated locally | |||||
| by users to express their model's statistical information, while the latter is assigned by the learnware market to accommodate and organize heterogeneous learnwares. | by users to express their model's statistical information, while the latter is assigned by the learnware market to accommodate and organize heterogeneous learnwares. | ||||
| Semantic Specification | Semantic Specification | ||||
| @@ -44,7 +44,7 @@ In the case of table learnwares, users should additionally provide descriptions | |||||
| Regular Specification | Regular Specification | ||||
| ====================================== | ====================================== | ||||
| The ``Learnware`` package provides a unified interface, ``generate_stat_spec``, for generating ``regular specification``\ s across different data types. | |||||
| The ``learnware`` package provides a unified interface, ``generate_stat_spec``, for generating ``regular specification``\ s across different data types. | |||||
| Users can use the training data ``train_x`` (supported types include numpy.ndarray, pandas.DataFrame, and torch.Tensor) as input to generate the ``regular specification`` of the model, | Users can use the training data ``train_x`` (supported types include numpy.ndarray, pandas.DataFrame, and torch.Tensor) as input to generate the ``regular specification`` of the model, | ||||
| as shown in the following code: | as shown in the following code: | ||||
| @@ -134,7 +134,7 @@ with particular learnware market implementations. | |||||
| - Learnware searchers perform helpful learnware recommendations among all table learnwares in the market, leveraging the ``system specification``\ s generated for users. | - Learnware searchers perform helpful learnware recommendations among all table learnwares in the market, leveraging the ``system specification``\ s generated for users. | ||||
| ``Learnware`` package now includes a type of ``system specification``, named ``HeteroMapTableSpecification``, made especially for the ``Hetero Market`` implementation. | |||||
| The ``learnware`` package now includes a type of ``system specification``, named ``HeteroMapTableSpecification``, made especially for the ``Hetero Market`` implementation. | |||||
| This specification is automatically given to all table learnwares when they are added to the ``Hetero Market``. | This specification is automatically given to all table learnwares when they are added to the ``Hetero Market``. | ||||
| It is also set up to be updated periodically, ensuring it remains accurate as the learnware market evolves and builds more precise specification worlds. | It is also set up to be updated periodically, ensuring it remains accurate as the learnware market evolves and builds more precise specification worlds. | ||||
| Please refer to `COMPONENTS: Hetero Market <../components/market.html#hetero-market>`_ for implementation details. | Please refer to `COMPONENTS: Hetero Market <../components/market.html#hetero-market>`_ for implementation details. | ||||
| @@ -4,11 +4,11 @@ Installation Guide | |||||
| ======================== | ======================== | ||||
| ``Learnware`` Package Installation | |||||
| ``learnware`` Package Installation | |||||
| =================================== | =================================== | ||||
| .. note:: | .. note:: | ||||
| ``Learnware`` package supports `Windows`, `Linux`. It's recommended to use ``Learnware`` in `Linux`. ``Learnware`` supports Python3, which is up to Python3.11. | |||||
| The ``learnware`` package supports `Windows`, `Linux`. It's recommended to use ``Learnware`` in `Linux`. ``Learnware`` supports Python3, which is up to Python3.11. | |||||
| Users can easily install ``Learnware`` by pip according to the following command: | Users can easily install ``Learnware`` by pip according to the following command: | ||||
| @@ -16,7 +16,7 @@ Users can easily install ``Learnware`` by pip according to the following command | |||||
| pip install learnware | pip install learnware | ||||
| In the ``Learnware`` package, besides the base classes, many core functionalities such as "learnware specification generation" and "learnware deployment" rely on the ``torch`` library. Users have the option to manually install ``torch``, or they can directly use the following command to install the ``learnware`` package: | |||||
| In the ``learnware`` package, besides the base classes, many core functionalities such as "learnware specification generation" and "learnware deployment" rely on the ``torch`` library. Users have the option to manually install ``torch``, or they can directly use the following command to install the ``learnware`` package: | |||||
| .. code-block:: bash | .. code-block:: bash | ||||
| @@ -26,7 +26,7 @@ In the ``Learnware`` package, besides the base classes, many core functionalitie | |||||
| However, it's crucial to note that due to the potential complexity of the user's local environment, installing ``learnware[full]`` does not guarantee that ``torch`` will successfully invoke ``CUDA`` in the user's local setting. | However, it's crucial to note that due to the potential complexity of the user's local environment, installing ``learnware[full]`` does not guarantee that ``torch`` will successfully invoke ``CUDA`` in the user's local setting. | ||||
| Install ``Learnware`` Package From Source | |||||
| Install ``learnware`` Package From Source | |||||
| ========================================== | ========================================== | ||||
| Also, Users can install ``Learnware`` by the source code according to the following steps: | Also, Users can install ``Learnware`` by the source code according to the following steps: | ||||
| @@ -20,7 +20,7 @@ Learnware is currently hosted on `PyPI <https://pypi.org/>`_. You can easily int | |||||
| pip install learnware | pip install learnware | ||||
| In the ``Learnware`` package, besides the base classes, many core functionalities such as "learnware specification generation" and "learnware deployment" rely on the ``torch`` library. Users have the option to manually install ``torch``, or they can directly use the following command to install the ``learnware`` package: | |||||
| In the ``learnware`` package, besides the base classes, many core functionalities such as "learnware specification generation" and "learnware deployment" rely on the ``torch`` library. Users have the option to manually install ``torch``, or they can directly use the following command to install the ``learnware`` package: | |||||
| .. code-block:: bash | .. code-block:: bash | ||||
| @@ -32,7 +32,7 @@ In the ``Learnware`` package, besides the base classes, many core functionalitie | |||||
| Prepare Learnware | Prepare Learnware | ||||
| ==================== | ==================== | ||||
| In learnware ``Learnware`` package, each learnware is encapsulated in a ``zip`` package, which should contain at least the following four files: | |||||
| In learnware ``learnware`` package, each learnware is encapsulated in a ``zip`` package, which should contain at least the following four files: | |||||
| - ``learnware.yaml``: learnware configuration file. | - ``learnware.yaml``: learnware configuration file. | ||||
| - ``__init__.py``: methods for using the model. | - ``__init__.py``: methods for using the model. | ||||
| @@ -135,7 +135,7 @@ combine ``HeteroMapAlignLearnware`` with the homogeneous reuse methods ``Averagi | |||||
| Reuse with ``Model Container`` | Reuse with ``Model Container`` | ||||
| ================================ | ================================ | ||||
| ``Learnware`` package provides ``Model Container`` to build executive environment for learnwares according to their runtime dependent files. The learnware's model will be executed in the containers and its env will be installed and uninstalled automatically. | |||||
| The ``learnware`` package provides ``Model Container`` to build executive environment for learnwares according to their runtime dependent files. The learnware's model will be executed in the containers and its env will be installed and uninstalled automatically. | |||||
| Run the following codes to try run a learnware with ``Model Container``: | Run the following codes to try run a learnware with ``Model Container``: | ||||
| @@ -51,7 +51,7 @@ Hetero Search | |||||
| For table-based user tasks, | For table-based user tasks, | ||||
| homogeneous searchers like ``EasySearcher`` fail to recommend learnwares when no table learnware matches the user task's feature dimension, returning empty results. | homogeneous searchers like ``EasySearcher`` fail to recommend learnwares when no table learnware matches the user task's feature dimension, returning empty results. | ||||
| To enhance functionality, ``Learnware`` package includes the heterogeneous learnware search feature, whose processions is as follows: | |||||
| To enhance functionality, the ``learnware`` package includes the heterogeneous learnware search feature, whose processions is as follows: | |||||
| - Learnware markets such as ``Hetero Market`` integrate different specification islands into a unified "specification world" by assigning system-level specifications to all learnwares. This allows heterogeneous searchers like ``HeteroSearcher`` to find helpful learnwares from all available table learnwares. | - Learnware markets such as ``Hetero Market`` integrate different specification islands into a unified "specification world" by assigning system-level specifications to all learnwares. This allows heterogeneous searchers like ``HeteroSearcher`` to find helpful learnwares from all available table learnwares. | ||||
| - Searchers assign system-level specifications to users based on ``UserInfo``'s statistical specification, using methods provided by corresponding organizers. In ``Hetero Market``, for example, ``HeteroOrganizer.generate_hetero_map_spec`` generates system-level specifications for users. | - Searchers assign system-level specifications to users based on ``UserInfo``'s statistical specification, using methods provided by corresponding organizers. In ``Hetero Market``, for example, ``HeteroOrganizer.generate_hetero_map_spec`` generates system-level specifications for users. | ||||
| @@ -83,7 +83,7 @@ Please note that module imports between Python files within the zip package shou | |||||
| Learnware Statistical Specification ``stat.json`` | Learnware Statistical Specification ``stat.json`` | ||||
| --------------------------------------------------- | --------------------------------------------------- | ||||
| A learnware consists of a model and a specification. Therefore, after preparing the model, you need to generate a statistical specification for it. Specifically, using the previously installed ``Learnware`` package, you can use the training data ``train_x`` (supported types include numpy.ndarray, pandas.DataFrame, and torch.Tensor) as input to generate the statistical specification of the model. | |||||
| A learnware consists of a model and a specification. Therefore, after preparing the model, you need to generate a statistical specification for it. Specifically, using the previously installed ``learnware`` package, you can use the training data ``train_x`` (supported types include numpy.ndarray, pandas.DataFrame, and torch.Tensor) as input to generate the statistical specification of the model. | |||||
| Here is an example of the code: | Here is an example of the code: | ||||