元控制器: 元任务和元数据集合元模型

介绍

元控制器 为 预测模型 提供指导，其旨在学习一系列预测任务中的规律模式，并使用学到的模式指导即将进行的预测任务。用户可以基于 元控制器 模块来实现自己的元模型实例。

元任务

元任务 实例是元学习框架中的基本元素。它保存了可以用于 元模型 的数据。多个 元任务 实例可以共享相同的 数据处理器，由 元数据集 控制。用户应该使用 prepare_task_data() 来获取可以直接输入 元模型 的数据。

class qlib.model.meta.task.MetaTask(task: dict, meta_info: object, mode: str = 'full')

A single meta-task, a meta-dataset contains a list of them. It serves as a component as in MetaDatasetDS

The data processing is different

  • the processed input may be different between training and testing

    • When training, the X, y, X_test, y_test in training tasks are necessary (# PROC_MODE_FULL #) but not necessary in test tasks. (# PROC_MODE_TEST #)

    • When the meta model can be transferred into other dataset, only meta_info is necessary (# PROC_MODE_TRANSFER #)

__init__(task: dict, meta_info: object, mode: str = 'full')

The __init__ func is responsible for

  • store the task

  • store the origin input data for

  • process the input data for meta data

参数:

  • task (dict) – the task to be enhanced by meta model

  • meta_info (object) – the input for meta model

get_meta_input() → object

Return the processed meta_info

元数据集

元数据集 控制元信息生成过程。它负责提供训练 元模型 所需的数据。用户应该使用 prepare_tasks 来获取 元任务 实例的列表。

class qlib.model.meta.dataset.MetaTaskDataset(segments: Dict[str, Tuple] | float, *args, **kwargs)

A dataset fetching the data in a meta-level.

A Meta Dataset is responsible for

  • input tasks(e.g. Qlib tasks) and prepare meta tasks

    • meta task contains more information than normal tasks (e.g. input data for meta model)

The learnt pattern could transfer to other meta dataset. The following cases should be supported

  • A meta-model trained on meta-dataset A and then applied to meta-dataset B

    • Some pattern are shared between meta-dataset A and B, so meta-input on meta-dataset A are used when meta model are applied on meta-dataset-B

__init__(segments: Dict[str, Tuple] | float, *args, **kwargs)

The meta-dataset maintains a list of meta-tasks when it is initialized.

The segments indicates the way to divide the data

The duty of the __init__ function of MetaTaskDataset - initialize the tasks

prepare_tasks(segments: List[str] | str, *args, **kwargs) → List[MetaTask]

Prepare the data in each meta-task and ready for training.

The following code example shows how to retrieve a list of meta-tasks from the meta_dataset:

# get the train segment and the test segment, both of them are lists
train_meta_tasks, test_meta_tasks = meta_dataset.prepare_tasks(["train", "test"])
参数:

segments (Union[List[Text], Tuple[Text], Text]) – the info to select data

返回:

A list of the prepared data of each meta-task for training the meta-model. For multiple segments [seg1, seg2, … , segN], the returned list will be [[tasks in seg1], [tasks in seg2], … , [tasks in segN]]. Each task is a meta task

返回类型:

list

元模型

一般元模型

Meta Model（元模型） 实例是控制工作流程的部分。使用 Meta Model（元模型） 的方式包括： 1. 用户使用 fit 函数对 Meta Model（元模型） 进行训练。 2. Meta Model（元模型） 实例通过 inference 函数提供有用的信息来指导工作流程。

class qlib.model.meta.model.MetaModel

The meta-model guiding the model learning.

The word Guiding can be categorized into two types based on the stage of model learning - The definition of learning tasks: Please refer to docs of MetaTaskModel - Controlling the learning process of models: Please refer to the docs of MetaGuideModel

abstract fit(*args, **kwargs)

The training process of the meta-model.

abstract inference(*args, **kwargs) → object

The inference process of the meta-model.

返回:

Some information to guide the model learning

返回类型:

object

元任务模型

这种类型的元模型可以直接与任务定义进行交互。然后，Meta Task Model（元任务模型） 是它们继承的类。它们通过修改基本任务定义来指导基本任务。可以使用 prepare_tasks 函数来获取修改后的基本任务定义。

class qlib.model.meta.model.MetaTaskModel

This type of meta-model deals with base task definitions. The meta-model creates tasks for training new base forecasting models after it is trained. prepare_tasks directly modifies the task definitions.

fit(meta_dataset: MetaTaskDataset)

The MetaTaskModel is expected to get prepared MetaTask from meta_dataset. And then it will learn knowledge from the meta tasks

inference(meta_dataset: MetaTaskDataset) → List[dict]

MetaTaskModel will make inference on the meta_dataset The MetaTaskModel is expected to get prepared MetaTask from meta_dataset. Then it will create modified task with Qlib format which can be executed by Qlib trainer.

返回:

A list of modified task definitions.

返回类型:

List[dict]

元指导模型

这种类型的元模型参与基本预测模型的训练过程。元模型可以在训练期间指导基本预测模型以提高性能。

class qlib.model.meta.model.MetaGuideModel

This type of meta-model aims to guide the training process of the base model. The meta-model interacts with the base forecasting models during their training process.

abstract fit(*args, **kwargs)

The training process of the meta-model.

abstract inference(*args, **kwargs)

The inference process of the meta-model.

返回:

Some information to guide the model learning

返回类型:

object

示例

Qlib 提供了 Meta Model（元模型） 模块的实现，DDG-DA，它适应于市场动态。

DDG-DA 包括四个步骤：

  1. 计算元信息并将其封装到 Meta Task（元任务） 实例中。所有的元任务形成一个 Meta Dataset（元数据集） 实例。

  2. 基于元数据集的训练数据训练 DDG-DA。

  3. 进行 DDG-DA 的推理以获取指导信息。

  4. 将指导信息应用于预测模型以提高性能。

可以在 examples/benchmarks_dynamic/DDG-DA/workflow.py 中找到 上述示例 。