`asset`¶

class AssetScope(*values)[source]¶

runner = 'runner'¶: Asset persists through the entire lifespan of the runner. Can be modified and passed through different epochs.

process = 'process'¶: Asset persists through the a single process call. Can be modified and passed through different nodes but are re-instantiated at the beginning of each epoch.

node = 'node'¶: Asset is re-instantiated every node call.

pydantic model AssetSpecification[source]¶

Specification for a single asset within a tube .yaml file.

Show JSON schema

{
   "title": "AssetSpecification",
   "description": "Specification for a single asset within a tube .yaml file.",
   "type": "object",
   "properties": {
      "id": {
         "title": "Id",
         "type": "string"
      },
      "type": {
         "title": "Type",
         "type": "string"
      },
      "scope": {
         "$ref": "#/$defs/AssetScope"
      },
      "params": {
         "anyOf": [
            {
               "additionalProperties": true,
               "type": "object"
            },
            {
               "type": "null"
            }
         ],
         "default": null,
         "title": "Params"
      },
      "depends": {
         "anyOf": [
            {
               "items": {
                  "type": "string"
               },
               "type": "array"
            },
            {
               "type": "null"
            }
         ],
         "default": null,
         "title": "Depends"
      }
   },
   "$defs": {
      "AssetScope": {
         "enum": [
            "runner",
            "process",
            "node"
         ],
         "title": "AssetScope",
         "type": "string"
      }
   },
   "required": [
      "id",
      "type",
      "scope"
   ]
}

Fields:

depends (list[str] | None)
id (str)
params (dict | None)
scope (noob.asset.AssetScope)
type_ (str)

field depends: list[Annotated[str, AfterValidator(func=_is_absolute_identifier)]] | None = None¶

Roundtrip dependency. Should point to the last node in a given epoch that manipulates the asset.

May only be used with scope == “runner”

Typically this is used with assets that are mutated by multiple nodes in a tube. In that case, nodes should use dependencies to structure the order of mutation: The first node that should have it should depend directly on the asset, and then it and each node should emit the asset so that the successive node can depend on that signal. The node signal that this asset specification depends on will be the version of the asset stored and used in the next processing epoch.

field id: Annotated[str, AfterValidator(func=_is_identifier), AfterValidator(func=_not_reserved)] [Required]¶

The unique identifier of the asset

Constraints:

func = <function _not_reserved at 0x763d55716200>

field params: dict | None = None¶: Initialization parameters

field scope: AssetScope [Required]¶: The scope of the asset. See AssetScope

field type_: Annotated[str, AfterValidator(func=_is_absolute_identifier)] [Required] (alias 'type')¶

The python path to the location of the asset e.g. package_name.module_name.ClassName

Constraints:

func = <function _is_absolute_identifier at 0x763d55715e40>

pydantic model Asset[source]¶

An asset within a processing tube.

Show JSON schema

{
   "title": "Asset",
   "description": "An asset within a processing tube.",
   "type": "object",
   "properties": {
      "id": {
         "title": "Id",
         "type": "string"
      },
      "spec": {
         "$ref": "#/$defs/AssetSpecification"
      },
      "scope": {
         "$ref": "#/$defs/AssetScope"
      },
      "params": {
         "additionalProperties": true,
         "title": "Params",
         "type": "object"
      },
      "obj": {
         "anyOf": [
            {},
            {
               "type": "null"
            }
         ],
         "default": null,
         "title": "Obj"
      }
   },
   "$defs": {
      "AssetScope": {
         "enum": [
            "runner",
            "process",
            "node"
         ],
         "title": "AssetScope",
         "type": "string"
      },
      "AssetSpecification": {
         "description": "Specification for a single asset within a tube .yaml file.",
         "properties": {
            "id": {
               "title": "Id",
               "type": "string"
            },
            "type": {
               "title": "Type",
               "type": "string"
            },
            "scope": {
               "$ref": "#/$defs/AssetScope"
            },
            "params": {
               "anyOf": [
                  {
                     "additionalProperties": true,
                     "type": "object"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "title": "Params"
            },
            "depends": {
               "anyOf": [
                  {
                     "items": {
                        "type": "string"
                     },
                     "type": "array"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "title": "Depends"
            }
         },
         "required": [
            "id",
            "type",
            "scope"
         ],
         "title": "AssetSpecification",
         "type": "object"
      }
   },
   "additionalProperties": false,
   "required": [
      "id",
      "spec",
      "scope"
   ]
}

Config:

extra: str = forbid

Fields:

id (str)
obj (Any | None)
params (dict[str, Any])
scope (noob.asset.AssetScope)
spec (noob.asset.AssetSpecification)

field id: Annotated[str, AfterValidator(func=_is_identifier), AfterValidator(func=_not_reserved)] [Required]¶

The unique identifier of the asset

Constraints:

func = <function _not_reserved at 0x763d55716200>

field obj: Any | None = None¶: Instantiated asset instance

field params: dict[str, Any] [Optional]¶: Initialization parameters

field scope: AssetScope [Required]¶: The scope of the asset. See AssetScope

field spec: AssetSpecification [Required]¶: The specs of the asset. See AssetSpecification

classmethod from_specification(spec: AssetSpecification) → Asset[source]¶

Create an asset from its spec

resolve the asset type
- if a subclass of Asset, just instantiate it
- if not a subclass Asset, wrap it in WrapClassAsset
- if a function, wrap it in WrapFuncAsset

deinit() → None[source]¶

Deinitialize the asset instance.

Default is a no-op. Subclasses do not need to override if they have no deinit logic.

init() → None[source]¶

Initialize the asset instance.

Default is a no-op. Subclasses do not need to override if they have no initialization logic.

update(obj: Any) → None[source]¶

pydantic model WrapClassAsset[source]¶

Wrap a non-Asset class.

Wrapping allows us to use arbitrary classes as Assets within noob. Initializes the inner class to hold the class instance as an asset object.

After instantiating the outer wrapping class, instantiate the inner wrapped class using the params given to the outer wrapping class during init() .

Show JSON schema

{
   "title": "WrapClassAsset",
   "type": "object",
   "properties": {
      "id": {
         "title": "Id",
         "type": "string"
      },
      "spec": {
         "$ref": "#/$defs/AssetSpecification"
      },
      "scope": {
         "$ref": "#/$defs/AssetScope"
      },
      "params": {
         "additionalProperties": true,
         "title": "Params",
         "type": "object"
      },
      "obj": {
         "anyOf": [
            {},
            {
               "type": "null"
            }
         ],
         "default": null,
         "title": "Obj"
      },
      "cls": {
         "default": null,
         "title": "Cls"
      }
   },
   "$defs": {
      "AssetScope": {
         "enum": [
            "runner",
            "process",
            "node"
         ],
         "title": "AssetScope",
         "type": "string"
      },
      "AssetSpecification": {
         "description": "Specification for a single asset within a tube .yaml file.",
         "properties": {
            "id": {
               "title": "Id",
               "type": "string"
            },
            "type": {
               "title": "Type",
               "type": "string"
            },
            "scope": {
               "$ref": "#/$defs/AssetScope"
            },
            "params": {
               "anyOf": [
                  {
                     "additionalProperties": true,
                     "type": "object"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "title": "Params"
            },
            "depends": {
               "anyOf": [
                  {
                     "items": {
                        "type": "string"
                     },
                     "type": "array"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "title": "Depends"
            }
         },
         "required": [
            "id",
            "type",
            "scope"
         ],
         "title": "AssetSpecification",
         "type": "object"
      }
   },
   "additionalProperties": false,
   "required": [
      "id",
      "spec",
      "scope"
   ]
}

Config:

extra: str = forbid

Fields:

cls (type)
obj (noob.asset.T | None)

field cls: type [Required]¶

field obj: T | None = None¶: Instantiated asset instance

deinit() → None[source]¶

init() → None[source]¶

pydantic model WrapFuncAsset[source]¶

Wrap a function to build an Asset.

The function effectively takes the role of __init__(), with the outer wrapping class params being injected as function parameters. The output of the function becomes the asset object.

Show JSON schema

{
   "title": "WrapFuncAsset",
   "type": "object",
   "properties": {
      "id": {
         "title": "Id",
         "type": "string"
      },
      "spec": {
         "$ref": "#/$defs/AssetSpecification"
      },
      "scope": {
         "$ref": "#/$defs/AssetScope"
      },
      "params": {
         "additionalProperties": true,
         "title": "Params",
         "type": "object"
      },
      "obj": {
         "anyOf": [
            {},
            {
               "type": "null"
            }
         ],
         "default": null,
         "title": "Obj"
      },
      "fn": {
         "default": null,
         "title": "Fn"
      }
   },
   "$defs": {
      "AssetScope": {
         "enum": [
            "runner",
            "process",
            "node"
         ],
         "title": "AssetScope",
         "type": "string"
      },
      "AssetSpecification": {
         "description": "Specification for a single asset within a tube .yaml file.",
         "properties": {
            "id": {
               "title": "Id",
               "type": "string"
            },
            "type": {
               "title": "Type",
               "type": "string"
            },
            "scope": {
               "$ref": "#/$defs/AssetScope"
            },
            "params": {
               "anyOf": [
                  {
                     "additionalProperties": true,
                     "type": "object"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "title": "Params"
            },
            "depends": {
               "anyOf": [
                  {
                     "items": {
                        "type": "string"
                     },
                     "type": "array"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "title": "Depends"
            }
         },
         "required": [
            "id",
            "type",
            "scope"
         ],
         "title": "AssetSpecification",
         "type": "object"
      }
   },
   "additionalProperties": false,
   "required": [
      "id",
      "spec",
      "scope"
   ]
}

Config:

extra: str = forbid

Fields:

fn (collections.abc.Callable)

field fn: Callable [Required]¶

deinit() → None[source]¶

init() → None[source]¶

asset¶

`asset`¶