modalities.inference.text package

Submodules

modalities.inference.text.config module

class modalities.inference.text.config.TextInferenceComponentConfig(**data)[source]

Bases: BaseModel

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Parameters:

  • model (Annotated[Module, <modalities.config.pydantic_if_types.PydanticThirdPartyTypeIF object at 0x7f84b6c2e910>])

  • tokenizer (Annotated[TokenizerWrapper, <modalities.config.pydantic_if_types.PydanticThirdPartyTypeIF object at 0x7f84b6c2ebd0>])

  • prompt_template (str)

  • sequence_length (int)

  • temperature (float | None)

  • eod_token (str | None)

  • device (Annotated[device, <modalities.config.pydantic_if_types.PydanticThirdPartyTypeIF object at 0x7f84b6c2f210>])

device: Annotated[device]
eod_token: Optional[str]
model: Annotated[Module]
model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

classmethod parse_device(device)[source]
Return type:

device

prompt_template: str
sequence_length: int
temperature: Optional[float]
tokenizer: Annotated[TokenizerWrapper]

modalities.inference.text.inference_component module

class modalities.inference.text.inference_component.TextInferenceComponent(model, tokenizer, prompt_template, sequence_length, temperature, eod_token, device)[source]

Bases: object

Parameters:
generate_tokens(context)[source]
Parameters:

context (str)

run()[source]

Module contents