modalities.inference.text package

Submodules

modalities.inference.text.config module

modalities.inference.text.inference_component module

class modalities.inference.text.inference_component.TextInferenceComponent(model, tokenizer, prompt_template, sequence_length, temperature, eod_token, device)

Bases: object

Parameters:

• model (Module)

• tokenizer (TokenizerWrapper)

• prompt_template (str)

• sequence_length (int)

• temperature (float)

• eod_token (str)

• device (device)

generate_tokens(context)

Parameters:

context (str)

run()

Module contents