generation_filters

Text generation response filters

These filters are used to clean up the text generated by the model before returning back to the client. Filters are used when models are not returning properly formatted output and typical fizes sich as fine-tuning or multiple generations to accomidate format are not feasible.

Classes

ChatMLEndFilter

Bases: GenerationFilter

ChatML end tag filter

Wrote this filter for (perhaps) an edge case where Midnight-Miqu-70B-v1.0 generates ChatML just fine but the ChatML tags are not considered special tokens. There is some weirdness in this model anyway where it tends to dender the last chatmp tag partially.

Functions

filter

filter(data: str, **kwargs) -> str

ChatMLStartFilter

Bases: GenerationFilter

ChatML start tag filter

Functions

filter

filter(data: str, **kwargs) -> str

GenerationFilter

Bases: ABC

Abstract class for generation response filters

Functions

filter abstractmethod

filter(data: str, **kwargs) -> str

NamePrefixFilter

Bases: GenerationFilter

Strip out agent name prefixes

Functions

filter

filter(data: str, **kwargs) -> str