Skip to content

mistral_instruct

Mistral-instruct family to canonical string parser

Attributes

STATE_INITIAL module-attribute 

STATE_INITIAL = 'INITIAL'

STATE_USER_MESSAGE module-attribute 

STATE_USER_MESSAGE = 'user'

states module-attribute 

states = ((STATE_USER_MESSAGE, 'exclusive'))

t_user_error module-attribute 

t_user_error = t_error

tokens module-attribute 

tokens = (
    "BOS",
    "BEGINTAG",
    "AI_MESSAGE",
    "USER_MESSAGE",
    "ENDTAG",
    "EOS",
)

Classes

MistralInstruct 

MistralInstruct(env: Environment)

Bases: BaseCanonical

Attributes

name property 
name: str

Functions

to_canonical 
to_canonical(data: str, **kwargs) -> CanonicalChat

Functions

t_AI_MESSAGE 

t_AI_MESSAGE(t: LexToken)

[\s\S]+?(?=(|||$))

t_BEGINTAG 

t_BEGINTAG(t: LexToken)
\[INST\]

t_BOS 

t_BOS(t: LexToken)

t_ENDTAG 

t_ENDTAG(t: LexToken)
\[/INST\]

t_EOS 

t_EOS(t: LexToken)

t_error 

t_error(t: LexToken)

t_user_ENDTAG 

t_user_ENDTAG(t: LexToken)
\[/INST\]

t_user_USER_MESSAGE 

t_user_USER_MESSAGE(t: LexToken)

[\s\S]+?(?=(|$))