Skip to content

service

Agent AI chat service module

Attributes

Classes

AIService 

AIService(sa_session: SASession, neo4j_session: NJSession)

Bases: BaseService

AI Service Layer

Functions

chat_completion 
chat_completion(
    request: ChatCompletionRequest,
) -> ChatCompletionResponse
chat_completion_stream 
chat_completion_stream(
    request: ChatCompletionRequest,
) -> Iterator[str]

OpenAI API compatible streaming chat completion implementation.

Note that for this method to work properly a LangChain monkeypatch needs to be applied. See: https://github.com/langchain-ai/langchain/issues/19185

Parameters:

Yields:

  • str ( str ) –

    A string representing a chat completion.

Returns:

  • Iterator[str] –

    Iterator[str]: An iterator that yields chat completions.

EleanorAIGenerationStats

Bases: BaseDataModel

Attributes

index class-attribute instance-attribute 
index: int = Field(..., description='Generation index')
orig_text class-attribute instance-attribute 
orig_text: str = Field(
    ...,
    description="Original generation text before filters were applied",
)
perplexity class-attribute instance-attribute 
perplexity: Optional[float] = Field(
    default=None, description="Generation perplexity score"
)

FilteringStreamBuffer 

FilteringStreamBuffer(
    output_filters: List[GenerationFilter],
    stream_buffer_flush_hwm: int = 50,
    filter_kwargs: Optional[Dict[str, Any]] = None,
)

Attributes

stream_buffer_flush_hwm instance-attribute 
stream_buffer_flush_hwm = stream_buffer_flush_hwm

Functions

add 
add(chunk: ChatCompletionChunk) -> None
check_and_filter 
check_and_filter(
    generation_index: int, skip_length_check: bool = False
) -> Optional[ChatCompletionChunk]
flush 
flush() -> List[ChatCompletionChunk]

StreamingOpenAICallback

Bases: BaseCallbackHandler

Attributes

last_chunk class-attribute instance-attribute 
last_chunk: Optional[ChatCompletionChunk] = None

Functions

on_llm_new_token 
on_llm_new_token(
    token: str,
    *,
    chunk: Optional[
        Union[GenerationChunk, ChatGenerationChunk]
    ] = None,
    run_id: UUID,
    parent_run_id: Optional[UUID] = None,
    **kwargs: Any
) -> Any

Run on new LLM token. Only available when streaming is enabled.

Parameters:

  • token (str) –

    The new token.

  • chunk (GenerationChunk | ChatGenerationChunk, default: None ) –

    The new generated chunk,

Functions

calculate_perplexity 

calculate_perplexity(logprobs)

stream_worker 

stream_worker(
    env,
    formatted_prompt,
    response_queue,
    stream_done,
    response_id,
    system_fingerprint,
    fsb,
)