You can use Quotient’s SDK to log inputs and outputs from your retrieval or search-augmented AI applications.

​
Initialize the Logger

Initialize the Quotient logger with configuration settings for your application:

from quotientai import QuotientAI, DetectionType

quotient = QuotientAI(api_key="your-quotient-api-key")

logger = quotient.logger.init(
    app_name="my-first-app",
    environment="dev",
    detections=[DetectionType.HALLUCINATION, DetectionType.DOCUMENT_RELEVANCY],
    detection_sample_rate=1.0,
)

​
Parameters:

  • app_name (string): Name of your application. Core parameter used to identify the source of logs in the Quotient dashboard.
  • environment (string): Environment where your application is running (e.g., “dev”, “staging”, “prod”). Core parameter used to help segregate logs by deployment environment.
  • tags (object): Key-value pairs for categorizing and filtering logs. Useful for slicing analytics by dimensions like customer, feature, model version, etc. Example: {"model": "gpt-4", "feature": "customer-support"}
  • sample_rate (float): Value that determines what percent of logs to persist. Valid range of 0.0 to 1.0. Defaults to 1.0.
  • detections (array): List of detection types to run. Available options:
    • DetectionType.HALLUCINATION - Detects potential hallucinations in model outputs
    • DetectionType.DOCUMENT_RELEVANCY - Evaluates how relevant retrieved documents are to the user query
  • detection_sample_rate (float): Value that determines what percent of logs to run detections on. Valid range of 0.0 to 1.0. Defaults to 0.0.

​
Send Logs

Log model interactions with context and metadata for analysis.

log_id = quotient.log(
    user_query="How do I cook a goose?",
    model_output="The capital of France is Paris",
    documents=["Here is an excellent goose recipe..."]
)

​
Parameters:

  • user_query (string): The input query or prompt sent to the model. Required when hallucination or document relevancy detection is enabled.
  • model_output (string): The response generated by the model. Required when hallucination detection is enabled.
  • documents (array): List of document contents used as context for the model. Can be strings or dictionaries with page_content and optional metadata. Used in hallucination detection and attribution analysis. Required when document relevancy detection is enabled. If a dictionary is passed, it must have the key page_content, containing the document text, and optionally metadata, containing any document metadata you want to capture about the document. Example: 
    [
    "Sample Document 1 without metadata",
    {"page_content": "Sample Document 2 with metadata", "metadata": {"source": "google.com"}},
    {"page_content": "Sample Document 3 without metadata"}
]
    NOTE: One of documents, message_history, or instructions is required when hallucination detection is enabled.
  • message_history (array): Previous messages in the conversation, following the OpenAI message format with role and content keys. Used in hallucination detection and attribution analysis. Example: 
    [
    {"role": "system", "content": "You are an expert on geography."},
    {"role": "user", "content": "What is the capital of France?"},
    {"role": "assistant", "content": "The capital of France is Paris"}
]
    NOTE: One of documents, message_history, or instructions is required when hallucination detection is enabled.
  • instructions (array): List of instructions provided to the model. Used in hallucination detection and attribution analysis. Example: 
    [
    "You are a helpful assistant that answers questions about the world.",
    "Answer the question in a concise manner. If you are not sure, say 'I don't know'."
]
    NOTE: One of documents, message_history, or instructions is required.
  • tags (object): Additional tags to associate with each log entry. Example: {"model": "gpt-4", "feature": "customer-support"}

​
Returns:

  • log_id (string): A UUID representing the unique identifier for the logged event.

​
Retrieve Logs

Retrieve logs sent to Quotient using the client.

logs = quotient.logs.list()

​
Returns:

  • logs (array): An array of Log objects containing the following fields:
    • id (string): Unique identifier for the log entry.
    • app_name (string): Name of the application that generated the log.
    • environment (string): Environment where the log was generated (e.g., “dev”, “prod”).
    • hallucination_detection (boolean): Whether hallucination detection was enabled for this log.
    • user_query (string): The original user query or prompt that was logged.
    • model_output (string): The model’s response that was logged.
    • documents (array): List of documents used as context for the model. Can be strings or LogDocument objects.
    • message_history (array): Previous messages in the conversation, following the OpenAI message format.
    • instructions (array): List of instructions provided to the model.
    • tags (object): Dictionary of tags associated with the log entry.
    • created_at (datetime): Timestamp when the log was created.
    • status (string): Current status of the log entry.
    • updated_at (datetime): Timestamp when the log was last updated.
    • has_hallucination (boolean): Whether the model output was detected to contain hallucinations.
    • hallucination_detection_sample_rate (float): Sample rate used for hallucination detection.
    • evaluations (array): List of evaluation results for the log entry.

​
Asynchronous Python client

You can also use the asynchronous Python client to log data and retrieve logs.

from quotientai import AsyncQuotientAI, DetectionType

quotient = AsyncQuotientAI(api_key="your-quotient-api-key")

quotient.logger.init(
    app_name="my-first-app",
    environment="dev",
    sample_rate=1.0,
    detections=[DetectionType.HALLUCINATION, DetectionType.DOCUMENT_RELEVANCY],
    detection_sample_rate=1.0,
)

log_id = await quotient.log(
    user_query="How do I cook a goose?",
    model_output="The capital of France is Paris",
    documents=["Here is an excellent goose recipe..."]
)

logs = await quotient.logs.list()
print(logs)