The Quotient MCP Server is our synchronous API for steering agent behavior in real-time and in production using models we develop. Quotient MCP is available in two ways:

Tools

evaluate_tool_call

Evaluates whether an AI agent correctly used available tools in a conversation context. Parameters:
  • available_tools: Array of tool definitions with schemas
  • message_history: Array of conversation messages
Returns:
  • score: "correct", "incorrect_tool", "incorrect_parameter_names", or "incorrect_parameter_values"
  • reason: Detailed explanation list

Example Usage

Input:
{
  "available_tools": [
    {
      "name": "google-play-developer",
      "description": "Get apps by a developer on Google Play",
      "input_schema": {
        "type": "object",
        "properties": {
          "devId": {"type": "string", "description": "Developer ID"},
          "num": {"type": "number", "default": 60, "description": "Number of results"},
          "lang": {"type": "string", "default": "en", "description": "Language code"},
          "country": {"type": "string", "default": "us", "description": "Country code"}
        },
        "required": ["devId"]
      }
    }
  ],
  "message_history": [
    {
      "role": "user", 
      "content": "Get apps by developer 'com.example.dev' with 50 results in English for US market"
    },
    {
      "role": "assistant",
      "content": "I'll fetch the apps for that developer.",
      "tool_calls": [
        {
          "function": {
            "name": "google-play-developer",
            "arguments": {
              "devId": "com.example.dev",
              "num": 50,
              "lang": "en",
              "country": "us"
            }
          }
        }
      ]
    }
  ]
}
Output:
{
  "score": "correct",
  "reason": []
}

Underlying Model

The evaluation is powered by the Limbic Tool Use Evaluator, a fine-tuned Qwen 2.5 0.5B model specifically designed for MCP function call evaluation.

Integration Examples

Cursor IDE

You can use the Quotient MCP server in Cursor IDE by adding the following to your .cursor/mcp.json:
{
  "mcpServers": {
    "quotient-mcp-server": {
      "url": "https://mcp.quotientai.co/mcp/"
    }
  }
}
Then, add an evaluation rule in .cursor/rules/evaluate-tool-call.mdc to always apply the evaluate_tool_call function after any agent tool call.
---
alwaysApply: true
---
# Evaluate Tool Calls Rule

## Critical Rule

- Always use the `evaluate_tool_call` function after any agent tool call to evaluate proper usage of the tool
- Use this function to validate that tools are being called correctly with proper parameters
- Apply this evaluation when assessing agent performance or troubleshooting tool usage issues

## When to Apply

- After Any Tool Call
- When reviewing agent conversations that involve tool calls
- When validating that an AI agent correctly used available tools
- When checking if tool parameters were set correctly
- When assessing overall agent tool calling performance

## Example Usage

<example>
When evaluating if an agent correctly used tools, always call:
evaluate_tool_call(available_tools, message_history)

This ensures proper validation of tool usage patterns and parameter correctness.
</example>

Amp

You can use the Quotient MCP server in Amp through both CLI and editor extension configurations. Configure the server in your editor extension settings or CLI configuration:
{
  "amp.mcpServers": {
    "quotient-mcp": {
      "url": "https://mcp.quotientai.co/mcp/"
    }
  }
}

CLI Configuration

The CLI configuration file location varies by operating system:
  • Windows: %APPDATA%\amp\settings.json
  • macOS: ~/.config/amp/settings.json
  • Linux: ~/.config/amp/settings.json
Add the MCP server configuration to your platform-specific settings file.

Editor Extension Configuration

For editor extensions that support Amp, add the configuration to your editor settings (e.g., .vscode/settings.json). All settings use the amp. prefix. Then, add evaluation rules in your project directory as AGENT.md:
# Evaluate Tool Calls Rule

## Critical Rule

- Always use the `evaluate_tool_call` function after any agent tool call to evaluate proper usage of the tool
- Use this function to validate that tools are being called correctly with proper parameters
- Apply this evaluation when assessing agent performance or troubleshooting tool usage issues

## When to Apply

- After Any Tool Call
- When reviewing agent conversations that involve tool calls
- When validating that an AI agent correctly used available tools
- When checking if tool parameters were set correctly
- When assessing overall agent tool calling performance

## Example Usage

<example>
When evaluating if an agent correctly used tools, always call:
evaluate_tool_call(available_tools, message_history)

This ensures proper validation of tool usage patterns and parameter correctness.
</example>

<example type="invalid">
Reviewing agent tool usage without using evaluate_tool_call function to validate the behavior.
</example>

Claude Desktop

Complete setup guide for integrating Quotient MCP with Claude Desktop using the stdio transport.

Prerequisites

Setup Steps

1. Clone the Repository
git clone https://github.com/quotient-ai/quotient-mcp
cd quotient-mcp
2. Configure Claude Desktop Add the following configuration to your Claude Desktop config file: Access your config file:
  • via UI: Settings → Developer → Local MCP → Edit Config
  • macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
  • Windows: %APPDATA%\Claude\claude_desktop_config.json
Add this configuration:
{
  "mcpServers": {
    "quotient-mcp": {
      "command": "uv",
      "args": [
        "run",
        "--with", "fastmcp>=2.10.6",
        "--with", "mcp==1.12.2", 
        "--with", "requests>=2.28.0",
        "fastmcp",
        "run",
        "/absolute/path/to/your/quotient-mcp/server.py"
      ]
    }
  }
}
Important: Replace /absolute/path/to/your/quotient-mcp/server.py with the full absolute path to your server.py file.
3. Configure Personal Preferences (Recommended) To automatically evaluate tool usage, add this rule to your Claude Desktop preferences:
  1. Go to Settings → Profile → Personal Preferences
  2. Copy and paste the following:
After making any tool call always use quotient-mcp evaluate_tool_call to evaluate proper tool use.
4. Restart Claude Desktop Completely quit Claude Desktop and reopen it for the changes to take effect. Once configured, you’ll have access to the quotient-mcp tools in Claude Desktop.

Claude Code

You can use the Quotient MCP server in Claude Code by adding the following to your project:
claude mcp add --transport http https://mcp.quotientai.co/mcp/
Then, add to your project instructions to always use the evaluate_tool_call function after any agent tool call.
# Project Instructions

## Tool Usage Requirements

- Always use `mcp__http-server__evaluate_tool_calls` after every tool call to evaluate the correctness of tool usage

OpenAI Agents SDK

You can use the Quotient MCP server in various agent SDKs such as the OpenAI Agents SDK by adding the following to your agent configuration:
from agents.mcp import MCPServerStreamableHttp

# custom prompt prefix for tool use evaluation
TOOL_USE_EVAL_PROMPT_PREFIX = """
After using any tool, you should use the tool_use_evaluator from quotient_mcp to evaluate the effectiveness and accuracy of the tool usage.
"""

# initialize MCP server connection
quotient_mcp = MCPServerStreamableHttp(
    params={
        "url": "https://mcp.quotientai.co/mcp/",
    },
    name="Quotient MCP Server",
)

# use in your agent configuration
agent = Agent(
    name="customer_service",
    instructions=TOOL_USE_EVAL_PROMPT_PREFIX + "Your agent instructions here...",
    mcp_servers=[quotient_mcp]
)

Open Source

Quotient MCP is open sourced at https://github.com/quotient-ai/quotient-mcp.

Transports

Quotient MCP supports both HTTP and stdio transports:

HTTP Transport (Remote Hosting)

The HTTP transport is used for remote hosting and is available at https://mcp.quotientai.co/mcp/.

Stdio Transport (Local Integration)

The stdio transport is designed for direct MCP client integration where the client manages the server process through standard input/output. Refer to the Claude Desktop section for an example implementation.