Empowering LLMs: Elasticsearch and OpenSearch Integration via the Model Context Protocol

This document details the implementation of a Model Context Protocol (MCP) server designed to seamlessly integrate Elasticsearch and OpenSearch with Large Language Model (LLM) applications. By leveraging MCP, this server provides LLMs with real-time access to indexed data, enabling richer, more informed interactions and AI-driven workflows.

Unlocking the Power of Context for LLMs

LLMs excel at generating human-quality text, but their effectiveness is often limited by their lack of access to current, specific data. This MCP server bridges this gap by providing a standardized interface for LLMs to interact with Elasticsearch and OpenSearch clusters. This integration unlocks a wide range of possibilities, including:

• AI-Powered Search: Enable users to perform complex searches using natural language, leveraging the LLM's understanding of context and intent.
• Intelligent Data Analysis: Allow LLMs to analyze indexed data and generate insights, reports, and summaries.
• Dynamic Content Generation: Use LLMs to create personalized content based on real-time data retrieved from Elasticsearch/OpenSearch.
• Automated Workflows: Integrate LLMs into automated workflows that require access to and manipulation of indexed data.

Core Functionality: A Deep Dive

This MCP server provides a comprehensive set of tools for interacting with Elasticsearch and OpenSearch, categorized into the following key areas:

Index Management

• list_indices: Retrieves a comprehensive list of all indices within the cluster, providing an overview of available data sources.
• get_index: Provides detailed information about specific indices, including mappings (schema), settings (configuration), and aliases (alternative names). This allows LLMs to understand the structure and properties of the data.
• create_index: Enables the creation of new indices, allowing LLMs to dynamically manage data storage based on application needs.
• delete_index: Provides the ability to remove indices, facilitating data lifecycle management and cleanup.

Document Manipulation

• search_documents: The cornerstone of the integration, this function allows LLMs to execute complex search queries against the indexed data. It returns relevant documents based on the query, providing the LLM with the context it needs.
• index_document: Enables the creation or updating of documents within an index. This allows LLMs to write data back to Elasticsearch/OpenSearch, facilitating dynamic data management.
• get_document: Retrieves a specific document by its ID, providing direct access to individual data points.
• delete_document: Removes a document from the index based on its ID, enabling precise data removal.
• delete_by_query: Provides a powerful mechanism for deleting multiple documents based on a query. This allows LLMs to remove data that matches specific criteria.

Cluster Monitoring

• get_cluster_health: Returns essential information about the overall health of the Elasticsearch/OpenSearch cluster, including status, node count, and shard allocation. This allows LLMs to monitor the health of the underlying data infrastructure.
• get_cluster_stats: Provides a high-level overview of cluster statistics, including resource usage, indexing rates, and search performance. This allows LLMs to gain insights into the performance and capacity of the cluster.

Alias Management

• list_aliases: Lists all aliases defined within the cluster, providing an overview of alternative index names.
• get_alias: Retrieves information about a specific alias, including the index it points to and any associated filters.
• put_alias: Creates or updates an alias for a specific index, allowing for flexible index management and routing.
• delete_alias: Removes an alias, providing control over index naming and access.

Getting Started: Configuration and Deployment

Prerequisites

• Docker and Docker Compose (for local cluster setup)
• Python 3.7+

Configuration

1. Environment Variables: Copy the .env.example file to .env and configure the following environment variables:

   • ELASTICSEARCH_HOSTS or OPENSEARCH_HOSTS: The URL(s) of your Elasticsearch/OpenSearch cluster. Include the protocol (e.g., https://) and port (e.g., 9200). Multiple hosts can be specified as a comma-separated list.
   • ELASTICSEARCH_USERNAME or OPENSEARCH_USERNAME: The username for authenticating with the cluster.
   • ELASTICSEARCH_PASSWORD or OPENSEARCH_PASSWORD: The password for authenticating with the cluster.

2. Start Elasticsearch/OpenSearch Cluster (Optional): For local development and testing, you can use Docker Compose to start a cluster:

   # For Elasticsearch
   docker-compose -f docker-compose-elasticsearch.yml up -d
   
   # For OpenSearch
   docker-compose -f docker-compose-opensearch.yml up -d

   The default Elasticsearch username is elastic and password is test123. The default OpenSearch username is admin and password is admin. You can access Kibana/OpenSearch Dashboards from http://localhost:5601.

Integration with Claude Desktop

This MCP server is designed to integrate seamlessly with Claude Desktop, providing a natural language interface to your Elasticsearch/OpenSearch data. There are three primary methods for integration:

Option 1: Smithery Installation (Recommended)

Smithery provides a streamlined installation process:

npx -y @smithery/cli install elasticsearch-mcp-server --client claude

This command automatically downloads and configures the server for use with Claude Desktop.

Option 2: uvx Package Manager

uvx automatically installs the package from PyPI, eliminating the need for local cloning. Add the following configuration to Claude Desktop's claude_desktop_config.json file:

// For Elasticsearch
{
  "mcpServers": {
    "elasticsearch-mcp-server": {
      "command": "uvx",
      "args": [
        "elasticsearch-mcp-server"
      ],
      "env": {
        "ELASTICSEARCH_HOSTS": "https://localhost:9200",
        "ELASTICSEARCH_USERNAME": "elastic",
        "ELASTICSEARCH_PASSWORD": "test123"
      }
    }
  }
}

// For OpenSearch
{
  "mcpServers": {
    "opensearch-mcp-server": {
      "command": "uvx",
      "args": [
        "opensearch-mcp-server"
      ],
      "env": {
        "OPENSEARCH_HOSTS": "https://localhost:9200",
        "OPENSEARCH_USERNAME": "admin",
        "OPENSEARCH_PASSWORD": "admin"
      }
    }
  }
}

Option 3: Local Development with uv

For local development, clone the repository and specify the path to the source code in the claude_desktop_config.json file:

// For Elasticsearch
{
  "mcpServers": {
    "elasticsearch-mcp-server": {
      "command": "uv",
      "args": [
        "--directory",
        "path/to/src/elasticsearch_mcp_server",
        "run",
        "elasticsearch-mcp-server"
      ],
      "env": {
        "ELASTICSEARCH_HOSTS": "https://localhost:9200",
        "ELASTICSEARCH_USERNAME": "elastic",
        "ELASTICSEARCH_PASSWORD": "test123"
      }
    }
  }
}

// For OpenSearch
{
  "mcpServers": {
    "opensearch-mcp-server": {
      "command": "uv",
      "args": [
        "--directory",
        "path/to/src/elasticsearch_mcp_server",
        "run",
        "opensearch-mcp-server"
      ],
      "env": {
        "OPENSEARCH_HOSTS": "https://localhost:9200",
        "OPENSEARCH_USERNAME": "admin",
        "OPENSEARCH_PASSWORD": "admin"
      }
    }
  }
}

The claude_desktop_config.json file is located in:

• macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
• Windows: %APPDATA%/Claude/claude_desktop_config.json

After modifying the configuration file, restart Claude Desktop to load the new MCP server.

Interacting with Claude

Once configured, you can interact with your Elasticsearch/OpenSearch cluster through Claude using natural language commands. Examples include:

• "List all indices in the cluster"
• "How old is the student Bob?" (assuming you have student data indexed)
• "Show me the cluster health status"

Usage with Anthropic MCP Client

For programmatic access, you can use the Anthropic MCP Client:

uv run mcp_client/client.py src/server.py

This command runs a sample client that interacts with the MCP server.

Conclusion

This Elasticsearch/OpenSearch MCP server empowers LLMs with real-time access to indexed data, enabling a new generation of intelligent applications. By providing a standardized interface and a comprehensive set of tools, this server simplifies the integration of LLMs with Elasticsearch and OpenSearch, unlocking the full potential of AI-driven data analysis and automation.