Dataproc

You are running a production-ready MCP server that enables Google Cloud Dataproc operations with intelligent parameter injection, strong security, and rich tooling. This server integrates with client applications like Claude.ai and Roo (VS Code) to provide seamless access to 22 production-ready MCP tools for managing clusters, jobs, profiles, analytics, and more.

How to use

You connect your MCP client (such as Claude.ai or Roo) to the Dataproc MCP Server to perform cluster management, job submissions, analytics, and configuration tasks. Use the client to invoke the available MCP tools, benefit from automatic parameter injection, and leverage semantic search and analytics features. Ensure you have authentication configured if you enable OAuth or service account-based access.

How to install

Prerequisites: ensure Node.js is installed on your machine. Install Node.js from the official site and verify versions with node -v and npm -v.

Step-by-step commands to get started locally using a standard npm-based workflow.

# Global installation (recommended for quick starts)
npm install -g @dipseth/dataproc-mcp-server

dataproc-mcp --setup

# Generate config and start the server
# Edit the generated config file at the indicated path if needed
# Start the server with your desired options

dataproc-mcp

Configuration and operation notes

You can run the MCP server with short, explicit commands that inject parameters automatically. You can also provide a custom config file path to control behavior and integration details.

Key configuration steps include generating a config, adjusting authentication, and starting the server. The configuration file allows you to set the Dataproc config path, logging level, and other operational parameters.

Claude Web App Compatibility

The Dataproc MCP Server supports full Claude.ai integration with HTTPS tunneling and OAuth. You can run the server in a way that makes all 22 MCP tools accessible to Claude.ai, with secure external access.

To start and expose the server for Claude.ai, you typically configure OAuth and TLS, then establish an external tunnel for Claude.ai to connect to your local server. You will run the server with appropriate CLI flags to enable HTTP and OAuth support, and you may use a tunnel service to provide a stable public URL.

Security best practices

Use service account impersonation for enterprise-grade authentication when possible. Enable robust input validation with schemas, implement rate limiting, rotate credentials regularly, and enable audit logging to track security events.

Troubleshooting tips

If you encounter connectivity issues, verify your OAuth configuration, TLS certificates, and firewall rules. Check server logs for authentication failures or validation errors and ensure the MCP client is configured to communicate with the correct URL and port.

Developer notes

This server supports multi-environment configurations (dev/staging/production), semantic search for knowledge retrieval, and a generic type conversion system that reduces boilerplate code. It includes extensive testing, monitoring, and CI/CD quality gates to maintain reliability.

Available tools

start_dataproc_cluster

Create and start new clusters with smart defaults and profiling integration.

create_cluster_from_yaml

Create a cluster from a YAML configuration file using template-driven setup.

create_cluster_from_profile

Create a cluster using predefined profiles with reduced parameters.

list_clusters

List clusters with semantic filtering and pagination.

list_tracked_clusters

List MCP-created clusters with profile filtering.

get_cluster

Get detailed information for a specific cluster with reduced params.

delete_cluster

Delete existing clusters with safe defaults and region/project fallbacks.

get_zeppelin_url

Retrieve the Zeppelin notebook URL for a cluster.

submit_hive_query

Submit Hive queries to clusters with async support and timeouts.

submit_dataproc_job

Submit Spark, PySpark, or Presto jobs with multi-engine support and local file staging.

cancel_dataproc_job

Cancel running or pending jobs to enable emergency cost control.

get_job_status

Get real-time status of a running job using minimal parameters.

get_job_results

Fetch job outputs with auto-pagination and formatting.

get_query_status

Track Hive query status with minimal input.

get_query_results

Retrieve Hive query results with smart pagination.

list_profiles

List available cluster profiles with category filtering.

get_profile

Get detailed cluster profile configuration by profile ID.

query_cluster_data

Query stored cluster data using natural language queries.

check_active_jobs

Provide a quick status overview of all active jobs across projects.

get_cluster_insights

Provide analytics on cluster configurations and performance.

get_job_analytics

Analyze job performance with success rates and error patterns.

query_knowledge

Query the knowledge base for clusters, jobs, errors using natural language.