Issues with executing Copilot actions on self-hosted server using Gemini API compared to public service

Other Questions langchain
1

I’m working on a task management app built with React and CopilotKit. The app incorporates features to handle message responses, read data, and execute user actions such as adding tasks.

Setup Overview:

Initially, I was using the public Copilot service with its API key, and everything functioned without problems. All features, including actions and responsive messages, worked smoothly. However, after the trial period ended, I transitioned to a self-hosted server setup using the LangChain adapter along with my own Gemini API key.

Current Issue:

In my custom server setup, I can successfully handle messages and process readable data, but the useCopilotAction functionality does not work at all. While the AI responds to messages appropriately, the action triggers simply won’t activate.

Current Code Implementation:

import express from "express";
import {
  CopilotRuntime,
  LangChainAdapter,
  copilotRuntimeNodeHttpEndpoint,
} from "@copilotkit/runtime";
import { ChatGoogleGenerativeAI } from "@langchain/google-genai";
import dotenv from "dotenv";

dotenv.config();

const server = express();
const aiModel = new ChatGoogleGenerativeAI({
  model: "gemini-1.5-flash",
  apiKey: process.env.GEMINI_API_KEY,
  region: "us-central1",
});

const adapter = new LangChainAdapter({
  chainFn: async ({ messages, tools }) => {
    return aiModel.bindTools(tools).stream(messages);
  },
});

server.use("/api/copilot", (req, res, next) => {
  const runtime = new CopilotRuntime();
  const handler = copilotRuntimeNodeHttpEndpoint({
    endpoint: "/api/copilot",
    runtime,
    serviceAdapter: adapter,
  });

  return handler(req, res, next);
});

server.listen(3000, () => {
  console.log("Server running on http://localhost:3000/api/copilot");
});

What steps can I take to ensure the action functionality works on my self-hosted server setup with the Gemini API? Are there specific configurations that differ from the public service?

2

Been there multiple times with complex API integrations and runtime persistence. This is super common with self-hosted setups.

The other suggestions about runtime persistence and tool handling work, but you’ll hit scaling issues and maintenance headaches later. Every CopilotKit update or Gemini API change means more debugging.

That’s why I switched to automation tools for these workflows. Build the entire task management flow visually - handle message processing, data reading, and action triggers without wrestling with runtime contexts or tool binding.

You get built-in error handling, retry logic, and state management without custom middleware. When APIs change, just update the connection instead of rewriting adapter code.

Migrated several similar projects this way. Maintenance overhead drops to almost zero.

Check it out: https://latenode.com

3

First, check your CopilotKit version compatibility - I hit major breaking changes between versions with custom adapters. LangChain integration changed big time in recent updates. Also check if you’re hitting Gemini API quota or rate limits silently. The hosted service handles retries well, but raw Gemini API calls can timeout without proper error reporting. Your chainFn looks right, but add explicit error handling around the tool binding. Gemini sometimes fails to recognize tool schemas if they’re not perfectly formatted. The hosted service has built-in schema validation that catches this automatically.

4

check your gemini api permissions - actions need different scopes than basic chat. also log the tools array before bindTools() to make sure it’s passing correctly. sometimes gemini ignores tool calls if your model version doesn’t support them right.

5

Had this exact same problem when I switched from CopilotKit’s hosted service to my own Gemini setup. Your issue is probably in how the LangChain adapter handles tool calling with Gemini. In your chainFn, you’ve got to explicitly handle tool responses. Don’t just return the stream directly - await it first and process any tool calls that come back. Also double-check your client-side useCopilotAction has proper error handling. Gemini fails silently sometimes when it can’t parse action parameters. What helped me was adding more specific type definitions in the action parameters. Gemini needs clearer guidance on which tools to trigger. The hosted service does all this processing automatically, which is why it worked there but breaks with raw Gemini.

6

Hit this exact issue last year during a similar migration. You’re creating a new CopilotRuntime instance on every request, which destroys the action context.

Move runtime creation outside your middleware:

const runtime = new CopilotRuntime();

server.use("/api/copilot", (req, res, next) => {
  const handler = copilotRuntimeNodeHttpEndpoint({
    endpoint: "/api/copilot",
    runtime, // use the same instance
    serviceAdapter: adapter,
  });
  
  return handler(req, res, next);
});

Add CORS headers if client and server run on different ports:

server.use((req, res, next) => {
  res.header('Access-Control-Allow-Origin', '*');
  res.header('Access-Control-Allow-Methods', 'GET, POST, OPTIONS');
  res.header('Access-Control-Allow-Headers', 'Content-Type, Authorization');
  next();
});

Hosted service handles runtime persistence automatically. Self-hosted doesn’t - you manage it yourself. Messages work but actions fail because actions need persistent runtime context to keep state between calls.

7

had a similar problem too. gemini can be a bit finicky with triggering actions. try lowering the temperature in your ChatGoogleGenerativeAI settings to 0. also, make sure your action descriptions are really clear and check if you’re passing the tools correctly in chainFn. logging could help!