Azure AI Agent Service maintains reference to removed connections despite no active usage

Other Questions AI Agent
1

I’m experiencing a strange issue with Azure AI Agent Service where it seems to keep references to deleted connections.

Currently, my Azure AI Agents service has no active agents. When I execute client_project.agents.get_agents_list(), it returns the expected empty response:

{'object': 'list', 'data': [], 'first_id': None, 'last_id': None, 'has_more': False}

The problem occurs after I remove an Azure AI search connection using this CLI command:

az ml connection delete -n search-hub-connection-v2 --resource-group resource-group-01 --workspace-name workspace-hub-main

Once the connection is deleted, the client_project.agents.get_agents_list() command fails with a JSON decode error:

DecodeError: JSON is invalid: Expecting value: line 1 column 1 (char 0)
ServiceInvocationException: Service invocation failed!
Status Code: 404 NotFound
Error: Connection search-hub-connection-v2 can't be found in this workspace

The issue is that even though I’m not actively using any agents and have created new connections with different names, operations like get_agents_list() and get_files_list() still fail because the service tries to reference the deleted connection.

My current workaround is to avoid deleting the original connection entirely. Has anyone encountered this behavior before? Is there a way to clear these phantom references?

2

Yeah, I’ve hit this exact bug before. The Azure AI Agent Service has some weird caching behavior where it holds onto connection metadata even after deletion.

What’s happening is that the service is storing connection references at the workspace level, and these aren’t getting properly cleaned up when you delete connections. The API calls are still trying to validate against those phantom references.

Here’s what worked for me:

First, try clearing the workspace cache by restarting the Azure ML workspace compute. Sometimes the service just needs a kick.

If that doesn’t work, you can force clear the references by creating a dummy connection with the same name as the deleted one, then properly detaching it from any workspace-level configs before deleting it again.

Another approach is to use the REST API directly to check for any lingering workspace metadata that references the old connection:

az rest --method GET --url "https://management.azure.com/subscriptions/{subscription}/resourceGroups/{rg}/providers/Microsoft.MachineLearningServices/workspaces/{workspace}/metadata"

I also found this helpful video that covers some of the internal workings of Azure AI Agent Service:

Honestly though, this feels like a service bug that Microsoft should fix. The workspace shouldn’t be holding stale connection references like this.