The problem starts when I remove an Azure AI Search connection through the CLI command: az ml connection delete -n search-connection-name --resource-group my-resource-group --workspace-name my-workspace
After deleting this connection, the same agent_client.agents.get_agents() command that worked before now fails with a JSON decode error:
DecodeError: JSON is invalid: Expecting value: line 1 column 1 (char 0)
Exception Type: ServiceInvocationException
Message: Service invocation failed!
Status Code: 404 NotFound
Error Code: UserError/NotFoundError
Reason: Connection search-connection-name can't be found in this workspace
This is strange because no agents are using this connection, yet the service still tries to reference it. Even after creating a new connection with a different name, operations like get_agents() or list_files() continue to fail because the service is looking for the original deleted connection name.
Currently my only solution is to avoid deleting the original connection entirely. Has anyone encountered this behavior before or know how to properly clean up these connection references?
This looks like a caching issue within the Azure AI Agent Service. I encountered something similar when working with Azure ML workspaces where service references weren’t properly invalidated after resource deletion. The service appears to be maintaining stale metadata about your workspace configuration, specifically the connection references that should have been cleaned up. Since your workspace shows no active agents, this is definitely a backend caching problem rather than an actual dependency issue. Try restarting your agent client session completely and reinitializing the connection. If that doesn’t work, you might need to create a support ticket with Microsoft as this seems like a service-level bug where the deletion operation isn’t properly propagating through all the internal service components. I’ve seen similar issues resolve themselves after 24-48 hours, but that’s obviously not an ideal solution for active development work.
This is definitely a workspace state synchronization bug I’ve dealt with before. The agent service is checking connection validity during API calls even when no agents exist, which makes zero sense architecturally.
Here’s what actually fixes it in my experience - you need to force the workspace to rebuild its connection registry. Instead of working around the API, go directly through the Azure portal to your ML workspace and check the “Connections” section. Sometimes deleted connections still show up there as “disconnected” or in an error state.
If you see any remnants there, delete them through the portal UI. If not, the nuclear option that always works is updating your workspace configuration. Add any new connection (doesn’t matter what type), then remove it immediately. This forces the workspace metadata to refresh and clears the internal connection cache.
I’ve also seen this happen when connection deletions fail silently due to permission issues. Double check your RBAC permissions include Microsoft.MachineLearningServices/workspaces/connections/delete. Sometimes the CLI reports success but the backend operation actually failed.
The real problem is Azure AI Agent Service doing eager connection validation instead of lazy loading. Pretty poor design choice that creates exactly this kind of orphaned reference situation.
sounds like the agent service is holding onto workspace-level connection metadata even after deletion. i had this happen with cosmos db connections too. try using the REST API directly to check if theres any hidden workspace config still refrencing that connection - sometimes the cli doesnt clean everything up properly but the api shows whats actually cached internally.
I’ve run into this exact problem before and it turned out to be related to how Azure AI Agent Service handles workspace metadata internally. The issue occurs because the service stores connection references at the workspace level, not just at the agent level, so even empty workspaces can have lingering references. What worked for me was forcing a workspace metadata refresh by temporarily creating a dummy agent with a valid connection, then immediately deleting it. This seems to trigger the service to rebuild its internal connection registry and clear out the stale references. Alternatively, you can try using the Azure ML SDK to explicitly clear the workspace cache by calling workspace.sync_keys() if you have access to the workspace object. The 404 error happens because the service is trying to validate all previously registered connections during the get_agents() call, which is honestly poor API design on Microsoft’s part.