How can I implement cached Gemini content with VertexAI API in Kotlin or Java?

Other Questions API
1

I’m having trouble creating cached content for Gemini using the VertexAI API in Kotlin. Here’s what I’ve tried:

val cachedContent = CachedContent.newBuilder()
    .setSystemInstruction(Content.newBuilder().addParts(Part.newBuilder().setText(longInstructions)).build())
    .setName("my-cache-example")
    .setTtl(Duration.newBuilder().setSeconds(3600).build())
    .setModel("gemini-1.5-flash")
    .build()

val request = CreateCachedContentRequest.newBuilder().setCachedContent(cachedContent).build()
GenAiCacheServiceClient.create().createCachedContent(request)

But I’m getting a 404 error. The weird thing is, I can use the API for normal model responses just fine. I’ve also successfully created cached content using Python.

I’m using the latest versions of google-cloud-vertexai (1.18.0) and google-cloud-aiplatform (3.59.0). My service account seems to have the right permissions too.

What am I missing in my Kotlin code? Any ideas on how to fix this?

2

I’ve encountered similar issues when working with the VertexAI API in Kotlin. One thing to check is whether you’ve explicitly enabled the GenAI Cache API in your Google Cloud project. Sometimes it’s not automatically activated, even if you have access to other VertexAI services.

Also, make sure you’re using the correct endpoint for the GenAI Cache service. The default endpoint might not work for all regions. Try specifying the endpoint explicitly when creating your client:

val settings = GenAiCacheServiceSettings.newBuilder()
    .setEndpoint("genai-cache.googleapis.com:443")
    .build()
val client = GenAiCacheServiceClient.create(settings)

If that doesn’t solve it, double-check your service account permissions. It might need the ‘Vertex AI User’ role specifically for cached content operations. Hope this helps!

3

hey there, i’ve run into this too. make sure ur using the right region for ur project. also, double check if the GenAI Cache API is actually enabled in ur GCP console. sometimes it’s not on by default. if that don’t work, try using a different model version like gemini-1.0-pro instead of 1.5-flash. good luck!

4

I’ve dealt with similar challenges integrating VertexAI’s Gemini in Java. One thing that often trips people up is the API versioning. Make sure you’re using the latest client libraries and API version. Sometimes, the documentation lags behind the actual API changes.

Another potential issue could be related to your project configuration. Double-check that you’ve set up your application default credentials correctly. You might need to explicitly set the GOOGLE_APPLICATION_CREDENTIALS environment variable pointing to your service account key file.

If you’re still hitting a wall, try enabling debug logging for the VertexAI client. It can provide more detailed error messages that might pinpoint the exact issue. You can do this by adding a logging configuration to your application.

Lastly, if all else fails, reaching out to Google Cloud support can be surprisingly helpful. They might be aware of known issues or have specific workarounds for your use case.