Getting 404 Error When Connecting to Local LLM API

Other Questions OpenAI
1

I’m trying to connect QlikSense to a local Llama3 model but keep running into issues. Every time I test my API connection, I get a 404 Not Found error from the server.

Here’s my setup:

Endpoint: http://localhost:5000/api/v1/completions
HTTP Method: POST

Request payload:

{
  "conversation": [
    {
      "speaker": "user",
      "message": "Hi there! I go by the name Charlie. Can you tell me what my name is?"
    }
  ],
  "format": "chat",
  "template": "ChatML"
}

I’ve double checked that my local server is running on port 5000, but the connection still fails. Has anyone encountered this before? What could be causing the 404 response?

2

You’re mixing up API formats. Most local Llama3 setups use OpenAI’s structure, not what you’ve got there. Drop that conversation array with speaker/message - you need a messages array with role/content pairs instead. Try this: {“messages”: [{“role”: “user”, “content”: “Hi there! I go by the name Charlie. Can you tell me what my name is?”}], “model”: “llama3”}. Those template and format fields aren’t standard OpenAI parameters either. Test it with a simple REST client using the proper OpenAI format first, then worry about QlikSense integration.

3

check ur llama3 server startup - most use diff paths than what ur hitting. mine runs on /v1/completions not /api/v1/completions. also that payload looks wrong for llama3, should prob use “prompt” field instead of “conversation”. try curl first to test the endpoint.

4

This looks like a server config issue, not the endpoint. I hit the same thing when my local LLM wasn’t exposing the API properly. Did you start your Llama3 server with the right flags? Most need specific parameters to turn on HTTP API access. Try hitting http://localhost:5000/docs or http://localhost:5000/openapi.json - if there’s API docs, you’ll see what endpoints are actually running. If nothing loads, the API module probably isn’t enabled. Also check your server console when you make the request. It should log incoming connections even with wrong paths, so you can tell if it’s routing vs connectivity.

5

Had this exact issue a few months ago - turned out to be port config despite everything looking right. First, check if your Llama3 server is binding to all interfaces. Sometimes it only binds to 127.0.0.1, which screws up routing. Try hitting the base URL without any path to see if you get anything back. Most local LLM servers don’t enable API endpoints by default either. You’ll probably need flags like --api or --enable-api when starting it, depending on your setup. Also check your server logs to see what requests are coming through - that’ll tell you if QlikSense is actually reaching your server or if it’s a network problem.

6

That 404 error indicates that the endpoint cannot be found on your server. I encountered a similar issue with my local LLM setup before. Different servers can have varying API paths, so if you’re using text-generation-webui, you might want to try using /v1/chat/completions instead of /api/v1/completions. It’s also essential to ensure that your server supports the completions endpoint, as some may only have generate or chat endpoints. You can verify the available endpoints by accessing http://localhost:5000 in your browser or by consulting your server documentation for the correct API paths. Additionally, make sure that your request payload is formatted correctly according to your server’s specifications.