Custom Template Not Working with /v1/chat/completions Endpoint #1

New issue

Closed

opened 2025-10-23 14:21:55 -05:00 by hendkai · 3 comments

hendkai commented

2025-10-23 14:21:55 -05:00

Issue Summary

When using a custom template with the /v1/chat/completions endpoint, Sortana fails to send a properly formatted request. LM Studio responds with 'messages' field is required, indicating that the request body is malformed.

Steps to Reproduce

Setup:
- LLM Server: LM Studio with openai/gpt-oss-20b model
- Endpoint URL: http://192.168.1.157:1234/v1/chat/completions
- Prompt Template: Custom

Custom Template Used:

[
  { "role": "system", "content": "You are an email-classification assistant. Determine whether the email satisfies the user's criterion. Return ONLY a JSON object on a single line of the form:\n{\"match\": true} - if the email satisfies the criterion\n{\"match\": false} - otherwise\n\nDo not add any other keys, text, or formatting." },
  { "role": "user", "content": "Email:\n{{email}}\n\nQuestion: {{query}}" }
]

Expected Behavior:
- Request should be sent with proper JSON structure to /v1/chat/completions
- Request body should contain a "messages" array as per OpenAI Chat API spec
Actual Behavior:
- Sortana converts the custom template to a string and puts it in the "prompt" field (Completion API format) instead of parsing it as "messages" (Chat API format)
- LM Studio receives: {"prompt": "[{\"role\": \"system\", ...}]", "max_tokens": ..., ...}
- LM Studio rejects with: Error: 'messages' field is required

Debug Output (Sortana)

Last Request Payload:

{
  "prompt": "[\n  { \"role\": \"system\", \"content\": \"...\", ... }\n]\n",
  "max_tokens": 4096,
  "temperature": 0.6,
  "top_p": 0.95,
  "seed": -1
}

LM Studio Log:

2025-10-23 21:06:53 [INFO] [LM STUDIO SERVER] Received OPTIONS request. {}
2025-10-23 21:06:53 [ERROR] 'messages' field is required

Expected Request Format

For /v1/chat/completions, the request should be:

{
  "model": "openai/gpt-oss-20b",
  "messages": [
    { "role": "system", "content": "..." },
    { "role": "user", "content": "..." }
  ],
  "max_tokens": 4096,
  "temperature": 0.6
}

Root Cause

When using a custom template with /v1/chat/completions, Sortana:

Treats the template as a string instead of parsing it as JSON
Inserts this string into the "prompt" field (Completion API format)
Does not recognize that /v1/chat/completions requires a properly formatted "messages" array

Workaround

Use the /v1/completions endpoint with the OpenAI/ChatML standard template instead. This works correctly but may have other limitations depending on the model configuration.

Additional Info

Sortana Version: Latest from git.jordanwages.com
LLM Server: LM Studio
Model: openai/gpt-oss-20b
Endpoints Tested:
- /v1/completions (works with ChatML template)
- /v1/chat/completions (fails with custom template)
Note: The model works fine in LM Studio's built-in chat interface, confirming the model itself is functional

### Issue Summary When using a **custom template** with the `/v1/chat/completions` endpoint, Sortana fails to send a properly formatted request. LM Studio responds with `'messages' field is required`, indicating that the request body is malformed. ### Steps to Reproduce 1. **Setup:** - LLM Server: LM Studio with `openai/gpt-oss-20b` model - Endpoint URL: `http://192.168.1.157:1234/v1/chat/completions` - Prompt Template: **Custom** 2. **Custom Template Used:** ```json [ { "role": "system", "content": "You are an email-classification assistant. Determine whether the email satisfies the user's criterion. Return ONLY a JSON object on a single line of the form:\n{\"match\": true} - if the email satisfies the criterion\n{\"match\": false} - otherwise\n\nDo not add any other keys, text, or formatting." }, { "role": "user", "content": "Email:\n{{email}}\n\nQuestion: {{query}}" } ] ``` 3. **Expected Behavior:** - Request should be sent with proper JSON structure to `/v1/chat/completions` - Request body should contain a `"messages"` array as per OpenAI Chat API spec 4. **Actual Behavior:** - Sortana converts the custom template to a **string** and puts it in the `"prompt"` field (Completion API format) instead of parsing it as `"messages"` (Chat API format) - LM Studio receives: `{"prompt": "[{\"role\": \"system\", ...}]", "max_tokens": ..., ...}` - LM Studio rejects with: `Error: 'messages' field is required` ### Debug Output (Sortana) **Last Request Payload:** ```json { "prompt": "[\n { \"role\": \"system\", \"content\": \"...\", ... }\n]\n", "max_tokens": 4096, "temperature": 0.6, "top_p": 0.95, "seed": -1 } ``` **LM Studio Log:** ``` 2025-10-23 21:06:53 [INFO] [LM STUDIO SERVER] Received OPTIONS request. {} 2025-10-23 21:06:53 [ERROR] 'messages' field is required ``` ### Expected Request Format For `/v1/chat/completions`, the request should be: ```json { "model": "openai/gpt-oss-20b", "messages": [ { "role": "system", "content": "..." }, { "role": "user", "content": "..." } ], "max_tokens": 4096, "temperature": 0.6 } ``` ### Root Cause When using a **custom template** with `/v1/chat/completions`, Sortana: 1. Treats the template as a string instead of parsing it as JSON 2. Inserts this string into the `"prompt"` field (Completion API format) 3. Does not recognize that `/v1/chat/completions` requires a properly formatted `"messages"` array ### Workaround Use the `/v1/completions` endpoint with the **OpenAI/ChatML** standard template instead. This works correctly but may have other limitations depending on the model configuration. ### Additional Info - **Sortana Version:** Latest from git.jordanwages.com - **LLM Server:** LM Studio - **Model:** `openai/gpt-oss-20b` - **Endpoints Tested:** - `/v1/completions` (works with ChatML template) - `/v1/chat/completions` (fails with custom template) - **Note:** The model works fine in LM Studio's built-in chat interface, confirming the model itself is functional

wagesj45 self-assigned this

2025-11-08 17:39:46 -06:00

wagesj45 commented

2026-01-06 21:47:45 -06:00

Owner

I think I got to the bottom of this. The way the engine is set up, it assumes using a /v1/completions API. I have added some logic to make the endpoint configuration value a "base" value, using it to create a completed completions API URL.

I think I got to the bottom of this. The way the engine is set up, it assumes using a `/v1/completions` API. I have added some logic to make the endpoint configuration value a "base" value, using it to create a completed `completions` API URL.

wagesj45 added the

enhancement

label

2026-01-06 23:23:54 -06:00

wagesj45 commented

2026-01-07 14:06:14 -06:00

Owner

Currently testing a build with new parsing logic, model templates, and endpoint URI evaluation. If all is good will be releasing as v2.3.3.