Programmatically parse sentence using homeassistant/debug agent?

[MostHated]

Hello, I was wondering if there happen to exist a way to accomplish the same as the Developer Tools -> Assist -> Sentence Parser, but programmatically? I see that testing functionality exists in this repo itself, but I am wanting to do it in the same manner that exists within HA, so I can see which/how my actual entities match, without performing the actual actions against them.

When submitting the sentence on the page, I can see the following in the debugger:

Object { type: "conversation/agent/homeassistant/debug", sentences: (1) […], language: "en", device_id: undefined, id: 35 }
device_id: undefined
id: 35
language: "en"
sentences: Array [ "Turn off bedroom light" ]
type: "conversation/agent/homeassistant/debug"

When trying to look up 'conversation/agent/homeassistant/debug' via github I don't see anyone else having used it via any sort of non-internal api call, or in the developer docs searching for 'agent', 'conversation', 'intent', etc I can't find anything relating to this specific method of debugging, or how I might be able to submit a sentence to it and receive back the same data.

Is there any way currently I can accomplish the same result as the Assist page sentence parser outside of on the page itself?

Thanks,
-MH

[tetele]

The code is there, but what exactly are you trying to achieve?

[MostHated]

Just as a fun side project, I am working on a voice assistant that can try more than one avenue to accomplish a goal, using ollama tools/function calling. If the call to conversation.process ends up without a match, it will try pulling info via community hassio cli to try and locate whatever it was, etc and try again.

I have a lot of entities added via mqtt, and most of their names are things like downstairs_desk_lamp, or upstairs_bathroom_light_1_c, so I am using the llm to compare that with the existing entities to figure out the right one, and it would be much more efficient and helpful to be able to call a live endpoint in a debugging manner that will not actually change any state, but simply return back if it matched with what I was expecting it to match with. While I am greatful that it exists, having to constantly go to Developer Tools -> Assist, and manually enter the resulting phrase that the llm came up with to see if it works as expected is a pain.

I was testing things at first using the above mentioned hass-cli since I could easily just run, for example: hass-cli call service conversation.process --arguments text="Turn of upstairs lights".

Typing that into the Sentence Parser gives the following (because I manually renamed the HA name of that particular item, so I knew it matched), which is exactly what I am hoping to get, without actually turning off any lights while I am testing things:

intent:
  name: HassTurnOff
slots:
  name: bedroom light
details:
  name:
    name: name
    value: bedroom light
    text: bedroom light
targets:
  light.upstairs_bedroom_light_1:
    matched: true
match: true
sentence_template: <turn> off (<area> <name>|<name> [in <area>])
unmatched_slots: {}
source: builtin

Hopefully that all makes sense. What it boils down to, is just trying to find a way to test the live intent parser running on my HA instance in order to get the same info as above programmatically, without actually changing any state. I don't do much web work (any), so if a way to make the conversation.process as a debug only call exists that isn't documented, I am not sure how to go about it.

[MostHated]

Ok, I got it figured out. It had to be accessed via websocket.

In case anyone else ends up looking for it in the future, here is an excerpt from my testing script:

Calling code

 def debug_intent(self, sentences):
    """Debug conversation intents for given sentences using Home Assistant's debug functionality."""

    import websockets
    import json
    import asyncio

    if isinstance(sentences, str):
      sentences = [sentences]
    elif not isinstance(sentences, list):
      raise ValueError("sentences must be a string or list of strings")

    async def connect_and_debug():
      uri = f"{self._base_url.replace('http', 'ws')}/api/websocket"
      msg_id = 1  # Start with message ID 1

      async with websockets.connect(uri) as websocket:
        # Auth phase 1: Wait for auth required message
        auth_required = json.loads(await websocket.recv())
        if auth_required['type'] != 'auth_required':
          raise Exception("First message should be auth_required")

        # Auth phase 2: Send auth token
        await websocket.send(json.dumps({
          "type": "auth",
          "access_token": HA_TOKEN,
        }))

        # Auth phase 3: Wait for auth_ok
        auth_ok = json.loads(await websocket.recv())
        log.debug(auth_ok)

        if auth_ok['type'] != 'auth_ok':
          raise Exception("Error in authentication")

        # Send debug request
        debug_request = {
          "id": msg_id + 1,
          "type": "conversation/agent/homeassistant/debug",
          "sentences": sentences,
          "language": "en"
        }
        log.debug(f"Sending debug request: {json.dumps(debug_request, indent=2)}")
        await websocket.send(json.dumps(debug_request))

        # Wait for result
        while True:
          response = json.loads(await websocket.recv())
          log.debug(f"Received response: {json.dumps(response, indent=2)}")

          if response.get("id") == msg_id + 1:
            if not response.get("success", True):
              log.error(f"Error from Home Assistant: {response.get('error', {}).get('message')}")
              return None
            return response.get("result")

    return asyncio.run(connect_and_debug())

Result

2024-12-01 22:00:34,853 - DEBUG - Sending debug request: {
  "id": 2,
  "type": "conversation/agent/homeassistant/debug",
  "sentences": [
    "turn off downstairs desk lamp"
  ],
  "language": "en"
}
2024-12-01 22:00:34,853 - DEBUG - > TEXT '{"id": 2, "type": "conversation/agent/homeassis...mp"], "language": "en"}' [125 bytes]
2024-12-01 22:00:34,974 - DEBUG - < TEXT '{"id":2,"type":"result","success":true,"result"...,"source":"builtin"}]}}' [411 bytes]
2024-12-01 22:00:34,974 - DEBUG - Received response: {
  "id": 2,
  "type": "result",
  "success": true,
  "result": {
    "results": [
      {
        "intent": {
          "name": "HassTurnOff"
        },
        "slots": {
          "name": "downstairs desk lamp"
        },
        "details": {
          "name": {
            "name": "name",
            "value": "downstairs desk lamp",
            "text": "downstairs desk lamp"
          }
        },
        "targets": {
          "light.downstairs_desk_lamp": {
            "matched": true
          }
        },
        "match": true,
        "sentence_template": "<turn> off (<area> <name>|<name> [in <area>])",
        "unmatched_slots": {},
        "source": "builtin"
      }
    ]
  }
}