Control Plane API to control EVI session settings from another client
in progress
R
Roman Kupkovic
Currently, only being able to update the settings of a live session/connection over the frontend client means making the prompts used visible to the user.
If we have sensitive business logic related prompts and actions we use for steering the hume conversation we might want to apply those without the user client knowing.
This could be solved with some form of hook or proxy, I am not sure. But it would greatly benefit bigger production projects that rely on their business logic being not easy to instantly copy.
Rob Hughes
in progress
Rob Hughes
Merged in a post:
Way to update existing chats session context via API
G
Garrett Fitzgerald
Right now, I think we need to create a custom language model to be able to do this.
I'm not sure how worthwhile this feature would be given the amount of effort it might take to build this, but...
What would be nice to have - is a way to have an existing socket connection open in my React app, but be able to update session settings via a different process in the backend.
My use case could use a more lightweight solution. Why this would be helpful for me is to not increase latency with a custom language model, and to obfuscate the data from the backend that would be delivered to the frontend to update session settings the my react sdk, because sometimes it can be sensitive.
Rob Hughes
under review
We are considering implementing a Control Plane API that could send session settings outside of the current EVI session.
Adam Tzagournis, CPA
Rob Hughes that would be great, as it serves Roman's use case of keeping context updates sent in session settings confidential too (and I'd like that as well)
G
Garrett Fitzgerald
Rob Hughes
excited to see this! much needed!
Adam Tzagournis, CPA
This is mission critical for my use case as well, especially given the fact that Hume is now passing supplemental LLM costs onto the user.
R
Roman Kupkovic
Adam Tzagournis, CPA
Maybe they make the separation for this at Custom LLM vs Hume-side LLM.
If you use Custom LLM you can do backend context injections as much as you want but the hume-side feature is still cool and would be nice to be used.
R
Roman Kupkovic
UPDATE:
This can be made possible by having the session_settings message type accept an object (to reference a stored Hume prompt ID) for the system prompt (instead of just a string), same as with the config definition.