Summary:

Vision has officially gone AI with some major enhancements. Not only has it implemented an AI so that any input it takes in will have adequate output, it also recognizes when the LLM is NOT needed and uses only the Command Handling script for faster function calling/ actions!

(Also it Streams word for word!)

Current Changes:

• Installed Ollama and Mistral
• Built the LLM_Fallback script to intercept any unknown input and run it through the promped LLM
  • Streams the LLM Response live
• Modified Command_Handler: uses fuzzy and substring matching as a priority
  • Long/Complex/Unknown Phrases sent to the LLM

Next Steps:

• Upgrade LLM prompt to not only respond verbally, but also be able to run functions based off of response
• Begin Vision's Memory System (Hybrid)
  • Save user Data
  • Remember past conversations

Future Visions:

• Dynamic Command Learning - Learn and Store new commands
• TTS MODEL (I forgot let me get on that soon)
• Intent Recognition
• Start Physical Home Automation Integrations

Honestly, I'm really looking to upgrade my system right now from my t480s. That will just be something that happens in the future when I stumble upon an opportunity.