It would be very interesting to have something like BakLLaVA's image description fed from a webcam used as a context for the LLM. "You can see: <description of scene, description of changes from last snapshot>" or something along those lines in the system prompt.