Hello! With the recent discussions of LLMs and DeepSeek, I thought I would share how to set up a personal assistant with Home Assistant. I have been using Home Assistant for years now because I have an auto-immune disease and I need to conserve all of the energy I can. I have found it invaluable and with the recent addition of ollama integration, it has become even easier to use.

Preparation: voice assistant please Hardware

First thing we need is something to run Home Assistant on. Any computer will do for right now, but we’ll talk about about LLM hardware a little later. Any computer is fine, including raspberry pis. If you don’t have any spare computers,you can purchase a Home Assistant Green which will come pre-loaded with Home Assistant. If you are using your own hardware, there are a couple of options:

• Install HAOS, their own distribution of Debian with Home Assistant pre-installed
• Install the Home Assistant container, which is limited in functionality, but if you do not need add-ons, it is an option.
• Install Home Assistant Supervised. This option is only for Debian devices, and they are strict about it on the forums, but I have it running on Ubuntu and it works fine, it just gives me an unsupported OS error sometimes. It will install in a couple of docker containers.

All of the installation instructions can be found Here

Finally, you need some IoT devices. Home Assistant has thousands of Integrations, so just search for the device you want to connect. There are even more custom integrations, but these are the supported options. As an aside, if you have very cheap wifi devices such as light bulbs or switches, see if you can control them with an app called Tuya Smart. If so, they are manufactured by a company called Tuya. Go to Walmart and pick up a few of These, these and one of these and you will be set to control your nearly everything in your home. While there is an official integration for the tuya devices, using a custom one allows for local control that does not call back to their servers. I will go into this later.

Step 1: Installation

I already touched on this, but if you are installing supervised on a non-supported OS, this thread is now for troubleshooting. Once installed, just follow the on screen prompts and it will tell you what devices available for home assistant on your network already.

Step 2: Okay, now I get voice assistant right? Advanced set up

Now, what I would suggest is installing the Home Assistant app on your phone and setting up areas in your house. With areas, you can assign your devices to a location in your house and control everything in that location at once. This can be found in Settings->Areas, Labels, and Zones.

I would then suggest finding a local weather service in the integrations and installing it, so that you can have that information available to you at all times.

For custom integrations, such as LocalTuya, you will need HACS, this is not officially supported, but it gives you more privacy, so I suggest it.

Finally we need to talk about a sticking point: bluetooth. Home Assistant tends to not play so nice with bluetooth, so it is suggested that you purchase an ESP32 and set up a bluetooth proxy if you need BLE. Which leads to my next step.

Please, I just want voice assist. Why are you doing this? Custom devices

Home Assistant has another project called ESPHome. With this, you can make custom devices that connect with bluetooth or wifi to control whatever you want with a simple yaml scripting interface. I would suggest installing the ESPHome add-on as well. This new device will integrate seamlessly with home assistant.

That last section shouldn’t be before VA, that was just cruel. Automation

Under settings-> Automations and scenes, you can automate your house if a simple wizard. Simply select what you want to be your trigger. it can be time, a voice assistant sentence, scanning a qr code, and more! Sometimes, to get some thing done, you need to store a state in a dummy device, and those can be made in the Devices and Services menu, under helpers. These open your options tremendously. You can use them to change the device type of a switch, group similar devices, have a counter, or more. I use them to track my medication, and to control a heater like a thermostat.

Actually, I’m interested in more than VA now, can you tell me about controlling Music

Another great project is Music Assistant. It’s very new, so it is still not super feature rich, but it lets you control multiple music sources and multiple speakers from home assistant.

You can also just get the Spotify integration as well.

Alright, let’s talk about setting up Voice Assistant Fucking finally

Voice Assistant is build into Home Assistant and can be controlled from the top right hamburger menu. It uses sentence matching to control your devices, and it has a lot built in already, but you can write more with automations. With it, you can set times, change device states, and trigger automations.

But you probably want more help than that, so lets talk about integrating a complete solution. We want to be able to shout at a box and have it talk back. First, we need a box to shout at. I just use my phone, but there are several options, from a (low cost option)[https://www.home-assistant.io/voice_control/thirteen-usd-voice-remote/] to a Whole smart speaker. You can also just connect a microphone and speaker to your server directly.

Next, we need add-ons, which can be found under settings. You will want Piper for text to speech, Whisper for Speech to Text, and openWakeWord if you are using the server to process wake words.

Now we need integrations. You need Wyoming Protocol and if you want to use an LLM, ollama. Search for them under the integrations menu.

You can now set all these up in the Assist menu, but I’m going to diverge into LLMs for a moment.

LLMs

if you want to use ollama, you need to install ollama from here. Now to run an LLM you will need some better hardware, but we have some options. I have no doubt that a 7 billion parameter model will work fine for Assist, but they require at least an RTX 3060 to run. If you do not have a spare graphics card and do not want to pay for one, You can try running a smaller model on your CPU. Keep in mind the smaller the model the more likely it will make a mistake. Search the ollama website for models that use tools. Llama 3.1 and 3.2 are good models for this.

You’ll notice I did not suggest DeepSeek, this is for a reason. two actually. 1) It is slow and 2) It cannot use tools.

The olama integration provides a good system prompt but you can change the model file directly to integrate it into your model and speed up the model a little.

Home assistant does allow you to have the LLM as a fallback if it cannot match what you say to an existing sentence. This way you get the speed of sentence matching and only need to use the LLM if you say something complex. My personal setup uses this and a small LLM (llama3.2 3b) with a six core 12 thread ryzen CPU.

Then you need to expose your devices to the voice assistant. Keep in mind the more devices you expose, The longer it will take the LLM to process. When you expose a device, You have the option to set an alias which is useful when you want to refer to something with a brand name or a custom name by a generic term. I use the android tv remote integration, and I alias it to ‘the TV’ so I do not have to call it ‘remote’.

From here you should have everything you need to have your own smarthome. Have fun and enjoy the automation without giving up your privacy!