You need to configure Windows to set the default audio device to be the primary speakers that you want everything to go through normally. Any additional outputs to TVs or other devices should not be the default. All software by default will output to the default audio device unless you configure it to use an alternate audio device instead. If your output is configured to HDMI TV and the TV isn't on, most programs will be sending the audio to the TV which is not on so you wont hear it. Additionally, not all games or other programs have a way to let you configure another audio device, which is why it is important to make Windows default to your normal computer speakers that are always on and available.
If you like to play games on your TV sometimes also, then you'll either have the game on the TV and audio coming from your computer speakers, or you'll need to go in the game's settings and configure the game to use the HDMI audio if it has the ability to do that, then switch it back to the computer manually when you want it to work on the computer speakers again. If a game does not have the option in the configuration screens, then you might need to search the web to see if there is a way to override it via a config file or something. Otherwise the only way to switch audio between multiple outputs for programs that only support using the Default device, is to constantly reconfigure the Default device in WIndows every time you use those games/programs, and change it back when you're done.
Hope this helps.