where to find Steam game music?

Depends on the game. When you start Steam's music player you can right-click on the tracks and use "view track in system". That will open the folder where this specific track is.

There are two locations that I know of. First you need to know where Steam installs your games/dlcs/soundtracks, then adjust my examples accordingly.

1. If the soundtrack say "Doesn't require base game" on its store page it will be found the \steamapps\ music folder (ie. "D:\Steam\steamapps\music\soudntrackfolder\")

2. If the base game is required for the soundtrack it will usually be someplace in the games install folder. These are usually found in \steamapps\common\ (ie. "D:\Steam\steamapps\common\gamefolder\soundtrackfolder\")

If you can't find what you're looking for in those places, do as Golwar suggests with the internal music player.

Cant you also right click on the downloaded album in your library, press manage and browse local files like you can with games?