If you also find the Leave button in Slack Huddles moves about a lot at the end of meetings, you can click on the headphone toggle in the bottom of the left sidebar to leave a huddle. (or try your luck at russian roulette and press the shortcut which either starts, joins, leaves or ends a huddle)