Ah the mines. Those men are hard at work pulling up the random node for me to mine and mining ore to be stacked by the mining shed outside. When you log into your garrison for the first time in a 24 hour period, you'll see all these lovely yellow dots sprinkled about your mini-map. For me, those dots are a distraction. Even if I don't need ore, they compel me to mine it just to get the mini map cleared. There is the same issue with the herb garden. Lots of dots that scream, "Pick me!"
What I'd like to see is the opportunity to give the gardeners and the miners a day off. For heavy population servers, not having more stacks of ore or herbs dumped into the Auction House could improve the selling price for both, given the nature of supply and demand. For those of us who just want a clean mini-map without having stacks of stuff we aren't going to use, this would be lovely. Maybe the crew in the mine hangs out in the garrison barracks or around the table in your town hall. There are a number of places within the garrison you could have groups of people congregate.
I like to think I'm a benevolent ruler. It does kind of pain me to run into the mine and realize these men (And why are they all human males? Hmmmmm? Dwarves love to mine. There are no dwarven miners, no women, no gnomes or worgen or pandas or night elves. Mining is only a human male trait?) never get to see the light of day. If you've ever run into your garrison during an invasion, they are working away as if they haven't a care in the world. I'd really like for them to have a day off.