Removable inserts certainly make this easier. For my cup holders, I use a rag, ID, and some patience/elbow grease to get into the corners. Really, nothing special...but mine are all DDs. For a garage queen, my approach probably wouldn't be enough.
Have you considered the Detail Swabs? I haven't used them before, but they look tailor made for this type of thing. I wouldn't be surprised if you've watched the video, but in case not, and for others reading this, check out how Adam uses LIC, the cockpit brush, MF utility towel, Sidekick, and swabs for smaller nooks and crannies in the interior:
Let us know if you figure out the magic touch!