I've had good luck with leaving the cloth on the speaker board and spraying on carpet cleaner. I let it sit for a while, then rinse it off and dry it with a blow drier right away.
I did remove the cloth once on another set and let it soak in carpet cleaner + warm water in a sink. Unfortunately, it shrunk before I reattached it
