It was mostly a marketing ploy. Often, there was nothing wired to the socket. And I don't think anyone actually made the adapters in any real quantity. And when NTSC color came out, it turned out better and cheaper(!) to buy a color TV set. Coupled with the lack of a lot of color programming, few people even thought about it.
It was somewhat similar to radio sets claiming to be "Television ready" when they had an audio input jack on the back. As if TV sets were not gonna have audio amps?