I left mine right where it was. It does some good as it blocks the radiant heat from any starter. I can't imagine why anyone would choose to remove it.
I wouldn't remove it either but, it was not installed by the person who restored the car before I bought it. I was wondering if I should install one I've got in my parts stash. Guess I will when the new motor gets here.
Be aware that virtually all "new" starters are bottom-feeder Made In Communist China crap. A used-but-usable Treasure Yard starter is what I look for. They tend to be priced right, too.
There are--or were--some "new" starters that were high-quality. Often service replacements direct from GM. I don't know if that is still true. In general, "new" starters and alternators are Asian; often Communist Chinese. They work the "new" angle, because it's cheaper to have crap made new in China than to rebuild used OEM-quality stuff in Mexico. Doesn't help that the gullible public thinks "new" junk is better than "rebuilt" OEM. Depending on the rebuilder, sometimes "new" junk IS better than rebuilt OEM. Thus my preference for used-but-usable from the Treasure Yard.