He's a former ECW World Heavyweight Champion.
As far as credibility with the ECW fans, I think he had more than Jericho, Benoit, Guerrero, Misterio, Malenko, and Storm. All of which people probably remember more from their WCW days. And none of them were considered to be as hardcore as Rhyno either when it came to the action in ECW.
Again, it was a terribly stupid move by Vince. He can't afford to be dropping even more ECW wrestlers, especially since it looks like the show is on life support already.