He may have pointed a finger at an adversary and sicked his dog Tony Spilotro on them but he was about 125 lbs and was purely a money guy ' the golden jew ' to use Scorsese's more politically correct term. Sam is based on Frank Rosenthal who really was a professional bookie and one of the best statistics and predictions guy under the Chicago outfit in the 50s and 60s.