I've always thought he made a bit of a mistake in that "Exercise," scene apart from the anger, and that's that he accused Milo very directly, in front of witnesses. I don't know about these things, but if there were no arrest later on, could that be considered slander? As for being tempted to hit him, I doubt that "out of shape" Columbo (his own words) would hit a famous health expert, unless he were VERY angry. (Let alone one played by Robert Conrad, considering his tough guy image, on and off television!)
It took me the longest while to think of this, but one reason for his anger at Dr. Mayfield was that his original "victim" (the other doctor) was still alive, but "hanging in the balance." So for a change, Columbo wasn't just angry about something the other character HAD DONE, but about something else, that could be PREVENTED. So in a way, it was almost a one-of-a-kind scene for THAT reason too. Someone mentioned very mild cases of anger. In "Blueprint For Murder", Columbo goes from apologizing to Markham for those reporters (typical of him) to getting just a little mad when he finds out that Markham HIMSELF called the reporters (which is, of course, less typical).