Following up on Cindy’s point about “A Friend in Deed”, maybe the opposite is true when they are caught. Halperin’s relief over the death of his wife turns into grief that he is caught; Caldwell’s grief over the death of his wife turns into relief that he is caught. Although we don’t see Caldwell at the end of the episode, we can imagine he might feel a sense of relief that it’s all over. He even said to Halperin that he should have just gone to the police in the first place.
Knowing Columbo, he might find some way or other to not arrest him in the first place. It wouldn't exactly hurt his pride, since he knows Diamond is innocent, and since he knows the whole thing will fall apart anyway.