It seems, from my trusty copy of "The Columbo Phile" -- in which Mark Dawidziak collects a bunch of these quotes, in chronological order -- that the exact quote "Just one more thing" was first spake in "Lady In Waiting" (1971).

But variations of it were these from the first:

"Oh one more thing" - Prescription Murder
"There's one detail that bothers me" - Prescription Murder
"There's just one thing I'm not clear about" - Murder By The Book
"Oh, one thing I almost forgot" - Dead Weight

Also from Murder by the Book: "There is one thing, not that it makes much difference..." when Columbo asks Ken why he drove back instead of flying.

GRANT:

Just gleaning through episodes in my mind I picture Riley Greenleaf, saying One more thing to Columbo.

I cant remember it specifically but that what comes to mind... Correct?

That's one I hadn't even thought of, so I'm glad to see it here.

The one I actually meant was Abigail Mitchell.

In "Lovely But Lethal" when he leaves Viveca's studio, and she picks up the telephone to call somebody: "There's a Lt. Columbo.." and BAM, he's RIGHT there in her face, looking up through his brow at her. Fantastic.

Another one like this is in "Swan Song" when Columbo gets done questioning Tommy Brown about something and starts to leave his home. Tommy turns to answer the phone and tells someone to cancel the Tabernacle and pay everybody off to date. He hangs up, sits there for about 2 seconds only to have Columbo reveal that he never left.

Yeah, that was a classic scene ... "I knew you were stopping the project."