no you are not digging way too deep....we all do it!
we talked about this a little while back...i'll try and find the thread and bump it up for you.
he actually says "charlie's got us a horror film"...so maybe charlie just got the film for them..and didn't attend the party.....
....or maybe charlie was in the bathroom at the time.
If charlie was in the bathroom, he would be the only one there without a date. Of course, we do see a glimpse of Milo's maid in this scene, walking by in the background so maybe he's hanging with the maid....Or maybe he's doing the "maids-a-milking" thing....or maybe my mind is in the gutter or just plain shot.