You didn't plan to get caught. He didn't plan to care. Now you're stuck in a custody arrangement neither of you asked for.