Punishment is about receiving forgiveness for me as well. When I have been punished I have felt incredibly sorrowful for having done something that warranted being punished. I could have wallowed in that misery of shame and regret, but that isn't what was expected of me. He gives the punishment, he forgives me, he gives aftercare and it's to be forgotten and life resumes.
I do think doing something special for you Dom is a wonderful idea. However, beating yourself up for doing wrong is not your responsibility.