Meditations 8:10

Remorse is annoyance at yourself for having passed up something that's to your benefit. But if it's to your benefit it must be good—something a truly good person would be concerned about.

But no truly good person would feel remorse at passing up pleasure.

So it cannot be to your benefit, or good.