Regret and Repentance
Repentance is the sincere regret and remorse we feel when we've done something dishonest, immoral, or otherwise harmful to others or ourselves. A repentant person naturally wishes they could undo whatever wrong they've done, or make up for the misdeed by doing something good.
However, we can't change the past, and good deeds don't nullify bad ones.
Some Vedic teachings prescribe atonement—prayaschitta—to get free from the consequences of past sins, but to prevent future misbehavior we need to correct the mentality that produces it. Only the process of Krishna consciousness has the power to completely change the way we see and act on the world.
Krishna says in the Bhagavad-gita that if we submit to Him as the supreme authority He frees us from all reactions to past misdeeds (karma). He also says there that even if His devotee commits some abominable activity, the surest route to righteousness is to continue on the path of bhakti-yoga, devotional service.