The idea of guilt in "The Scarlet Letter", based on 'the difference between shaming someone in public and allowing him or her to suffer the consequences of an unjust act privately', can be seen in Hestern Prynne and Arthur Dimmesdale.
Hertern bears on her chest the scarlet letter 'A' the represents the act of adultery that she has committed. The scarlet letter 'A' is a symbol of her sin for everybody to see, her public shaming, her punishment.
Arthur Dimmesdale (a theologian) and Hestern Prynne became lovers and he never confessed it publicly (he never confessed that he was the father of Pearl either). He deals with his guilt by tormenting himself, and, because of that, develops a heart condition as a result.
As we can see, while Hestern Prynne faces a public shaming, Arthur Dimmesdale suffers the consequences privately.