I found this place some years ago, a quiet little town in a fjord, only accessible over a mountain road, and with the most beautiful place to watch puffins.
In Icelandic, a puffin is called Lundi, if you have more than one they're called Lundar. Höfn is the Icelandic word for port, thus the name - the Haven of Puffins, Lundarhöfn.
The best season to watch the puffins is between the beginning of July and mid of August, that's when the puffins breed.
Please do only park in the designated parking space for visitors, not along the boats - the fishermen don't like that.