Well, destroyers were for killing torpedo boats, originally. The name 'destroyer' is short for 'torpedo boat destroyer', after all. Even in real life, though, they were never super effective in that role but were slightly better at being torpedo boats than torpedo boats were and eventually supplanted them in their own niche.
But yes, ASW is where they really shone, though using them as a long, sharp stick to keep the enemy from closing with your heavy units is a good job in the meantime..