Two things:
1. In Trafalgar, once the grand melee started, musket fire was actually incredibly effective. You had ships literally bumping sides, and the men in the tops were clearing the decks like crazy. It's also how Nelson died.
2. Muskets are inaccurate, but not as inaccurate as people seem to believe. At 50 yards, you will hit a man-sized target, it is not 50%. I know this because I own a Brown Bess and use it frequently, and in my reenactment regiment, we have taken muskets to the range and fired as a group as well as individually, and no one really misses at 50 yards.