From my playthrough something that i always found prevalent in naval battles is that the fleet that can board the most always wins, which in turn kinda makes using canons redundant.
From what i can say is that if the enemy has 1 or 2 ships of the line (3rd/4th rate lets say) with a high enough crew count then the battle is lost.
In the mission "Objects in Motion" the enemy fleet had some two ships that where either 3rd or 4th rate (i cant know for sure) as opposed to my highest being a few 5th rate with a couple 6th rate. I tried outmaneuvering them and attempt to rake them to no effect to its guns, crew and hull. Plus the broadside is just stupid, instead of the canons waiting for them actually have sight on the target they shoot all at the same time missing most, what should be raking shots, or having the balls bounce off because they shot at an angle that no ship cannon could possibly achieve instead of "waiting for their turn". With that in mind the only other option to deal with ships is by boarding them.
The issue that im left with is, as most of my ships are 5th rate i cant employ enough crew to board an enemy ship and then fight the rest of the enemy fleet with whats left of my crew, assuming i win the boarding. the enemy ships of the line had 500 and something in one and around 650 men if my memory serves me right, and they could still keep a decent speed and maneuverability.
Long story short, it is simpler to play the boarding game than actually shooting your cannons at the enemy, especially since cannons (round shot) dont do much against the crew if it penetrates, even if its a full ship.
In order to balance this i suggest increasing shot damage to crewmates relative to their amount, ship size, broadside intensity and point of impact.
The bigger the crew, the higher the chances of casualty of penetrating shots, due to the fact that there is a higher chance of hitting someone with the shot or with spalling.
Have cannons only fire when they actually have the enemy ship in sights, and not waste their broadside by shooting the ocean
Bigger ships shouldnt be able to hit smaller ships if those same smaller vessels are below their canons maximum depression