Aquiring meaningful Data for calculating Battle Rating

[Tom Farseer]

@admin
I have the following suggestion to introduce a way to generate data for modeling a meaningful Battle Rating.

Taking data from the logs you have now will result in a bad starting position for any iterative process, whether it is applied by hand or automated. It will require many iterations to get to a stable solution.

The reason for this is, the fact that you have (if I understand it correctly) data that encompasses different phases of the game with several buffs and nerfs along the way. This would at least explain why the Wasa has the same BR as the Bellona, despite being worse in every aspect except for a light advantage in turning rate and a negligibly better sailing profile. Also ships not being used does not mean they are weak (e.g. the Agamemnon,) there are just better alternatives that are as easy to obtain (why sail an Aggy if you can afford and crew a Bellona?)

My suggestion is as follows:
Create a build of a battle instance. Load one AI ship on both sides, with equivalent starting postions (eg. facing each other at a beam reach).
Pair each ship with every other ship. Run the battle as often as you can, considering resources, for every pairing. This will create a set of data that shows each ships relative performance without taking into account different skill levels and knowledge stuff or fleet composition etc..

Use this ranking as a base for a BR order, which can be adjusted on data that will be accumulated from then on. You will get a starting point for the proess that resembles the current situation without the many tunings of the past.

EDIT: This post is solely about the way that data is acquired, not about how it is used to calculate BR!

Edited August 30, 2018 by Tom Farseer
Spelling

[Angus MacDuff]

isn't it already a done deal?

Edited August 29, 2018 by Oberon74

[Tom Farseer]

It can be reevaluated at any time. Right now I'd say the data set that is used is flawed, because of it's history. @admin may well correct me on that. My suggestion is to do it again with a cleaner set of data.

[Angus MacDuff]

Fair enough, but I have to say I would be very interested in how this, in its new format, affects the PBs.  Two Bucs instead of a L'Ocean....

[Percival Merewether]

I think the only reason the new system appears “flawed” is because it has seen no active use yet. In time the BRs should balance out due to more varied use.

The Agamemnon now is rated very low. I bet it won’t take long for that to change. The new system actively works against new powerful “metas” by discouraging mass-use of popular ships.

I think this is the most positive thing from patch 26.

[Tom Farseer]

Just now, Percival Merewether said:

In time the BRs should balance out due to more varied use.

If the algorithm works, yes it should. My suggestion just aims at feeding said algorithm a better suited starting point, to make it converge on it's finite solution faster.
 

[Percival Merewether]

3 minutes ago, Tom Farseer said:

If the algorithm works, yes it should. My suggestion just aims at feeding said algorithm a better suited starting point, to make it converge on it's finite solution faster.

Alright, makes sense.!

As you were

[z4ys]

1 hour ago, Tom Farseer said:

@admin
I have the following suggestion to introduce a way to generate data for modeling a meaningful Battle Rating.

Taking data from the logs you have now will result in a bad starting position for any iterative process, wether it is applied by hand or automated. It will require many iterations to get to a stable solution.

The reason for this is, the fact that you have (if I understand it correctly) data that encompasses different phases of the game with several buffs and nerfs along the way. This would at least explain why the Wasa has the same BR as the Bellona, despite being worse in every aspect except for a light advantage in turning rate and a negligibly better sailing profile. Also ships not being used does not mean they are weak (e.g. the Agamemnon,) there are just better alternatives that are as easy to obtain (why sail an Aggy if you can afford and crew a Bellona?)

My suggestion is as follows:
Create a build of a battle instance. Load one AI ship on both sides, with equivalent starting postions (eg. facing each other at a beam reach).
Pair each ship with every other ship. Run the battle as often as you can, considering resources, for every pairing. This will create a set of data that shows each ships relative performance without taking into account different skill levels and knowledge stuff or fleet composition etc..

Use this ranking as a base for a BR order, which can be adjusted on data that will be accumulated from then on. You will get a starting point for the proess that resembles the current situation without the many tunings of the past.

Wasnt that already done by calculating the BR because of hull hp and dps?

AI performance and player performance is different.

[Tom Farseer]

9 minutes ago, z4ys said:

Wasnt that already done by calculating the BR because of hull hp and dps?

I have not once in my post talked about the algorithm that is used to calulate BR. Just about it's starting condition.

Hull HP and DPS obviously are not enough parameters by a long shot for such a complex model.

From what admin said in the Patch Notes thread and the Requin feedback thread, the BR in it's new form is chosen by the devs according to overall metadate of the ships, meaning win/loss ratio and some such. So we don't know if there is an actual mathematical algorithm at all, or if it's just sense of proportion by the devs.

My suggestion is using a set of such data that is not polluted by changes in ship stats over time.

[Old Crusty]

I think your idea to get a base to work from is a great idea.

[Tom Farseer]

...sigh... once again:
This post is not about changing the system with which the BR is calculated. It is ONLY about starting the process with clean data and not with logs that encompass the times when ships where blatantly overpowered and thus sailed very much, as opposed their stats ingame now after being nerfed.

Take the Wasa as an example:
The only reason it was used in PBs with it's current stats was because the BR was a lot lower than the Bellona's (305<365). In OW it was underpowerd compared to the rest of it's class and saw next to no use. However the time when it was very powerful makes for a higher rate of usage in Ow in the logs. That data however is irrelevant, but it still contributed to the Wasa BR being the same as the Bellona (both at 450). One can already tell that makes no sense, because the Wasa is worse in every aspect except a very minor advantage in turnrate. It just generates the need for more iterations until the BR fits.

The BR will at some point align to a good place, if their method works. It will just do so quicker if we start at a point that already makes some sense, e.g. use unpolluted data as a base.

[Old Crusty]

My question is, with this system for calculating BR can the Developers do manual changes? With your example of Wasa and Bellona for instance, can they do a manual adjustment if thy find that the Wasa BR isn’t falling into line like it should.

I am excited to see over the next couple of months how this works out. I think that this is a huge step in the right direction for the game.

[admin]

2 minutes ago, Old Crusty said:

My question is, with this system for calculating BR can the Developers do manual changes? With your example of Wasa and Bellona for instance, can they do a manual adjustment if thy find that the Wasa BR isn’t falling into line like it should.

I am excited to see over the next couple of months how this works out. I think that this is a huge step in the right direction for the game.

Yes its fully manual now

Maybe its also a time for wind power finally that will change speeds of vessels every day by 1-2 knots depending on wind strength  - with weather predictions for next week in API (giving rise to community weathermen).

on the other hand - it lacks stability because you learn the ship and if ship speed changes every day - it might be too dynamic and could upset a lot of players.

[z4ys]

2 minutes ago, admin said:

Yes its fully manual now

Maybe its also a time for wind power finally that will change speeds of vessels every day by 1-2 knots depending on wind strength  - with weather predictions for next week in API (giving rise to community weathermen).

on the other hand - it lacks stability because you learn the ship and if ship speed changes every day - it might be too dynamic and could upset a lot of players.

would at least favour sometimes small ships so more player would actual sail those ships or nobody (only a few) would play during that days

[Salty Dog]

I would make sense to come up with a calculation to figure BR...

(Weight of broadside + armor of ship)/some ratio= BR

yes, I guess that would mean that different builds of a ship would result in different BR values but at least it would be rational.  otherwise, we're just making an opinion call.  

Personally, I think all ships of a class should have BR within a few points of each other.   So all ships that were 3rd rate ships would be nearly identical depending on broadside and armor.

[Old Crusty]

1 hour ago, Salty Dog said:

I would make sense to come up with a calculation to figure BR...

(Weight of broadside + armor of ship)/some ratio= BR

yes, I guess that would mean that different builds of a ship would result in different BR values but at least it would be rational.  otherwise, we're just making an opinion call.  

Personally, I think all ships of a class should have BR within a few points of each other.   So all ships that were 3rd rate ships would be nearly identical depending on broadside and armor.

Are you aware of the huge difference in 5th rates? You want a Cerberus to have a BR value close to the Endymion? I don’t think so.

[Guest]

My brain is getting tired from having to figure out so much in this game. Used to be simple and fun. Now i get the feeling that to play this game, you have to learn way way way too much.

[Hethwill]

I never got beyond the "attack" button regarding "figuring out stuff".

I think the BR order of battle stuff might be useful for the "strategos" of RvR.

[Powderhorn]

I love the idea of variable wind strength.

[Jean Ribault]

2 minutes ago, Powderhorn said:

I love the idea of variable wind strength.

Agree, that would be a nice feature.  Might throw wrenches into certain meta too in addition to just being fun.

[AeRoTR]

What a fantastic idea which is starting to be considered. Hope it happens, few knots of speed change can make things much more interesting, more choices. A day in KPR when we won’t see that little red sails.

Edited August 30, 2018 by AeRoTR

[Cecil Selous]

I would vote for a more frequent change of wind strength. Every two hours? And not always from one extreme to the other. Completely dynamic over the whole map would also be good And no need for a complete dead calm.

[Powderhorn]

Then we just need stealing the wind modelled..............

[Hethwill]

2 minutes ago, Powderhorn said:

Then we just need stealing the wind modelled..............

... and tides...

[Tom Farseer]

should i change the tags?