On comparing scoring systems using simulations

Event and Tournament talk
1

Dear all, this is my first post in the kings of war forum!
It was suggested to me just now, I did not know it existed, and I must say I m very happy to find an alternative to facebook!

I worked for a while on a statistical approach to compare scoring systems in Kings of War.
I came up with the article linked below.

I posted this article on FB as well, but given the initial reactions I realised that the title of my article suggests that I have a bias against the Blackjack system. I decided to keep the title anyways for transparency and I hope we can have a nice chat about the topic here! I ll copy the FB post here:

’Dear KOW fanatics, I would like to share a piece of work that I spent some time working on… It is about comparing objectively scoring systems and helping TOs make their choice for their events.

It involved some coding and some maths, which I share along this (I hope easy to read) piece about scoring systems.

I know this does not cover all systems and can be extended, but if there is interest in the topic this could motivate me to work a bit more on the topic…

Here is the link

https://github.com/…/kows…/blob/main/polished_article.md

TLDR:

+ Positive scoring systems such as Northern Kings are actually quite good at providing balanced games and identifying the best players in a tournament reasonably, but favour players who submarine and face weaker opponents

+ Combining NK + robin pools for the first rounds is the best system for finding the “best” player, to consider for instance if you are organising the masters

+ Strength Of Schedule can reduce submarines etc quite well, but more fairness correlates with more variance on who ends up on the podium

+ Based on the study I don’t think anyone should use Blackjack.

Thanks for reading me !

Steve

7 Likes
2

Very interesting, and thanks for doing a deep dive. :slight_smile:
Personally I have used Blackjack for many years at my Kings of Oslo GT. Northern Kings was created later and never quite convinced me to change my ways although I have played in several events using it and I have no problem with it.
SOS has always been interesting but I miss a scoring software that caters to it.

1 Like
3

Steve, fantastic article–thanks for digging into this. I have data from a handful of 2026 tournaments and compared the real outcomes to your simulations. The tl,dr is that the real-world validates your simulations as well. I think there are good lessons in here for TOs: what are you trying to do with your event? Depending on how you answer that, you may want to adjust your scoring system.

1 Like
4

Great article and interesting finds.

From my personal point of view, I have always preferred WDL over scoring points, for the simple reason that those games I played that way where more fun as people played a little more risky (outside the top players of course) as they were more likely to sacrifice units for a victory over trying to preserve and score points.

But overall the one big problem here is that with people needing to do things on their own, they most often just use the system they are familiar with or go with the default one the software provides they use.

So for different Systems to have a chance to be considered, at least either the Companion would need the option to choose between different scoring systems (or set different scores for ranking, ie with the option if WDL or TP is set first etc.) or another (offline) software would need to provide this

also different pairing systems need to be supported by software, otherwise people hesitate to use anything more complicated

2 Likes
5

Would you see value in new tournament software that has different options? Potentially I could build something new, but it has seemed to me that the space is pretty congested right now. I wouldn’t want to create something unless there was a need–but also happy to do it if it would benefit the community.

6

From my point yes, specially of it works offline so it can replace the excel sheets some are using in that case

But at the same time it would also need to work with the companion for easy import/export of attendees and results so you don’t have to do it manually.

(and in case with any of the local sites used to make things as easy as possible.
As an example, central Europe still uses tabletopturniere.de (which has a very outdated offline pairing program) as the go to because it offers options for TOs others don’t.)

But not sure if this fits the needs of the wider community like the US

1 Like
7

thanks a lot for your answer, really interesting read on your end as well! Thank you for comparing the results to real data.

I don’t understand your ELO gap comparison however, did you have an ELO record for the players in the tournaments?

1 Like
8

Kind of–I calculate elo within a particular tournament, but not across tournaments (the name normalization combined with small sample size makes that hard to do when so few events publish game by game data).

1 Like
9

I am a little bit behind with podcasts but just finished an episode of counter charge where using the Companion and the lack of options for different scoring systems was briefly covered

@RobP might can add some input on what a 3rd party tool would need so people would start using it in their regions (outside a companion import/export)

And @BrG might have some input for the ELO rating

1 Like
10

I do compute an Elo rating for individual players, based on w/d/l over 7500+ games at the moment. I do scrap all events on the companion and TOs send me some data that are not on the companion (mainly spanish, french and swiss tournaments, some from the UK as well)

Only for EU+GB though because there is enough cross play between countries to be relevant. I do not have enough data to do a relevant North America Elo, I m starting to have some volume for Australia and SE Asia but I am not sure the quantity is very representative of what is happening over there. (and not sure it interests people, whereas I had very positive feedback from the European one)

Right now there are 800+ players, 300+ with a rating because I do not consider a publication before 15 tournaments game to be relevant),

details are there : ELO | Masters Kings of War France

2 Likes
11

oh yeah and name normalization requires a bit of osint, I also got some players that gave me all their aliases

the accuracy of the score varies from the number of recorded games for the said player, at 15 games when you enter the rating your score is still not very stable but, eg Chris Lynch rating with over 100 games is more accurate than Tom Robinson with only about 40.
All in all it is best to look at it as a broad approximation which is getting better and better the more games are in

Oh and thanks to COK, that is not the only cross play event but the largest one that helps maintain an healthy amount of crossplay between all local communities, a big mixing cluster :smiley:

1 Like
12

From the Clash of Kings UK 2026 pack:

Clash of Kings 2026 will be using the
Clash of Kings Scoring System.

Tournament Points (TPs)
Win: 10
Draw: 5
Loss: 1

Scenario Points (SPs)
There will be a maximum of 7 Scenario Points available each game and will be based on the VPs scored by each player.

Kill Points (KPs)
A player will score 1 KP for every 20% of the opponent’s army points routed, with a maximum of 4 KPs available.

Your total score for the round is RPs+SPs+KPs, with a max score of 21 and a min score of 1.
When calculating placings, the order of deciders will be:
Tournament Points; Scenario Points; Points Routed; Scenario Points Attrition; Routed Attrition; Strength of Schedule

2 Likes