Jump to content


BlitzAnalysiz.com - In-depth analysis of Blitz statistics

BlitzAnalysis.com new site stats statistics

  • Please log in to reply
45 replies to this topic

JDBC567 #21 Posted 10 June 2020 - 07:57 AM

    First Lieutenant

  • Member
  • 2163 battles
  • 566
  • [13RT]
  • Member since:
    06-02-2019
Nice website!
What's relative winrate?


BossArdnutz #22 Posted 10 June 2020 - 08:12 AM

    First Lieutenant

  • Member
  • 30431 battles
  • 743
  • [-IT-]
  • Member since:
    03-30-2018

View PostJylpah, on 10 June 2020 - 07:19 AM, said:

 

Excellent point! I should. I have been hesitating to put too much formulas / math there not to alienate all those four visitors, but I could run Anova tables and put those in the spoiler/expand boxes. Let me think about this.

 

You don't need to expose the math. Just stick a note under the chart if the comparison reaches p < .05. For multiple comparisons, just put a "*" above the column for the ones that are significant. 



JDBC567 #23 Posted 10 June 2020 - 09:36 AM

    First Lieutenant

  • Member
  • 2163 battles
  • 566
  • [13RT]
  • Member since:
    06-02-2019

View PostJylpah, on 10 June 2020 - 07:44 AM, said:

 

Hello, I am about to add tanks/tier/type pages showing similar graphs (and more) for all the same tier same type tanks. And finally I plan to add tank-specific pages for all the tanks and compare to the means of the same tier/type.

 

It would be also great to allow users to pick whatever tanks and compare them, but that would require interactive charts. At the moment the site is 100% static content: HTML + PNG graphs. If I find a way to use interactive charts (HTMLwidgets?) on the site like BlitzStars, I might be able to let the visitors to do more tailored comparisons there. If someone happens to have good ideas how to do interactive graphs on static web pages (GitHub Pages + Hugo at the moment), I am happy to hear. So far I have been told to look at gatsby.js (React/JS page generator), Nivo charts and Chart.js. This is not a high prio thing at the moment, but definitely something I will look into it once I get some time to study. I am not a web developer at all, but just learning by doing.

 

I'm not sure if it will work but you could try using python tkinter to create the interactive content and embed the python code in the HTML code 

I've not used tkinter it HTML too much so it might not work.



Jylpah #24 Posted 10 June 2020 - 12:06 PM

    Sergeant

  • Member
  • 19754 battles
  • 184
  • [SKUX]
  • Member since:
    08-06-2015

View PostJDBC567, on 10 June 2020 - 09:57 AM, said:

Nice website!
What's relative winrate?

 

It reads in the article ;)

 

Relative WR = Average over players of the tank( Player WR in the tank - their average WR at the tier of tank)


Blitzanalysiz.com - In-depth analysis of Blitz game statistics  


Jylpah/blitz-tools      Python tools to upload and analyze replays
Jylpah[SKUX] @Asia       Jylpah[AFK] @EU       

 


Jylpah #25 Posted 10 June 2020 - 07:41 PM

    Sergeant

  • Member
  • 19754 battles
  • 184
  • [SKUX]
  • Member since:
    08-06-2015

View PostBossArdnutz, on 10 June 2020 - 10:12 AM, said:

 

You don't need to expose the math. Just stick a note under the chart if the comparison reaches p < .05. For multiple comparisons, just put a "*" above the column for the ones that are significant. 

 

Hello, I went add scary raw t-test prints inside expand-boxes and mentioning whether the result is statistically significant. I think your proposal is good and I might implement a template function to generate such tables and mark significant differences.


Blitzanalysiz.com - In-depth analysis of Blitz game statistics  


Jylpah/blitz-tools      Python tools to upload and analyze replays
Jylpah[SKUX] @Asia       Jylpah[AFK] @EU       

 


BossArdnutz #26 Posted 11 June 2020 - 02:45 PM

    First Lieutenant

  • Member
  • 30431 battles
  • 743
  • [-IT-]
  • Member since:
    03-30-2018
Great work Jylpah! Is a good demonstration of the effect of sample size on statistical power - even with samples in the hundreds, and quite large percentage differences in relative win rates, many of the comparisons failed to achieve high confidence significance.

Particularly relevant given discussion in another thread here that advances a sample of 3 games as conclusive proof of MM manipulation...

Jylpah #27 Posted 12 June 2020 - 12:21 PM

    Sergeant

  • Member
  • 19754 battles
  • 184
  • [SKUX]
  • Member since:
    08-06-2015

View PostBossArdnutz, on 11 June 2020 - 04:45 PM, said:

Great work Jylpah! Is a good demonstration of the effect of sample size on statistical power - even with samples in the hundreds, and quite large percentage differences in relative win rates, many of the comparisons failed to achieve high confidence significance.

Particularly relevant given discussion in another thread here that advances a sample of 3 games as conclusive proof of MM manipulation...

 

At the moment there is requirement to have at least 20 battles in a tank to be counted into Relative WR calculations that do not separate how many battles there is behind the WR. 20 battles is very little causing large variance. It is a compromise between rWR variance and sample size. I have picked the 20 battle requirement out of the thin air, but I could run the numbers with different battle count requirements and optimize for example the mean variance of the Relative WR results. What do you think?

 

What comes to the visualization, I think adding 95% confidence intervals to the graphs in some nice & understandable  way would do the job. Most of the site content is (bar) graphs.


Blitzanalysiz.com - In-depth analysis of Blitz game statistics  


Jylpah/blitz-tools      Python tools to upload and analyze replays
Jylpah[SKUX] @Asia       Jylpah[AFK] @EU       

 


BossArdnutz #28 Posted 13 June 2020 - 01:09 AM

    First Lieutenant

  • Member
  • 30431 battles
  • 743
  • [-IT-]
  • Member since:
    03-30-2018

We're pushing the limits of my understanding here, but would a valid approach be to determine the required sample size for your desired power, then work out what battle count gives that sample size? Or is that too much like working backwards from the desired result instead of actually testing the hypothesis properly? I guess it also introduces all the confounding factors of low battle count (honeymoon period, stock tanks, inexperienced driver for the tank etc).

On the other hand, non-significant results are still meaningful. If it's not possible to be confident that a rWR difference is really due to tank characteristics, does that suggest the tanks may actually be balanced quite well?

 

95% CI would be a good approach, I agree.

 

Just a fantastic resource all together, thank you Jylpah! 


Edited by BossArdnutz, 13 June 2020 - 01:10 AM.


Jylpah #29 Posted 15 June 2020 - 07:29 AM

    Sergeant

  • Member
  • 19754 battles
  • 184
  • [SKUX]
  • Member since:
    08-06-2015

Hello Boss,

 

The issue is more the fact that in graphs there are often several (10-20) tanks in the graphs so what exactly is the null hypothesis? The Null hypothesis works for pairwise comparisons, but not when there is many-to-many relationship. Yes, I can get the sample sizes. Just need to change the way the templates work, but it should be doable. I am more concerned about the readability of the graphs since not everyone is a statistician. 

 

And thanks for encouragement!


Blitzanalysiz.com - In-depth analysis of Blitz game statistics  


Jylpah/blitz-tools      Python tools to upload and analyze replays
Jylpah[SKUX] @Asia       Jylpah[AFK] @EU       

 


BossArdnutz #30 Posted 15 June 2020 - 02:24 PM

    First Lieutenant

  • Member
  • 30431 battles
  • 743
  • [-IT-]
  • Member since:
    03-30-2018
I'm thinking mostly about the relative win rate charts. The null hypothesis there is that the tank is balanced, I.e. the win rate of players in that tank is not different from the win rate of the same players in other tanks of the same tier. So that's what you're checking for significance - not tank vs tank, but tank vs player base.

Jylpah #31 Posted 16 June 2020 - 09:02 AM

    Sergeant

  • Member
  • 19754 battles
  • 184
  • [SKUX]
  • Member since:
    08-06-2015

View PostBossArdnutz, on 15 June 2020 - 04:24 PM, said:

I'm thinking mostly about the relative win rate charts. The null hypothesis there is that the tank is balanced, I.e. the win rate of players in that tank is not different from the win rate of the same players in other tanks of the same tier. So that's what you're checking for significance - not tank vs tank, but tank vs player base.

 

Yes you are right. The right test here is to check whether the Relative WR is significantly different from zero (= "perfectly balanced= ) i.e. t-test.

 

Nerdy stuff 

Edited by Jylpah, 17 June 2020 - 11:36 AM.

Blitzanalysiz.com - In-depth analysis of Blitz game statistics  


Jylpah/blitz-tools      Python tools to upload and analyze replays
Jylpah[SKUX] @Asia       Jylpah[AFK] @EU       

 


Aky_Seargent #32 Posted 16 June 2020 - 11:24 AM

    Sergeant

  • Member
  • 1618 battles
  • 191
  • Member since:
    07-26-2019
That "nerdy stuff" is a whole lot of python(am I right? Is this python?) I also program a lot of games in c++ and c#, so hoped that I would understand python too.... But I didn't, call me a nub programmer, everyone!

"If you can, shoot at the rear, if you can't, shoot at the side. Don't bother with the front." 

- US Army General

Currently a UNICUM in the EU server(Formerly a SUPERUNICUM) (EU Server ID - TheMonk_74)


BossArdnutz #33 Posted 16 June 2020 - 10:47 PM

    First Lieutenant

  • Member
  • 30431 battles
  • 743
  • [-IT-]
  • Member since:
    03-30-2018

@Aky_sargent, this is more about probability and statistics (kind of the same thing really) than about coding. You can do the same analysis using Excel spreadsheets, MATLAB etc. Regardless of the tool used, it won't make sense unless you understand the statistical techniques in play.

 

@Jylpah, good work and well reasoned as always. 


Edited by BossArdnutz, 16 June 2020 - 10:51 PM.


Jylpah #34 Posted 17 June 2020 - 11:35 AM

    Sergeant

  • Member
  • 19754 battles
  • 184
  • [SKUX]
  • Member since:
    08-06-2015

View PostAky_Seargent, on 16 June 2020 - 01:24 PM, said:

That "nerdy stuff" is a whole lot of python(am I right? Is this python?) I also program a lot of games in c++ and c#, so hoped that I would understand python too.... But I didn't, call me a nub programmer, everyone!

 

Hello! I use Python on data fetching scripts, but R for statistical analysis and to generate the site content together with Hugo static site generator. Python and its data analytics modules could have been another choice, but R is better for research type of analysis together with Rstudio. Python analytics modules (NumPy/Pandas) suit better to server-side code IMO. C++ / C is far more hard core than Python. So I am the nab here. Python is nowadays my favorite scripting language.

 


Blitzanalysiz.com - In-depth analysis of Blitz game statistics  


Jylpah/blitz-tools      Python tools to upload and analyze replays
Jylpah[SKUX] @Asia       Jylpah[AFK] @EU       

 


Jylpah #35 Posted 17 June 2020 - 10:30 PM

    Sergeant

  • Member
  • 19754 battles
  • 184
  • [SKUX]
  • Member since:
    08-06-2015

2020-06-18:

  • Added stats for update 6.10:
  • Fixed a bug in stats filtering code that caused stats being ignored without a reason. It had minor effect on the stats.

 

It seems that on average Sheridan that got nerfed in 7.0 did not perform that well.

 

https://blitzanalysiz.com/update/6.10/tanks/10/#the-best-performing-tanks

 

Best oerforming tier X tanks

 

However, it did perform better for the better players that WG uses for balancing, but it seemed still "perfectly balanced" even for the super unicums. Just comparing it the to M48 Patton here since those are roughly equally performing tanks - on average.

 

I guess the reason for the Sheridan nerf was the big outcry that mixed ATGM hatred (well deserved, IMO) and Sheridan's assumed OPness. Also Sheridan has been a ridiculously popular tanks and WG prefers to maintain diversity among the tanks played. Super-popular tanks like FV183 do get nerfed more than the deserve.


Edited by Jylpah, 19 June 2020 - 07:15 AM.

Blitzanalysiz.com - In-depth analysis of Blitz game statistics  


Jylpah/blitz-tools      Python tools to upload and analyze replays
Jylpah[SKUX] @Asia       Jylpah[AFK] @EU       

 


Jylpah #36 Posted 16 July 2020 - 02:15 PM

    Sergeant

  • Member
  • 19754 battles
  • 184
  • [SKUX]
  • Member since:
    08-06-2015

Hello,

 

Stats for update 7.0 has been added to the site. Interestingly, the tracked player base played random battles significantly less than during 6.10 despite the update period being longer than usual.

 

 

 

 

In addition, there are now pages for each tank type per tier. Now all the tanks' data is now viewable on the site.

 

 


Blitzanalysiz.com - In-depth analysis of Blitz game statistics  


Jylpah/blitz-tools      Python tools to upload and analyze replays
Jylpah[SKUX] @Asia       Jylpah[AFK] @EU       

 


erwin10001 #37 Posted 17 July 2020 - 02:19 AM

    Captain

  • Member
  • 49240 battles
  • 1,267
  • [NETS]
  • Member since:
    11-12-2016

great work. interesting that the 'average player' has a winrate of 51%. I guess there are many new players who got a winrate boost by playing bots in the first 100 games.

 

could you provide details on 'ow long the measurement period was for  7.0? (you mentioned that 7.0 was longer than others)  this will allow us to compare stats for different versions.



TnkDztroyer #38 Posted 18 July 2020 - 01:32 AM

    Captain

  • Member
  • 10 battles
  • 1,044
  • Member since:
    02-04-2019

View Posterwin10001, on 17 July 2020 - 12:19 PM, said:

great work. interesting that the 'average player' has a winrate of 51%. I guess there are many new players who got a winrate boost by playing bots in the first 100 games.

 

could you provide details on 'ow long the measurement period was for  7.0? (you mentioned that 7.0 was longer than others)  this will allow us to compare stats for different versions.

 

Ya! very impressive website. But what is the median WR I wonder? 

 


Greens 0 / Reds 7 situation normal..  


Jylpah #39 Posted 18 July 2020 - 07:58 AM

    Sergeant

  • Member
  • 19754 battles
  • 184
  • [SKUX]
  • Member since:
    08-06-2015

View Posterwin10001, on 17 July 2020 - 04:19 AM, said:

great work. interesting that the 'average player' has a winrate of 51%. I guess there are many new players who got a winrate boost by playing bots in the first 100 games.

 

could you provide details on 'ow long the measurement period was for  7.0? (you mentioned that 7.0 was longer than others)  this will allow us to compare stats for different versions.


Hello, the reason for higher than 50% average WR is the account sampling. WG API does not have a method to list all the players, but those account_ids need to be “found”, that is the account_id need to be in replays posted to WoTinspector. The likelihood for a random player to end up into a replay posted to WoTinspector.com depends on the number of battles he plays (on higher tiers where replays are usually posted about).
 

As you can see from the battles per update per player histogram there, Blitz has huge number of inactive players who just play few games per update. Those are likely less-experienced/less-skilled players who lose far more than the average, of whom only small share is recorded in the database. 
 

The measurement period has been the whole update. It is just that WG’s release schedule is uneven. 


Blitzanalysiz.com - In-depth analysis of Blitz game statistics  


Jylpah/blitz-tools      Python tools to upload and analyze replays
Jylpah[SKUX] @Asia       Jylpah[AFK] @EU       

 


Jylpah #40 Posted 09 August 2020 - 04:46 PM

    Sergeant

  • Member
  • 19754 battles
  • 184
  • [SKUX]
  • Member since:
    08-06-2015

Update 2020-08-08

 


Edited by Jylpah, 09 August 2020 - 04:58 PM.

Blitzanalysiz.com - In-depth analysis of Blitz game statistics  


Jylpah/blitz-tools      Python tools to upload and analyze replays
Jylpah[SKUX] @Asia       Jylpah[AFK] @EU       

 





1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users