Minor Scoring Adjustments

EDIT: I’m rolling back the changes. I thought I had figured it out, but the more I thought about it the more I realized this is going to need additional analysis and additional data. I’m going to leave these values alone until after the arc is complete, unless I come up with an improvement that I’m absolutely certain of. By the way, you’re strongly encouraged to help with balance testing matches if you’d like to contribute to this effort. It’s also a fun opportunity to tangle with your Factions friends, enemies, and frenemies in a no-holds-barred, all-Champions-allowed context.


 

I’ve made some updates to how tiers are valued.

The initial values were set based on a couple large surveys I did in which I asked LoL players to estimate the strength-values of various ranks in a mixed-rank game:

The survey posed the question using the following hypothetical scenario (paraphrased).

You have 5,000 RP riding on the outcome of a match. You are not allowed to participate in this match, but you do get to build your team by selecting the tiers of its Summoners, using a limited budget of credits. You don’t get to select individual Summoners, or specify which division they’re in: if you “buy” a Silver Summoner, you get a random draw from the entire pool of Silver Summoners across LoL’s playerbase. Your team does not get to practice together: they’re just thrown into the match.

The price of a Silver Summoner is set at 100 credits. Given that a Silver Summoner costs 100 points, how much would you be willing to pay for:

      • A Bronze Summoner?
      • A Gold Summoner?
      • A Platinum Summoner?
      • A Diamond Summoner?

This is a way of getting at the question of how much impact rank differences have in a mixed-rank game. In thinking about this, remember to distinguish between “difficulty of climbing from one rank to the next” and “difference in game-impact from one rank to the next”. (For example, it might be much harder to go from being a college basketball all-star to an NBA all-star, but the gap between them might be much smaller than the gap between the college all-star and an amateur player.)

I essentially just took the average valuation and used that to create the starting point values. In other words, the starting point values were based entirely on perception, not direct empirical data. The values I started with were:

Bronze Silver Gold Platinum Diamond
60 100 180 270 410

Under this scheme:

  • The Silver–Bronze gap (40 points) is half the size of the Gold–Silver gap (80 points).
  • The Platinum–Gold gap (90 points) is about the same size as the Gold–Silver gap (80 points).
  • The Diamond–Platinum gap (140 points) is about twice the size of the Gold–Silver gap (80 points).

I think using general perceptions of rank differences is better than just using my personal guesswork. (In full disclosure, my personal hunch was that there was a big Silver–Bronze gap, a small Gold–Silver gap, a large Platinum–Gold gap, and a small Diamond–Platinum gap.) However, mass guesswork is still guesswork, and most people outside of Factions don’t have a lot of experience playing mixed-rank matches.

Getting some data into the picture is a high priority of mine. I tried running controlled balance-testing matches, but we just weren’t getting the data we needed in the quantities we needed. Currently, my plan is to use actual Factions results data instead. Obviously, there are some confounding variables (the different factions) involved.

Still, I think that even the most primitive look at the data is going to provide some improvements relative to the purely perception-based valuation. Thus, I’ve done a preliminary review — primarily descriptive and qualitative — and come to some initial findings, based on analysis of about 200 matches.

In summary:

  • The Silver–Bronze gap seems the largest.
  • The Platinum–Gold gap seems about as large as the Silver–Bronze gap.
  • The Gold–Silver gap seems virtually nonexistent, as does the Diamond–Platinum gap.

I cannot stress enough that this is pretty much all voodoo so far. I will not be surprised if these conclusions are utterly overturned once I get more data in. But given the choice between guesswork and even crude data, I’d prefer the latter. I think there’s just enough strength in this analysis to warrant a moderate adjustment.

Here are the new values:

Bronze Silver Gold Platinum Diamond
30 100 130 200 250

As for how these rank valuations translate into changes in match value: at present, for every 100 points in skill gap between teams, the value of the match is adjusted up or down by 30%.

Finally, for the curious, here’s a comparison of where the Balance of Power would be if we had no skill adjust whatsoever:

PILTOVER BANDLE CITY ZAUN DEMACIA
with skill adjust 36 -16 37 38
without skill adjust 20 -11 35 51

Of course, this is a counterfactual scenario; if we had no skill adjust, it’s possible that people might be more or less careful about balancing matches than they are now.

—CupcakeTrap

Caitlyn, you monster.

Posted in Uncategorized
2 comments on “Minor Scoring Adjustments
  1. Banane des bois says:

    Although I agree on the general tendencies (bronze and silver are quite similar, gold gap being more important and so on), I believe the silver-gold gap is too big for now.

    A 100pt gap means a team could suffer/benefit of a 30% “discount” on the match value just by having a gold over a silver. A team made of 3 silvers/2 gold vs a team of 2 silvers/3 gold would have a huge benefit – be it a loss or a win – while the difference in skill is overall not so big. Especially if it’s a Silver 1 over a Gold 5. Imo, a gap of 70 (or 80) would be enough, while it’ll still be 3 to 4 times more than the bronze-silver gap (20pts).

    Also, maybe the plat-diamond gap isn’t big enough? From what I’ve seen, diamond players tend to not play the more “carry-potential” role, often filling up what’s left. But when they do take these roles, they end up having a high influence on the game, much more than a plat would have, given the same position.
    On that point, I’m not really sure so I’d gladly discuss about it but I feel the gap isn’t big enough.

    • CupcakeTrap says:

      On the Silver-Gold gap: so far, the data indicate a very strong gap there.

      On Diamonds: confirmation bias is, I think, a major concern here. People always remember when a Diamond hyper-carries. But there are plenty of matches where one team had a Diamond, and the other didn’t, and the Diamond team still lost.

      I’m probably going to make one final adjustment to the skill valuations in the near future, as more data comes in.

      Speaking of, the most deliciously high-quality data is also the easiest to gather: https://leaguefactions.net/balance-testing/

      If you have a moment, please consider setting up a couple balance matches. It’s easier than setting up a factions match, both because there are no defined “sides” and because we’re totally fine with imba matches: indeed, imba matches offer valuable information.

      Although I welcome any match data, if you want to make it extra-useful, try to limit the total number of tiers per match.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: