EarthWyrm

Based on a PM he sent me yesterday, his 5 runs on the internal server all involved him going through the maps and manually defeating all the mobs. I don't know whether or not he used the [I Win Button] on his test server runs, but he definitely used the old-fashioned method of beating stuff up in at least 5 of his 15 runs.

If someone could wander over to Test, do a few runs, then change their global name and do the exact same few runs, that would at least rule out some sort of weird global-name seed as part of the bug.

Although the account-based testing above will probably have the same type of information, unless the globals on those two accounts are highly similar.

Same. On the off chance that something had changed on test, I hopped back on and used the same Abyss mission I'd given Synapse to test. He ran it 15 times and got 14-19 recipe drops each time. I've run it over 20 times on a single character and never gotten above 6. My average is three.

In 344 kills, I again got 3 recipes.

I'm sorry he can't reproduce it. It's real and it's consistent, but not across characters/accounts in a way that we've been able to isolate the common factor.

I was basing my comment on a PM conversation I had going with Synapse. Based on his posting, however, it appears that he's concluded that we're imagining the problem.

Given the fact that the only way I could possibly offer him to replicate the problem would be, "Log into my main villain and run this map," I understand where he's coming from. I believe that the problem is a combination of character and map, and could be based off anything from number of consecutive consonants in your global name to the height slider. But if we can't give him a way to make the problem happen, there's nothing for him to fix.

Ah, cool. I was hoping that if I made a mistake, someone would point it out quickly, and it wouldn't be fatal. I follow the math, and knew that if I was making an incorrect leap at any point it was probably in the weighting. Thank you for the correction.

This is why I don't feel like a total moron.

So, it's not as extreme as my response to myself (ubermegageeky thing to do ) would indicate, but it's still pretty unlikely.

Absolutely. And I still believe that his data don't show that there is no problem, but instead suggest that the problem may be masked by half of the people whose drop distributions are off being incredibly happy with the new system.

I very much appreciate the correction.

Translated to remove geekery:

If drops were working as intended, based on published drop rates, Synapse would get an average of 14 recipes/run over 15 runs between 1 time in 100 and 1 time in 1000.

Conclusion: Based on Synapse's own data, drops are likely not working as intended, but are also not uniformly low. If some people have "hot" versions of a map, while others have "cold" versions, then the average drop rate when datamined will look to be exactly where it's supposed to be.

His numbers are actually higher for recipes than the predicted. The map uniformly spawns 344 enemies. 268 are minions. 68 are lieutenants. 8 are bosses who downgrade to lieutenants. This is why I kept using it for testing - because of the invariant spawns.

The predicted number of Mob-drop recipes for that number and distribution of spawns is 11.2.

Set for 0/8 or +1/8, I have not (since I16 beta started) gotten more than 5 or 6 mob-dropped recipes for clearing that map with my brute. My average is about 3.

As others have noted, this is not true for everyone. I also don't feel like it's a perception issue. I think there's a factor other than drop rates at work; I have no doubt that the drop rates have not been changed.

This. The problem is not the drop rate - it's the drop distribution. As Starcloud said in the section I snipped, there's something wonky happening in the drop logic. Given that it's not something that replicably happens with changes to the virtual teammate slider, I don't know what else to test.

We know what the drop rates are supposed to be. I trust that they haven't been changed. We don't know how the drop distribution is supposed to work, so we can't test it, and now we've got a top-heavy set of runs from a redname whose lower-bound of 14... hey, wait a second...

15 runs on a map with 344 enemies is 5160 enemies.

The per-map weighted drop rate should be 3.26% (((268*.026667)+(76*.053333))/344).

Even if Synapse only got 14 recipes/run, his overall drop rate would be (14*15)/5160 = 4.07%.

The margin of error for an expected drop rate of 3.26% with 5160 observations is +/- .48%, so a range from 2.78 to 3.74 would be expected 95% of the time. 4.08 is outside that range. I'm pretty certain that if you computed it using 99% confidence values, these results would show up as aberrant as well.

That means that SYNAPSE'S DROP RATE IS OUTSIDE THE MARGIN OF ERROR FOR THE MAP if he only got the LOWEST number of drops out of his 15 runs each time (which he didn't). Yes, it's outside in the high direction, but that's completely consistent with people saying, "I don't see anything wrong with the drops." If there's something wrong with drop distribution, sometimes it will be off to the low end (where people will complain), and other times it will be off to the high end (where they will not).

Welcome to the high end.

I've used mainly "real" missions and gotten low drop rates.

I've heard people get really bad drop rates on the BM map. I've heard people get really good drop rates on the BM map. Same for the Behemoth map. I've heard people get bad Lib TV drop rates. I've gotten good ones, and some less-good.

I know the devs are looking at this, but suspect that the lack of consistency we're seeing is going to make it really hard for them to isolate the problem, or even conclude a problem is present. For all that we've been collecting data since beta, the people most likely to read and contribute to threads on drop rates at the ones who feel like something's wrong. I don't think I'm imagining the problem, but I also can't come up with a reliable way to demonstrate what's going on.

What you describe is exactly the reason it's going to be so hard for this bug - and I'm convinced there really is one - to be tracked down. It doesn't seem to replicate consistently across accounts, characters, or maps, but enough people are seeing it and enough data have been gathered to show that something is going on. Glad you're relatively unaffected.

Out of curiosity, what map did you use for your farming runs? (If you'd rather not publicize, feel free to PM it to me.) I ask not because I wanna steal yer tech, but because I'm still trying to figure out whether there are replicably "good" maps that the devs might be able to use to set against maps like the one I've used that has been, at least for my characters on test and live, replicably bad.

When I've run that map I've either gotten right around 100% of expected (spawned for 8 with no bosses, that's around 26-28 recipes) or right around 50% of expected. I also got my only purple since I16 dropped on one of those runs. It's a relatively small number of runs, but the consistency of what I've seen in terms of realized vs. expected Pool A (common/uncommon/rare/purple treated as a single pool) has been noteworthy.

Even the random number god must obey the laws of probability.

http://boards.cityofheroes.com/showthread.php?t=188370

Pool A recipe drops are demonstrably low, or at least, lower than published numbers would suggest they ought to be. With 30-40k observations, the difference is outside what would be expected by chance.

Not impossible. But highly, highly, highly improbable.

The best evidence I've seen came from the bugged Cargo ship map, a while ago. When there was a set pattern of exactly which mobs would drop Pool A recipes, one of the things that could drop at the "Pool A" spot was Purples. Those same spots in the sequence never dropped costume recipes, which happened at different points in the sequence.

So, yes. I'm fairly confident based on observations of that map that Purples are low-probability drops in Pool A. If I had to guess, I'd say to get a purple there has to be a Pool A drop, it has to be the 1/41 that's rare, and then there is another roll to determine if it's purple.

I no longer have easy access to that map. How variable are the spawns? What I mean is, there is a Council warehouse map I used villainside, because it requires a glowie click to complete. Killing every enemy on the map resulted in 268 minions, 68 lieutenants, and 8 bosses (who auto-downgraded).

Every. Time. Those exact numbers replicated every time.

And on that map, I have never gotten more than 5 recipes when solo but set to the equivalent of 8 villains. The average is about 3, for 344 defeats, on at least 20 runs over beta and live.

There are maps that are less extreme. Beber seems to get around 70% of advertised drop rates. I've found other maps that are very "feast or famine" and either give 50% or 100+%. I strongly suspect that there's something that happens when the map is spawned that determines where drops are going to fall, but have run out of things to guess at. It may also have something to do with the nature of the spawns; the BP map is unique in high-level play. Those mobs didn't even count as BP for badge purposes for a long time. The Axis Amerika map is not unique, but the mobs don't occur that often. The Council map includes Bounty Hunter bosses who appear nowhere else in the game. The Battle Maiden map includes low-occurrence high-level mobs.

None of which explains BillZ's lack of drops in paper missions, which I've also observed, and in Borea missions, and in Cimerora missions. Flailing guesswork is guesswork.

Downloaded it and it looks great! Thanks again for creating this. I did have one question, though, and it's not specific to the new version.

I'd been assuming that since bosses who've been downgraded to lieutenants have their inf-award set to lieutenant level, their drop status turned them into lieutenants as well (that is, downgraded bosses have a .053333% chance to drop a pool a recipe, just like any other lieutenant). If that's the case, I'm guessing that the expected ranges dropstats calculates will be off on the high end for people who are running on the "no bosses" setting, since it probably treats anything with a boss-name as having boss-level drop rates. Is that correct?

It's often not going to be a major issue, but my runs from yesterday included 350 downgraded bosses, so the expected numbers Dropstats gives me are very different from the expected numbers I calculated by hand.

But highly consistent. Looks like Pool A drops were 10/10/7 (or 8, if temp powers are pool A; does anyone know that for sure?), and salvage was always in the 30-35 range.

What I'm currently seeing is a great deal of within-map consistency, but not necessarily much across maps.

My Council map has an average of about 3 recipe drops per time that I clear it (344 enemies).

I have another map that has 400-500 enemies and my 4 runs on it have yielded 22/20/20/17 Pool A drops.

The thing is, different people seem to get different results with the same map, and I've sometimes gotten different results when I use the same maps with different characters, which makes it look like the RNG seed derives from both the map and the character. To my knowledge, no one else has run the Council warehouse map (it's the first mission in Abyss's "Light and Darkness" arc, if anyone with a 50 vill wants to go in and give it a run to test and see if they can get better than a 1% overall drop rate on it) to see if it's just generally bad, or if it really was specific to the character I happened to be testing with.

I'm going to try and write up my cross-map results to send to Synapse. I think that overall drop rates look fine because the thing that's gone wrong would have no effect on teams and no effect on soloists who don't bump their spawn sizes, and won't affect everyone who does bump their spawn sizes, if they're on the "right" map.

The fly in the ointment is the data from up-thread that showed diminished drop rates for someone fighting at 0/1, but I think Archie's overall data showed that solo at 1 was pretty close to expected across 5k or so observations, so it's hard to know. But I'll be trying to write it up some time very soon.

I'm really starting to think there may be some truth to this. I found a map - ONE map - where my drop rates are astronomically good. It's replicated once. I need to try it again and see whether I've had 2 fluke runs out of 2 runs on it, but it's the only map where I've gotten above 50-70% of expected recipes in a month of testing. (I mention "map" rather than "instance" because of how many instances of an identical map I used in testing, with consistently bad results; if there's something in the map identifier that is part of the seed, maybe... it would also make sense if part of the seed for drops identified team position as an element of determining who gets the drops and that this is determined when the map is spawned, so that as you move from map to map in a team the "favored spot" moves across teammates, and most maps don't favor position 1... but I'm shooting in the dark.)

I may have data to report tonight. I suck at taking breaks. My (fairly mild) OCD won't let me leave unanswered questions alone.

What I really want to know is what's different, in terms of RNG seeds or whatever, when people get really good runs. I had myself one, and only one, really good run (and the 18 recipes on 641 defeats isn't even that good - it's an overall 2.8% drop rate, which is about what it ought to be if things were working consistent with published drop rates), so it seems like there has to be something going on with how the RNG is seeded, what place you have in the virtual "team," or something that we simply don't have access to because we don't get to see the underlying architecture of how drops are assigned.

I may just take a break from playing the game until this gets fixed. Not because drops matter that much to me, but because communication from the dev team does.

Agreed. The difficulty settings may or may not be having an effect, but it's not as simple as "You're getting 1/8 if you're set for 8, and the other 7 disappear into the aether". If that were the case, drop rates would consistently be 1/8 of what they're supposed to be. At my worst, I only end up at around 1/4, and most folks who have trouble seem to be in the 50-70% range.

Random thought: What character-level settings changed by default when I16 went live? One is that everyone got dropped back down to the base difficulty. Could that have had an effect like what we're seeing, where some people are just fine on drops based on where they were pre-release, while others are not? How are drops on characters created after I16 went live?

That's a great list. The only thing I would add is that the /whereami command allows you to know precisely *which* map you're on, if we feel the need to record that as well. For instance, the "Retrieve Void Hunter Rifles for Abyss" mission I used for a lot of my testing is on one Council warehouse map in the version I had on Live, but one of a couple of different Council warehouse maps thru Ouroboros. If we think map might be a culprit, being precise on which map may be useful.

I've gotten somewhat sloppy on some of this, I'll admit. I think it frustration at hearing nothing for so long led me to just want to keep collecting data points to show that there's really something going on.

A final note from my end - all my testing leads me to believe that there is nothing wrong with salvage drop rates. Continuing to track them is fine, but my tests have all showed salvage rates easily within margin of error for whatever sample size I collected, barring perhaps the smallest.

I got 5 or 6 purples over the course of Beta. I also had an overall drop rate substantially less than 50% of the published drop rates over several thousand mob defeats for Pool A recipes. Purples seemed, ironically enough, to be less affected than anything else in my testing.

Edit: FPARN :P

I can't even get a replicable "good" map. I should go back to Ouroboros and get the TV Don map again. That was the only map since beta started that I got greater than or equal to the expected number of drops for ANY run.

I ran the demon map once tonight. Got 3 common recipes (I think), no sets. Spawned at 0/6 to minimize the number of scaled-down bosses.

Then swapped back to my brute and ran a few more missions at +1/8.

433 minions, 143 lieutenants.
Got 13 salvage (if temp powers are in Pool A; if not, 12). Projected number was 19.17. So I did twice as well as in my last set of runs, but twice as well as "cruddy" still puts me below 70% of published drop rates. That's not outside margin of error for a sample size of 576, but included in the aggregate data collected since beta started, it continues to make the case that the drop rates are lower than they should be.

To note: This set of data was collected with myself set to "No XP/double inf when exemplared" rather than "Earn XP", and while there was an improvement, it's my suspicion that this setting is also not the culprit.

There are a lot of people who put more time into this beta than I did. Just on this topic, UberGuy and Archie Gremlin put in a ton of work (to name two of several), with Archie developing a great tool to help everyone quantify what we're seeing. That was a ton easier than me manually copy-pasting one screen of drop logs at a time out of the program and into word.

If there's a bug here, I just wish I'd been able to figure out what it was.