Message from @meglide

Discord ID: 776176335516205096


2020-11-11 19:09:55 UTC  

my takeways from this are:

1. you need a high sigma (a large range of order of magnitudes) to see a Benford-looking graph.

2. you need a large number of samples to reliably reproduce a Benford-looking graph (you can even find the 1s bar lower than the 2s bar at 150 samples if you repeat the simulation over and over , at 150 samples).

3. If you play around with the numbers, you can make or break the law; the law seems to apply only to certain shapes and centerings of the normal distribution

2020-11-11 19:10:43 UTC  

No issue I was just poking around since the site seem to allow me to edit, I changed 1 value then ran it, as expected it broke but I put it back. I just wanted to apologize if it affected other users.

2020-11-11 19:11:25 UTC  

oh no it can't

2020-11-11 19:11:43 UTC  

it is not collaborative, if you change it, then it is local

2020-11-11 19:11:48 UTC  

if you save, it forks to a new copy

2020-11-11 19:12:04 UTC  

Ahh similar to github ?

2020-11-11 19:12:48 UTC  

right

2020-11-11 19:13:14 UTC  

Thats neat I like the interface on mobile :)

2020-11-11 19:19:29 UTC  

I added a regenerate button to make it easy to click fast to see the variations of the bars from simulation to simulation

2020-11-11 19:24:48 UTC  

me playing with the sliders

https://cdn.discordapp.com/attachments/771201221145919499/776165551608430682/benford-sliders.gif

2020-11-11 19:26:26 UTC  

anyway

2020-11-11 19:26:57 UTC  

I think the conclusion here is: if the variance in magnitude is low, benford's doesn't indicate anything

2020-11-11 19:27:25 UTC  

I think the conclusion here is:


1. if the variance in magnitude is low, Benford's Law doesn't say anything should be expected.
2. Though perhaps one could ask question about why the variance in magnitude is low, and how does this compare to previous elections and so on; I imagine simply looking at the political history of the districts involved etc. would answer such questions.
3. But if the variance in magnitude is high, _and_ there are enough samples, then if the shape of the graph does not follow Benford, then there is something to analyze further.
4. Even in the best case, % deviation from the expected Benford graph is not as useful as it seems, unless you factor in the number of samples (the more samples, the less expected deviation).

2020-11-11 19:32:42 UTC  

All that being said, I bet Mark Nigrini knows more than me

2020-11-11 19:32:57 UTC  

I don't think he'd disagree though

2020-11-11 19:33:03 UTC  

IIRC he did use a lot of samples

2020-11-11 19:34:59 UTC  

2 hours đź‘€

2020-11-11 19:39:06 UTC  

YES thank you for this.

2020-11-11 20:07:39 UTC  

@realz I think you've put this Benford's law issue to rest ... I work with numbers every day and have for many decades, generally those statistical methods are good for evaluating measurements of physical processes (temperature, wind velocity, etc.) but when you look at numbers associated with human behavior then you'll find many time statistical methods don't readily apply, just take the polling predictions versus the election results both in 2016 and 2020

2020-11-11 20:08:07 UTC  

mhm

2020-11-11 20:08:18 UTC  

that is yet another issue

2020-11-11 20:09:19 UTC  

@realz agreed, that is a separate issue than Bendfords law.

2020-11-11 20:10:13 UTC  

The reason that the first digit appears in ratios is because we have a system of numbers based in 10

2020-11-11 20:11:04 UTC  

So 1 should appear more often than other numbers. It would be different than if it was evaluating the last digit that appears.

2020-11-11 20:11:22 UTC  

I imagine that benford's law actually applies in every base

2020-11-11 20:11:26 UTC  

which is interesting

2020-11-11 20:11:31 UTC  

as for the last number!

2020-11-11 20:11:53 UTC  

watch Parker's (Stand-up Maths) video

2020-11-11 20:11:58 UTC  

the last number also has a law

2020-11-11 20:12:15 UTC  

he demonstrates that the last 2 numbers are uniformly distributed in chicago

2020-11-11 20:12:19 UTC  

except for a blip

2020-11-11 20:12:45 UTC  

there are so few trump voters in some, that with just 1,2 or 3 digits, the last 2 digits are not uniformly distributed

2020-11-11 20:12:56 UTC  

The last 2 numbers I would expect to be uniform in distribution

2020-11-11 20:13:04 UTC  

yep he runs that test

2020-11-11 20:13:12 UTC  

and they are, except for this blip I just explained

2020-11-11 20:13:20 UTC  

` there are so few trump voters in some, that with just 1,2 or 3 digits, the last 2 digits are not uniformly distributed`

2020-11-11 20:13:41 UTC  

Yeah but I am sharing that Benfords law isn’t about what’s being measured. Though what’s being measured could bias what you would expect to see.

2020-11-11 20:14:27 UTC  

you are referring back to the base

2020-11-11 20:15:30 UTC  

> https://youtu.be/CMMbZH-H4ks
@Michele411 the problem I have with this sort of thing is that there is no evidence ... it sounds like a conspiracy theory and I cannot differentiate it from something from a crazy person