Message from @Watching the Watchers

Discord ID: 781069078374776832


2020-11-25 08:01:57 UTC  

its like handling astronomy data lol

2020-11-25 08:02:06 UTC  

only time I've had such large photos

2020-11-25 08:02:55 UTC  

yea this is the thing, I dunno offhand anything online that can handle graphs this large, except maybe some esoteric webgl graph rendering libs

2020-11-25 08:03:07 UTC  

so I'm not sure if this will be viewable online

2020-11-25 08:03:17 UTC  

I might have to use Gephi (desktop application) to view this

2020-11-25 08:03:29 UTC  

in fact I'll try that now

2020-11-25 08:04:03 UTC  

There are a few especially with an Amazon instance to back it

2020-11-25 08:04:27 UTC  

I don't have access to one at the moment sadly...

2020-11-25 08:07:17 UTC  

first I just need to see if this looks sane

2020-11-25 08:07:38 UTC  

if I get an interesting result (CONSPIRACY or BUNK) then I'll figure a way to show it to everyone

2020-11-25 08:07:51 UTC  

right now I just want to just _see_ it at all lol

2020-11-25 08:08:25 UTC  

LOL

2020-11-25 08:08:27 UTC  

exactly

2020-11-25 08:08:33 UTC  

https://github.com/d3/d3 is a JS library that is designed for heavy duty graphs charting and data visualization

2020-11-25 08:08:45 UTC  

yea I've been using d3

2020-11-25 08:08:55 UTC  

but I don't think its meant to handle 90K-1.5M nodes

2020-11-25 08:09:00 UTC  

and even more edges

2020-11-25 08:09:00 UTC  

You'd ahve to do some massive filtering

2020-11-25 08:09:27 UTC  

yea without knowing what it looks like, I don't know what is safe to filter

2020-11-25 08:09:39 UTC  

Data optimization is a must...

2020-11-25 08:09:39 UTC  

@AntiFish03, you just advanced to level 1!

2020-11-25 08:10:30 UTC  

OK Gephi froze

2020-11-25 08:10:38 UTC  

Wow...

2020-11-25 08:10:46 UTC  

its doing a layout

2020-11-25 08:10:50 UTC  

so it makes sense lol

2020-11-25 08:10:57 UTC  

I just wish it was async and not so rude

2020-11-25 08:11:25 UTC  

and my latest svg is still running 40 mins 😦

2020-11-25 08:11:29 UTC  

Lol not everything can play as nice at async as Python ot JS

2020-11-25 08:11:38 UTC  

OK fine I'll do the clumping edge labeling

2020-11-25 08:11:47 UTC  

it results in small number of edges

2020-11-25 08:14:25 UTC  

Sadly, I don't have a lot of experience with data analysis... I mean my attempt at try to make an example at how simple it would be to hide code in a vote tabulator for someone is generally as far as I have to go, it's either data input or display of analyzed data.

2020-11-25 08:16:12 UTC  

It's nothing special and I put maybe an hour into it to prove what I was thinking. But it has no place around real data...

2020-11-25 08:17:32 UTC  

Not, that it couldn't be used as a trainer for an AI to look for similar style inconsistencies.

2020-11-25 08:18:02 UTC  

cool

2020-11-25 08:18:13 UTC  

gephi is still forzed

2020-11-25 08:18:32 UTC  

What's your resource monitor say?

2020-11-25 08:19:24 UTC  

it says Gephi is using 1 full core

2020-11-25 08:19:33 UTC  

6 G of RAM

2020-11-25 08:19:49 UTC  

and running very inefficiently w.r.t power 🙂

2020-11-25 08:20:47 UTC  

man that 500k edge graph is taking almost an hour to turn into an svg