Exactly, C++ is mostly scraping the websites and also partially building search indexes

Huh, I thought you were scraping in C#, and was asking since I tried making a scraper before in C#

what exactly is this pornspider doing

@sum confirmed for jedimocro

But aside from linkin shit I didn't exactly manage how to get data to work properly

@Spicy it indexes a lot of porn websites and provides a cross site search over all of them

I made that while I was temporarily jobless because I really had nothing to do

how much effort did it take to not get banned everywhere

and then some of the users' data like os gets stored to do some analytics?

probably just regular logs

Well that's the Haskell part. My girlfriend is doing a lot of data analysis in haskell. Removes illegal content, keeps the huge amount of data consistent

Just search logs, yes. I don't even log IPs

and its just a web interface?

Site interface?

web

Ah the only GUI is web

pretty neat

But the web interface is pretty basic

I'm not that much of a web dev

fuck it

Time to try and make my scrape shit work

Why? You can just contribute to mine and we can make pornspider great

Why make your own?

Because I want to scrape other things

Mainly Boorus and sometime 4chin

Imagine a desktop client for 4chin

Since too many times I see few nice images in a thread

Save it, only for it to pop up to 500 and go into archive at fireden

And browsing fireden images is reaaaaaaally ***fun***

https://github.com/avplayer/avhtml

I used this

To build the crawler

and a nip nop ping pong to you

chinks write a lot of OSS

Learn chink and profit from it

Well damn

4Chin blocks WebClient class

@kontroll ordering tomorrow

wish me luck I guess

tfw all pornspider traffic is csg

i hope i dont get fucked™