Message from @Jacob

Discord ID: 500767752340897812


2018-10-13 05:59:13 UTC  

Does
```
if {"immigrant", "immigration"}.intersection(comment_text)
```
work?

2018-10-13 05:59:21 UTC  

would that work for plurals?

2018-10-13 05:59:35 UTC  

comment_text is a string, so no

2018-10-13 05:59:44 UTC  

intersection is for sets

2018-10-13 05:59:47 UTC  

ah

2018-10-13 05:59:57 UTC  

so I can't check for certain phrases in order?

2018-10-13 06:00:17 UTC  

You can, look at the "white people" condition as an example

2018-10-13 06:00:24 UTC  

oh that's right

2018-10-13 06:00:33 UTC  

I just can't do it as an intersection

2018-10-13 06:00:52 UTC  

right

2018-10-13 06:00:55 UTC  

so, if I want to check for one out of a list of phrases, I just have to use ```and``` a lot?

2018-10-13 06:02:01 UTC  

I mean, I have a list of phrases, and I want to see if one of those phrases is in a comment

2018-10-13 06:02:19 UTC  

that is a straightforward way, a less verbose way would be:
```python
if any(key_phrase in my_string for key_phrase in list_of_phrases)
```

2018-10-13 06:02:58 UTC  

thanks

2018-10-13 13:24:40 UTC  

what do you guys think of DuckDuckGo? I'm sick of google

2018-10-13 20:24:08 UTC  

@Deleted User at first I was hesitant to use it because its owner and founder is a "you know who" but I find DDG gives different results from Google often so it's a good other source to check out. However, if you like Google results and are just worried about privacy then you can use startpage.

2018-10-13 20:28:00 UTC  

@ThisIsChris Any idea why the script skips over some comments?

2018-10-13 20:28:14 UTC  
2018-10-13 20:28:21 UTC  

If I use this, it shows comments that the script didn't find

2018-10-13 20:32:00 UTC  

@Jacob you could be being rate limited. Instead of silently handling the JSONDecodeError try printing the error to see what's happening. One thing you can do to mitigate is add
```python
import time
time.sleep(4)
```

2018-10-13 20:32:15 UTC  

I did try that

2018-10-13 20:32:19 UTC  

Into the while loop

2018-10-13 20:32:21 UTC  

but I think it misses a lot of comments, no?

2018-10-13 20:32:39 UTC  

I did time.sleep() with .5, 1, and 2

2018-10-13 20:32:45 UTC  

Try 4

2018-10-13 20:32:52 UTC  

That's the documented rate limit

2018-10-13 20:33:04 UTC  

Wouldn't 4 seconds be enough for a lot of comments to be posted?

2018-10-13 20:33:14 UTC  

Also I think the rate limit is actually 1 request every 2 seconds

2018-10-13 20:33:18 UTC  

Otherwise try printing the JSONDecodeError

2018-10-13 20:33:48 UTC  

E0602:Undefined variable 'JSONDecodeError'

2018-10-13 20:33:56 UTC  

@Jacob I think it's one per 2 seconds if you are using a dev account. One per 4 seconds if not

2018-10-13 20:34:04 UTC  

ah

2018-10-13 20:34:35 UTC  

would be interesting if we could find a way around this

2018-10-13 20:34:43 UTC  

like send requests from 4 different IP addresses

2018-10-13 20:35:07 UTC  

To print the JSONDecodeError rewrite the `except` statement to be
```python
except JSONDecodeError as e:
print(str(e))
```

2018-10-13 20:37:06 UTC  

Well, another way to look it is, do we *need* every comment? Running the script for an hour probably will yield more comments than I have time to respond to anyway.

2018-10-13 20:37:11 UTC  

@Jacob sure that shouldn't be a problem, you just need 4 proxies. Or just get a dev account, which is free and not too hard to set up. But I think from a development focus that it's more important to get your system working on just the comments you find first, and after that worry about increasing the throughput of the comment stream

2018-10-13 20:37:28 UTC  

@Jacob "do we need every comment" exactly

2018-10-13 20:37:58 UTC  

That can be a future development step, first step is just to get it working sometimes, then work on that part later

2018-10-13 20:38:13 UTC  

Ya, the function of the program is "scan Reddit for pro-immigration comments so we can respond to them" not "scan Reddit for literally every pro-immigration comment"

2018-10-13 20:38:22 UTC  

Yeeep