How to run a python script like pm2 for nodejs

Posted on Apr 7, 2022

Question

I've used pm2 for my Node.js script and I love it.
Now I have a python script which collect streaming data on EC2. Sometimes the script bombs out and I would like a process manager to restart itself like pm2.

Is there something the same as pm2 for python? I've been searching around and couldn't find anything.

Here's my error

  File "/usr/local/lib/python2.7/dist-packages/tweepy/streaming.py", line 430, in filter
    self._start(async)
  File "/usr/local/lib/python2.7/dist-packages/tweepy/streaming.py", line 346, in _start
    self._run()
  File "/usr/local/lib/python2.7/dist-packages/tweepy/streaming.py", line 286, in _run
    raise exception
AttributeError: 'NoneType' object has no attribute 'strip'
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:90:

It's a simple data collecting script

class StdOutListener(StreamListener):
def on_data(self, data):
    mydata = json.loads(data)
    db.raw_tweets.insert_one(mydata)
    return True

def on_error(self, status):
    mydata = json.loads(status)
    db.error_tweets.insert_one(mydata)

if name == ‘main’:

#This handles Twitter authetification and the connection to Twitter Streaming API
l = StdOutListener()
auth = OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
stream = Stream(auth, l)

#This line filter Twitter Streams to capture data by the keywords: 'python', 'javascript', 'ruby'
stream.filter(follow=[''])

That I would like it to just restart itself in case something happens.

Answer

You can actually run python scripts from within pm2:

pm2 start echo.py

If the script ends in a .py suffix it will use a python interpreter by default. If your filename doesn't end in .py you can do:

pm2 start echo --interpreter=python

I've found you have to be a little bit careful which python you are using, especially if you are using a virtualenv python with a different version to the 'default' python on your machine.