Need some help with this -- multiprocessing with gunicorn

[DevasiaThomas]

https://stackoverflow.com/questions/45697850/error-when-closing-master-process-multiprocessing
Sorry, I didn't know how else to contact you guys @benoitc . Any help is appreciated. The error traces are from gunicorn

[tilgovi]

I took a look at the stack traces but I don't know what you're seeing or what's causing it. None of those traces look to be from gunicorn. Gunicorn does not use multiprocessing to spawn workers, it uses os.fork().

My best guess is that maybe you're using multiprocessing to start your other process, and then just invoking Gunicorn. Maybe Gunicorn and multiprocessing are both catching and handling the keyboard interrupt.

If you're not already, can you try using multiprocessing to start two processes, one for your background process and one for gunicorn, and then just have the main processes join them both?

[DevasiaThomas]

Hi @tilgovi ,
Well the os.fork() explains it. It copies over the multiprocessing object and its state over to each worker from the listener process. Which causes each of the worker to try kill the the process I created when they die (because they think they are the parent process - but they are not -this happens because of the dumb copy during os.fork() - thats my best guess). The number of repeated stack traces match the total number of gunicorn processes .
Whatever I am doing happens in the sitecustomize.py file. So all this happens in the gunicorn process space but before any of the gunicorn code is executed.

Im thinking over your idea of using a separate process for gunicorn, but it may not help my process. I am not completely sure, but do you think if i do what i am doing in a separate thread, will the os.fork() copy the thread too? I need to be in gunicorns process space.

[tilgovi]

I do not know what to advise you to do here. Is there more I can do to help?

[DevasiaThomas]

No, I think not. Thanks or the clarification. I could have done os.fork() myself but that would make my code not be cross platform. I guess this maybe why (one of the reasons) gunicorn doesn't run on windows. I'll try something on my end. Thanks

[tilgovi]

It's true! If you watch #524 for updates, I expect we might move to multiprocessing in a future release in order to support Windows. Then, you might have an easier time with this.

[benoitc]

did you try to launch multiprocess in the when_ready hook instead?

[benoitc]

@tilgovi there are many reason we are not using multiprocessing though. one is to have a better supervision that the one offered by this module.

[DevasiaThomas]

@benoitc Thank you for that suggestion. I did see it, And I did consider it at one point. But I am limited by my own constraints

1. I need to do everything before my target application loads up (in this case gunicorn). So whatever I do, I do within sitecustomize.py
2. I can't make any changes to config-files/code that are in place.

I could run my code as threading.Thread threads, but that may hinder the listener process performance - I haven't checked the numbers yet. Also I'm a bit afraid about this here

[DevasiaThomas]

@benoitc Unless there is a way to dynamically tell what when_ready is to do

[benoitc]

i guess you can enrich the config directly before loading gunicorn.

on the other hand i think the issue comes from having the atexit function which is apparently shared between the processes. I think gunicorn should take care of it when a worker is spawned

[DevasiaThomas]

Yeah, the atexit functioning is done by the multiprocessing library. I'll let you guys know how I handled it. All this happens because of the dumb copy that os.fork() does. I agree that it is on the developer to take care of the dumbness but since this os.fork() is inside gunicorn - It's something I cannot control. It might b a week before I can get anything done on this issue though.