remote_exec hangs forever when remotely using gevent model only at python3

[pcw109550]

The problem only occurs at python3.

Environment:

• OS: Linux faa410b1f09b 4.19.121-linuxkit #1 SMP Thu Jan 21 15:36:34 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
• Python version: Python 3.7.5

Dependencies:

• gevent version: 21.1.2
• execnet version: 1.8.0

Code:

import execnet
# hangs forever when remote is gevent
execnet.set_execmodel('thread', 'gevent')
gw = execnet.makegateway()
print(gw)
print(gw.remote_status())

print('channel.send')
print (gw.remote_exec("channel.send(333)").receive())
print('everything done')

Problem: Above code hangs forever.

Stdout:

<Gateway id='gw0' receive-live, thread model, 0 active channels>
<RInfo 'execmodel=gevent, numchannels=0, numexecuting=0'>
channel.send

Debug log:

[89653] gw0 sent <Message STATUS channel=1 lendata=0>
[89653] gw0 [receiver-thread] RECEIVERTHREAD: starting to run
[89655] creating workergateway on <execnet.gateway_base.Popen2IO object at 0x1106a5820>
[89655] gw0-worker [serve] spawning receiver thread
[89655] gw0-worker [serve] joining receiver thread
[89655] gw0-worker waiting for receiver thread to finish
[89655] gw0-worker [receiver-thread] RECEIVERTHREAD: starting to run
[89655] gw0-worker [receiver-thread] received <Message STATUS channel=1 lendata=0>
[89655] gw0-worker sent <Message CHANNEL_DATA channel=1 lendata=73>
[89653] gw0 [receiver-thread] received <Message CHANNEL_DATA channel=1 lendata=73>
[89653] gw0 [receiver-thread] received <Message CHANNEL_CLOSE channel=1 lendata=0>
[89655] gw0-worker sent <Message CHANNEL_CLOSE channel=1 lendata=0>
[89653] gw0 1 channel.__del__
[89653] gw0 sent <Message CHANNEL_EXEC channel=3 lendata=31>
[89655] gw0-worker [receiver-thread] received <Message CHANNEL_EXEC channel=3 lendata=31>

The code hangs while executing remote_exec. The problem only occurs when remote thread model is gevent.

I guess the problem occurred because while gevent is rapidly syncing with python3, execnet was not maintained :C.

Maybe related to function mapping when making ExecModel:

execnet/execnet/gateway_base.py

Lines 112 to 125 in 894533a

	elif backend == "gevent":
	importdef = {
	"get_ident": ["gevent.thread::get_ident"],
	"_spawn_n": ["gevent::spawn"],
	"threading": ["threading"],
	"queue": ["gevent.queue"],
	"sleep": ["gevent::sleep"],
	"subprocess": ["gevent.subprocess"],
	"socket": ["gevent.socket"],
	# XXX
	"_fdopen": ["gevent.fileobject::FileObjectThread"],
	"_lock": ["gevent.lock"],
	"_event": ["gevent.event"],
	}

I know this repo is unmaintained. Hoping for receiving someone's help!

[RonnyPfannschmidt]

turns out adding a remote_status after the remote_exec gets it running

this need s ome extra help

[arpitgupta-it]

@RonnyPfannschmidt I would like to take this up, in case it is still open.

[RonnyPfannschmidt]

Please go ahead

I suspect the issue is similar to the one we worked around with threats