advanced-download-and-upload-abort-restart Question #151
Comments
|
I tried setting something up like: remote_stat = await client.stat(path)
remote_size = int(remote_stat['size'])
if await aos.path.exists(dest):
stat = await aos.stat(dest)
size = stat.st_size
else:
size = 0
while not remote_size == size:
async with client.download_stream(path, offset=size) as stream:But that will only work if the local file is exactly the size of the remote file. How can I check |
|
In first post I see some problems: Unfortunately aioftp is a little chaotic project, so you wont get empty block on iteration end. Maybe this should be made as you expected. So your check and break call will never happend. Iteration internaly ends when empty block comes from socket, so you need just to iterate as much as you can. |
|
Thanks for your reply. I had a long debugging session yesterday and can confirm if not block:
breakis never evaluated while max_attempts<30:
try:
logging.info(f'Starting Attempt {max_attempts}')
async with async_timeout.timeout(20):
async with aioftp.Client.context('192.12.137.7', user='anonymous', port=21) as client:
if remote_size is None:
logging.info(f'Getting remote stats for file {path}')
remote_stat = await client.stat(path)
remote_size = int(remote_stat['size'])
logging.info(f' Remote file has size {remote_size}')
async with aiofiles.open(dest, mode='ab', ) as local_file:
#Check to see if local_file exists
if await aos.path.exists(dest):
stat = await aos.stat(dest)
size = stat.st_size
else:
size = 0
logging.info(f'Starting at postition {size}')
local_file.seek(size)
if remote_size == size:
break
async with client.download_stream(path, offset=size) as stream:
async for block in stream.iter_by_block():
await local_file.write(block)
except aioftp.StatusCodeError as ftp_e:
max_attempts +=1
logging.info(f'Found aioftp error, trying another attempt')
if ftp_e.received_codes ==( '426',):
logging.info(f'Forced timeout error, trying another attempt')
if ftp_e.received_codes != ( '426',):
logging.info('new code')
asyncio.sleep(1)
continue
except asyncio.exceptions.TimeoutError as asy_e:
logging.info(f'found time out exception')
max_attempts +=1
continueAdding a time out of 20 seconds creates a lot of |
|
This issue should be opened, since I don't understand the nature of failure is ot ftp client or server problelm. |
|
All I can say is during my debugging when I got to a point where When |
|
@danhamill |
|
In my comment,
This loop sits inside an I have been extensively using this routine and have found some of the files I download locally are larger than what is provided on the server. Do you have any idea why this could happen? Thanks! |
|
Not enough info to make predictions. You should try to reproduce this on some big text file and then compare local «bigger» and remote «smaller» files with |
I am trying to implement a the advanced-download-and-upload-abort-restart example for my use and seem to have come across an infinte loop when my first file completes the download:
Logging Info
As you can see the code appears to be working properly, and I am getting a complete transfer. However the code does not seem to move on to the next file.
How can I write a conditional statement that checks for
[aioftp.client] 226 Transfer complete?Which bit of code is that message coming from?
The text was updated successfully, but these errors were encountered: