new examples : a parallel pipeline for image processing(resize) #1755
Conversation
examples/image-processing/main.py
Outdated
| # a job is an image to resize and a path where the resized_image will be saved | ||
| # once, the image was resied, the worker can ask a new job | ||
| # server will keep sending job to workers until there is no images left or user hit ctl+c | ||
| NOTE: opencv, numpy, click, loguru must be installed |
There was a problem hiding this comment.
Probably makes sense to add a requirements.txt to the directory
There was a problem hiding this comment.
Hello, as you suggest, i've added the README.md and a requirements.txt(click, pyzmq, pillow).
examples/image-processing/log.py
Outdated
| @@ -0,0 +1,17 @@ | |||
| from sys import stdout | |||
|
|
|||
| from loguru import logger | |||
There was a problem hiding this comment.
Maybe not worth it to use a third-party logger for a simple example? Let's stick to the standard library.
There was a problem hiding this comment.
i also changed the loguru with the standard logging library.
examples/image-processing/main.py
Outdated
| start = time.time() | ||
| for cursor, path2source_image in enumerate(image_filepaths): | ||
| bgr_image = cv2.imread(path2source_image) | ||
| resized_image = cv2.resize(bgr_image, dsize=size) |
There was a problem hiding this comment.
If we're just resizing images, pillow is a far lighter dependency than a full computer vision library like opencv.
There was a problem hiding this comment.
This new version use pillow for image resizing, hope this example will help the community to have a better understanding on how to combine zeromq and python-multiprocessing for fast image processing tasks.
this example shows how to use zeromq and multiprocessing to set up a parallel system capable of processing(resize) a large volume of images. the architecture is based on 4 types of sockets : server-side: router/publisher, client-side : dealer/subscriber.