How to reduce memory usage of Dask Scheduler
·
682 words
·
4 mins
Few useful tips how to configure Dask Scheduler with the aim of reducing the memory usage and its growth over time.
dask.distributed
Dask.distributed is a lightweight library for distributed computing in Python. It extends both the
concurrent.futuresanddaskAPIs to moderate sized clusters.
Resources #
2022
2021
Setting up, configuring and deploying Dask Distributed
·
·
1716 words
·
9 mins
A collection of tips and tricks for setting up, configuring and deploying
dask.distributed in Python projects. Most of topics covered focus on single tenant access pattern, where potentially multiple instances of the same application or a project submit tasks directly to Dask’s scheduler.