The NetBSD Project

CVS log for pkgsrc/devel/py-joblib/Makefile

[BACK] Up to [cvs.NetBSD.org] / pkgsrc / devel / py-joblib

Request diff between arbitrary revisions


Keyword substitution: kv
Default branch: MAIN


Revision 1.19: download - view: text, markup, annotated - select for diffs
Sat May 11 19:08:53 2024 UTC (5 months, 3 weeks ago) by adam
Branches: MAIN
CVS tags: pkgsrc-2024Q3-base, pkgsrc-2024Q3, pkgsrc-2024Q2-base, pkgsrc-2024Q2, HEAD
Diff to: previous 1.18: preferred, colored
Changes since revision 1.18: +2 -2 lines
py-joblib: updated to 1.4.2

Release 1.4.2 -- 2024/05/02

Due to maintenance issues, 1.4.1 was not valid and we bumped the version to 1.4.2

- Fix a backward incompatible change in ``MemorizedFunc.call`` which needs to
  return the metadata. Also make sure that ``NotMemorizedFunc.call`` return
  an empty dict for metadata for consistency.
  https://github.com/joblib/joblib/pull/1576

Revision 1.18: download - view: text, markup, annotated - select for diffs
Sat Apr 13 05:27:48 2024 UTC (6 months, 3 weeks ago) by adam
Branches: MAIN
Diff to: previous 1.17: preferred, colored
Changes since revision 1.17: +2 -3 lines
py-joblib: updated to 1.4.0

Release 1.4.0 -- 2024/04/08

- Allow caching co-routines with `Memory.cache`.

- Try to cast ``n_jobs`` to int in parallel and raise an error if
  it fails. This means that ``n_jobs=2.3`` will now result in
  ``effective_n_jobs=2`` instead of failing.

- Ensure that errors in the task generator given to Parallel's call
  are raised in the results consumming thread.

- Adjust codebase to NumPy 2.0 by changing ``np.NaN`` to ``np.nan``
  and importing ``byte_bounds`` from ``np.lib.array_utils``.

- The parameter ``return_as`` in ``joblib.Parallel`` can now be set to
  ``generator_unordered``. In this case the results will be returned in the
  order of task completion rather than the order of submission.

- dask backend now supports ``return_as=generator`` and
  ``return_as=generator_unordered``.

- Vendor cloudpickle 3.0.0 and end support for Python 3.7 which has
  reached end of life.

Revision 1.17: download - view: text, markup, annotated - select for diffs
Wed Jan 10 14:38:25 2024 UTC (9 months, 4 weeks ago) by bacon
Branches: MAIN
CVS tags: pkgsrc-2024Q1-base, pkgsrc-2024Q1
Diff to: previous 1.16: preferred, colored
Changes since revision 1.16: +2 -1 lines
devel/py-joblib: Unbreak hardware core count

Patch in platform-specific cases for NetBSD and FreeBSD
Without this, the core count routine errors out
No other changes

Maintainer feedback timeout (12 days)
Issue was discussed on tech-pkg and upstream
https://github.com/joblib/joblib/issues/1535

Revision 1.16: download - view: text, markup, annotated - select for diffs
Fri Nov 3 10:35:53 2023 UTC (12 months ago) by adam
Branches: MAIN
CVS tags: pkgsrc-2023Q4-base, pkgsrc-2023Q4
Diff to: previous 1.15: preferred, colored
Changes since revision 1.15: +5 -6 lines
py-joblib: updated to 1.3.2

Release 1.3.2 -- 2023/08/08

Fix a regression in joblib.Parallel introduced in 1.3.0 where explicitly setting n_jobs=None was not interpreted as "unset".
Fix a regression in joblib.Parallel introduced in 1.3.0 where joblib.Parallel logging methods exposed from inheritance to joblib.Logger didn't work because of missing logger initialization.
Various maintenance updates to the doc, the ci and the test.

Release 1.3.1 -- 2023/06/29

Fix compatibility with python 3.7 by vendor loky 3.4.1 which is compatible with this version.

Release 1.3.0 -- 2023/06/28

Ensure native byte order for memmap arrays in joblib.load.
Add ability to change default Parallel backend in tests by setting the JOBLIB_TESTS_DEFAULT_PARALLEL_BACKEND environment variable.
Fix temporary folder creation in joblib.Parallel on Linux subsystems on Windows which do have /dev/shm but don't have the os.statvfs function
Drop runtime dependency on distutils. distutils is going away in Python 3.12 and is deprecated from Python 3.10 onwards. This import was kept around to avoid breaking scikit-learn, however it's now been long enough since scikit-learn deployed a fixed (verion 1.1 was released in May 2022) that it should be safe to remove this.
A warning is raised when a pickling error occurs during caching operations. In version 1.5, this warning will be turned into an error. For all other errors, a new warning has been introduced: joblib.memory.CacheWarning.
Avoid (module, name) collisions when caching nested functions. This fix changes the module name of nested functions, invalidating caches from previous versions of Joblib.
Add cache_validation_callback in :meth:`joblib.Memory.cache`, to allow custom cache invalidation based on the metadata of the function call.
Add a return_as parameter for Parallel, that enables consuming results asynchronously.
Improve the behavior of joblib for n_jobs=1, with simplified tracebacks and more efficient running time.
Add the parallel_config context manager to allow for more fine-grained control over the backend configuration. It should be used in place of the parallel_backend context manager. In particular, it has the advantage of not requiring to set a specific backend in the context manager.
Add items_limit and age_limit in :meth:`joblib.Memory.reduce_size` to make it easy to limit the number of items and remove items that have not been accessed for a long time in the cache.
Deprecate bytes_limit in Memory as this is not automatically enforced, the limit can be directly passed to :meth:`joblib.Memory.reduce_size` which needs to be called to actually enforce the limit.
Vendor loky 3.4.0 which includes various fixes.
Various updates to the documentation and to benchmarking tools.
Move project metadata to pyproject.toml.
Add more tests to improve python nogil support.

Revision 1.15: download - view: text, markup, annotated - select for diffs
Mon Nov 21 09:35:31 2022 UTC (23 months, 2 weeks ago) by adam
Branches: MAIN
CVS tags: pkgsrc-2023Q3-base, pkgsrc-2023Q3, pkgsrc-2023Q2-base, pkgsrc-2023Q2, pkgsrc-2023Q1-base, pkgsrc-2023Q1, pkgsrc-2022Q4-base, pkgsrc-2022Q4
Diff to: previous 1.14: preferred, colored
Changes since revision 1.14: +2 -3 lines
py-joblib: updated to 1.2.0

Release 1.2.0
Fix a security issue where eval(pre_dispatch) could potentially run arbitrary code. Now only basic numerics are supported. https://github.com/joblib/joblib/pull/1327
Make sure that joblib works even when multiprocessing is not available, for instance with Pyodide https://github.com/joblib/joblib/pull/1256
Avoid unnecessary warnings when workers and main process delete the temporary memmap folder contents concurrently. https://github.com/joblib/joblib/pull/1263
Fix memory alignment bug for pickles containing numpy arrays. This is especially important when loading the pickle with mmap_mode != None as the resulting numpy.memmap object would not be able to correct the misalignment without performing a memory copy. This bug would cause invalid computation and segmentation faults with native code that would directly access the underlying data buffer of a numpy array, for instance C/C++/Cython code compiled with older GCC versions or some old OpenBLAS written in platform specific assembly. https://github.com/joblib/joblib/pull/1254
Vendor cloudpickle 2.2.0 which adds support for PyPy 3.8+.
Vendor loky 3.3.0 which fixes several bugs including:
robustly forcibly terminating worker processes in case of a crash (https://github.com/joblib/joblib/pull/1269);
avoiding leaking worker processes in case of nested loky parallel calls;
reliability spawn the correct number of reusable workers.

Revision 1.14: download - view: text, markup, annotated - select for diffs
Tue Jan 4 20:53:13 2022 UTC (2 years, 10 months ago) by wiz
Branches: MAIN
CVS tags: pkgsrc-2022Q3-base, pkgsrc-2022Q3, pkgsrc-2022Q2-base, pkgsrc-2022Q2, pkgsrc-2022Q1-base, pkgsrc-2022Q1
Diff to: previous 1.13: preferred, colored
Changes since revision 1.13: +2 -1 lines
*: bump PKGREVISION for egg.mk users

They now have a tool dependency on py-setuptools instead of a DEPENDS

Revision 1.13: download - view: text, markup, annotated - select for diffs
Thu Oct 7 19:38:31 2021 UTC (3 years, 1 month ago) by adam
Branches: MAIN
CVS tags: pkgsrc-2021Q4-base, pkgsrc-2021Q4
Diff to: previous 1.12: preferred, colored
Changes since revision 1.12: +2 -2 lines
py-joblib: updated to 1.1.0

Release 1.1.0
Fix byte order inconsistency issue during deserialization using joblib.load in cross-endian environment: the numpy arrays are now always loaded to use the system byte order, independently of the byte order of the system that serialized the pickle. https://github.com/joblib/joblib/pull/1181
Fix joblib.Memory bug with the ignore parameter when the cached function is a decorated function. https://github.com/joblib/joblib/pull/1165
Fix joblib.Memory to properly handle caching for functions defined interactively in a IPython session or in Jupyter notebook cell. https://github.com/joblib/joblib/pull/1214
Update vendored loky (from version 2.9 to 3.0) and cloudpickle (from version 1.6 to 2.0) https://github.com/joblib/joblib/pull/1218

Revision 1.12: download - view: text, markup, annotated - select for diffs
Wed Feb 10 08:24:36 2021 UTC (3 years, 8 months ago) by adam
Branches: MAIN
CVS tags: pkgsrc-2021Q3-base, pkgsrc-2021Q3, pkgsrc-2021Q2-base, pkgsrc-2021Q2, pkgsrc-2021Q1-base, pkgsrc-2021Q1
Diff to: previous 1.11: preferred, colored
Changes since revision 1.11: +2 -2 lines
py-joblib: updated to 1.0.1

1.0.1
dask: avoid redundant scattering of large arguments to make a more efficient
use of the network resources and avoid crashing dask with “OSError: [Errno 55]
No buffer space available” or “ConnectionResetError: [Errno 104] connection
reset by peer”.

Revision 1.11: download - view: text, markup, annotated - select for diffs
Sat Jan 16 21:16:35 2021 UTC (3 years, 9 months ago) by adam
Branches: MAIN
Diff to: previous 1.10: preferred, colored
Changes since revision 1.10: +2 -2 lines
py-joblib: updated to 1.0.0

1.0.0
Make joblib.hash and joblib.Memory caching system compatible with numpy >= 1.20.0. Also make it explicit in the documentation that users should now expect to have their joblib.Memory cache invalidated when either joblib or a third party library involved in the cached values definition is upgraded. In particular, users updating joblib to a release that includes this fix will see their previous cache invalidated if they contained reference to numpy objects.
Remove deprecated check_pickle argument in delayed.

Revision 1.10: download - view: text, markup, annotated - select for diffs
Sat Oct 3 06:03:18 2020 UTC (4 years, 1 month ago) by adam
Branches: MAIN
CVS tags: pkgsrc-2020Q4-base, pkgsrc-2020Q4
Diff to: previous 1.9: preferred, colored
Changes since revision 1.9: +2 -2 lines
py-joblib: updated to 0.17.0

Release 0.17.0

- Fix a spurious invalidation of `Memory.cache`'d functions called with
  `Parallel` under Jupyter or IPython.

- Bump vendored loky to 2.9.0 and cloudpickle to 1.6.0. In particular
  this fixes a problem to add compat for Python 3.9.

Revision 1.9: download - view: text, markup, annotated - select for diffs
Mon Aug 10 14:41:36 2020 UTC (4 years, 2 months ago) by adam
Branches: MAIN
CVS tags: pkgsrc-2020Q3-base, pkgsrc-2020Q3
Diff to: previous 1.8: preferred, colored
Changes since revision 1.8: +2 -2 lines
py-joblib: updated to 0.16.0

Release 0.16.0
--------------

- Fix a problem in the constructors of of Parallel backends classes that
  inherit from the `AutoBatchingMixin` that prevented the dask backend to
  properly batch short tasks.

- Fix a problem in the way the joblib dask backend batches calls that would
  badly interact with the dask callable pickling cache and lead to wrong
  results or errors.

- Prevent a dask.distributed bug from surfacing in joblib's dask backend
  during nested Parallel calls (due to joblib's auto-scattering feature)

- Workaround for a race condition after Parallel calls with the dask backend
  that would cause low level warnings from asyncio coroutines:

Revision 1.8: download - view: text, markup, annotated - select for diffs
Thu May 28 12:06:28 2020 UTC (4 years, 5 months ago) by adam
Branches: MAIN
CVS tags: pkgsrc-2020Q2-base, pkgsrc-2020Q2
Diff to: previous 1.7: preferred, colored
Changes since revision 1.7: +2 -2 lines
py-joblib: updated 0.15.1

Release 0.15.1
- Make joblib work on Python 3 installation that do not ship with the lzma
  package in their standard library.

Revision 1.7: download - view: text, markup, annotated - select for diffs
Sat May 16 17:20:21 2020 UTC (4 years, 5 months ago) by adam
Branches: MAIN
Diff to: previous 1.6: preferred, colored
Changes since revision 1.6: +6 -5 lines
py-joblib: updated to 0.15.0

Release 0.15.0

- Drop support for Python 2 and Python 3.5. All objects in
  ``joblib.my_exceptions`` and ``joblib.format_stack`` are now deprecated and
  will be removed in joblib 0.16. Note that no deprecation warning will be
  raised for these objects Python < 3.7.

- Fix many bugs related to the temporary files and folder generated when
  automatically memory mapping large numpy arrays for efficient inter-process
  communication. In particular, this would cause `PermissionError` exceptions
  to be raised under Windows and large leaked files in `/dev/shm` under Linux
  in case of crash.

- Make the dask backend collect results as soon as they complete
  leading to a performance improvement:

- Fix the number of jobs reported by ``effective_n_jobs`` when ``n_jobs=None``
  called in a parallel backend context.

- Upgraded vendored cloupickle to 1.4.1 and loky to 2.8.0. This allows for
  Parallel calls of dynamically defined functions with type annotations
  in particular.

Revision 1.6: download - view: text, markup, annotated - select for diffs
Sat Dec 14 10:54:16 2019 UTC (4 years, 10 months ago) by adam
Branches: MAIN
CVS tags: pkgsrc-2020Q1-base, pkgsrc-2020Q1, pkgsrc-2019Q4-base, pkgsrc-2019Q4
Diff to: previous 1.5: preferred, colored
Changes since revision 1.5: +2 -2 lines
py-joblib: updated to 0.14.1

Release 0.14.1

Configure the loky workers' environment to mitigate oversubsription with nested multi-threaded code in the following case:

allow for a suitable number of threads for numba (NUMBA_NUM_THREADS);
enable Interprocess Communication for scheduler coordination when the nested code uses Threading Building Blocks (TBB) (ENABLE_IPC=1)
https://github.com/joblib/joblib/pull/951

Fix a regression where the loky backend was not reusing previously spawned workers. https://github.com/joblib/joblib/pull/968

Revert https://github.com/joblib/joblib/pull/847 to avoid using pkg_resources that introduced a performance regression under Windows: https://github.com/joblib/joblib/issues/965

Revision 1.5: download - view: text, markup, annotated - select for diffs
Thu Oct 31 10:24:52 2019 UTC (5 years ago) by adam
Branches: MAIN
Diff to: previous 1.4: preferred, colored
Changes since revision 1.4: +3 -3 lines
py-joblib: updated to 0.14.0

Release 0.14.0
Improved the load balancing between workers to avoid stranglers caused by an excessively large batch size when the task duration is varying significantly (because of the combined use of joblib.Parallel and joblib.Memory with a partially warmed cache for instance). https://github.com/joblib/joblib/pull/899
Add official support for Python 3.8: fixed protocol number in Hasher and updated tests.
Fix a deadlock when using the dask backend (when scattering large numpy arrays). https://github.com/joblib/joblib/pull/914
Warn users that they should never use joblib.load with files from untrusted sources. Fix security related API change introduced in numpy 1.6.3 that would prevent using joblib with recent numpy versions. https://github.com/joblib/joblib/pull/879
Upgrade to cloudpickle 1.1.1 that add supports for the upcoming Python 3.8 release among other things. https://github.com/joblib/joblib/pull/878
Fix semaphore availability checker to avoid spawning resource trackers on module import. https://github.com/joblib/joblib/pull/893
Fix the oversubscription protection to only protect against nested Parallel calls. This allows joblib to be run in background threads. https://github.com/joblib/joblib/pull/934
Fix ValueError (negative dimensions) when pickling large numpy arrays on Windows. https://github.com/joblib/joblib/pull/920
Upgrade to loky 2.6.0 that add supports for the setting environment variables in child before loading any module. https://github.com/joblib/joblib/pull/940
Fix the oversubscription protection for native libraries using threadpools (OpenBLAS, MKL, Blis and OpenMP runtimes). The maximal number of threads is can now be set in children using the inner_max_num_threads in parallel_backend. It defaults to cpu_count() // n_jobs.

Revision 1.4: download - view: text, markup, annotated - select for diffs
Thu Feb 14 09:01:45 2019 UTC (5 years, 8 months ago) by adam
Branches: MAIN
CVS tags: pkgsrc-2019Q3-base, pkgsrc-2019Q3, pkgsrc-2019Q2-base, pkgsrc-2019Q2, pkgsrc-2019Q1-base, pkgsrc-2019Q1
Diff to: previous 1.3: preferred, colored
Changes since revision 1.3: +3 -3 lines
py-joblib: updated to 0.13.2

0.13.2:
Add a non-regression test, reporting that cloudpickle versions between 0.5.4 and 0.7 introduced a bug where global variables changes in a parent process between two calls to joblib.Parallel would not be propagated into the workers

0.13.1:
Memory now accepts pathlib.Path objects as location parameter. Also, a warning is raised if the returned backend is None while location is not None.

Make Parallel raise an informative RuntimeError when the active parallel backend has zero worker.

Make the DaskDistributedBackend wait for workers before trying to schedule work. This is useful in particular when the workers are provisionned dynamically but provisionning is not immediate (for instance using Kubernetes, Yarn or an HPC job queue).

Revision 1.3: download - view: text, markup, annotated - select for diffs
Fri Nov 30 12:20:44 2018 UTC (5 years, 11 months ago) by adam
Branches: MAIN
CVS tags: pkgsrc-2018Q4-base, pkgsrc-2018Q4
Diff to: previous 1.2: preferred, colored
Changes since revision 1.2: +2 -2 lines
py-joblib: updated to 0.13.0

Release 0.13.0:
   Fix nested backend in SequentialBackend to avoid changing the default
   backend to Sequential.

    Fix nested_backend behavior to avoid setting the default number of
    workers to -1 when the backend is not dask.


Release 0.12.5

    Include loky 2.3.1 with better error reporting when a worker is
    abruptly terminated. Also fixes spurious debug output.

    Include cloudpickle 0.5.6. Fix a bug with the handling of global
    variables by locally defined functions.


Release 0.12.4

    Include loky 2.3.0 with many bugfixes, notably w.r.t. when setting
    non-default multiprocessing contexts. Also include improvement on
    memory management of long running worker processes and fixed issues
    when using the loky backend under PyPy.

    Raises a more explicit exception when a corrupted MemorizedResult is loaded.

    Loading a corrupted cached file with mmap mode enabled would
    recompute the results and return them without memmory mapping.


Release 0.12.3

    Fix joblib import setting the global start_method for multiprocessing.

    Fix MemorizedResult not picklable.

    Fix Memory, MemorizedFunc and MemorizedResult round-trip pickling +
    unpickling.

    Fixed a regression in Memory when positional arguments are called as
    kwargs several times with different values.

    Integration of loky 2.2.2 that fixes issues with the selection of the
    default start method and improve the reporting when calling functions
    with arguments that raise an exception when unpickling.

    Prevent MemorizedFunc.call_and_shelve from loading cached results to
    RAM when not necessary. Results in big performance improvements

Revision 1.2: download - view: text, markup, annotated - select for diffs
Tue Aug 14 17:39:32 2018 UTC (6 years, 2 months ago) by adam
Branches: MAIN
CVS tags: pkgsrc-2018Q3-base, pkgsrc-2018Q3
Diff to: previous 1.1: preferred, colored
Changes since revision 1.1: +3 -3 lines
py-joblib: updated to 0.12.2

Release 0.12.2:
Integrate loky 2.2.0 to fix regression with unpicklable arguments and functions reported by users.
Loky 2.2.0 also provides a protection against memory leaks long running applications when psutil is installed.
Joblib now includes the code for the dask backend which has been updated to properly handle nested parallelism and data scattering at the same time.
Restored some private API attribute and arguments (MemorizedResult.argument_hash and BatchedCalls.__init__'s pickle_cache) for backward compat.
Fix a deprecation warning message (for Memory's cachedir).

Release 0.12.1:
Make sure that any exception triggered when serializing jobs in the queue will be wrapped as a PicklingError as in past versions of joblib.
Fix kwonlydefaults key error in filter_args

Revision 1.1: download - view: text, markup, annotated - select for diffs
Fri Jul 6 03:13:36 2018 UTC (6 years, 4 months ago) by minskim
Branches: MAIN
devel/py-joblib: Import version 0.12.0

Joblib is a set of tools to provide lightweight pipelining in Python.
In particular, joblib offers transparent disk-caching of the output
values and lazy re-evaluation (memoize pattern), easy simple parallel
computing, and logging and tracing of the execution.  Joblib is
optimized to be fast and robust in particular on large data and has
specific optimizations for numpy arrays.

Packaged by Kamel Ibn Aziz Derouiche for pkgsrc-wip and updated by me.

Diff request

This form allows you to request diffs between any two revisions of a file. You may select a symbolic revision name using the selection box or you may type in a numeric name using the type-in text box.

Log view options

CVSweb <webmaster@jp.NetBSD.org>