CVS log for pkgsrc/pkgtools/pbulk/Makefile

Up to [cvs.NetBSD.org] / pkgsrc / pkgtools / pbulk

Request diff between arbitrary revisions

Keyword substitution: kv
Default branch: MAIN

pbulk: extend txt (mail) report

If there are any packages where the dependencies could not be resolved,
there is now additional section listing them.

The summary at the top is also extended to list the number of packages
broken this way.

Bump version.

pbulk, pbulk-base: bump version to 0.74/0.58

When a dependency can not be resolved, mark packages using it as
broken using PKG_FAIL_REASON, but don't fail the scan/resolve phase.

This obsoletes the ignore_missing_dependencies configuration variable,
add a warning about that.

Improve the man pages while here.

pbulk: Update to 0.73.

Ensure the digest command is correctly quoted.  Fixes config when digest
is "mktool digest".

pbulk: Update to 0.72.

Add more fields to the meta/status file to improve support for bulk build
reporting tools.

pbulk-0.71: Report culprit in html report for indirect-prefailed too.

pbulk: make rsync dependency a default-on option

pbulk-0.70: store end of build in epoche format too

This is nicer for using bulk builds as benchmark as it doesn't require
parsing dates.

Bump version for pbulk script fixes.

pbulk: add missing backslash

Fix pkg_admin expansion due to lost local change.

pbulk-0.67: Support both $NetBSD$ style RCS IDs and digests in +BUILD_INFO.
Bump configuration version for the new pkg_admin field.

pbulk-0.66: make it simpler to compute the duration of the scan phase

This is handy when benchmarking whole system performance.

*: Move SUBST_STAGE from post-patch to pre-configure

Performing substitutions during post-patch breaks tools such as mkpatches,
making it very difficult to regenerate correct patches after making changes,
and often leading to substituted string replacements being committed.

pbulk-0.65: Add basic support for specifying chroot'd clients by
absolute path. Introduce hook for setting up the chroot
(client_prepare_chroot) similar to client_prepare_action for ssh
clients.

pbulk-0.64:
Ensure to reset port after each loop, so that "client" and "client:port"
notation can be mixed.

pbulk-0.63:
Introduce a separate client-prepare script. The default configuration
runs the renamed client_prepare_action for each scan/build client once,
dropping duplicates in the list. Clients are prepared in parallel, but
the master waits for all before starting the actual scan. Make
bulkbuild-restart re-prepare the clients, i.e. in case they have a
pkgsrc tree in a local tmpfs.

pbulk-0.62: cleanup if either the initial pkg_add round or the package
upload failed.

pbulk-0.61: don't create category symlinks in the default package sync
implementation.

Drop support for USE_DESTDIR=no. Bump version.

pbulk-0.59: Belatedly bump PBULK_CONFIG_VERSION to ensure that
sync_packages etc are present.

pbulk-0.58: Create All if it doesn't exist yet. Noticed by asau@

pbulk-0.57: Reorder build phases slightly to make better use of destdir
builds. Introduce hooks for sending bulklog and package explicitly the
master. The default bulklog action is a nop and the package action
mirrors the normal package step, i.e. copying to ${packages}/All and
adding the category symlinks.

When copying the meta directory, preserve time stamps. Otherwise changes
won't be picked up correctly.

pbulk-0.55: Add option for running the checksum phase unprivileged. This
makes it easy to use DISTDIR=${WRKDIR}/.distfiles or so. Not the default
as it changes the required permissions on ${DISTDIR}.

Allow clients to be specified with port number, simplifying multi-chroot
setups.

Bump version for package list format change.

Just define PKGNAME and use it.

pbulk-0.52: ensure sort(1) is called with LC_ALL=C

Without this, a LANG of e.g. en_US.UTF-8 would sort "- *" before "+ <file>",
resulting in no packages being uploaded.

Update pbulk to 0.51 and pbulk-base to 0.46. Add support for reusing old
scan results to speed up the process. Fix a bug that overwrote the
package.log during installation.

"user-destdir" is default these days

pbulk-0.50: Portability fix, use 'id' instead of 'whoami'.

Pullup ticket #3792 - requested by joerg
pkgtools/pbulk: bug fix update

Revisions pulled up:
- pkgtools/pbulk/Makefile                                       1.63
- pkgtools/pbulk/files/pbulk/scripts/compute-packages.awk       1.4

---
   Module Name:	pkgsrc
   Committed By:	joerg
   Date:		Sun May 20 20:12:02 UTC 2012

   Modified Files:
   	pkgsrc/pkgtools/pbulk: Makefile
   	pkgsrc/pkgtools/pbulk/files/pbulk/scripts: compute-packages.awk

   Log Message:
   pbulk-0.49: Fix a bug in the computation of the restricted subset.

pbulk-0.49: Fix a bug in the computation of the restricted subset.

pbulk-0.48: If pkg_add fails, remove binary package

pbulk-0.47:
- Explicitly call make create-usergroup if USERGROUP_PHASE is specified
  and DESTDIR is active
- If the current user is not root, don't use su.
- Skip lines starting with # in the limited bulk build list.

pbulk-0.46: Use stage-install instead of install.
[Change in pbulk-0.45, but be explicit here]

pbulk-0.45:
Make the age check for packages an option, but keep it enabled by
default.

pbulk-0.44: Remove a second call to pscan from a failed merge.

pbulk-0.43:
Remove the content of bulklog, not the directory. Much nicer if nullfs
is used. Log error output of pscan.

pbulk-0.41:
- Consider packages silently outdated if recorded files are missing.
- Explicitly pass down tar to pbulk-save-wrkdir, TAR is unset at that
  point.

Pullup ticket #2881 - requested by joerg
pbulk-base: bug fix update
pbulk: bug fix update

Revisions pulled up:
- pkgtools/pbulk-base/Makefile			1.2
- pkgtools/pbulk/Makefile			1.55
- pkgtools/pbulk/files/pbulk/lib/event.c	1.6
- pkgtools/pbulk/files/pbulk/scripts/pkg-build	1.18
--
Module Name:	pkgsrc
Committed By:	joerg
Date:		Sun Aug 23 18:02:04 UTC 2009

Modified Files:
	pkgsrc/pkgtools/pbulk-base: Makefile
	pkgsrc/pkgtools/pbulk/files/pbulk/lib: event.c

Log Message:
pbulk-base-0.39:
Fix ordering of event handler vs queue modification. This can result in
queue corruption or use after free when the master can't handle a
request before the next arrives. From Matt Dillon.
---
Module Name:	pkgsrc
Committed By:	joerg
Date:		Fri Sep  4 22:06:18 UTC 2009

Modified Files:
	pkgsrc/pkgtools/pbulk: Makefile
	pkgsrc/pkgtools/pbulk/files/pbulk/scripts: pkg-build

Log Message:
pbulk-0.41:
Create & chown work.log for destdir builds too, as the normal build is
run unprivileged.

pbulk-0.41:
Create & chown work.log for destdir builds too, as the normal build is
run unprivileged.

pbulk-0.40:
Use plain sed to extract RCS IDs. GNU ident has issues with unexpanded
RCS IDs.

Split pbulk into pbulk-base (the backend programs) and pbulk (rest).

pbulk-0.38:
Skip lines starting with # in the limited list.

Remove redundant NO_CHECKSUM and EXTRACT_ONLY definitions.

pbulk-0.37:
- replace libevent dependency with internal event loop to make the pbulk
  core standalone
- open success and error log with O_APPEND, which allows e.g. cleaning
  up the error build during build without breaking it if done with care.

PKGREVISION bump for libevent shlib name change.

pbulk-0.36:
Avoid grep -q and use awk instead. This has the side effect of parsing
presolve only once for each file. grep -q doesn't exist on Solaris.

pbulk-0.35:
Sync Dewey with pkg_install(-renovation) and add a fast check if a
pattern could ever match. This reduces the time for pbulk-resolve on a
full tree on my laptop from 12.9s to 5.3s.

pbulk-0.34:
- version configuration file, object if the version doesn't match
- add an option for pbulk-resolve to ignore unresolvable dependencies
similiar to the incremental mode
- use this option to run pbulk-resolve again if ignore_missing_depencies
is set to yes. Report this explicitly to un-confuse readers and point
them to the logfile.

pbulk-0.34:
Isolate rules to compute the restricted subset in the upload script.
Make the upload script independent from the report file to allow
uploading partial builds.
Only depend on NO_BIN_ON_FTP to decide what to upload and what not.

Bump PKGREVISION for libevent users due to 1.4.3->1.4.5 shlib name change.

pbulk-0.33: Don't try to install/deinstall bootstrap packages.
This allows a destdir build to package bootstrap components.

pbulk-0.32:
Make bulkbuild-rebuild over 200 times faster when running against a full
tree scan by using awk to filter out the data instead of the while read
shell builtin.

Use full path for chown.

pbulk-0.31:
- For direct build failures, make the package location in the first
column a hyper link to the last (supposedly failing) phase.
- For indirect failures, list the first 10 failing dependencies.

Mark this package MAKE_JOBS_SAFE=no.

pbulk-0.30:
Print the collected PKG_FAIL_REASON in the HTML version of the report.

pbulk-0.29:
Add bulkbuild-restart script that checks if error/success files exist
and makes sense and continues the previous build if they do.

pbulk-0.28:
- don't try to deinstall (and fail) on packages marked not for uninstall

Needs GNU nroff to format catpages.
(But for this case it would make life simpler for bulk builders if the
 catpages were pregenerated.)

pbulk-0.27:
When running the start script in master mode, already accept incoming
connections. This makes the start script more useful when using more
clients than the default listen back log.

pbulk-0.26:
Add first pile of patches for cross-compiling support.
This still depends on the packages being pre-installed to
the normal prefix, but otherwise works. Cross-compiling
needs pkg_install-20080126. While here, simplify configuration
and allow using one pkg_install installation for everything.
As a side effect remove the pkgdb limitation for NetBSD builds.
Add a message that tells users to check the configuration file
on updates.

pbulk-0.25:
Add bulkbuild-rebuild which can be used to build packages individually.
It assumes that all dependencies are built and the results of the scan
phase exists. That means it can be used after a bulk build to try to
rebuild a failing package. Fixes PR 37407.

pbulk-0.24:
When all jobs are processed in master mode, close the listen socket
and shutdown(2) all peers. Give them a second to close(2) the
connection themselve, so that the port remains usable on the master.
This is the standard compliant fix for PR 37002.

Fixed a problem in pkg-build, bump rev.

No revision bump for pkgsrc native tools, update version.

PKGREVISION++ for the invalid definition of shell functions.

pbulk-0.21:
Don't try to remove a non-existing file if no package had an error
during the build.

pbulk-0.20:
- run checksum stage of a build with the privileged user, as DISTDIR
  might not be writeable
- create the actual package as privileged user as well
- pre-create work.log when using user-destdir
- in pre-build, only wait if the meta subdirectory exists. bulklog itself
  can often be a mount point.

pbulk-0.19:
- add support for building using DESTDIR.

pbulk 0.18:
- If not running on NetBSD or not using /usr/bin/make, set the default
  of pbulk's make to ${prefix}/bin/bmake.
- Make path to neato an option and don't install it on demand.
- Fix a bug in the master mode that should fix the strange faults seen
  with native libevent on NetBSD.
- Add option to skip checksum computation. Don't upload a checksum file
  if that option is active.

Force dependency on libevent from pkgsrc on NetBSD, the native version
is broken. This shows e.g. as querying the status can trigger a signal
10. Bump version.

Skip empty lines in the limited list. Bump to 1.16.

Bump version or revision of all packages that have a runtime dependency
on gzip.

Make language lawyers happy and rename report_recipient to plural.

Convert to use the features framework.

close is a function, fix C&P errors. From Aleksey Cheusov.
Bump version, remove pkgrevision.

Work around broken default visibility in Linux (PR 36790).
Linux has sin_len either (PR 36791).

Revbump sweep of all libevent consumers due to update to libevent-1.3d.

Don't hard-code tar as tar, but make it an option and use the tool
version correctly. Fixes issues on platforms where the first tar
in path doesn't support -z. Bump to pbulk-0.12.

Don't compare a function name against NULL, master_mode is meant here.
Thanks tnn@.

Reverse the test for extracting the bootstrap kit as the old version
hits set -e error handling.

Bump to pbulk-1.11.

Do SUBST handling in post-patch instead of pre-configure. This makes
substitution (and any substitution warnings) run before the build starts,
rather than after building libnbcompat. The reason for this is that
libnbcompat is _built_ in the pre-configure stage when in inplace mode.

Some people actually watch the pbulk-build client output, so add
a newline after each message. Bump to pbulk-0.10.

pkgsrc_base_dir is now without leading /, compensate. GC it in the HTML
report script. Bump to pbulk-0.9.

Remove an additional ) and correct the assignment of a variable.
Bump to 0.8.

Always call setprogname for platforms that don't have __progname.
Don't depend on date to implement %s or even %F and %R or support -r,
compute dates as one comment and split. This works even with the very
special HP-UX date. Requested and tested by tnn@

Bump to pbulk-0.7.

Add two variables to specify the prefix of the report mail's subject
and the recipient. Bump to pbulk-0.6.

bulkdir was an old configuration variable and replaced with bulklog
in the example config. Catch up in the scripts as well.
Create the log directory first for limited bulk builds.

Bump to pbulk-0.5.

#!/usr/bin/awk -f -
works for some reasons on NetBSD, but not with other versions
of one-true-awk or gawk. Bump to 0.4.

Scripts get installed without suffix, so fix default configuration
to match the installed names. Bump version to 0.3.

Use libnbcompat and fix build issues on Solaris.
Don't depend on sin_len. Slightly factor out code for setting
non-blocking IO.

Bump to pbulk-0.2. Tested by dmcmahill@.

Force devel/libevent on NetBSD 2, as it is far too old. Found by Dan on
NetBSD/alpha.

This package uses BSD-style Makefiles.

Initial import of pbulk, the new pkgsrc bulk build framework.

Discussion of various parts of the design with jlam@, wiz@, tls@
and many other developers. Special thanks to David Maxwell for
testing the initial prototype and finding some bugs with
Coverity Prevent.

Supported by Google's Summer of Code 2007 project.

OK for import during the freeze: jlam@

From DESCR:
pbulk is the modular bulk build framework for pkgsrc.

This package contains:

- pbulk-scan, a program to scan the entire pkgsrc tree
  or a list of directories therein for packages and
  dependencies. Distributed operation using a master/client
  mode is supported.

- pbulk-resolve, a program to resolve the dependencies
  from the output of pbulk-scan

- pbulk-build, the build scheduler. Distributed builds via TCP
  are supported.

- bulkbuild and related scripts to implement full and limited
  bulk builds on top of those programs.

The pbulk framework is considered experimental.

Initial revision