Up to [cvs.NetBSD.org] / pkgsrc / pkgtools / pkg_install / files / lib
Request diff between arbitrary revisions
Keyword substitution: kv
Default branch: MAIN
pkg_install: Hide stdout if upgrading/refreshing. Helps avoid problems where users see pkg_delete messages about removing configuration files even though they are still in use and required, as well as annoyingly repeating MESSAGE files on every upgrade or refresh. Packages that do not have an existing version (i.e. new installs) continue to show install script output as well as MESSAGE files, and explicit pkg_delete operations will obviously show deinstall script output too. From Jonathan Perkin <jperkin@smartos.org> via drecklypkg commit $64df139d33d2af103f51a832c2f253ff068dd743
pkg_install: Introduce support for CHECK_OS_VERSION. When set to "no", pkg_add will not issue a warning if the host OS version does not exactly match the OS version the package was built on. This can be useful on many OS where the kernel release version has nothing to do with userland compatibility, or where it may differ without being ABI incompatible. Ultimately it would be ideal if the version checks were smart enough across all our supported OS to not need this, but until then this is useful for users who know what they're doing. Bump version to 20201211.
pkg_install: mark show_version as noreturn since it exits
pkg_install: Fix and speed up "pkg_admin rebuild-tree". In the pkg_admin front end, instead of adding +REQUIRED_BY entries as they are found, which previously led to duplicate entries, cache the results and write out the files at the end. Underneath, add a caching version of iterate_pkg_db() that avoids the same pkgdb directory lookup for every installed package, but is only suitable for reads. Also add a cache for best_match lookups to avoid expensive matches each time. For all caches, use a simple hashing function to improve lookup performance. In summary, as well as fixing +REQUIRED_BY files, these patches reduce the wall/user/system time of "pkg_admin rebuild-tree" on a system with 12,762 packages installed down from 13m52s/11m20s/2m32s to just 1m4s/1m3s/0m1s.
pkg_install-20180226: - add option to include IGNORE_URL pkg_install.conf entries in audit but flagged as "ignored". - bump version to 20180226
pkg_install-20170419: - Convert to libarchive 3.x interfaces, avoiding the legacy glue. - Rename local copy of netpgpverify to match installed headers. - Shuffle bootstrap code to reduce special cases in pkg_add. - Always read/write file names in tar archives as binary. While they are supposed to be UTF-8, locale conversion creates more problems than it solves here. - Fix const correctness in gpg_verify. - Add format string annotation for xasprintf for GCC-like compilers. - Restrict supported archive formats for binary packages to ar (signed packages) and tar (actual content) with uncompressed/gzip/bzip2/xz as compression choices. This reduces the exposed libarchive surface. - The pkg-vulnerability file correspondingly supports only the same compression choices.
Do not check for or include sys/param.h This resolves issues on platforms which MACHINE & MACHINE_ARCH is explicitly defined, such as OpenBSD/amd64 & Bitrig/amd64 where we would like to build with MACHINE_ARCH=x86_64 but can't as a platform check fails when installing packages after bootstrap pkg_add: Warning: package `digest-20121220' was built for a platform: pkg_add: OpenBSD/x86_64 5.8 (pkg) vs. OpenBSD/amd64 5.8 (this host) Tested on Mac OS X Tiger/PowerPC OpenBSD Bitrig Solaris 10/SPARC FreeBSD 10.2-RELEASE & 11-CURRENT DragonFlyBSD Debian 8 OmniOS Reviewed by joerg@
Implement inline package signature verification. This replaces calling out to an external gpg command for verification with inline verification using the security/netpgpverify library. Bump version to 20150901.
Remove pkg_views support, second part: infrastructure.
pkg_install-20100616: - Recognize -C for pkg_add like the rest of the tools do - Do the existing check for pkg_add -f, it makes the combination of -f and -U a bit less surprising - Fix option handling for CHECK_VULNERABILITIES (from spz@) - Make end-of-life check the default. pkg_install.conf(5) can be used to override the default. The existing admin -e & co continue to work as before.
pkg_install-20100405: - Try to detect common forms of pkgdb corruption and issue a warning in that case. - Refactor the pkg_vulnerabilities logic to use the compression support from libarchive. This reduces the amount zlib/bzip2 interaction to the linkage.
pkg_install-20100220: Try harder to give the user an idea which file failed. Addresses PR 42223.
pkg_install-20100204: Fix an enum to prevent crashes later (PR 42734)
pkg_install-20100203: - Restore PKG_PREFIX in pkg_delete (PR 42731) - Ensure that the current pkg_install version is at least as new as the version used to build the package
pkg_install-20100122:
- Refactor man pages:
- Move description of environmental variables into pkg_install.conf(5)
- Remove a bunch of dead references
- Avoid using hard-coded path names that can easily be overridden by
the user.
- Refactor pkgdb related logic, so that -K behaves consistently.
- Add some helper functions to isolate more places from the pkgdb
layout.
- Sync license list.
- Handle PKG_REFCOUNT_DBDIR consistently as option too.
- Utilize connection cache from libfetch.
- Add helper target for cat pages updates.
pkg_install-20090817: Add a new command for pkg_admin: findbest. It takes one or more patterns and searches for the best match in PKG_PATH, just like pkg_add would. It prints the URLs of the best match for each pattern to stdout. Rewrite the config file parser to read the file only once.
pkg_install-20090806: - pkg_add: add support to check license conditions before installation - pkg_delete: add -k option to skip over preserve packages.
pkg_install-20090727: Make pkg_install WARNS=4 clean and fix a number of lint warnings. Based on reports from veego about warnings with older GCC releases in pkg_delete, where a variable is potentially used uninitialized.
pkg_install-20090425: Add logic to evaluate license conditions. As frontend for pkgsrc, pkg_admin gets two new commands (check-license and check-single-license), which can be used to evaluate a given condition. pkg_add will be changed to honour licenses at a later point.
pkg_install-20090424: Make pkg_delete -d the default behavior, remove the option. Remove pkg_create -R support (reorder @dirrm entries). Introduce new plist command @pkgdir, which makes pkg_add add this directory if it doesn't exist already and pkg_delete remove it only if no other package has a @pkgdir entry for it. Otherwise directories are pruned when the last file or directory in them is removed. @dirrm is now a pure hint, if the directory doesn't exist, it will be silently skipped.
Constify.
Remove remaining parts of @mtree support.
Remove support and documentation for @ignore_inst.
pkg_install-20090307: Simplify archive handling by depending on archive_read_finish and the close callback where needed. Fixes a file descriptor leak as side effect as reported by wiz.
pkg_install-20090228: Integrate the PKG_PATH logic for tightly with the find_archive logic: - remember initial current working directory from the time PKG_PATH is processed, it will be used as reference for all relative entries - remove now redundant fchdir dance in pkg_add - pass down to find_archive if this is a top-level package (e.g. requested on the command line) or not; the location of top-level packages is searched for packages first and for URLs or path names the PKG_PATH itself is not processed (e.g. pkg_add foo/bar not look into PKG_PATH for bar). This addresses PR 33884 in a different way. - document the rules for finding packages more explicitly - allow specifying PKG_PATH in pkg_install.conf as well; environment takes precendence. - remove PKG_PATH related logic in pkg_info(1), it doesn't deal with PKG_PATH anyway - remove PKG_PATH, PKG_TMPDIR and TMPDIR description in pkg_info(1), they are no longer used
Close the archive on fatal errors during signature validation. This can happen e.g. if pkg_add finds a directory with that name. Remove a bunch of HAVE_SSL conditionals that no longer apply as GPG support is back in.
pkg_install-20090213: Fix pkg_delete -d: - ignore @dirrm commands, directories are removed already on demand - use local copy of dir before changing it, the caller expects it unmodified for the call to pkgdb_remove
Merge pkg_install-20090201 from pkg_install-renovation branch.
- DB support is always included from libnbcompat if needed
- pkg_view and linkfarm are not installed any more; they are not moved
into the attic yet, so they can easily be installed as separte package
- common configuration file to customise the behavior of various
components; this supersedes the old audit-packages.conf
- support for PKSC7 signatures (using X509 certs) and GPG signatures for
packages in a secure way. See pkg_admin(8) for how to create them and
pkg_install.conf(5) for the options to use them
- audit-packages and download-vulnerability-list are wrapper scripts
around pkg_admin. They try to mimic the classic options if used sanely.
"pkg_admin audit" is now an order of magnitude faster than before
- pkg_add uses libarchive and libfetch instead of external ftp and tar:
- progress bar is currently missing for downloads
- "pkg_add -" is no longer supported
- no adhoc check for conficts between dependencies and already
installed packages
- "pkg_add -s" has been replaced with an option in pkg_install.conf,
verification of plain detached GPG signatures is no longer supported
- optional check for vulnerabilities before adding a package
- if /var and /usr/pkg are on different fileystems it is twice as fast
now
- conflicts due to overlapping plists are checked before installation
- pkg_add no longer plays with the process limits
- pkg_add and pkg_delete have a new destdir option; scripts have to
either be modified to use PKG_DESTDIR or should be disabled
- pkg_add -u for now can't be used to update to the exact same version
- internal "rm -rf" and "mkdir_p" code
- all memory allocation failures are not explicitly fatal
- if a file is not removed due to a failed checksum, still remove the
entry from pkgdb
pkg_install-20090201: - sync DESCR and MESSAGE with HEAD and the branch state - sync Makefile - regen configure - cleanup __RCSID - merge audit logic in pkg_add and pkg_admin into new audit_packages function. As side effect, pkg_add will list all vulnerabilities before asking the user whether it should continue - sort config_variable array
pkg_install-20090127: Add options to specify user-id and keyrings for GPG operations.
pkg_install-20090108: pkg_add optionally checks for vulnerable packages and bails out.
pkg_install-20081230: Add initial support for secure GPG signatures.
pkg_install-20080916: Merge from changes for read_plist from pkg_install-renovation to always initialize the plist and add append_plist for the one case where this is not desired. Fixes PR 39276.
pkg_install-20080821: - Recognize file:// urls to prevent double quoting. - Fix some err/errx botchups. - Add configuration options for verbose logging of net IO, enabling of active FTP (switching to passive FTP as default) and for ignoring proxy settings. - When iterating over PACKAGES to find a match, warn if an entry can't processed (e.g. bad URL).
pkg_create actually does want to mess with the plist before reading it, so introduce append_plist to give the old behavior and unbreak pkg_create.
pkg_install-20080806: Provide and use internal implementation of rm -rf.
Use a predictable name for the +VIEWS temporary file.
Add another constant for temporary +REQUIRED_BY files. Don't use mkstemp in pkg_delete for that purpose. If more than one produce tries to change this file, breakage will happen anyway and this means the system is somewhat self-healing.
Most memory allocation failures were fatal already and the majority of the rest lacked an explicit check. Add the usual x* wrappers around malloc and friends that explicitly terminate on error and use them in all but Dewey.
pkg_install-20080730: Add destdir installation support for pkg_add and pkg_delete. This does not chroot install/deinstall scripts, they have to support it directly (using PKG_DESTDIR) or they have to be disabled.
Internal implementation of mkdir_p.
pkg_install-20080705: Based on a discussion with tls@, make the key validation stricter. Don't allow CA keys for packages or pkg-vulnerabilities. Require CODE SIGN as key usage attribute for packages. Require that no key usage attribute is set for pkg-vulnerabilities. The latter is not finalised and might change again.
Restore older (and somewhat bogus) behaviour of modifying the +CONTENTS file when -p is given to pkg_add.
Add variable to control how packages are installed. VERIFIED_INSTALLATION supports: never (default): print signature, but don't care about it. always: check for valid signature, abort otherwise trusted: ask for packages without valid signature interactive: always ask before installation
Add quick_pkg_match to decide whether a pattern can match a package at all. Use it to speed up pkg_match and the audit-history backend. This improves ``pkg_admin audit'' from 1.333s to 0.098s and ``pkg_admin audit php libpng'' from 0.034s to 0.029s on my laptop.
Add support PKCS7 signatures in packages and pkg-vulnerabilities. The support for non-native OpenSSL still needs to be done, e.g. ensure static linkage for that. Documentation is essentially missing.
Move configuration parser into libinstall. Add a new VERIFY_CMD to be used by pkg_add soon.
Add a clean pkg_add implementation on top of libarchive and libfetch. Known regressions: - "pkg_add -" (aka reading from stdin) is currently not supported - "pkg_add -s" is not supported either - no progress reports for the downloads - binary packages with hardlinks created by pkg_create before pkg_install-20080422 will not extract correctly (libarchive issue) - no adhoc check for potential conflicts between dependencies and already installed packages Features: - Twice as fast for the typical case of /var/tmp and /usr/pkg on different filesystems - Standalone - implicit conflict detection before actual installation.
Revert last change, it was not intended to go into HEAD.
Add a clean pkg_add implementation on top of libarchive and libfetch. Known regressions: - "pkg_add -" (aka reading from stdin) is currently not supported - "pkg_add -s" is not supported either - no progress reports for the downloads - binary packages with hardlinks created by pkg_create before pkg_install-20080422 will not extract correctly (libarchive issue) - no adhoc check for potential conflicts between dependencies and already installed packages Features: - Twice as fast for the typical case of /var/tmp and /usr/pkg on different filesystems - Standalone - implicit conflict detection before actual installation.
Move get_dash_string to pkg_create as only user. Remove str_lowercase.
pkg_install-20080404: Switch pkg_info to use libfetch for remote access.
pkg_install-20080219: Add new parser for pkg_vulnerabilities, that handles decompress transparently and internally. Preliminary tests with a modified audit-packages show a speed increase by 2 for common operations like listening all vulnerable installed packages.
pkg_install-20080202: Convert pkg_info to use libarchive instead of tar for binary package handling. As a side effect pkg_info on remote packages will use one FTP instance per argument, but will try to fetch only the meta data. This reduces time for pkg_info -X on a full binary repository from 36.9s to 18.1s for the cache hot case (PKG_TMPDIR on tmpfs for the old pkg_info). Thanks to tnn@ for testing.
Do not install a package if some of the installed packages would conflict with it. This fixes PR 37451.
* When adding or checking a package using pkg_admin(1), give a more
meaningful warning if the file is a symlink whose target doesn't exist.
The message has now been changed from:
pkg_admin: netbsd32_compat30-extras: File `/usr/pkg/emul/netbsd32/usr/lib/libm387.so' is in +CONTENTS but not on filesystem!
to:
pkg_admin: Symlink `/usr/pkg/emul/netbsd32/usr/lib/libm387.so' exists and is in +CONTENTS but target does not exist!
* Pass values for --sysconfdir (and --prefix) to the configure script
instead of passing them through CPPFLAGS. Both SYSCONFDIR and PREFIX
are only used by audit-packages, and the audit-packages Makefile
already handles passing these values inherited from the configure
script. This avoids compiler warnings that, e.g. SYSCONFDIR has been
redefined on the command line.
* Fix quoting for arguments to the configure script -- :Q instead of
\"\".
Bump version to 20070828. Reviewed by <joerg>.
Start abstracting pkgdb access by introducing pkgdb_pkg_file. The function returns the path name of the given file for the package. The memory needs to be freed by the caller.
Make _pkgdb_getPKGDB_DIR return const char * and fix fallout.
Restore ignorance of entry types for lsbest and lsall for now. The check for already installed packages depends on being able to run them in PKGDB_DIR. This is clearly obnoxius, but leave it for later.
- Introduce find_best_matching_file and match_local_files. This replace the functionality offered by findmatchingname and findbestmatching name. They optionally strip the suffix from the filename before matching it, instead of modifying the pattern directly. Drop the old functions. - Fix a bug in pkg_order where the version strings where inverted - Make pkg_admin lsbest and lsall use the new functions. - Make ftpio use pkg_order directly. Bump version to 20070812.
Change interface of is_automatic_installed and mark_as_automatic_installed to take a package name and not a full path. Add assertions to test for this. Drop a few islinktodir checks. Change pkg_info to use iterate_pkg_db instead of scanning the directory by hand. As a side effect don't try to check for the pkgdb dir first, let pkgdb_dump and iterate_pkg_db handle that. Make pkgdb_dump return failure if it can't open the package db.
Replace note_whats_installed, add_to_list_fn and generally most users of findbestmatchingname and findmatching name with more descriptive and easier to use iterator functions. This functions are a first step to abstract away pkgdb layout from most parts of the code. It also helps to reduce side effects and point out potential bugs in this code. Fix a potential, but practically irrelevant buffer overflow. No longer allow symbolic links directly in pkgdb to store the meta data of individual packages outside. E.g. /var/db/pkg/atk-1.18.0 must be a directory and not point to it. This is not yet enforced in all parts of the code, more changes will follow. Bump version to 20070808.
- only build pkg_admin, pkg_create and pkg_info for --enable-bootstrap - for pkg_create, split-off the package creation from the plist processing. package creation is disabled for --enable-bootstrap - change package creation to use libarchive - add two options (-u and -g) to override file system ownership - pkg_create now creates files in the tarball with correct owner/group based on -u/-g or @owner/@group - add a function to compute the +CONTENTS file in memory Bump version to 20070802. Review of a slightly older version by jlam@
Push pkg_perform prototype into the individual packages.
After the obsoletion on March, remove the support for require scripts. Also remove the documentation of the mtree parts as it is not supported.
Mostly deorbit mtree support: - pkg_create bails out with -m - pkg_add ignores mtree specs in packages and prints a warning. It does not remove the file like before. - remove now obsolete documentation. pkg_info still prints the information and pkg_admin ignores the files. Make the dependency on the tools a runtime dependency. Take maintainership of pkg_install and bump version to 20070725.
Split pattern related functions from lib/str.c off into lib/opattern.c. Rename pmatch to pkg_match and add new function pkg_order that implements the order logic used by pkg_add internally. Change the interface of findmatchingname's callback to take the pattern used for the matching as first argument. The new pkg_order function takes it and it doesn't hurt in other cases. Adjust callers and corresponding callbacks accordingly. Remove an unused matchfn typedef in dewey.c. Bump to 20070720. OK jlam@
Merge remaining small changes from NetBSD, most of all the pkgsrc.7 man page. Fix build on !NetBSD. Rebuild man pages with nroff -mandoc on NetBSD current for consistency. Make it DESTDIR safe. Remove the note that src/usr.sbin/pkg_install is the authoritive version, it won't be any longer.
Sync with src, version 20060424: Add support for creating summary file containing all information about a package repository needed by package managing software. This allows better use of remote binary package repositories. Discussed on tech-pkg.
Welcome to version 20060405. pkg_add also check for USE_ABI_DEPENDS. Check if USE_ABI_DEPENDS or IGNORE_RECOMMENDED was set when this package was built. IGNORE_RECOMMENDED is now historical. For the cat man page, I manually made it say "pkgsrc" instead of "NetBSD". I committed to NetBSD's src/usr.sbin/pkg_install/ first :)
Sync with src/usr.sbin: whitespace diffs only.
Specify the full path to tar when unpacking a package. Modify pfcexec so the new process file name can be different from argv[0]. This fix is from Peter Postma, and it addresses PR#32228
Correct PAX_CMD to be pax.
Remove unused variables, fix variable names to avoid shadowing globals, and add PAX_CMD fallback macro.
Reduce the use of the system() function in pkg_install to avoid quoting problems. Instead, use exec*() functions. Replace PUSHOUT() macro and string buffers with a function that operates on data structures. If it is necessary to copy files into place from staging area, then use pax to copy them. Add functions in pexec.c to create a pipe for sending data to a child process. Replace pipe code in create/perform.c with these functions. Use these functions instead of command-line arguments when copying files into place from staging area. Three system() references remain: @exec, @unexec, and vsystem().
Sync with base system as of Sat Nov 5 13:19:32 UTC 2005. Welcome to 20051103! Now with "automatic" support.
Update pkgtools/pkg_install to 20050204 (latest release from the HEAD
branch of src). Changes from version 20050106:
Teach pkg_add(1) and pkg_delete(1) to pass PKG_METADATA_DIR and
PKG_REFCOUNT_DBDIR in the environment when running the +* scripts.
PKG_METADATA_DIR is the location of the +* files after the package is
registered. PKG_REFCOUNT_DBDIR is the location of the reference counts
database directory. If PKG_REFCOUNT_DBDIR is left unset, then it
defaults the the location of the package database directory with
".refcount" appended to the path, e.g. /var/db/pkg.refcount.
pkgviews users should explicitly set PKG_REFCOUNT_DBDIR in the shell
environment to ensure that all packages will use the same refcount
database.
These changes allow the +INSTALL and +DEINSTALL script to keep state
in +* files within ${PKG_METADATA_DIR}, and to store reference counts
in ${PKG_REFCOUNT_DBDIR} to handle usage of resources outside of
${LOCALBASE}.
Pull in portability changes from src, and bump version to 20041226: Introduce a new abstraction in the C code, called MaxPathSize. All previous occurrences of MAXPATHLEN and FILENAME_MAX have been changed to use MaxPathSize instead. If MAXPATHLEN is not defined, then assume a default value of 1024 (this is primarily for use on the Hurd). The reason for this is that some older platforms define FILENAME_MAX to be 14, although MAXPATHLEN is 1024. On BSD-derived systems, FILENAME_MAX And MAXPATHLEN are both 1024. Bump pkg_install version to 20041226. These modifications have been tested so far on a NetBSD-current bulk build.
Make binary updates work better when there are dependant packages to update also. Specifying -u twice will now update the named package and its dependencies. Also make sure Fake really means Fake, even if update is specified.
sync with src: no trailing comma at end of enum.
Add a -N option to pkg_delete to delete registration information for a package but not its files. Override PKG_DELETE when building this package so -N can be used to upgrade an existing install. Bump version to 20041101.
Update pkg_install to 20040411, pulling up the changes in the NetBSD source tree: Make pkg_add recognise binary packages built with ignored recommendations (IGNORE_RECOMMENDED set in the presence of RECOMMENDED pre-requisites). This change is fully backward-compatible: binary packages have set the corresponding BUILD_INFO since RECOMMENDED was implemented. Packages built before that time will install as before. Bump PKGTOOLS_VERSION to 20040411. This closes PR pkg/24712 by Thomas Klausner.
Merge 20040409: Parameterize the umask calls so that defs.*.mk can change it on a per-opsys basis (and so that mk.conf can override it to match what the pkgsrc builder is using).
sync with NetBSD-current's pkg_install from 20040207.
Update pkgtools/pkg_install to 20040114. Changes from version 20040106
include:
* use ${MACHINE_ARCH} instead of what uname() puts into utsname.machine.
the former represents the CPU architecture, which is what is recorded
in binary packages; also allow the OS name to be overridden, as pkgsrc
doesn't always use the value of utsname.sysname.
* allow adding depoted packages if another package with the same
${PKGBASE} is already installed as this is explicitly allowed for in
pkgviews.
Sync pkgtools/pkg_install with src/usr.sbin/pkg_install from 20031220. New features include OS/arch/version checking and a large speed improvement to pkg_info.
Sync pkgtools/pkg_install with src/usr.sbin/pkg_install from 20031003. New features include fetching binary packages using HTTP (thanks wiz!) and some minor speed optimizations for pkg_delete.
sync with NetBSD-current after updates from wiz.
sync with NetBSD-current's pkg_install-20030923.
Update to pkg_install-20030824. Changes from the previous version include:
* Check for errors in reading from the file descriptor for the ftp
process, and correct the error messages.
* Nuke some code for older NetBSDs that's better handled in
pkgtools/libnbcompat.
* Don't prevent user from using views other than the default view.
Sync to latest pkg_install sources from the HEAD for pkg_delete optimizations and man page fixes. Also use recent src2nbcompat to reverse the order that "config.h" and <nbcompat.h> are included in *.[ch] files.
Update pkg_install to 20030823, which has the changes to support package views. Pkgsrc changes include: * Convert to use pkgtools/libnbcompat so that it's easier to keep this in sync with src/usr.sbin/pkg_install. * Add a src2nbcompat script that converts src/usr.sbin/pkg_install into an appropriate start for pkgtools/pkg_install. The script removes the Makefiles, adds CPP inclusion guards around each #include <...> line, and includes <nbcompat.h> at the top of each *.c file. This should make it even easier to keep sources in sync with the HEAD.