Up to [cvs.NetBSD.org] / pkgsrc / mk / tools
Request diff between arbitrary revisions
Keyword substitution: kv
Default branch: MAIN
mk: New CROSS_* variable scheme for cross-compilation.
This change brings in support for cross-OS-compilation.
Caveat: Existing mk.conf files used for cross-compilation no longer
work; they must be adapted to the new scheme.
Under this new scheme, if an mk.conf variable X can be different for
native and cross builds like OPSYS, MACHINE_ARCH, LOCALBASE, &c., you
just define X in mk.conf to set it for native builds, and CROSS_X in
mk.conf to set it for cross builds.
Example:
LOCALBASE= /home/riastradh/pkgsrc/cross/pkg
CROSS_LOCALBASE= /opt/pkg
CROSS_MACHINE_ARCH= powerpc
For now, _all_ OS-related variables must be defined in cross-builds;
none are filled in automatically. For NetBSD, this means:
CROSS_OPSYS= NetBSD
CROSS_OS_VERSION= 10.0
CROSS_OPSYS_VERSION= 100000
CROSS_LOWER_OPSYS= netbsd
CROSS_LOWER_OPSYS_VERSUFFIX=# empty
CROSS_LOWER_OS_VARIANT= # empty
CROSS_LOWER_VARIANT_VERSION=# empty
CROSS_LOWER_VENDOR= # empty
This new scheme may raise make(1) costs by making a lot of variables
like OPSYS expand to conditionals rather than fixed strings. It is
difficult to make these fixed strings because:
- they can only be committed _after_ mk.conf is loaded, before which
USE_CROSS_COMPILE isn't yet determined, but
- the conditionals must be applied _before_ the rest of bsd.own.mk
which uses these variables.
So we can't just have conditionals in bsd.prefs.mk after loading
bsd.own.mk -- we would need to put the conditionals in the middle of
bsd.own.mk between loading mk.conf and everything else.
As a followup commit, we can consider committing the variables after
bsd.own.mk as an optimization so that the conditional expansions are
only used inside bsd.own.mk and not in any other infrastructure. Or
we can consider putting a hook inside bsd.own.mk for committing the
variables there too. But let's see how this works out in practice
first.
mk/tools: add mandoc as tool Only nroff was handled as tool dependency for now, coming from the same source. Now the mandoc command is handled, too (libfido2 needs it, the change staged in wip right now). Platform-provided mandoc is only checked on NetBSD and Linux right now.
mk/tools: replace '!empty' conditions with '==' or '!=' Main work done by running 'pkglint -F --only "simpler" --only "simplified" *.mk', with manual improvements afterwards: * Removed several redundant ':U' modifiers, as pkglint doesn't know the exact order in which the infrastructure files include each other. * Removed redundant 'defined(X11_TYPE)' and the corresponding comment.
mk: Use OPSYS_VERSION
tools: Expose NATIVE_AR and NATIVE_RANLIB for cross-builds.
mk: Cross-eyed hacks to support cross-libtool.
For a long time, when cross-building, say from native=amd64 to
target=powerpc, it was necessary to:
1. cross-build a _powerpc_ package called cross-libtool-base-powerpc,
and then
2. install the powerpc package _natively_ with `pkg_add -m x86_64' to
override the architecture check that normally forbids this kind of
shenanigans,
in order to cross-build anything that uses libtool as a tool.
This is partly because libtool doesn't follow the normal GNU
convention of `./configure --build=<native platform> --host=<platform
package will run on> --target=<platform package is configured to
operate on>' -- in this example, build=amd64, host=amd64,
target=powerpc.
Instead, libtool expects to be cross-built itself, even if it's going
to run as a tool. It's not as bonkers as it sounds at first: libtool
is just a shell script, and it caches various information about the
(cross-building!) toolchain it is built with so it can use that
information later when it is run as a tool itself to cross-compile
other software.
To make this work, we need to create the toolchain wrappers for
libtool _as if_ we were cross-building even if we are building a
native package. So mk/tools uses a new flag TOOLS_USE_CROSS_COMPILE
instead of USE_CROSS_COMPILE, and libtool internally sets
MACHINE_ARCH=${TARGET_ARCH} (in the example above, powerpc) to make
it look like we're cross-building. The new TOOLS_CROSS_DESTDIR is an
alias for the (defaulted) CROSS_DESTDIR, which must now be set
unconditionally in mk.conf in order for libtool to know where the
cross-destdir will be; _CROSS_DESTDIR remains empty when building any
native packages (including the native cross-libtool package).
Finally, we need to make the resulting package be a native package,
with MACHINE_ARCH set to the one that it will be installed on (in the
example above, amd64), so I added an indirection _BUILD_DEFS.${var}
to replace var on its own in the build definitions that get baked
into the package, shown by `pkg_info -B'. Setting
_BUILD_DEFS.MACHINE_ARCH=${NATIVE_MACHINE_ARCH} ensures that this
mutant hybrid cross-built libtool still produces a native package.
All of this logic is gated on setting USE_CROSS_COMPILE in mk.conf or
LIBTOOL_CROSS_COMPILE in the package makefile, so it should be safe
for non-cross-builds -- when USE_CROSS_COMPILE=no and you're not
building cross-libtool, everything is as before.
mk/tools: Fix readelf for cross-compilation on NetBSD. This is needed by check-pie.
mk: Use native bsdtar on NetBSD 9.0 onwards. Relying on native variables like MKBSDTAR only works when using the native make, and should be avoided as they are not set when using a bootstrap. Should fix build of lang/go117 with bootstrapped NetBSD, as bsdtar from pkgsrc is unable to handle the distfile due to locale errors.
put back bsdtar as host tool, conditional on MKBSDTAR http://mail-index.netbsd.org/source-changes/2020/01/17/msg112935.html
tools/tools.NetBSD.mk: Register TOOLS_PLATFORM.realpath
remove bsdtar as tool because it was reverted in src for some reason ...
NetBSD 9 or later has bsdtar as /bin/tar
mk/tools: Add a gstrip tool on platforms where it exists. This will be used in places where GNU strip (or equivalent) is required, specifically for the "-g" flag.
mk: Add native GNU/BSD grep where they are known to exist.
Pullup ticket #5348 - requested by maya
mk/tools/tools.NetBSD.mk: bugfix for paxctl
Revisions pulled up:
- mk/tools/tools.NetBSD.mk 1.61,1.60
---
Module Name: pkgsrc
Committed By: maya
Date: Fri Apr 21 16:03:10 UTC 2017
Modified Files:
pkgsrc/mk/tools: tools.NetBSD.mk
Log Message:
Check for paxctl's existence rather than version+arch check.
It likely only matched amd64 before (looks like i386 had a typo),
but it's necessary on most netbsd archs since netbsd-7.99.x.
---
Module Name: pkgsrc
Committed By: ryoon
Date: Fri Apr 21 14:45:07 UTC 2017
Modified Files:
pkgsrc/mk/tools: tools.NetBSD.mk
Log Message:
Workaround for make(1) from recent NetBSD-current to detect paxctl
Check for paxctl's existence rather than version+arch check. It likely only matched amd64 before (looks like i386 had a typo), but it's necessary on most netbsd archs since netbsd-7.99.x.
Workaround for make(1) from recent NetBSD-current to detect paxctl
Handle optional ctfmerge/ctfconvert presence on NetBSD by explicitly passing down NOCTF when they don't exist.
Import mini-framework for paxctl(8) on NetBSD/{amd64,i386}
This allows setting flags for PaX on select binaries. Two new variables
are introduced for packages: NOT_PAX_ASLR_SAFE and NOT_PAX_MPROTECT_SAFE.
They both expect a list of binaries are known to not support PaX ASLR
and/or PaX MPROTECT, respectively.
"Please commit" wiz@
Remove NetBSD 0.x and 1.x settings, these have long since been unsupported.
Put back additions of some native gettext tools.
Revert "Add some missing builtin gettext-tools path.", proposed/discussed is required for such changes.
revert "Add unlink as a tool.", I cannot describe why is it on the Earth. proposed/discussed is required for such changes, from PMC gdt@.
Add unlink as a tool.
Add some missing builtin gettext-tools path.
Provide uniq(1) as a TOOLS.
sort
Add `xz' command as a TOOL.
NetBSD and Mac OS X both provide "readlink" under "/usr/bin/readlink".
Set NATIVE_CC/CC, NATIVE_CXX/CXX, and NATIVE_LD/LD for cross-builds. The NATIVE_xyz versions are for packages that build tools that they run natively but don't end up in the final product. This is a provisional scheme -- it should be replaced eventually by something more principled. ok agc
Only pick up /sbin/ldconfig as ldconfig tool for the case OBJECT_FMT=="a.out". Fixes PR 46445.
gsed related clean up.
* Stop to treat NetBSD's sed as GNU sed, not full compatible.
* Then, no need to reset TOOLS_PLATFORM.gsed for NetBSD if USE_TOOLS+=gsed and
real GNU sed is required.
* In addition, convert simple USE_TOOLS+=gsed to conditionally, without NetBSD.
* convert {BUILD_,}DEPENDS+=gsed to USE_TOOLS, all tools from gsed are real gsed.
Add objcopy to the list of cross-compiling tools.
use -current's builtin xzcat
Use unzip from base if it exists.
Add ftp as tool. On NetBSD and DragonFly, this will use the system version of tnftp, otherwise it will use net/tnftp. Require ftp as bootstrap dependency when the package has files to fetch. The dependency is currently unconditional and FETCH_CMD is set to TOOLS_PATH.ftp by default.
Fixed indentation.
Add readelf as tool, for only only the BSDs get it.
A number of packages implicitly assume that ar runs the right ranlib, so override it as well for DESTDIR.
Create ranlib override for cross-compilation as libtool doesn't use the target version. If we ever want to create archives for the native platform, NATIVE_RANLIB similiar to NATIVE_CC should be introduced.
Add core of the infrastructure support for cross-compilation.
- USE_CROSS_COMPILATION activates it, CROSS_DESTDIR specifies root of
the target filesystem
- derive _CROSS_DESTDIR from CROSS_DESTDIR or MAKEOBJDIR
- buildlink3.mk prefixes the files to symlink with _CROSS_DESTDIR
- compiler/gcc.mk knows about the target prefix (e.g. i386--netbsdelf)
- PKG_DBDIR is prefixed with _CROSS_DESTDIR
- package-install and bin-install are not called with su
- install and strip are redirected to the tool version
- links for the target specific ar, as, ld, nm, objdump, ranlib and
strip are added
- compiler wrapper detect if linking is requested or not
- special command sinks for CPP and CC/CXX add the cross-compile magic:
- modify include dirs to get the target /usr/include
- modify linker dirs and runpath to use target /usr/lib at link time,
but keep correct rpath entries
Supported-by: Google SoC 2007
Basic tests by he@ on Sparc. Review from jlam@.
Add bzip2 as tool. All platforms that had code to deal with bzcat also get bzip2, the rest falls back to archivers/bzip2. OK jlam.
Add ident as tool and use /usr/bin/ident on those platforms I know to have it. Fallback to devel/rcs otherwise. Commited during freeze as it is a dependency of pbulk.
Teach the tools framework about the following new tools:
itools, intltool, diff3, sdiff, msgmerge
* Adding USE_TOOLS+=itools to a package Makefile will cause the
tool-directory versions of imake, makedepend, mkdirhier and xmkmf
to point to the ones from the devel/nbitools package.
This change will remove the need for nbitools/buildlink3.mk, which
currently does a bit of hackery to force the "right" imake tools to
be used by packages that need it.
* Adding USE_TOOLS+=intltool to a package Makefile will cause the
local versions of intltool-* inside ${WRKSRC} to be replaced by
copies from the textproc/intltool package. If "intltool" is not
specified as a tool, then we create "broken" intltool-* tools in
the tools directory to help highlight hidden dependencies on the
intltool package.
In addition, modify the tools framework so that if "perl" is not
specified as a tool, then we create a "broken" perl tool in the
tools directory for the same reason as for "intltool".
These two changes together will remove the need for
intltools/buildlink3.mk and should also catch all cases where the
sources' intltools may have been silently used because perl was
found on the system.
* Adding USE_TOOLS+=diff3, USE_TOOLS+=sdiff, or USE_TOOLS+=msgmerge
to a package Makefile will cause the corresponding tool to be pulled
into the tools directory.
These are convenience tools to help simplify dependencies for some
packages.
Teach the tools framework about "openssl".
EOL the tv-derecurse branch; it has become too outdated. The last live revision can be fetched at the tv-derecurse-eol tag.
Add back "gsoelim" as a NetBSD tool since it supports "-I directory". Mark this as merely "GNUish" in the comment so that we know it's not really GNU soelim.
NetBSD actually doesn't have GNU soelim, but rather BSD soelim.
Teach the tools framework about "groff" as a tool.
Rename msgfmt.mk to gettext.mk. The rest of the gettext tool handling will be moved here from replace.mk shortly.
Overhaul the way packages can ask for "msgfmt". If a package needs msgfmt, then it should set the following in the package Makefile: USE_TOOLS+= msgfmt To deal with message files that use the "msgid_plural" statement, which isn't supported in NetBSD<=3.x and also in gettext<=0.10.35, we determine if the built-in "msgfmt" is sufficiently new enough to understand "msgid_plural". If it isn't, then we use the msgfmt.sh script to transform the msgid_plural statements to an equivalent construct that's understood by older msgfmt tools. The msgfmt.sh script is a straightforward translation of the original perl script msgfmt.pl script by Julio M. Merino Vidal into shell and awk, which are more lightweight dependencies than perl. We remove the USE_MSGFMT_PLURALS bits in gettext-lib/builtin.mk as they are made obsolete by the new code in mk/tools/msgfmt.mk. BUILD_USE_MSGFMT is still supported but will be removed in a separate commit.
Teach the tools framework how to supply a pkgsrc version of install-info if a native one isn't available. We ensure that the "install-info" tool in the tools directory is a no-op since the real info file registration is handled by the INSTALL/DEINSTALL script in pkgsrc/mk/pkginstall/install-info.
* Teach the tools framework how to supply the pkgsrc version of
makeinfo if no native makeinfo executable exists. Honor TEXINFO_REQD
when determining whether the native makeinfo can be used.
* Remove USE_MAKEINFO and replace it with USE_TOOLS+=makeinfo.
* Get rid of all the "split" argument deduction for makeinfo since
the PLIST module already handles varying numbers of split info files
correctly.
NOTE: Platforms that have "makeinfo" in the base system should check
that the makeinfo entries of pkgsrc/mk/tools.${OPSYS}.mk are
correct.
Pullup ticket 1019 - requested by Johnny C. Lam bugfixes for the tools framework Revisions pulled up: - pkgsrc/mk/tools/replace.mk 1.145-1.155 - pkgsrc/mk/tools/tools.Darwin.mk 1.22 - pkgsrc/mk/tools/tools.NetBSD.mk 1.23, 1.24 - pkgsrc/mk/tools/tools.OpenBSD.mk 1.22 - pkgsrc/mk/tools/tools.Linux.mk 1.28 - pkgsrc/mk/tools/defaults.mk 1.43 Module Name: pkgsrc Committed By: reed Date: Wed Dec 28 04:24:37 UTC 2005 Modified Files: pkgsrc/mk/tools: replace.mk Log Message: Add makepsres to _TOOLS.x11-clients. (makepsres used for lyx install for example.) --- Module Name: pkgsrc Committed By: reed Date: Wed Dec 28 16:46:38 UTC 2005 Modified Files: pkgsrc/mk/tools: replace.mk Log Message: For the x11-clients section, use "XFree86-clients" instead of "imake" for the TOOLS_PREFIX (for the XFree86-clients provided tool). (Asked about on tech-pkg and jlam said to commit.) --- Module Name: pkgsrc Committed By: jlam Date: Wed Dec 28 22:21:19 UTC 2005 Modified Files: pkgsrc/mk/tools: replace.mk Log Message: Consolidate xmkmf with the other "imake" tools. --- Module Name: pkgsrc Committed By: jlam Date: Wed Dec 28 22:26:50 UTC 2005 Modified Files: pkgsrc/mk/tools: replace.mk Log Message: Whitespace changes to align with other sections. --- Module Name: pkgsrc Committed By: jlam Date: Wed Dec 28 22:41:27 UTC 2005 Modified Files: pkgsrc/mk/tools: replace.mk Log Message: Teach the tools framework about ttmkfdir and type1inst. --- Module Name: pkgsrc Committed By: jlam Date: Sun Jan 8 22:09:05 UTC 2006 Modified Files: pkgsrc/mk/tools: replace.mk Log Message: Consolidate groff-supplied tools into one section. --- Module Name: pkgsrc Committed By: jlam Date: Sun Jan 8 23:00:00 UTC 2006 Modified Files: pkgsrc/mk/tools: defaults.mk replace.mk tools.Linux.mk tools.NetBSD.mk Log Message: Teach the tools framework about "gsoelim", which is the soelim tool from a groff distribution. --- Module Name: pkgsrc Committed By: jlam Date: Mon Jan 9 18:32:52 UTC 2006 Modified Files: pkgsrc/mk/tools: replace.mk tools.Darwin.mk tools.NetBSD.mk tools.OpenBSD.mk Log Message: Teach the tools framework about the ``ksh'' tool, provided by the shells/pdksh package if it doesn't exist on the base system. --- Module Name: pkgsrc Committed By: jlam Date: Thu Jan 12 18:29:03 UTC 2006 Modified Files: pkgsrc/mk/tools: replace.mk Log Message: Re-implement some code that was lost in revision 1.144 so that IMAKE may be overridden to point to the path of the imake binary to use. --- Module Name: pkgsrc Committed By: jlam Date: Thu Jan 12 19:26:03 UTC 2006 Modified Files: pkgsrc/mk/tools: replace.mk Log Message: Fix a typo that was causing imake, mkdirhier and xmkmf to not be created as tools in the tools directory.
Teach the tools framework about the ``ksh'' tool, provided by the shells/pdksh package if it doesn't exist on the base system.
Teach the tools framework about "gsoelim", which is the soelim tool from a groff distribution.
Add "soelim", also part of groff if replacement is needed.
Teach the tools framework about "sleep" which can be provided by the sysutils/coreutils package.
Teach the tools framework about ``printf'', which can be replaced by the printf in sysutils/coreutils.
Teach the tools framework about "byacc", which is a Berkeley-compatible YACC. This should be used by packages that require NOT using bison (maybe because bison can't handle the .y file) but still need a yacc.
Teach the tools framework about csh, and replace with tcsh if necessary.
Note locations for native diff tools.
Teach the tools framework about mktemp. Replace explicit tests for mktemp with USE_TOOLS+=mktemp in the samba packages.
Teach the tools framework about nroff.
Teach the tools framework about "flex" so that packages can specify that they require flex instead of any old lex.
sync [all but bsd.pkg.mk] with HEAD
file tools.NetBSD.mk was added on branch tv-derecurse on 2005-05-17 18:29:46 +0000
Change the EXTRACT_USING_PAX defined/undefined option into a EXTRACT_USING
which can take multiple values -- "pax" or "gtar". The default value
of EXTRACT_USING is "pax", which more closely matches reality since
before, we were using bootstrap "tar" for ${GTAR} and it was actually
pax-as-tar. Also, stop pretending pax-as-tar from the bootstrap kit
or on NetBSD is GNU tar. Lastly, in bsd.pkg.extract.mk, note whether
we need "pax" or "gtar" depending on what we need to extract the
distfiles.
Consider ${TAR} differently from ${GTAR}. Currently, mark pkgsrc down
as needing both (no impact since they're both satisfied by the tar
binary installed by the bootstrap kit). There's some funniness in
the extraction code where we check for pax or GNU tar that needs to
be resolved. Remove the TAR=${GTAR} hack since it's no longer needed
after these changes.
XXX Later, a sweep needs to be made to see where we actually need GTAR
XXX and where we only need TAR, probably triggered by whether we call it
XXX with the "z" option or not. Packages that need GTAR should explicitly
XXX add USE_TOOLS+=gtar to the package Makefile.
Split out ldconfig handling from the USE_TOOLS processing in replace.mk.
This makes more sense since there is never going to be a pkgsrc
replacement for ldconfig. We now always create an ldconfig tool in
${TOOLS_DIR} that either calls the system-supplied one if it exists,
or is a no-op.
Remove conditional expression from tools.*.mk files. Strip handling has moved to its own file, and we only need to define TOOLS_PLATFORM.strip if it actually exists on the system.
Teach the new tools framework about ${STRIP}, which is used in some
packages to strip installed executables. If INSTALL_UNSTRIPPED ==
"yes", then we create a "strip" wrapper in ${TOOLS_DIR} that just
calls ${TRUE} by considering ${TRUE} the system-supplied strip command.
Teach the new tools framework about INSTALL and set the appropriate
default value for each platform. Currently, the replacement tools
comes from sysutils/coreutils, but where there is no native BSD install
program, bootstrap-pkgsrc should probably be made to provide an install
shell script as an alternative, and mk/tools/bootstrap.mk should be
amended accordingly.
Also remove one use of ${TYPE} in pkgsrc (bsd.pkg.mk) under the new tools
framework.
Teach the new tools framework about MAIL_CMD. We use mail/nail as a "mail" replacement in case TOOLS_PLATFORM.mail is empty.
Teach the new tools framework about ldconfig. This must be provided by
tools.${OPSYS}.mk, so at the very worst, it must be set to "true".
Teach the new tools framework about USE_TOOLS+=bzcat.
Note the "[" is natively available everywhere (usually as a shell builtin).
Rename PLATFORM_TOOLS.* to TOOLS_PLATFORM.* to bring the variable names under the same namespace as the other parts of the new tools framework.
The BSDs all have lex, tbl, and yacc in the base system.
Initial stab at creating lists of system-supplied tools for each platform.
These were culled from pkgsrc/mk/${OPSYS}.mk. These files should only be
listing utilities that aren't installed by pkgsrc.