Up to [cvs.NetBSD.org] / pkgsrc / mk / wrapper
Request diff between arbitrary revisions
Keyword substitution: kv
Default branch: MAIN
mk: remove Interix support
mk: Cross-eyed hacks to support cross-libtool.
For a long time, when cross-building, say from native=amd64 to
target=powerpc, it was necessary to:
1. cross-build a _powerpc_ package called cross-libtool-base-powerpc,
and then
2. install the powerpc package _natively_ with `pkg_add -m x86_64' to
override the architecture check that normally forbids this kind of
shenanigans,
in order to cross-build anything that uses libtool as a tool.
This is partly because libtool doesn't follow the normal GNU
convention of `./configure --build=<native platform> --host=<platform
package will run on> --target=<platform package is configured to
operate on>' -- in this example, build=amd64, host=amd64,
target=powerpc.
Instead, libtool expects to be cross-built itself, even if it's going
to run as a tool. It's not as bonkers as it sounds at first: libtool
is just a shell script, and it caches various information about the
(cross-building!) toolchain it is built with so it can use that
information later when it is run as a tool itself to cross-compile
other software.
To make this work, we need to create the toolchain wrappers for
libtool _as if_ we were cross-building even if we are building a
native package. So mk/tools uses a new flag TOOLS_USE_CROSS_COMPILE
instead of USE_CROSS_COMPILE, and libtool internally sets
MACHINE_ARCH=${TARGET_ARCH} (in the example above, powerpc) to make
it look like we're cross-building. The new TOOLS_CROSS_DESTDIR is an
alias for the (defaulted) CROSS_DESTDIR, which must now be set
unconditionally in mk.conf in order for libtool to know where the
cross-destdir will be; _CROSS_DESTDIR remains empty when building any
native packages (including the native cross-libtool package).
Finally, we need to make the resulting package be a native package,
with MACHINE_ARCH set to the one that it will be installed on (in the
example above, amd64), so I added an indirection _BUILD_DEFS.${var}
to replace var on its own in the build definitions that get baked
into the package, shown by `pkg_info -B'. Setting
_BUILD_DEFS.MACHINE_ARCH=${NATIVE_MACHINE_ARCH} ensures that this
mutant hybrid cross-built libtool still produces a native package.
All of this logic is gated on setting USE_CROSS_COMPILE in mk.conf or
LIBTOOL_CROSS_COMPILE in the package makefile, so it should be safe
for non-cross-builds -- when USE_CROSS_COMPILE=no and you're not
building cross-libtool, everything is as before.
mk: MKPIE is not GCC-specific. Should help ObjC packages.
Add a package-settable variable PKGSRC_OVERRIDE_MKPIE This is needed by packages that require hand-holding in building PIE. Also a post-build check for MKPIE is performed by default when PKG_DEVELOPER=YES.
mk/wrapper: allow "unwrap" to be a no-op SUBST block Discovered in pkgtools/pkg, where libpkg/pkg.pc was left unmodified.
mk: allow "bmake clean depends" as shortcut When "bmake clean depends" was called for a package where the various cookie files already existed, these would enable different rules than a clean package directory. Since "bmake clean" deletes all the cookie files before "bmake depends" starts, in these combined command lines the cookie files must be treated as absent.
Remove clauses 3,4 from TNF-only copyright blocks. This is based on the decision The NetBSD Foundation made in 2008 to do so, which was already applied to src. This change has been applied to code which is likely not in other repositories. ok board@, reviewed by riastradh@
Introduce a command sink for ld(1) with MKPIE When not using cwrappers, so far PKGSRC_MKPIE was only automatically applied when linking using gcc(1) (when enabled). This is now also the case for packages using ld(1) to link executables.
Obtain the MKPIE flags from the environment This solves an issue with the command sink component of the MKPIE wrapper for GCC, where the contents of the _MKPIE_CFLAGS.gcc and _MKPIE_LDFLAGS.gcc variables was guessed. It is now communicated to cmd-sink-mkpie-gcc through the environment instead.
Add support for a number of security features - Revisit (and rename) support for FORTIFY as PKGSRC_USE_FORTIFY (instead of PKGSRC_USE_FORT) for easier support outside NetBSD/gcc; - PKGSRC_USE_SSP is no longer enabled by default when PKGSRC_USE_FORTIFY is enabled; - PKGSRC_MKPIE builds executables as PIE (to leverage userland ASLR) - PKGSRC_USE_RELRO builds with a read-only GOT to prevent some exploits from functioning. Tested on NetBSD/amd64 by myself, in every combination, with and without pkgtools/cwrappers. MKPIE is not supported at the moment with cwrappers. Also, MKPIE is known to still break a number of packages when enabled (and actually supported). Tested on SunOS by jperkin@, thank you! As discussed on tech-pkg@, the default behavior is not changed, except where noted above. ok bsiegert@
pass correct ABI via OBJECT_MODE environment variable on AIX It turns out a handful of AIX binutil-like utilities are particular about type of object files they should examine. Instead of piping through flags for each utility everywhere, it is easier to just export 'OBJECT_MODE=[32|64]' instead. From Eric N. Vander Weele.
pass correct ABI flags to ar(1) on AIX AIX is particular about the type of object files `ar` should examine. This should be set explicitly to coincide with the user's defined $ABI. Contributed by Eric N. Vander Weele.
Fixed subst.mk not to corrupt files Up to now, using subst.mk may have led to file corruption during active package development. This happened when a sed(1) command had a syntax error, in which case the whole sed(1) command was terminated, leaving an empty original file behind. This commit changes that behavior by applying the sed(1) commands to the original file and saving the result in a temporary file. Only after that succeeded is the original file overwritten. During this rewrite, SUBST_POSTCMD has been removed, since it was only used in one place (mk/wrapper), and since it relied on the exact sequence of the internal commands. No package in either main pkgsrc or pkgsrc-wip uses this variable right now.
Split cwrappers from legacy wrappers. We no longer generate the legacy wrappers when USE_CWRAPPERS is enabled, saving a reasonable amount of I/O during builds, mostly due to avoiding the transform/untransform sed file generations. WRAPPER_DIR and WRAPPER_BINDIR are used by various packages to override or point to specific wrappers, and these now point to the cwrappers directory when enabled, removing the need for CWRAPPERS_BIN_DIR duplication and fixing packages which previously were using legacy wrappers by accident. A number of targets are now duplicated between bsd.wrapper.mk and cwrappers.mk, the intention being that the legacy wrappers will be deprecated once cwrappers is verified on all supported platforms. If that turns out to take longer than expected, we will probably want to introduce a wrapper.mk to abstract them away before loading the appropriate back-end.
Remove ECHO_WRAPPER_MSG and just use ECHO_BUILDLINK_MSG, which was set to the same value anyway. Also removes a comment from 2005 which was possibly wrong at the time it was committed, given the same construct has been used in bsd.buildlink3.mk unchanged since 2004.
Separate dependency from nada-rule.
Add cwrappers glue as developed during Google's Summer of Code 2007.
Wrap not only ld but also ar and ranlib.
Create a wrapper for gfortran.
compiler.mk/gcc.mk: Add support for USE_LANGUAGES+= ada
All recent packages featuring Ada code have a hard dependency on the
lang/gnat-aux compiler package. The valid values for USE_LANGUAGES
are c, c99, c++, fortran, fortran77, java, objc, so specifying a
specific compiler was necessary up into now.
One problem with lang/gnat-aux is that it is installed at ${LOCALBASE}
where the lang/gccXX compilers are installed at ${LOCALBASE}/gccXX.
The latter compilers have no possibility of sharing conflicting files
unlike lang/gnat-aux. Rather than fundamentally update the GCC 4.6-based
lang/gnat-aux to avoid these conflicts, a new Ada-capable compiler
based on GCC 4.7 was created with the intent of being supported by
mk/compiler.mk and mk/compiler/gcc.mk.
The Ada packages will be effectively migrated from lang/gnat-aux to the
new lang/gcc-aux compiler, but lang/gcc-aux will remain as a standalone
package as it is the only GCC 4.6-based compiler that builds on
DragonFly and serves it as a world and kernel compile option.
In addition to the current language wrappers, lang/gcc-aux adds
wrappers for "ada" (unique to gcc-aux, hardlinked to gcc driver),
and the gnat, gnatmake, gnatbind, gnatlink, gnatchop, gnatprep,
and gnatls programs. Supporting all of these allows the wrapper
system to be used with Ada packages; currently wrappers are mostly
disabled on them.
The lang/gcc47 implicitly adds support for the "objc-c++" language by
adding it to the USE_LANGUAGES list, but it wasn't really supported.
An attempt was made to better support objc-c++, but this new enumeration
probably still needs work or needs to be removed completely.
Logic for Ada support:
1) All lang/gccXX compilers have version numbers ranging from 2.8.1 to 9.
2) lang/gcc-aux uses the release date as its version number in the form of
YYYYMMDD with a minimum value of 20120614, so there is no version
overlap.
3) When at least one element of USE_LANGUAGES is "ada", the value of
20120614 is added to the set of GCC_REQD which selects lang/gcc-aux.
4) The _NEED_NEWER_GCC check is disabled. It fails and isn't relevant;
unless a package sets GCC_REQD over 20120614, the only way to select
lang/gcc-aux is to specify the Ada language and only one compiler
known to gcc.mk can support it.
Add support of C++ crossbuild with g++, based on CC with gcc.
Force to use symbolic link for wrapper if WRAPPER_USE_SYMLINK is defined. Workaround for missing hard link support on Haiku, PR#43398.
Make wrapper generation code work properly if there is a directory with a whitespace in the command search path.
Avoid one fork per wrapper call if no cleanup is necessary and debugging is disabled. In that case, the post processing is doing nothing anyway.
Also wrap the cpp command like cc for AIX/XLC.
Improve AIX/XLC support: - provide cpp wrapper using cc -E - provide c++_r wrapper working like c++ - provide -rpath handling for cc - fix some debug messages to provide the correct script name
Don't use the CC wrapper for ld on AIX with XLC. From Jens Rehsack.
Improve AIX/XLC support based on discussions with Jens Rehsack: - Add cc_r/xlc_r wrapper using the same rules as xlc itself. It is used for example by Perl. - Improve the RPATH emulation: - Always set -blibpath, use /usr/lib:/lib as default - If -blibpath is exlicitly given, add to the default - Additionally append any -Wl,-rpath given.
Merge aix-xlc block into the xlc block and distinguish AIX and Darwin. The PKG_FAIL_REASON isn't executed by default, no idea why, but this makes it a bit cleaner why the compiler wrapper will fail and fixes the more important AIX case.
On AIX don't overwrite the XLC definitions with the platform logic, the former are already more specific.
Add IRIX command sinks, needed to get rpath fixup when using GCC.
Do not load the $cache file for every argument. Instead, load it once and then call it as a shell function. Note that the shell function is not called "cache", since some shells have problems when a function has the same name as a variable. This speeds up the wrapper by as little as 75 percent for the final link command of editors/abiword on SunOS-5.10-sparc (before: 20 seconds, after: 5 seconds).
Oops, don't overflow the terminal.
Using a single awk process instead of a pipe of four grep processes should be both faster and easier to read.
When CPP is defined as "cc -E", the wrapper framework created the wrapper for gcc and cc as type "CPP", although these aren't C preprocessors. By preventing the code from overwriting files, this works now.
Add core of the infrastructure support for cross-compilation.
- USE_CROSS_COMPILATION activates it, CROSS_DESTDIR specifies root of
the target filesystem
- derive _CROSS_DESTDIR from CROSS_DESTDIR or MAKEOBJDIR
- buildlink3.mk prefixes the files to symlink with _CROSS_DESTDIR
- compiler/gcc.mk knows about the target prefix (e.g. i386--netbsdelf)
- PKG_DBDIR is prefixed with _CROSS_DESTDIR
- package-install and bin-install are not called with su
- install and strip are redirected to the tool version
- links for the target specific ar, as, ld, nm, objdump, ranlib and
strip are added
- compiler wrapper detect if linking is requested or not
- special command sinks for CPP and CC/CXX add the cross-compile magic:
- modify include dirs to get the target /usr/include
- modify linker dirs and runpath to use target /usr/lib at link time,
but keep correct rpath entries
Supported-by: Google SoC 2007
Basic tests by he@ on Sparc. Review from jlam@.
Changed the plain "mkdir" into ${MKDIR}, like in all the other cases.
Apparently, the tools directory isn't in the PATH at the point where
this code is executed.
Fixes PR 35487.
Replaced ${_PKG_SILENT}${_PKG_DEBUG} with a simple ${RUN} to save some
space.
fix typo
Teach pkgsrc about HP-UX.
Add OSF1 wrapper quirks to always enable posix sockets and iso C++ iostreams. Fixes PR 31444. Fixes PR 31447. Both problems reported by Rumi Szabolcs.
The ABI for which the code should be generated can be queried from the wrapper_abi variable. This is used in the MIPSpro wrapper to reject any package that tries to compile a file for the wrong ABI. This is not just a warning because such a package most likely has more severe problems. The gcc -Wall option seems to be similar to the MIPSpro -fullwarn option, so map them.
Documented WRAPPER_REORDER_CMDS.
Added a new variable wrapper_type to the wrappers, so that the transformations can decide whether they happen in a C or a C++ compiler. Now the wrappers don't depend on the basename of the executed compiler wrapper anymore. Fixes PR 35555.
The sunpro C++ compiler needs its own cache, since cc and CC transform -Wall to different arguments.
Bugfix: Only regular files can be unwrapped.
Removed the separate files for the cache header and footer. This makes the code a lot more readable.
Merged the transformations from the Solaris-specific gcc part into the generic one. That way, the unknown options are reported properly on all platforms.
Added a transformation file for gcc to check for unknown options.
Added wrapper-subr.sh, which provides shell functions that are useful to
make the wrapper code less frightening. For example, instead of
arg="${arg#-Wl,}"
$debug_log $wrapperlog " (transform-sunpro-cc) to: $arg"
addtocache=yes
one can simply write
transform_to "${arg-Wl,}"
The functions that are currently implemented are: transform_to,
transform_discard, transform_to_nocache, transform_discard,
transform_pass and transform_pass_unknown.
Removed some code duplication, which results in 120 lines less code.
Fixed the "inconsistent update" problem when using the pkgsrc wrapper cache together with MAKE_FLAGS=-j. The cache file is now regenerated into a temporary file and then renamed (atomically) to the "real" cache file. Fixed some code that would fail when run in "set -e" mode. If you want to test the "set -e" mode, just uncomment the line in wrapper.sh.
When CPP is defined as "cc -E" on Solaris with sunpro, the cc wrapper had been overwritten with the cpp wrapper, and the latter didn't define the necessary transformation rules. Fixed that.
When $debug_log is set to ":" and at the same time $IFS is ":", the
command line
$debug_log $arguments
tries to execute the empty command. This results in error messages of
the form
cc[50]: : cannot execute (IRIX)
bash: : command not found (Bash)
: permission denied (NetBSD /bin/sh)
Setting IFS to the original value as soon as possible fixes these bugs.
Fixes PR 34135.
From the second point in Krister's mail to tech-pkg:
http://mail-index.netbsd.org/tech-pkg/2006/07/30/0005.html
2. Too much is logged in the .wrapper.log
The wrappers log quite a lot of detailed information by
default, with the result that my .work.log file is 176236028
bytes {for ghc}.
The wrapper functionality is mature, so this detailed
information is seldom needed, and you can in general determine
what happened by looking at the input/output-lines (i.e.
those lines marked as "[*]" and "<.>" in the .work.log).
I would therefore like to change the default behavior to
only log the input/output-lines.
Flip the switch for WRAPPER_DEBUG from "yes" to "no" by default so
that the detailed and rather verbose "push", "pop", "transform", etc.
messages are not printed. While here, also make WRAPPER_DEBUG a
pkgsrc-private variable by prepending with an underscore.
After auditing all the packages the ensure that they never invoke the compiler toolchain until the configure stage, move the "wrapper" stage back to between "patch" and "configure". This makes "make extract NO_DEPENDS=" work again for all packages.
Modify the barrier so that we always invoke the recursive make process when passing through the barrier. This ensures the PATH (passed via PKGSRC_MAKE_ENV) is correctly set for all phases after the barrier. This fixes a bug in "interactive" pkgsrc use, where if you have no work directory and type "make build && make install", then the "install" step does not have a PATH set to include all the wrapper and tools directories.
Rearrange sequence so that "wrapper" occurs between "tools" and "extract". This allows all user-definable targets (pre-*, do-*, post-*) to have access to the wrapper scripts.
Rename <phase>_COOKIE variables to _COOKIE.<phase>. These are private variables so there are no user-visible changes. This change just makes it a little easier to write for loops.
The cookie files are indirectly made using *-cookie targets, so verify that they are never being created more than once by inserting a check into the *-cookie targets.
Never skip the wrapper step, even if NO_BUILD is set. There are packages that invoke the compiler in either the configure or install steps that we still need to guard against.
Flatten out recursive makes into a single re-invocation of make by introducing the concept of a "barrier". We separate the user-invokable targets into ones that must happen before the barrier, and ones that must happen after the barrier. The ones that happen after the barrier are run in a sub-make process. In this case, the targets that must be run after the barrier are from the "wrapper" step and beyond. We rewrite the various "flow" targets, e.g. wrapper, configure, build, etc., so that they of the right form to use the barrier target. This now completely removes the concept of PKG_PHASE from pkgsrc. It is replaced with the concept of "before" and "after" the barrier, and this state can be checked by testing for the existence of the barrier cookie file. Because we've removed most of the recursive makes, there is now nowhere to hook the PKG_ERROR_HANDLER.* commands, so remove them for now. As part of this commit, put back the logic that conditionalized the sources for the various cookie files. Because the sources are all "phony" targets, they were always run, regardless of whether or not the cookie file already existed. Now, if a cookie file exists, then that entire phase associated with that cookie file is skipped. Lastly, fix a thinko in configure/bsd.configure.mk where setting NO_CONFIGURE in a package Makefile would manage to skip the "wrapper" step altogether. Fix this by correctly noting "wrapper" and not "patch" as the preceding step to "configure".
Make the check-vulnerable target more self-sufficient, by moving some of the logic from fetch/fetch.mk into flavor/pkg/check.mk, so that check-vulnerable can be used as a source target. Make check-vulnerable a source target for every phase of the build workflow, which ensures that it is always run if the user starts a new phase from the command line. Fix the cookie-generation targets so that they don't append, only overwrite to the cookie file. This works around potential problems due to recursive makes. Move the cookie checks so that they surround the corresponding phase target. The presence of the cookie should now inform the make process to avoid doing any processing of phases that occur before the phase corresponding to the cookie.
Refactor configure, build, test and wrapper phases out of bsd.pkg.mk
and into their own directories. Also do some cleanups with build/_build
and pkginstall -- we get rid of _build and simply run pkginstall as
part of the "build" target.
Introduce a new mechanism to handle varying directory depths under
${WRKSRC} in which we find files to override, e.g. configure, config.*,
libtool, etc. OVERRIDE_DIRDEPTH is a package-settable variable that
specifies how far under ${WRKSRC} the various targets should look,
and it defaults to "2". We preserve the
meaning of the various *_OVERRIDE variables, so if they are defined,
then their values supersede the OVERRIDE_DIRDEPTH mechanism.
devel/tla will need to specially set OVERRIDE_DIRDEPTH to 3 (see log
for revision 1.1857 for bsd.pkg.mk -- to be done in a separate commit.
Teach the tools framework about the following new tools:
itools, intltool, diff3, sdiff, msgmerge
* Adding USE_TOOLS+=itools to a package Makefile will cause the
tool-directory versions of imake, makedepend, mkdirhier and xmkmf
to point to the ones from the devel/nbitools package.
This change will remove the need for nbitools/buildlink3.mk, which
currently does a bit of hackery to force the "right" imake tools to
be used by packages that need it.
* Adding USE_TOOLS+=intltool to a package Makefile will cause the
local versions of intltool-* inside ${WRKSRC} to be replaced by
copies from the textproc/intltool package. If "intltool" is not
specified as a tool, then we create "broken" intltool-* tools in
the tools directory to help highlight hidden dependencies on the
intltool package.
In addition, modify the tools framework so that if "perl" is not
specified as a tool, then we create a "broken" perl tool in the
tools directory for the same reason as for "intltool".
These two changes together will remove the need for
intltools/buildlink3.mk and should also catch all cases where the
sources' intltools may have been silently used because perl was
found on the system.
* Adding USE_TOOLS+=diff3, USE_TOOLS+=sdiff, or USE_TOOLS+=msgmerge
to a package Makefile will cause the corresponding tool to be pulled
into the tools directory.
These are convenience tools to help simplify dependencies for some
packages.
Fixed an obvious typo: .if !target$(${_WRAP_EMPTY_FILE}) contains a
dollar character before the opening parenthesis. This should really have
been detected by bmake, but somehow the parser seems to be tolerant with
respect to syntax errors.
EOL the tv-derecurse branch; it has become too outdated. The last live revision can be fetched at the tv-derecurse-eol tag.
Remove superfluous quotes around SUBST_MESSAGE.
Applied all quoting fixes found by "pkglint --autofix".
added support for the SGI IDO (IRIX Development Option) cc used by IRIX 5.3. changes approved by jschauma and recht.
Pullup ticket 709 - requested by Grant Beattie portability fixes for compiler wrapper Revisions pulled up: - pkgsrc/mk/wrapper/arg-pp-sunpro-cxx 1.1, 1.2 - pkgsrc/mk/wrapper/bsd.wrapper.mk 1.36, 1.37 - pkgsrc/mk/wrapper/transform-sunpro-cc 1.9 Module Name: pkgsrc Committed By: grant Date: Tue Jul 26 07:36:54 UTC 2005 Modified Files: pkgsrc/mk/wrapper: bsd.wrapper.mk Log Message: icc 9.0 wants the same options as 8.1. --- Module Name: pkgsrc Committed By: grant Date: Sat Aug 20 02:11:15 UTC 2005 Modified Files: pkgsrc/mk/wrapper: transform-sunpro-cc Log Message: add -xnolibmopt after -fast to disable the use of libsunmath (until it can be sanely statically linked). -- Module Name: pkgsrc Committed By: grant Date: Sat Aug 20 14:37:53 UTC 2005 Modified Files: pkgsrc/mk/wrapper: bsd.wrapper.mk Added Files: pkgsrc/mk/wrapper: arg-pp-sunpro-cxx Log Message: strip leading "-Qoption ld" from -R args for sunpro C++ compiler, to avoid duplicate "-Qoption ld" args when the -R args is later stripped. thanks to jlam@ for the hint. --- Module Name: pkgsrc Committed By: grant Date: Sun Aug 21 02:39:52 UTC 2005 Modified Files: pkgsrc/mk/wrapper: arg-pp-sunpro-cxx Log Message: preserve -Qoption arg if the 2nd next arg isn't -R.
Pullup ticket 708 - requested by Grant Beattie portability fix for compiler wrapper Revisions pulled up: - pkgsrc/mk/wrapper/bsd.wrapper.mk 1.35 - pkgsrc/mk/wrapper/cmd-sink-solaris-imake 1.1 - pkgsrc/mk/wrapper/transform-solaris-gcc 1.1 Module Name: pkgsrc Committed By: grant Date: Sat Jul 23 04:45:30 UTC 2005 Modified Files: pkgsrc/mk/wrapper: bsd.wrapper.mk Added Files: pkgsrc/mk/wrapper: cmd-sink-solaris-imake transform-solaris-gcc Log Message: ensure gcc options are always passed to imake when using gcc on Solaris. fixes PR pkg/29608.
strip leading "-Qoption ld" from -R args for sunpro C++ compiler, to avoid duplicate "-Qoption ld" args when the -R args is later stripped. thanks to jlam@ for the hint.
Pullup ticket 630 - requested by grant beattie
infrastructure fixes for sunpro
Revisions pulled up:
- pkgsrc/mk/wrapper/bsd.wrapper.mk 1.33-1.34
- pkgsrc/mk/wrapper/cmd-sink-sunpro-cxx 1.1
Module Name: pkgsrc
Committed By: grant
Date: Wed Jul 6 03:31:24 UTC 2005
Modified Files:
pkgsrc/mk/wrapper: bsd.wrapper.mk
Added Files:
pkgsrc/mk/wrapper: cmd-sink-sunpro-cxx
Log Message:
add cmd-sink for sunpro C++ which links shared libraries with the
standard C++ runtime libraries.
we need to explicitly do this because ${CXX} only links against the
C++ runtime libraries when building an executable, not a shared
object. this results in executables linked against libraries linked by
${CXX} unable to resolve C++ symbols at runtime.
fixes packages which link shared libraries using ${CXX}, including
graphics/glu, databases/db4, and packages which use shared libraries
provided by these packages.
----
Module Name: pkgsrc
Committed By: grant
Date: Wed Jul 6 03:35:28 UTC 2005
Modified Files:
pkgsrc/mk/wrapper: bsd.wrapper.mk
Log Message:
update copyright year
icc 9.0 wants the same options as 8.1.
ensure gcc options are always passed to imake when using gcc on Solaris. fixes PR pkg/29608.
update copyright year
add cmd-sink for sunpro C++ which links shared libraries with the
standard C++ runtime libraries.
we need to explicitly do this because ${CXX} only links against the
C++ runtime libraries when building an executable, not a shared
object. this results in executables linked against libraries linked by
${CXX} unable to resolve C++ symbols at runtime.
fixes packages which link shared libraries using ${CXX}, including
graphics/glu, databases/db4, and packages which use shared libraries
provided by these packages.
Made sure the :sh operator does not result in an empty string. Rewrote the variable substitution for the find(1) command to be more readable.
Change instances where we stored `shell command` in a make variable into variables that use the :sh modifier. This still causes expansion to only happen when referenced, and has the advantage of being :Q-safe. Bring back the changes from revision 1.19 of mk/subst.mk now that the problem noted above has been fixed. This passes the buildlink-unwrap regression test.
sync [all but bsd.pkg.mk] with HEAD
Rename MAKE_VARS to MAKEVARS so that it more closely resembles "MAKEFLAGS". Both "MAKEVARS" and "MAKEFLAGS" affect the package-level make process, not the software's own make process.
Teach bsd.pkg.mk to create a phase-specific "makevars.mk" file that caches variable definitions that were computed by make. These variables are specified by listing them in MAKE_VARS, e.g., .if !defined(FOO) FOO!= very_time_consuming_command .endif MAKE_VARS+= FOO bsd.pkg.mk will include only the one generated during the most recent phase. A particular phase's makevars.mk file consists of variable definitions that are a superset of all of the ones produced in previous phases of the build. The caching is useful because bsd.pkg.mk invokes make recursively, which in the example above has the potential to run the very time-consuming command each time unless we cause FOO to be defined for the sub-make processes. We don't cache via MAKE_FLAGS because MAKE_FLAGS isn't consistently applied to every invocation of make, and also because MAKE_FLAGS can overflow the maximum length of a make variable very quickly if we add many values to it. One important and desirable property of variables cached via MAKE_VARS is that they only apply to the current package, and not to any dependencies whose builds may have been triggered by the current package. The makevars.mk files are generated by new targets fetch-vars, extract-vars, patch-vars, etc., and these targets are built during the corresponding real-* target to ensure that they are being invoked with PKG_PHASE set to the proper value. Also, remove the variables cache file that bsd.wrapper.mk was generating since the new makevars.mk files provide the same functionality at a higher level. Change all WRAPPER_VARS definitions that were used by the old wrapper-phase cache file into MAKE_VARS definitions.
In line with the unconditional inclusion in bsd.pkg.mk, bsd.buildlink3.mk must always bee included too.
USE_BUILDLINK3 is no longer optional (and cannot be turned off). Per mail to tech-pkg: ===== * USE_BUILDLINK3=YES will be unconditional. (In fact, USE_BUILDLINK3 will be ignored altogether by mk/; but see below.) * NO_BUILDLINK and NO_WRAPPER will be ignored by mk/. If a build happens, these phases will happen. * The existing NO_BUILD will imply the previous NO_BUILDLINK and NO_WRAPPER. If no build happens, those phases are not needed. * NO_TOOLS will be ignored by mk/. The tools phase, which provides much more than just the C compiler, will always happen regardless of package. This will make metapackage builds only slightly slower, in trade for far less user error.
sync with HEAD
fix previous to work with pkgsrc bmake, and add a comment explaining why it must be done this way.
Also echo the wrapper stages if PKG_VERBOSE is defined, not just when PKG_DEBUG_LEVEL > 1.
sync with HEAD
add wrappers to transform arguments for AIX's ld(1), notably, if we are dynamically linking, the -brtl flag must be passed to the linker so that the runtime linker is used. tested on AIX 4.3 and 5.1 by garbled@.
properly detect icc 8.1, which needs a different argument to statically link the Intel provided libraries.
sync with HEAD
add support for icc, the Intel C++ Compiler (Linux). icc is a high performance compiler suite for Linux/i386 and ia64, and is free for non-commercial use. see: http://www.intel.com/software/products/compilers/clin/ for more information. tested with icc version 8.0 and 8.1 on Slackware 10/i386.
The libtool wrapper should always do the compiler wrapper transformations, so move the setting for _WRAP_ARG_PP.LIBTOOL out of conditional areas of bsd.wrapper.mk and set it globally in bsd.buildlink3.mk with the rest of the libtool wrapper variables.
Teach the libtool wrapper about the same types of transformations that the compiler wrapper already knows to do. This should protect "-install_name ..." from wrapper transformations within the libtool wrapper on Darwin and fixes PR pkg/29215. Software that properly use libtool don't pass an explicit "-install_name ..." because libtool always generates its own such argument when building the (Darwin) shared library. However, not every piece of software, including the ones like lang/tcl that pkgsrc converted to use libtool, will use libtool in the documented fashion. Longer term, those packages should be fixed properly, but we will still need the libtool wrapper to avoid transforming arguments that the compiler wrapper itself doesn't transform, e.g. for MIPSpro and -LANG:*.
sync with HEAD
Split out the argument pre-processing logic into its own script arg-pp-main, and allow for wrapper-specific hooks to extend the pre-processing through _WRAP_ARG_PP.<wrappee>. Move the Darwin GCC-related preprocessing into arg-pp-darwin-gcc, and create a arg-pp-mipspro-cc that understands -LANG:<feature> and -LIST:<feature> so that they're not confused with the usual -Ldir options.
sync with HEAD
Really skip the transformations for imake. This shows the danger in having variables that take a "yes" value to turn off behavior.
Fix error in previous that turned off all transformations in all wrappers. "Oops". We now do transformations by default, and turn them off explicitly for wrappers that don't want them, e.g. IMAKE.
sync with HEAD
Avoid the silly optimization in the make logic and push it into the shell script instead. We can't avoid invoking sed at least once since we need to protect the arguments against shell expansion, but we avoid invoking sed for transformations unless there actually are transformations to perform. The code is simpler, more straightforward, and logically correct now. This fixes problems with using a non-GCC compiler with packages that haven't been converted to use buildlink3, noted in PR pkg/28896.
Merge from HEAD.
add handling for some required -Wl, transformations for packages that assume Apple gcc on Darwin. thanks, Johnny :) this allows perl to build with xlc.
merge from HEAD
Remove the trailing whitespace in the values of the toolchain variables (CC, LD, CXX, etc.) if the default values have no arguments. Now, CC == "cc" and not "cc ".
merge from HEAD
Modify the way that the toolchain variables, e.g. CC, CPP, CXX, LD, etc.,
are handled. The idea now is to simply remove the paths in the values
of these variables, leaving behind only the basename plus any arguments,
e.g.:
CC= /usr/local/bin/gcc becomes CC= gcc
CPP= /usr/local/bin/gcc -E becomes CPP= gcc -E
The wrapper scripts are generated for every unique executable mentioned
by the toolchain variables, so for the example above, only a "gcc"
wrapper script is generated for ${CC} and ${CPP}. PKG_{CC,CPP,CXX,etc.}
are the paths to the executables wrapped by the wrapper scripts.
Note that it's now possible to set "CC" to something more than just the
path to the compiler, e.g.
CC= cc -ffast-math -funroll-loops -fomit-frame-pointer
and the full value of ${CC} will be passed through via CONFIGURE_ENV
and MAKE_ENV.
Factor out package data export bits which are not used in the normal build process into a new fragment, bsd.pkg.data.mk. Fix some include ordering so that most of the dependency logic works correctly again (though fetch-list is still broken).
So that I don't lose all this work, stash the work-in-progress derecursion cleanup onto the tv-derecurse branch (only branched in pkgsrc/mk).
add framework support for Tru64 and the Compaq C compiler. patches provided by Tobias Nygren <tnn at netilium dot org> with minor changes by me.
Insert the default values for WRAPPER_DEBUG and WRAPPER_UPDATE_CACHE as defaults within the wrapper script. This avoids requiring those two variables to be defined in the shell environment for the correct defaults to take effect.
Oy. For some reason, when doing :O:u, -L and -R options are stripped completely (they say "cached" but never come out the other side). So revert previous, hopefully to find a resolution later....
Big speedup hack: Sort and uniqify ${_WRAP_TRANSFORM_CMDS}. Eliminates
many (hundreds, in some packages) multiple identical transformations.
Move -D_ALL_SOURCE for Interix, required for nearly every compilation, from the funky CPPFLAGS assignment into the new wrapper framework.
add initial support for IBM's XL C/C++ compiler. tested with version 6.0 on Mac OS X 10.3.5. to use XL C, set PKGSRC_COMPILER=xlc in mk.conf. XLCBASE defaults to /opt/ibmcmp/vacpp/6.0 (the default installation location on OS X), this can be overridden in mk.conf too. this is a work in progress - some simple packages can be built, but there are still lots of issues that need to be worked through.
Add a scan script that checks for "-v" on the command-line, and prevents the wrappers from adding extra flags to the command-line if "-v" exists. This makes "g++ -v" return the same error code for both the wrapper and the real executable and fixes the problem with the libtool-base build not correctly detecting the C++ compiler and thus not adding the CXX configuration tag to the final libtool script. Downgrade LIBTOOL_REQD to 1.5.10 since the newest version isn't needed with this change.
Move some wrapper definitions into a separate file wrapper-defs.mk that is included by bsd.prefs.mk. This allows the following variables to be used before bsd.wrapper.mk is included: WRAPPER_DIR WRAPPER_SRCDIR WRAPPER_BINDIR WRAPPER_SHELL WRAPPER_TMPDIR
Make _WRAPPEES into a public variable so that we can more easily create wrapper scripts in package-land.
Add Dan Winship's workaround to allow building on Darwin. This is only a temporary fix until I can find time to do this better.
Move @_WRAP_TRANSFORM_SED@ from the logic script into wrapper.sh so that it can be customized per wrappee. This fixes the imake wrapper script to not do any transformations to -I... options.
Initial commit of a new wrapper script framework that encapsulates
the non-buildlink-related code and moves it out of mk/buildlink3 into
mk/wrapper. The buildlink3 code is modified to simply hook its
transformations into the wrapper script framework.
The wrapper script framework has some new features:
* Support automatically passing "ABI" flags to the compiler and linker
depending on the value of ${ABI}. Currently supports the SunPro
compiler with ${ABI} == 64 and the MIPSPro compiler with ${ABI} as
any of 32, n32, o32, and 64.
* making UnixWare GCC accept -rpath options and silently converting
them into an appropriate LD_RUN_PATH
* Add cmd-sink-interix-gcc and cmd-sink-interix-ld that errors out
when it sees -fpic/-fPIC and -shared/-Bshareable, respectively
(requested by <tv>).
* Much improved debugging output. It's possible to output the wrapper
work log in-line with normal output by setting WRAPPER_LOG to
"stderr".
Important differences in behaviour from the old buildlink3 code include:
* Only move the -l options to the end of the command line, leaving the
-L options in-place.
* Extend the autodetection of the libtool mode to detect "compile" and
"uninstall".
* Fix problem noted in both PR pkg/24760 and PR pkg/25500, where
-L/usr/lib/* was being mangled improperly.
* Remove the top-level "buildlink" target; instead, make buildlinking
occur as part of the "wrapper" target.
* mangle and sub-mangle are only meant to transform directories in
-I, -L, and rpath options, so remove the lines in
buildlink3/gen-transform.sh that transformed bare directories.
* Add the ability for the libtool wrapper to be called just to unwrap
an existing libtool archive by running:
libtool --mode=unwrap -o libfoo.la
The old --fix-la syntax no longer works.
20040818
========
* Initial release of a new wrapper script framework that encapsulates
the non-buildlink-related code and moves it out of mk/buildlink3.
These features include:
* making MIPSpro accept GCC options
* making MIPSpro "ucode" accept GCC options
* making SunPro accept GCC options
* making "ld" accept -Wl,* options and silently removing the "-Wl,"
* (NEW) making UnixWare GCC accept -rpath options and silently
converting them into an appropriate LD_RUN_PATH
One major benefit of this is that the buildlink3 code is now much
tighter and easier to understand since it concerns itself solely
with buildlink-related details. I haven't yet optimized the wrapper
cache, so the new wrapper scripts may take slightly longer to execute
than the old buildlink3 wrapper scripts, but I'll be improving this
over time.
20040821
========
* Move the inclusion of $cmd_sink outside of the main loop in wrapper.sh
so that the $cmd_sink script can be used to globally scan and process
the arguments. Move the LD_RUN_PATH code to a cmd-sink-unixware-gcc
script. Garbage-collect the now unused export_vars-related code.
* Add cmd-sink-aix-xlc for AIX xlc that munges -Wl,-R* into an
appropriate -blibpath option.
* Add cmd-sink-interix-gcc and cmd-sink-interix-ld that errors out
when it sees -fpic/-fPIC and -shared/-Bshareable, respectively
(requested by <tv>).
* Move the code that converts full paths to shared libraries into the
"-Ldir -llib" equivalents from the buildlink3 code into wrapper/logic.
Remove the same from bsd.buildlink3.mk and gen-transform.sh.
* Move the code that checks for absolute rpaths from the buildlink3
code into wrapper/arg-source. Remove the same from bsd.buildlink3.mk
and gen-transform.sh.
* Only move the -l options to the end of the command line, leaving the
-L options in-place.
* Add more debugging code.
20040824
========
* Fix quoting problems after arguments are transformed. Remove the
hack that was inserted that magically made almost everything work
because we do it the right way now.
* Move the inclusion of $logic outside of the main loop in wrapper.sh
so that the $logic script doesn't have to worry about underflowing
the argument buffer.
* Encapsulate the loop in wrapper.sh that fills the argument buffer
entirely within the arg-source script.
* Move from the logic script into the arg-source script the
transformations that merge or split arguments.
* Fix bug where skipargs was effectively being ignored if it was more
than 1.
* Handle the whitespace in transformations in the logic script that
turn one library option into multiple library options, e.g.
"-lreadline" -> "-ledit -ltermcap".
* Allow you to specify an environment variable WRAPPER_SKIP_TRANSFORM
for whether you wish to skip the transformation step in the logic
script. This is intended for testing purposes.
* Added check_prog() and init_lib() functions to the shell code library
to make it more reusable outside of the wrapper framework.
* Allow the msg_log() function to output to "stdout" or "stderr". If
you want to have all of the logging appear on the screen, then you
can now set WRAPPER_LOG=stderr.
* Make some of the script components not overridable on a per-wrapper
basis.
* Add a gen-transform.sh script that generates transformation sedfiles.
The "transform" script is used to transform arguments, while the
"untransform" script is used to unwrap files. Move the no-rpath
logic from buildlink3/gen-transform.sh into wrapper/gen-transform.sh
since it's not buildlink3-specific.
* Check for a non-empty blibpath before adding the option in
cmd-sink-aix-xlc.
* Extend the autodetection of the libtool mode to detect "compile" and
"uninstall".
* Add a cmd-sink-libtool script that doesn't pass linker options to
libtool unless we're in "link" mode.
* Set _USE_RPATH to "yes" for UnixWare so that the wrappers will see the
rpath options and convert them to a LD_RUN_PATH definition.
* Add more debugging code.
20040826
========
* Rewrite buildlink3/gen-transform.sh to produce more precise sed commands.
Drop some unused commands from the mini-language, and add a few more
that are more restrictive in their scope.
* Fix problem where repeated options weren't properly handled by some
of sed commands. It's not enough that they're "global replace",
since some patterns match separator characters before and after each
option. We must repeat those patterns twice to catch all instances
correctly.
* Fix problem noted in both PR pkg/24760 and PR pkg/25500, where
-L/usr/lib/* was being mangled improperly.
* Remove the top-level "buildlink" target; instead, make buildlinking
occur as part of the "wrapper" target.
* Add more debugging code.
20040828
========
* Added a head_queue function to shell-lib that returns the head of the
named queue without popping it off the front of the queue.
* Strip consecutive, repeated library options from the command line when
we read it in the logic script.
* Be more careful about not underflowing the argument buffer.
20040906
========
* shell-lib was moved into pkgsrc/mk/scripts; correct references to that
file in the wrapper code.
* Use opt-sub instead of sub-mangle when protecting -I/usr/include/*
and -L/usr/lib/* from buildlink transformations. This avoids adding
lines that look like "-I-I..." in the transformation sedfiles.
* mangle and sub-mangle are only meant to transform directories in
-I, -L, and rpath options, so remove the lines in
buildlink3/gen-transform.sh that transformed bare directories.
* Fix bug in strip-slashdot where the "." wasn't backquoted and thus
matched all characters instead of only the "." character.
* Change the libtool wrapper to use a modified buildcmd script that
doesn't rearrange any of the arguments. This should fix spurious
problems where libtool doesn't understand how to parse the command
line when the -l options are moved to the end of the argument list.
* Fix bug in the logic script where the $cachearg and $cachedarg
weren't being properly set at all times, which caused the cache to
contain the wrong transformed argument.
20040907
========
* Support automatically passing "ABI" flags to the compiler and linker
depending on the value of ${ABI}. Currently supports the SunPro
compiler with ${ABI} == 64 and the MIPSPro compiler with ${ABI} as
any of 32, n32, o32, and 64.
* Move back the code that splits absolute paths to shared libraries
from arg-source back into logic. This allows us to correctly skip
splitting those paths based on the previous option. Also add a
sanity check that the library name in the split argument doesn't
contain a "/" since shell globs are not as precise as REs.
* Don't transform the path given after --dynamic-linker (used by GNU
ld for ELF linkage).
* Add the ability for the libtool wrapper to be called just to unwrap
an existing libtool archive by running:
libtool --mode=unwrap -o libfoo.la
20040914
========
* Add a loop in libtool-fix-la to ensure that all of the options listed
in the dependency_libs lines of *.lai files are processed. This fixes
a buildlink3 leakage bug.
* Merge the gen-transform.sh scripts between buildlink3 and wrapper and
place them all in wrapper. This makes sense since the commands simply
allow for many types of transformations, which buildlink3 takes
advantage of, but there is nothing inherently buildlink-ish about
those commands.
* Don't directly manipulate SUBST_SED.unwrap. Instead, create the
value of SUBST_SED.unwrap by combining several other variables
(currently just _UNWRAP_SED) to ensure that the correct ordering is
preserved.
* Correct some confusing debugging messages.