CVS log for pkgsrc/mk/fetch/fetch.mk
Up to [cvs.NetBSD.org] / pkgsrc / mk / fetch
Request diff between arbitrary revisions
Keyword substitution: kv
Default branch: MAIN
Revision 1.80: download - view: text, markup, annotated - select for diffs
Thu Dec 26 20:23:07 2024 UTC (22 months, 2 weeks ago) by maya
Branches: MAIN
CVS tags: pkgsrc-2026Q3-base,
pkgsrc-2026Q3,
pkgsrc-2026Q2-base,
pkgsrc-2026Q2,
pkgsrc-2026Q1-base,
pkgsrc-2026Q1,
pkgsrc-2025Q4-base,
pkgsrc-2025Q4,
pkgsrc-2025Q3-base,
pkgsrc-2025Q3,
pkgsrc-2025Q2-base,
pkgsrc-2025Q2,
pkgsrc-2025Q1-base,
pkgsrc-2025Q1,
HEAD
Diff to: previous 1.79: preferred, colored
Changes since revision 1.79: +7 -3
lines
Validate TLS when fetching.
This is not a security issue for users of pkgsrc as any downloaded files
are checked against a checksum, but it is for developers are updating a
package and generating the checksum for the first time.
We know this is a safe change to make as some bulk builds are already
running with TLS verification (e.g. the default fetch method on NetBSD 10
validates TLS, and there aren't issues shown by bulk builds).
If an upstream has TLS errors, we can work around it using
`make upload-distfiles`, which will upload it to our local distfile mirror,
and is a default fallback.
PR pkg/58891
Revision 1.79: download - view: text, markup, annotated - select for diffs
Tue Oct 22 06:29:21 2024 UTC (2 years ago) by jperkin
Branches: MAIN
CVS tags: pkgsrc-2024Q4-base,
pkgsrc-2024Q4
Diff to: previous 1.78: preferred, colored
Changes since revision 1.78: +16 -2
lines
mk: Add ARGMAX_REQ support for ARG_MAX limits.
The wip/grafana package now contains so many distfiles that any command
that operates on them now overflows ARG_MAX on NetBSD (262144). This
causes problems both in how bmake handles executing shell commands, as
well as the checksum.awk script that executes digest on them.
Modify checksum.awk to support reading distfiles from stdin, and process
files in batches if a digest command is over 64K in length. This value
strikes a good balance between performance and ensuring even the most
stringent modern ARG_MAX limit is avoided.
Introduce a new package variable ARGMAX_REQ that if set to a value higher
than ARG_MAX, and if we are unable to detect a bmake that has its own fix
for this problem, will use alternate targets that use temporary files for
input.
With this change, and ARGMAX_REQ=524288 set in wip/grafana, NetBSD's
native make is now able to both create and check its distinfo.
As an added benefit, on systems that do not breach the limit and are
running a newer bmake, the distinfo and makesum targets no longer use a
temporary input file.
Revision 1.78: download - view: text, markup, annotated - select for diffs
Mon Oct 14 08:02:40 2024 UTC (2 years, 1 month ago) by jperkin
Branches: MAIN
Diff to: previous 1.77: preferred, colored
Changes since revision 1.77: +2 -1
lines
mk: Provide a default _FETCH_CMD.mktool.
This should never be used, so set to false to slightly improve the error
case where FETCH_USING=mktool but TOOLS_PLATFORM.mktool is not defined.
Revision 1.77: download - view: text, markup, annotated - select for diffs
Fri Oct 11 11:52:23 2024 UTC (2 years, 1 month ago) by jperkin
Branches: MAIN
Diff to: previous 1.76: preferred, colored
Changes since revision 1.76: +16 -1
lines
mk: Add mktool support to fetch.
mktool is an optional external tool, available from pkgtools/mktool or
"cargo install mktool", that provides significant performance improvements
and an improved progress bar compared to other fetch commands.
After installing, set both:
FETCH_USING= mktool
TOOLS_PLATFORM.mktool= /path/to/mktool
to enable it. This is so that an alternate fetch command can still be used
if required, while still using mktool elsewhere in the infrastructure, as
not all network access methods (e.g. proxies) have been fully tested.
When testing the performance of "bmake fetch" inside wip/grafana on an M1
MacBook Pro running macOS, fetching distfiles from http://localhost/ backed
by www/caddy, FETCH_USING=curl took 47m 58s while FETCH_USING=mktool took
just 4 seconds, well over 500x faster!
This can be seen in realtime along with an example of the streamlined
progress bar at <https://asciinema.org/a/S4MWXHLSJmL4GKYAhOBIIHE31>.
Revision 1.76: download - view: text, markup, annotated - select for diffs
Wed Feb 21 10:53:28 2024 UTC (2 years, 8 months ago) by jperkin
Branches: MAIN
CVS tags: pkgsrc-2024Q3-base,
pkgsrc-2024Q3,
pkgsrc-2024Q2-base,
pkgsrc-2024Q2,
pkgsrc-2024Q1-base,
pkgsrc-2024Q1
Diff to: previous 1.75: preferred, colored
Changes since revision 1.75: +2 -2
lines
mk/fetch: Remove -O from curl's FETCH_AFTER_ARGS.
Perhaps this was required in some ancient version of curl, but in any modern
version having both "-o filename" and "-O" (which means to use the remote
filename) only results in "Warning: Got more output options than URLs" messages
for every single download.
Revision 1.75: download - view: text, markup, annotated - select for diffs
Tue Aug 4 21:54:46 2020 UTC (6 years, 3 months ago) by rillig
Branches: MAIN
CVS tags: pkgsrc-2023Q4-base,
pkgsrc-2023Q4,
pkgsrc-2023Q3-base,
pkgsrc-2023Q3,
pkgsrc-2023Q2-base,
pkgsrc-2023Q2,
pkgsrc-2023Q1-base,
pkgsrc-2023Q1,
pkgsrc-2022Q4-base,
pkgsrc-2022Q4,
pkgsrc-2022Q3-base,
pkgsrc-2022Q3,
pkgsrc-2022Q2-base,
pkgsrc-2022Q2,
pkgsrc-2022Q1-base,
pkgsrc-2022Q1,
pkgsrc-2021Q4-base,
pkgsrc-2021Q4,
pkgsrc-2021Q3-base,
pkgsrc-2021Q3,
pkgsrc-2021Q2-base,
pkgsrc-2021Q2,
pkgsrc-2021Q1-base,
pkgsrc-2021Q1,
pkgsrc-2020Q4-base,
pkgsrc-2020Q4,
pkgsrc-2020Q3-base,
pkgsrc-2020Q3
Diff to: previous 1.74: preferred, colored
Changes since revision 1.74: +2 -1
lines
mk/fetch/fetch.mk: add keywords for "bmake help"
Revision 1.74: download - view: text, markup, annotated - select for diffs
Thu Jun 18 17:05:55 2020 UTC (6 years, 4 months ago) by wiz
Branches: MAIN
CVS tags: pkgsrc-2020Q2-base,
pkgsrc-2020Q2
Diff to: previous 1.73: preferred, colored
Changes since revision 1.73: +2 -2
lines
mk: do not check vulnerability when just fetching distfiles
This is intended to reduce the log output on ftp.NetBSD.org when
fetching all distfiles.
Also, we call the target in basically every step of package creation
(extract, patch, configure, build, install, package) - perhaps we should trim
this down some more.
Revision 1.73: download - view: text, markup, annotated - select for diffs
Sat Apr 25 11:36:20 2020 UTC (6 years, 6 months ago) by js
Branches: MAIN
Diff to: previous 1.72: preferred, colored
Changes since revision 1.72: +2 -2
lines
mk/fetch/fetch.mk: Use -f with ofhttp
pkgsrc wants to be able to override files when the checksum didn't
match.
Revision 1.72: download - view: text, markup, annotated - select for diffs
Sat Apr 25 11:21:06 2020 UTC (6 years, 6 months ago) by js
Branches: MAIN
Diff to: previous 1.71: preferred, colored
Changes since revision 1.71: +7 -1
lines
mk/fetch/fetch.mk: Add ofhttp as fetcher
Revision 1.71: download - view: text, markup, annotated - select for diffs
Mon Mar 30 22:44:26 2020 UTC (6 years, 7 months ago) by rillig
Branches: MAIN
Diff to: previous 1.70: preferred, colored
Changes since revision 1.70: +2 -2
lines
mk/fetch/fetch.mk: don't print double-slash in pathname
Revision 1.70: download - view: text, markup, annotated - select for diffs
Thu Jan 11 08:54:37 2018 UTC (8 years, 10 months ago) by alnsn
Branches: MAIN
CVS tags: pkgsrc-2020Q1-base,
pkgsrc-2020Q1,
pkgsrc-2019Q4-base,
pkgsrc-2019Q4,
pkgsrc-2019Q3-base,
pkgsrc-2019Q3,
pkgsrc-2019Q2-base,
pkgsrc-2019Q2,
pkgsrc-2019Q1-base,
pkgsrc-2019Q1,
pkgsrc-2018Q4-base,
pkgsrc-2018Q4,
pkgsrc-2018Q3-base,
pkgsrc-2018Q3,
pkgsrc-2018Q2-base,
pkgsrc-2018Q2,
pkgsrc-2018Q1-base,
pkgsrc-2018Q1
Diff to: previous 1.69: preferred, colored
Changes since revision 1.69: +40 -6
lines
Add FETCH_PROXY and FETCH_PROXY.{ftp,http,https}.
Revision 1.69: download - view: text, markup, annotated - select for diffs
Tue Aug 1 13:15:32 2017 UTC (9 years, 3 months ago) by jperkin
Branches: MAIN
CVS tags: pkgsrc-2017Q4-base,
pkgsrc-2017Q4,
pkgsrc-2017Q3-base,
pkgsrc-2017Q3
Diff to: previous 1.68: preferred, colored
Changes since revision 1.68: +3 -2
lines
Do not perform checksum tests when explicitly calling a make target that
creates or updates the checksum. Allows users to set FAILOVER_FETCH in
mk.conf unconditionally and be able to update distinfo files without
first having to remove them.
Revision 1.68: download - view: text, markup, annotated - select for diffs
Wed Jan 6 07:38:25 2016 UTC (10 years, 10 months ago) by dholland
Branches: MAIN
CVS tags: pkgsrc-2017Q2-base,
pkgsrc-2017Q2,
pkgsrc-2017Q1-base,
pkgsrc-2017Q1,
pkgsrc-2016Q4-base,
pkgsrc-2016Q4,
pkgsrc-2016Q3-base,
pkgsrc-2016Q3,
pkgsrc-2016Q2-base,
pkgsrc-2016Q2,
pkgsrc-2016Q1-base,
pkgsrc-2016Q1
Diff to: previous 1.67: preferred, colored
Changes since revision 1.67: +2 -2
lines
Use :tA to make DISTINFO_FILE absolute when passing it to the fetch
and checksum scripts.
Fixes the problem where DISTINFO_FILE=../../foo/bar/distinfo doesn't
work without having an extraneous and ugly ${.CURDIR} stuffed into it
by hand.
Revision 1.67: download - view: text, markup, annotated - select for diffs
Mon Sep 14 08:27:13 2015 UTC (11 years, 2 months ago) by joerg
Branches: MAIN
CVS tags: pkgsrc-2015Q4-base,
pkgsrc-2015Q4,
pkgsrc-2015Q3-base,
pkgsrc-2015Q3
Diff to: previous 1.66: preferred, colored
Changes since revision 1.66: +3 -3
lines
Reorder and fix logic snafu.
Revision 1.66: download - view: text, markup, annotated - select for diffs
Sun Sep 13 15:03:57 2015 UTC (11 years, 2 months ago) by joerg
Branches: MAIN
Diff to: previous 1.65: preferred, colored
Changes since revision 1.65: +7 -2
lines
Introduce IGNORE_INTERACTIVE_FETCH to allow trying to fetch files from
the override site, even for INTERACTIVE_STATE=stage or FETCH_MESSAGE.
Ignore all backup sites as they won't have the distfile anyway, the
fetch wouldn't be interactive in first place otherwise.
Revision 1.65: download - view: text, markup, annotated - select for diffs
Tue Sep 8 11:13:12 2015 UTC (11 years, 2 months ago) by joerg
Branches: MAIN
Diff to: previous 1.64: preferred, colored
Changes since revision 1.64: +4 -1
lines
Add a hook for running an action after a successful fetch.
This is intended for keeping distfile archives synchronised between
different machines without requiring NFS mounts or the like.
Revision 1.64: download - view: text, markup, annotated - select for diffs
Tue Oct 7 07:27:18 2014 UTC (12 years, 1 month ago) by tron
Branches: MAIN
CVS tags: pkgsrc-2015Q2-base,
pkgsrc-2015Q2,
pkgsrc-2015Q1-base,
pkgsrc-2015Q1,
pkgsrc-2014Q4-base,
pkgsrc-2014Q4
Diff to: previous 1.63: preferred, colored
Changes since revision 1.63: +2 -2
lines
Pass "--remote-time" option to "curl" to get correct time stamps
on downloaded files.
Revision 1.63: download - view: text, markup, annotated - select for diffs
Fri Feb 14 07:43:46 2014 UTC (12 years, 9 months ago) by obache
Branches: MAIN
CVS tags: pkgsrc-2014Q3-base,
pkgsrc-2014Q3,
pkgsrc-2014Q2-base,
pkgsrc-2014Q2,
pkgsrc-2014Q1-base,
pkgsrc-2014Q1
Diff to: previous 1.62: preferred, colored
Changes since revision 1.62: +2 -2
lines
tell wget as a tool.
Revision 1.62: download - view: text, markup, annotated - select for diffs
Sat Nov 16 15:29:39 2013 UTC (13 years ago) by jakllsch
Branches: MAIN
CVS tags: pkgsrc-2013Q4-base,
pkgsrc-2013Q4
Diff to: previous 1.61: preferred, colored
Changes since revision 1.61: +2 -2
lines
Ensure failure on failure of depends-fetch and depends-checksum targets.
Revision 1.61: download - view: text, markup, annotated - select for diffs
Sat Nov 16 03:21:10 2013 UTC (13 years ago) by jakllsch
Branches: MAIN
Diff to: previous 1.60: preferred, colored
Changes since revision 1.60: +2 -2
lines
Use correct verb in depends-fetch target information message.
Revision 1.60: download - view: text, markup, annotated - select for diffs
Sat Feb 16 07:32:28 2013 UTC (13 years, 9 months ago) by obache
Branches: MAIN
CVS tags: pkgsrc-2013Q3-base,
pkgsrc-2013Q3,
pkgsrc-2013Q2-base,
pkgsrc-2013Q2,
pkgsrc-2013Q1-base,
pkgsrc-2013Q1
Diff to: previous 1.59: preferred, colored
Changes since revision 1.59: +2 -1
lines
Remove fetched file if fetch command succeeds to fetch but checksum mismatch
with PKG_RESUME_TRANSFERS=YES.
PR 39896.
Revision 1.59: download - view: text, markup, annotated - select for diffs
Thu Feb 7 13:59:10 2013 UTC (13 years, 9 months ago) by obache
Branches: MAIN
Diff to: previous 1.58: preferred, colored
Changes since revision 1.58: +1 -2
lines
FTPUSERAGENT is a environment variable for ftp(1), not a make variable.
No need to pass here.
related to PR 47525.
Revision 1.58: download - view: text, markup, annotated - select for diffs
Sun Jan 20 13:22:59 2013 UTC (13 years, 9 months ago) by morr
Branches: MAIN
Diff to: previous 1.57: preferred, colored
Changes since revision 1.57: +2 -2
lines
Allow overwritting of FTPUSERAGENT for some broken sites, which requires it.
Revision 1.57: download - view: text, markup, annotated - select for diffs
Fri Dec 14 16:46:24 2012 UTC (13 years, 11 months ago) by ryoon
Branches: MAIN
CVS tags: pkgsrc-2012Q4-base,
pkgsrc-2012Q4
Diff to: previous 1.56: preferred, colored
Changes since revision 1.56: +2 -1
lines
Clear FTPUSERAGENT environment variable.
When FTPUSERAGENT="Mozilla/1.0", for example, fetching from
MASTER_SITE_SOURCEFORGE fails.
Original problem is reported from tsutsui@.
Revision 1.56: download - view: text, markup, annotated - select for diffs
Fri Jul 27 10:48:36 2012 UTC (14 years, 3 months ago) by jperkin
Branches: MAIN
CVS tags: pkgsrc-2012Q3-base,
pkgsrc-2012Q3
Diff to: previous 1.55: preferred, colored
Changes since revision 1.55: +2 -2
lines
Make curl a tool. Add support for /usr/bin/curl, if available, on Solaris.
SmartOS and OmniOS users can now set FETCH_USING=curl and no longer require
pkgsrc dependencies for downloading distfiles.
Revision 1.55: download - view: text, markup, annotated - select for diffs
Mon Feb 6 12:49:38 2012 UTC (14 years, 9 months ago) by fhajny
Branches: MAIN
CVS tags: pkgsrc-2012Q2-base,
pkgsrc-2012Q2,
pkgsrc-2012Q1-base,
pkgsrc-2012Q1
Diff to: previous 1.54: preferred, colored
Changes since revision 1.54: +1 -0
lines
Revert runaway commit
Revision 1.54: download - view: text, markup, annotated - select for diffs
Mon Feb 6 12:41:29 2012 UTC (14 years, 9 months ago) by fhajny
Branches: MAIN
Diff to: previous 1.53: preferred, colored
Changes since revision 1.53: +1 -2
lines
Added devel/ruby-kgio, www/ruby-raindrops and www/ruby-unicorn
Revision 1.53: download - view: text, markup, annotated - select for diffs
Tue Dec 6 01:17:33 2011 UTC (14 years, 11 months ago) by obache
Branches: MAIN
CVS tags: pkgsrc-2011Q4-base,
pkgsrc-2011Q4
Diff to: previous 1.52: preferred, colored
Changes since revision 1.52: +2 -1
lines
Let wget to skip CERT check, same as curl doing.
Revision 1.52: download - view: text, markup, annotated - select for diffs
Thu Sep 8 20:17:16 2011 UTC (15 years, 2 months ago) by abs
Branches: MAIN
CVS tags: pkgsrc-2011Q3-base,
pkgsrc-2011Q3
Diff to: previous 1.51: preferred, colored
Changes since revision 1.51: +3 -3
lines
Introduce PKGSRC_SETENV, defaulting to SETENV. Can be set to ${SETENV} -i
to santise environment
Revision 1.51: download - view: text, markup, annotated - select for diffs
Tue May 17 16:38:18 2011 UTC (15 years, 6 months ago) by mspo
Branches: MAIN
CVS tags: pkgsrc-2011Q2-base,
pkgsrc-2011Q2
Diff to: previous 1.50: preferred, colored
Changes since revision 1.50: +2 -2
lines
he -> the typo in fetch.mk; reported by Snader_LB
Revision 1.50: download - view: text, markup, annotated - select for diffs
Thu Feb 10 10:46:45 2011 UTC (15 years, 9 months ago) by tron
Branches: MAIN
CVS tags: pkgsrc-2011Q1-base,
pkgsrc-2011Q1
Diff to: previous 1.49: preferred, colored
Changes since revision 1.49: +2 -2
lines
Disable certificate checks when using "curl". This fixes the build of
the "youtube-dl" package.
Revision 1.49: download - view: text, markup, annotated - select for diffs
Tue Feb 8 02:45:18 2011 UTC (15 years, 9 months ago) by obache
Branches: MAIN
Diff to: previous 1.48: preferred, colored
Changes since revision 1.48: +11 -4
lines
Add FETCH_USE_IPV4_ONLY option to force connecting to only IPv4 addresses.
Revision 1.48: download - view: text, markup, annotated - select for diffs
Tue Feb 8 02:40:57 2011 UTC (15 years, 9 months ago) by obache
Branches: MAIN
Diff to: previous 1.47: preferred, colored
Changes since revision 1.47: +2 -2
lines
Add `--location' option for curl to follow HTTP redirect.
Revision 1.47: download - view: text, markup, annotated - select for diffs
Tue Feb 8 02:02:32 2011 UTC (15 years, 9 months ago) by obache
Branches: MAIN
Diff to: previous 1.46: preferred, colored
Changes since revision 1.46: +2 -1
lines
Let curl to silent fail with server error.
fixes a part of PR#44528.
Revision 1.46: download - view: text, markup, annotated - select for diffs
Sat Jan 22 02:22:17 2011 UTC (15 years, 9 months ago) by obache
Branches: MAIN
Diff to: previous 1.45: preferred, colored
Changes since revision 1.45: +3 -3
lines
For curl, "=" is not required for longname options with value.
Revision 1.45: download - view: text, markup, annotated - select for diffs
Tue Jan 18 15:20:54 2011 UTC (15 years, 10 months ago) by reed
Branches: MAIN
Diff to: previous 1.44: preferred, colored
Changes since revision 1.44: +2 -2
lines
For PR #44396.
"FETCH_CMD is the program used to fetch files for FETCH_USING=custom."
Revision 1.43.8.1: download - view: text, markup, annotated - select for diffs
Sun Jul 4 07:22:09 2010 UTC (16 years, 4 months ago) by agc
Branches: pkgsrc-2010Q2
Diff to: previous 1.43: preferred, colored; next MAIN 1.44: preferred, colored
Changes since revision 1.43: +13 -5
lines
Pullup previous changes on HEAD to pkgsrc-2010Q2 branch to fix branching
error, and to sync with reality.
Revision 1.44: download - view: text, markup, annotated - select for diffs
Sun Jun 27 17:46:22 2010 UTC (16 years, 4 months ago) by joerg
Branches: MAIN
CVS tags: pkgsrc-2010Q4-base,
pkgsrc-2010Q4,
pkgsrc-2010Q3-base,
pkgsrc-2010Q3
Diff to: previous 1.43: preferred, colored
Changes since revision 1.43: +13 -5
lines
Add FETCH_TIMEOUT option to limit the time connections may hang during
fetch. Requested by spz@, OK by wiz@
Revision 1.43: download - view: text, markup, annotated - select for diffs
Wed Jul 15 09:40:30 2009 UTC (17 years, 4 months ago) by joerg
Branches: MAIN
CVS tags: pkgsrc-2010Q2-base,
pkgsrc-2010Q1-base,
pkgsrc-2010Q1,
pkgsrc-2009Q4-base,
pkgsrc-2009Q4,
pkgsrc-2009Q3-base,
pkgsrc-2009Q3
Branch point for: pkgsrc-2010Q2
Diff to: previous 1.42: preferred, colored
Changes since revision 1.42: +11 -1
lines
Add depends-fetch and depends-checksum. They work like
show-depends-options, e.g. recurse. make fetch-list does not provide the
same functionality as it doesn't deal well with broken mirror.
Revision 1.42: download - view: text, markup, annotated - select for diffs
Tue Mar 17 22:13:36 2009 UTC (17 years, 8 months ago) by rillig
Branches: MAIN
CVS tags: pkgsrc-2009Q2-base,
pkgsrc-2009Q2,
pkgsrc-2009Q1-base,
pkgsrc-2009Q1
Diff to: previous 1.41: preferred, colored
Changes since revision 1.41: +7 -7
lines
Replaced _PKG_SILENT and _PKG_DEBUG with RUN.
Revision 1.41: download - view: text, markup, annotated - select for diffs
Tue Feb 17 10:42:45 2009 UTC (17 years, 9 months ago) by joerg
Branches: MAIN
Diff to: previous 1.40: preferred, colored
Changes since revision 1.40: +2 -2
lines
Fix C&P error. Pointed out by OBATA Akio.
Revision 1.40: download - view: text, markup, annotated - select for diffs
Sat Feb 14 18:51:20 2009 UTC (17 years, 9 months ago) by joerg
Branches: MAIN
Diff to: previous 1.39: preferred, colored
Changes since revision 1.39: +3 -5
lines
Don't use += to append to an empty variable. If PASSIVE_FTP is not
defined, it would still leave the variable with a non-empty value and
that would be passed down to the fetch command when running fetch-list.
Revision 1.39: download - view: text, markup, annotated - select for diffs
Tue Feb 10 19:25:09 2009 UTC (17 years, 9 months ago) by joerg
Branches: MAIN
Diff to: previous 1.38: preferred, colored
Changes since revision 1.38: +2 -2
lines
Optionally request passive FTP for wget.
Revision 1.38: download - view: text, markup, annotated - select for diffs
Tue Feb 10 18:55:54 2009 UTC (17 years, 9 months ago) by joerg
Branches: MAIN
Diff to: previous 1.37: preferred, colored
Changes since revision 1.37: +26 -19
lines
Change the way the fetch process is organised as proposed on tech-pkg:
- Introduce FETCH_USING, which specifies the mechanism to use; possible
values are ftp, fetch, curl, wget, manual and custom.
- Depend on the correct tool/program if not using manual or custom.
- For manual, just use /usr/bin/false to bail out if a distfile is
missing.
- For custom, FETCH_CMD and related variables are used as before.
- Default value is ftp.
Revision 1.37: download - view: text, markup, annotated - select for diffs
Wed Oct 8 15:39:07 2008 UTC (18 years, 1 month ago) by joerg
Branches: MAIN
CVS tags: pkgsrc-2008Q4-base,
pkgsrc-2008Q4
Diff to: previous 1.36: preferred, colored
Changes since revision 1.36: +6 -1
lines
Allow use of fetch as FETCH_CMD.
Revision 1.36: download - view: text, markup, annotated - select for diffs
Wed Sep 24 05:33:17 2008 UTC (18 years, 1 month ago) by obache
Branches: MAIN
CVS tags: pkgsrc-2008Q3-base,
pkgsrc-2008Q3
Diff to: previous 1.35: preferred, colored
Changes since revision 1.35: +2 -2
lines
Fix a typo in comment.
Revision 1.35: download - view: text, markup, annotated - select for diffs
Sun Sep 7 08:17:39 2008 UTC (18 years, 2 months ago) by obache
Branches: MAIN
CVS tags: cube-native-xorg-base,
cube-native-xorg
Diff to: previous 1.34: preferred, colored
Changes since revision 1.34: +7 -1
lines
Add FETCH_*_ARGS templates for curl.
Should be solution for PR 39208.
Revision 1.34: download - view: text, markup, annotated - select for diffs
Thu May 22 16:27:22 2008 UTC (18 years, 5 months ago) by joerg
Branches: MAIN
CVS tags: pkgsrc-2008Q2-base,
pkgsrc-2008Q2,
cwrapper
Diff to: previous 1.33: preferred, colored
Changes since revision 1.33: +1 -25
lines
Move digest dependency into checksum and fetch part.
Make _ALL_FILES and in turn _CKSUMFILES available earlier.
Use it to only depend on digest if _CKSUMFILES is non-empty.
NO_CHECKSUM will now only skip the checksum and checksum-phase
targets. FAILOVER_FETCH will independently check the sums on
distfiles and depend on digest, fixing the remaining issues
originally raised in PR 34914.
Revision 1.33: download - view: text, markup, annotated - select for diffs
Thu Apr 3 14:07:51 2008 UTC (18 years, 7 months ago) by joerg
Branches: MAIN
CVS tags: pkgsrc-2008Q1-base,
pkgsrc-2008Q1
Diff to: previous 1.32: preferred, colored
Changes since revision 1.32: +2 -1
lines
Move handling of pkg_install version into flavor. Add an explicit
phase pkg_install-depends before bootstrap-depends that just tries to
install a new pkg_install if the current version is too old. Still
keep it as bootstrap dependency for the bulk build code.
For NetBSD, PKG_TOOLS_BIN has to be computed in shell code due to a make
bug.
OK: jlam@
Revision 1.32: download - view: text, markup, annotated - select for diffs
Sat Dec 1 11:11:56 2007 UTC (18 years, 11 months ago) by rillig
Branches: MAIN
CVS tags: pkgsrc-2007Q4-base,
pkgsrc-2007Q4
Diff to: previous 1.31: preferred, colored
Changes since revision 1.31: +1 -2
lines
Removed _INTERACTIVE_COOKIE, after asking on tech-pkg if anyone needs
it. It had only been used to change the "nerrors" variable in bulk
builds, but this variable is not used further anyway.
Revision 1.31: download - view: text, markup, annotated - select for diffs
Sat Dec 1 10:56:31 2007 UTC (18 years, 11 months ago) by rillig
Branches: MAIN
Diff to: previous 1.30: preferred, colored
Changes since revision 1.30: +2 -2
lines
If a package defines FETCH_MESSAGE, show that instead of the generic
instructions.
Revision 1.30: download - view: text, markup, annotated - select for diffs
Thu Sep 6 20:54:53 2007 UTC (19 years, 2 months ago) by joerg
Branches: MAIN
CVS tags: pkgsrc-2007Q3-base,
pkgsrc-2007Q3
Diff to: previous 1.29: preferred, colored
Changes since revision 1.29: +6 -1
lines
If DISTDIR points to a non-existing location and a package has neither
PATCHFILES nor DISTFILES, make fetch would complain about an unbuildable
dependency. Fix this by explicitly checking that _ALLFILES is not empty.
Reported by Sean Boudreau.
Revision 1.29: download - view: text, markup, annotated - select for diffs
Tue Aug 14 14:19:19 2007 UTC (19 years, 3 months ago) by joerg
Branches: MAIN
Diff to: previous 1.28: preferred, colored
Changes since revision 1.28: +2 -3
lines
Checksum checks only make sense when distinfo is also present, so
move this into the conditional.
Revision 1.28: download - view: text, markup, annotated - select for diffs
Wed Aug 1 15:24:51 2007 UTC (19 years, 3 months ago) by jlam
Branches: MAIN
Diff to: previous 1.27: preferred, colored
Changes since revision 1.27: +3 -1
lines
Teach mirror-distfiles to use emul-fetch if EMUL_PLATFORMS is defined so
that mirror sites will grab all of the distfiles, not just the ones that
are for the native platform.
Revision 1.27: download - view: text, markup, annotated - select for diffs
Sat Jul 14 05:51:11 2007 UTC (19 years, 4 months ago) by obache
Branches: MAIN
Diff to: previous 1.26: preferred, colored
Changes since revision 1.26: +1 -3
lines
No need to pass "." as directory, it is default value.
Revision 1.26: download - view: text, markup, annotated - select for diffs
Sat Jul 14 05:45:35 2007 UTC (19 years, 4 months ago) by obache
Branches: MAIN
Diff to: previous 1.25: preferred, colored
Changes since revision 1.25: +7 -3
lines
Allow empty DIST_SUBDIR, need to fix PR 35494.
Revision 1.25: download - view: text, markup, annotated - select for diffs
Tue Jun 12 09:54:52 2007 UTC (19 years, 5 months ago) by obache
Branches: MAIN
CVS tags: pkgsrc-2007Q2-base,
pkgsrc-2007Q2
Diff to: previous 1.24: preferred, colored
Changes since revision 1.24: +6 -1
lines
Define some variables for using FETCH_CMD=wget.
Revision 1.24: download - view: text, markup, annotated - select for diffs
Fri Dec 29 20:14:57 2006 UTC (19 years, 10 months ago) by schmonz
Branches: MAIN
CVS tags: pkgsrc-2007Q1-base,
pkgsrc-2007Q1,
pkgsrc-2006Q4-base,
pkgsrc-2006Q4,
jlam-pkgviews-base,
jlam-pkgviews
Diff to: previous 1.23: preferred, colored
Changes since revision 1.23: +2 -1
lines
Silence some longstanding noise in mail/qmail, caused by the fact
that its ${DISTFILES} is also a common ${PATCHFILES}, resulting in
the file being listed twice in ${_ALLFILES}. This one-line change
removes duplicate filenames from ${_ALLFILES}, mirroring an identical
transformation to ${ALLFILES} that used to do the trick. (The fetch
code, obviously, has since changed a bit.)
What the noise looked like:
$ cd pkgsrc/mail/qmail
$ make clean PKG_OPTIONS="netqmail"
make: "/home/build/trees/pkgsrc-current/mk/fetch/fetch.mk" line 133: warning: duplicate script for target "/home/build/trees/pkgsrc-current/distfiles/netqmail-1.05.tar.gz" ignored
make: "/home/build/trees/pkgsrc-current/mk/fetch/fetch.mk" line 133: warning: using previous script for "/home/build/trees/pkgsrc-current/distfiles/netqmail-1.05.tar.gz" defined here
===> Cleaning for qmail-1.03nb12
$
Tested in a bulk build by joerg@.
Revision 1.23: download - view: text, markup, annotated - select for diffs
Sat Nov 25 21:33:39 2006 UTC (19 years, 11 months ago) by jdolecek
Branches: MAIN
Diff to: previous 1.22: preferred, colored
Changes since revision 1.22: +3 -3
lines
set ORS to space for _MASTER_RAND_AWK, so that the output would be the
in the format MASTER_SORT_REGEX expects when MASTER_SORT_RANDOM is off,
in order for MASTER_SORT to actually work when MASTER_SORT_RANDOM=YES
(default nowadays)
XXX testing stuff before committing would surely be a good idea
Revision 1.22: download - view: text, markup, annotated - select for diffs
Sun Oct 8 11:37:38 2006 UTC (20 years, 1 month ago) by mishka
Branches: MAIN
Diff to: previous 1.21: preferred, colored
Changes since revision 1.21: +21 -7
lines
pkg/34695: Static list of package master sites may (and often will)
lead to overloads of very first distribution site. Moreover, if first
site in the list is not available (often seen for sourceforge mirrors)
you have to wait for timeout each time. To distribute load on master
distribution sites and to make second problem not so annoying randomly
intermix list of MASTER_SITES with MASTER_SORT_RANDOM feature. Any of
MASTER_SORT and MASTER_SORT_REGEX can be applied later.
The feature is turned ON by default and is disabled for PKG_DEVELOPERs
or if MASTER_SORT_RANDOM=no.
Revision 1.21: download - view: text, markup, annotated - select for diffs
Sat Oct 7 12:22:06 2006 UTC (20 years, 1 month ago) by rillig
Branches: MAIN
Diff to: previous 1.20: preferred, colored
Changes since revision 1.20: +5 -5
lines
Meta-packages that had DIST_SUBDIR set (for whatever reason) tried to
fetch files with an empty name, for example "geda/".
The problem was that the expression ${LIST:S,^,${DIST_SUBDIR}/,} results
in ${DIST_SUBDIR}/ when ${LIST} is empty. This is surprising but matches
the documentation in the manual page, so this cannot be called a bug.
The proper fix is to use ${LIST:@f@${DIST_SUBDIR}/${f}@} instead.
Noticed by Don Woodstock on #netbsd-code.
Revision 1.20: download - view: text, markup, annotated - select for diffs
Thu Jul 27 18:48:03 2006 UTC (20 years, 3 months ago) by jlam
Branches: MAIN
CVS tags: pkgsrc-2006Q3-base,
pkgsrc-2006Q3
Diff to: previous 1.19: preferred, colored
Changes since revision 1.19: +4 -7
lines
Rename "SITES_* to "SITES.*" for file-specific lists of sites from which
to fetch the file. This completes the renaming described in revision
1.1799 of bsd.pkg.mk.
Revision 1.19: download - view: text, markup, annotated - select for diffs
Thu Jul 27 16:06:27 2006 UTC (20 years, 3 months ago) by jlam
Branches: MAIN
Diff to: previous 1.18: preferred, colored
Changes since revision 1.18: +5 -5
lines
All of the other *SITES* variables are plural, so rename DYNAMIC_SITE_*
to DYNAMIC_SITES_* for consistency.
Revision 1.18: download - view: text, markup, annotated - select for diffs
Thu Jul 27 15:16:44 2006 UTC (20 years, 3 months ago) by jlam
Branches: MAIN
Diff to: previous 1.17: preferred, colored
Changes since revision 1.17: +5 -3
lines
Apply the "convention over configuration" principle:
If ${FILESDIR}/getsite.sh exists, then use it to determine the fetch
URL for each of the distfiles for the package. Otherwise, use
SITE_<file> and MASTER_SITES, in order, to determine the URL for each
distfile.
If the script path differs from ${FILESDIR}/getsite.sh, then set
DYNAMIC_SITE_SCRIPT to the full path to that script.
Remove the need to set DYNAMIC_MASTER_SITES explicitly in the package
Makefile for:
graphics/ns-cult3d
wm/sawfish-themes
www/apache-tomcat55
www/jakarta-tomcat4
www/jakarta-tomcat5
Revision 1.17: download - view: text, markup, annotated - select for diffs
Thu Jul 27 07:41:40 2006 UTC (20 years, 3 months ago) by rillig
Branches: MAIN
Diff to: previous 1.16: preferred, colored
Changes since revision 1.16: +12 -14
lines
Documented DIST_PATH and fixed two typos (${TARGET} is not ${.TARGET}).
Revision 1.16: download - view: text, markup, annotated - select for diffs
Wed Jul 19 14:25:59 2006 UTC (20 years, 4 months ago) by jlam
Branches: MAIN
Diff to: previous 1.15: preferred, colored
Changes since revision 1.15: +2 -2
lines
Only pass -v to the fetch script if PKG_VERBOSE is set.
Revision 1.15: download - view: text, markup, annotated - select for diffs
Wed Jul 19 14:22:47 2006 UTC (20 years, 4 months ago) by jlam
Branches: MAIN
Diff to: previous 1.14: preferred, colored
Changes since revision 1.14: +5 -3
lines
Restore original quoting from old fetch.mk for _ORDERED_SITES to resolve
the problem with MASTER_SORT*.
XXX The whole MASTER_SORT* thing should just be a standalone script.
Revision 1.14: download - view: text, markup, annotated - select for diffs
Wed Jul 19 14:03:59 2006 UTC (20 years, 4 months ago) by jlam
Branches: MAIN
Diff to: previous 1.13: preferred, colored
Changes since revision 1.13: +10 -10
lines
Make some variables "private":
MASTER_SORT_AWK
SORT_SITES_CMD
ORDERED_SITES
These variables are purely internal to pkgsrc.
Revision 1.13: download - view: text, markup, annotated - select for diffs
Wed Jul 19 14:01:29 2006 UTC (20 years, 4 months ago) by jlam
Branches: MAIN
Diff to: previous 1.12: preferred, colored
Changes since revision 1.12: +2 -2
lines
Work around incomprehensible quoting mess surrounding MASTER_SORT* by
in do-fetch-file (formerly they were inserted by the old _FETCH_FILE
inserting some quotes that are no longer added by the fetch command list
macro). This makes fetching with MASTER_SORT* set work again.
Revision 1.12: download - view: text, markup, annotated - select for diffs
Wed Jul 19 13:33:42 2006 UTC (20 years, 4 months ago) by jlam
Branches: MAIN
Diff to: previous 1.11: preferred, colored
Changes since revision 1.11: +7 -8
lines
PASSIVE_FETCH only affects "ftp", so move it to the ftp-related FETCH_*
variables.
Revision 1.11: download - view: text, markup, annotated - select for diffs
Tue Jul 18 23:27:14 2006 UTC (20 years, 4 months ago) by jlam
Branches: MAIN
Diff to: previous 1.10: preferred, colored
Changes since revision 1.10: +1 -4
lines
Convert _FETCH_MESSAGE to plain old FETCH_MESSAGE. Remove _FETCH_MESSAGE
processing from mk/fetch/*.mk.
Revision 1.10: download - view: text, markup, annotated - select for diffs
Tue Jul 18 22:41:06 2006 UTC (20 years, 4 months ago) by jlam
Branches: MAIN
Diff to: previous 1.9: preferred, colored
Changes since revision 1.9: +200 -338
lines
Overhaul fetch module:
* All the smarts is now encapsulated in the "fetch" script. The fetch
script understands how to use the distinfo file (if specified) to
look up the size and checksums of the file to fetch and will use
that information to verify checksums of the fetched files or resume
transfers of interrupted fetches.
* Move the default settings for FETCH_RESUME_ARGS and FETCH_OUTPUT_ARGS
for "ftp" from mk/defaults/mk.conf into mk/fetch/fetch.mk. We rewrite
it to avoid needing conditional statements.
* Avoid spawning a new make(1) process just to mirror a distfile.
* Split out fetch-list targets into a separate file fetch-list.mk.
These targets should probably be moved into a standalone script.
* Fix distclean target to properly remove partial downloads.
Revision 1.9: download - view: text, markup, annotated - select for diffs
Mon Jul 17 02:13:11 2006 UTC (20 years, 4 months ago) by schmonz
Branches: MAIN
Diff to: previous 1.8: preferred, colored
Changes since revision 1.8: +2 -1
lines
Remove duplicate files from ${ALLFILES}. This doesn't affect the
"fetch" target, but it does affect the "checksum" target's shell
script, which errors out on the second occurrence of a file. The
shell script should perhaps also be fixed, but it seems sensible
regardless for ${ALLFILES} not to contain duplicate filenames. As
a side effect, the file list is sorted.
Regression found by building mail/qmail with the "qmail-netqmail"
option, which adds to PATCHFILES a file that's already in ${DISTFILES}.
Arguably this is gross, but it worked before, and now works again.
Tested on my usual pkg_comp(8) build of 200+ packages, with an
initially empty ${DISTDIR} and ${PACKAGES}. Thanks seb@ for the
more idiomatic make(1) construction.
Revision 1.8: download - view: text, markup, annotated - select for diffs
Sat Jul 15 23:58:52 2006 UTC (20 years, 4 months ago) by rillig
Branches: MAIN
Diff to: previous 1.7: preferred, colored
Changes since revision 1.7: +2 -2
lines
Split the variable BUILD_DEFS into those that are defined by packages
and those that are defined by the infrastructure (_BUILD_DEFS). This
allows the build-defs-message target to be moved to the end of
bsd.pkg.mk. Now it prints the correct result even in unprivileged
builds, which had been wrong due to the order in which the files have
been included. For example, ${UNPRIVILEGED_USER} was displayed as (not
defined) although its value was defined, which could be checked with
"bmake show-var".
Tested with one package that _does_ define BUILD_DEFS and with one that
doesn't. The behavior stays the same.
Revision 1.7: download - view: text, markup, annotated - select for diffs
Thu Jul 13 15:23:18 2006 UTC (20 years, 4 months ago) by jlam
Branches: MAIN
Diff to: previous 1.6: preferred, colored
Changes since revision 1.6: +2 -2
lines
DIGEST is no longer defined, so use TOOLS_DIGEST instead.
Revision 1.6: download - view: text, markup, annotated - select for diffs
Thu Jul 13 14:02:34 2006 UTC (20 years, 4 months ago) by jlam
Branches: MAIN
Diff to: previous 1.5: preferred, colored
Changes since revision 1.5: +3 -3
lines
* Add a new stage "bootstrap-depends" that happens before all other
stages, and that installs dependencies listed in BOOTSTRAP_DEPENDS.
The bootstrap-depends step works just like the normal depends step
and honors the value of DEPENDS_TARGET. It's now possible to add
dependencies solely to facilitate fetching the distfiles, e.g.
BOOTSTRAP_DEPENDS+= curl-[0-9]*:../../www/curl
* Teach the tools framework about ":bootstrap" as a tools modifier
which indicates the tool should be added as a dependency via
BOOTSTRAP_DEPENDS.
* Add "digest" to the tools framework.
* Use USE_TOOLS+=digest:bootstrap to force pkgsrc to install digest
before anything else. Get rid of unused "uptodate-digest" target
and related digest version-checking code.
* Finish the refactoring work: split checksum-related code out of
bsd.pkg.mk and into pkgsrc/mk/checksum and replace the "checksum"
target command list with a script that does all the real work.
* Make DIGEST_ALGORITHMS and PATCH_DIGEST_ALGORITHM into private
variables by prepending them with an underscore. Also, rename
_PATCH_DIGEST_ALGORITHM to _PATCH_DIGEST_ALGORITHMS and adjust the
makepatchsum target to allow that variable to contain a list of
algorithms, all of which are used when creating the patch checksums
for ${DISTINFO_FILE}.
Revision 1.5: download - view: text, markup, annotated - select for diffs
Wed Jul 5 09:08:35 2006 UTC (20 years, 4 months ago) by jlam
Branches: MAIN
Diff to: previous 1.4: preferred, colored
Changes since revision 1.4: +2 -20
lines
Make the check-vulnerable target more self-sufficient, by moving some
of the logic from fetch/fetch.mk into flavor/pkg/check.mk, so that
check-vulnerable can be used as a source target.
Make check-vulnerable a source target for every phase of the build
workflow, which ensures that it is always run if the user starts a
new phase from the command line.
Fix the cookie-generation targets so that they don't append, only
overwrite to the cookie file. This works around potential problems
due to recursive makes.
Move the cookie checks so that they surround the corresponding phase
target. The presence of the cookie should now inform the make process
to avoid doing any processing of phases that occur before the phase
corresponding to the cookie.
Revision 1.4: download - view: text, markup, annotated - select for diffs
Tue Jun 20 14:54:03 2006 UTC (20 years, 5 months ago) by jlam
Branches: MAIN
CVS tags: pkgsrc-2006Q2-base,
pkgsrc-2006Q2
Diff to: previous 1.3: preferred, colored
Changes since revision 1.3: +2 -2
lines
If FAILOVER_FETCH is defined, then we need the digest binary to already
be installed, so let uptodate-digest do that crazy thing it does.
Revision 1.3: download - view: text, markup, annotated - select for diffs
Tue Jun 20 14:50:27 2006 UTC (20 years, 5 months ago) by jlam
Branches: MAIN
Diff to: previous 1.2: preferred, colored
Changes since revision 1.2: +2 -2
lines
Fix FAILOVER_FETCH usage to actually match documentation -- it's
supposed to be just defined/undefined, but was previously being used
as non-empty/emtpy.
Revision 1.2: download - view: text, markup, annotated - select for diffs
Tue Jun 6 03:36:00 2006 UTC (20 years, 5 months ago) by jlam
Branches: MAIN
Diff to: previous 1.1: preferred, colored
Changes since revision 1.1: +5 -5
lines
Improve the warning message if the vulnerabilities file is not found.
Revision 1.1: download - view: text, markup, annotated - select for diffs
Tue Jun 6 03:05:48 2006 UTC (20 years, 5 months ago) by jlam
Branches: MAIN
Refactor "fetch" and "extract" code into correspondingly named
subdirectories of pkgsrc/mk. Move the following files around for
locality:
pkgsrc/mk/scripts/extract -> pkgsrc/mk/extract/extract
pkgsrc/mk/bsd.sites.mk -> pkgsrc/mk/fetch/sites.mk
Also get rid of the recursive make for the "fetch" and "extract"
targets. This basically merges the "fetch" and "extract" phases into
the "patch" phase.
There is still much more work to do to simplify the fetch code, but
this is a good start.
CVSweb <webmaster@jp.NetBSD.org>