Annotation of htdocs/docs/pkgsrc/binary.html, revision 1.1

1.1     ! dsieger     1: <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
        !             2: <html>
        !             3: <head>
        !             4: <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
        !             5: <title>Chapter 6. Creating binary packages</title>
        !             6: <link rel="stylesheet" href="/NetBSD.css" type="text/css">
        !             7: <meta name="generator" content="DocBook XSL Stylesheets VX.X.X">
        !             8: <link rel="start" href="index.html" title="The pkgsrc guide">
        !             9: <link rel="up" href="users-guide.html" title="Part I. The pkgsrc user's guide">
        !            10: <link rel="prev" href="configuring.html" title="Chapter 5. Configuring pkgsrc">
        !            11: <link rel="next" href="files.html" title="Chapter 7. Directory layout of the installed files">
        !            12: </head>
        !            13: <body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
        !            14: <div class="navheader">
        !            15: <table width="100%" summary="Navigation header">
        !            16: <tr><th colspan="3" align="center">Chapter 6. Creating binary packages</th></tr>
        !            17: <tr>
        !            18: <td width="20%" align="left">
        !            19: <a accesskey="p" href="configuring.html">Prev</a> </td>
        !            20: <th width="60%" align="center">Part I. The pkgsrc user's guide</th>
        !            21: <td width="20%" align="right"> <a accesskey="n" href="files.html">Next</a>
        !            22: </td>
        !            23: </tr>
        !            24: </table>
        !            25: <hr>
        !            26: </div>
        !            27: <div class="chapter" lang="en">
        !            28: <div class="titlepage"><div><div><h2 class="title">
        !            29: <a name="binary"></a>Chapter 6. Creating binary packages</h2></div></div></div>
        !            30: <div class="toc">
        !            31: <p><b>Table of Contents</b></p>
        !            32: <dl>
        !            33: <dt><span class="sect1"><a href="binary.html#building-a-single-binary-package">6.1. Building a single binary package</a></span></dt>
        !            34: <dt><span class="sect1"><a href="binary.html#settings-for-creationg-of-binary-packages">6.2. Settings for creation of binary packages</a></span></dt>
        !            35: <dt><span class="sect1"><a href="binary.html#bulkbuild">6.3. Doing a bulk build of all packages</a></span></dt>
        !            36: <dd><dl>
        !            37: <dt><span class="sect2"><a href="binary.html#binary.configuration">6.3.1. Configuration</a></span></dt>
        !            38: <dt><span class="sect2"><a href="binary.html#other-environmental-considerations">6.3.2. Other environmental considerations</a></span></dt>
        !            39: <dt><span class="sect2"><a href="binary.html#operation">6.3.3. Operation</a></span></dt>
        !            40: <dt><span class="sect2"><a href="binary.html#what-it-does">6.3.4. What it does</a></span></dt>
        !            41: <dt><span class="sect2"><a href="binary.html#disk-space-requirements">6.3.5. Disk space requirements</a></span></dt>
        !            42: <dt><span class="sect2"><a href="binary.html#setting-up-a-sandbox">6.3.6. Setting up a sandbox for chrooted builds</a></span></dt>
        !            43: <dt><span class="sect2"><a href="binary.html#building-a-partial-set">6.3.7. Building a partial set of packages</a></span></dt>
        !            44: <dt><span class="sect2"><a href="binary.html#bulk-upload">6.3.8. Uploading results of a bulk build</a></span></dt>
        !            45: </dl></dd>
        !            46: <dt><span class="sect1"><a href="binary.html#creating-cdroms">6.4. Creating a multiple CD-ROM packages collection</a></span></dt>
        !            47: <dd><dl><dt><span class="sect2"><a href="binary.html#cdpack-example">6.4.1. Example of cdpack</a></span></dt></dl></dd>
        !            48: </dl>
        !            49: </div>
        !            50: <div class="sect1" lang="en">
        !            51: <div class="titlepage"><div><div><h2 class="title" style="clear: both">
        !            52: <a name="building-a-single-binary-package"></a>6.1. Building a single binary package</h2></div></div></div>
        !            53: <p>Once you have built and installed a package, you can create
        !            54:     a <span class="emphasis"><em>binary package</em></span> which can be installed on
        !            55:     another system with <a href="http://netbsd.gw.com/cgi-bin/man-cgi?pkg_add+1+NetBSD-current"><span class="citerefentry"><span class="refentrytitle">pkg_add</span>(1)</span></a>. This saves having to build
        !            56:     the same package on a group of hosts and wasting CPU time. It also
        !            57:     provides a simple means for others to install your package, should
        !            58:     you distribute it.</p>
        !            59: <p>To create a binary package, change into the appropriate
        !            60:     directory in pkgsrc, and run <span><strong class="command">make
        !            61:     package</strong></span>:</p>
        !            62: <pre class="screen">
        !            63: <code class="prompt">#</code> <strong class="userinput"><code>cd misc/figlet</code></strong>
        !            64: <code class="prompt">#</code> <strong class="userinput"><code>make package</code></strong>
        !            65:     </pre>
        !            66: <p>This will build and install your package (if not already done),
        !            67:     and then build a binary package from what was installed. You can
        !            68:     then use the <span><strong class="command">pkg_*</strong></span> tools to manipulate
        !            69:     it. Binary packages are created by default in
        !            70:     <code class="filename">/usr/pkgsrc/packages</code>, in the form of a
        !            71:     gzipped tar file. See <a href="logs.html#logs.package" title="B.2. Packaging figlet">Section B.2, &#8220;Packaging figlet&#8221;</a> for a
        !            72:     continuation of the above <a href="ftp://ftp.NetBSD.org/pub/pkgsrc/current/pkgsrc/misc/figlet/README.html" target="_top"><code class="filename">misc/figlet</code></a> example.</p>
        !            73: <p>See <a href="submit.html" title="Chapter 20. Submitting and Committing">Chapter 20, <i>Submitting and Committing</i></a> for information on how to submit
        !            74:     such a binary package.</p>
        !            75: </div>
        !            76: <div class="sect1" lang="en">
        !            77: <div class="titlepage"><div><div><h2 class="title" style="clear: both">
        !            78: <a name="settings-for-creationg-of-binary-packages"></a>6.2. Settings for creation of binary packages</h2></div></div></div>
        !            79: <p>See <a href="build.html#build.helpful-targets" title="16.17. Other helpful targets">Section 16.17, &#8220;Other helpful targets&#8221;</a>.</p>
        !            80: </div>
        !            81: <div class="sect1" lang="en">
        !            82: <div class="titlepage"><div><div><h2 class="title" style="clear: both">
        !            83: <a name="bulkbuild"></a>6.3. Doing a bulk build of all packages</h2></div></div></div>
        !            84: <p>If you want to get a full set of precompiled binary
        !            85:     packages, this section describes how to get them. Beware that
        !            86:     the bulk build will remove all currently installed packages from
        !            87:     your system!</p>
        !            88: <p>Having an FTP server configured either on the
        !            89:     machine doing the bulk builds or on a nearby NFS server can help
        !            90:     to make the packages available to other machines that can then
        !            91:     save time by installing only the binary packages. See <a href="http://netbsd.gw.com/cgi-bin/man-cgi?ftpd+8+NetBSD-current"><span class="citerefentry"><span class="refentrytitle">ftpd</span>(8)</span></a> for
        !            92:     more information. If you use a remote NFS server's storage, be
        !            93:     sure to not actually compile on NFS storage, as this slows
        !            94:     things down a lot.</p>
        !            95: <div class="sect2" lang="en">
        !            96: <div class="titlepage"><div><div><h3 class="title">
        !            97: <a name="binary.configuration"></a>6.3.1. Configuration</h3></div></div></div>
        !            98: <div class="sect3" lang="en">
        !            99: <div class="titlepage"><div><div><h4 class="title">
        !           100: <a name="binary.bulk.build.conf"></a>6.3.1.1. <code class="filename">build.conf</code>
        !           101: </h4></div></div></div>
        !           102: <p>The <code class="filename">build.conf</code> file is the main
        !           103:        configuration file for bulk builds. You can configure how your
        !           104:        copy of pkgsrc is kept up to date, how the distfiles are
        !           105:        downloaded, how the packages are built and how the report is
        !           106:        generated. You can find an annotated example file in
        !           107:        <code class="filename">pkgsrc/mk/bulk/build.conf-example</code>. To use
        !           108:        it, copy <code class="filename">build.conf-example</code> to
        !           109:        <code class="filename">build.conf</code> and edit it, following the
        !           110:        comments in that file.</p>
        !           111: </div>
        !           112: <div class="sect3" lang="en">
        !           113: <div class="titlepage"><div><div><h4 class="title">
        !           114: <a name="binary.mk.conf"></a>6.3.1.2. /etc/mk.conf</h4></div></div></div>
        !           115: <p>You may want to set variables in
        !           116:        <code class="filename">/etc/mk.conf</code>.
        !           117:        Look at <code class="filename">pkgsrc/mk/defaults/mk.conf</code> for
        !           118:        details of the default settings. You will want to ensure that
        !           119:        <code class="varname">ACCEPTABLE_LICENSES</code> meet your local policy.
        !           120:        As used in this example, <code class="varname">_ACCEPTABLE=yes</code>
        !           121:        accepts <span class="emphasis"><em>all</em></span> licenses.</p>
        !           122: <pre class="programlisting">
        !           123: PACKAGES?=      ${_PKGSRCDIR}/packages/${MACHINE_ARCH}
        !           124: WRKOBJDIR?=     /usr/tmp/pkgsrc   # build here instead of in pkgsrc
        !           125: BSDSRCDIR=      /usr/src
        !           126: BSDXSRCDIR=     /usr/xsrc         # for x11/xservers
        !           127: OBJHOSTNAME?=   yes               # use work.`hostname`
        !           128: FAILOVER_FETCH= yes               # insist on the correct checksum
        !           129: PKG_DEVELOPER?= yes
        !           130: _ACCEPTABLE=    yes
        !           131: </pre>
        !           132: <p>Some options that are especially useful for bulk builds
        !           133:        can be found at the top lines of the file
        !           134:        <code class="filename">mk/bulk/bsd.bulk-pkg.mk</code>. The most useful
        !           135:        options of these are briefly described here.</p>
        !           136: <div class="itemizedlist"><ul type="disc">
        !           137: <li><p>If you are on a slow machine, you may want to
        !           138:          set <code class="varname">USE_BULK_BROKEN_CHECK</code> to
        !           139:          &#8220;<span class="quote">no</span>&#8221;.</p></li>
        !           140: <li><p>If you are doing bulk builds from a read-only
        !           141:          copy of pkgsrc, you have to set <code class="varname">BULKFILESDIR</code>
        !           142:          to the directory where all log files are created. Otherwise the
        !           143:          log files are created in the pkgsrc directory.</p></li>
        !           144: <li><p>Another important variable is
        !           145:          <code class="varname">BULK_PREREQ</code>, which is a list of packages that
        !           146:          should be always available while building other
        !           147:          packages.</p></li>
        !           148: </ul></div>
        !           149: <p>Some other options are scattered in the pkgsrc
        !           150:        infrastructure:</p>
        !           151: <div class="itemizedlist"><ul type="disc">
        !           152: <li><p><code class="varname">ALLOW_VULNERABLE_PACKAGES</code>
        !           153:          should be set to <code class="literal">yes</code>. The purpose of the bulk
        !           154:          builds is creating binary packages, no matter if they are
        !           155:          vulnerable or not. When uploading the packages to a public
        !           156:          server, the vulnerable packages will be put into a directory of
        !           157:          their own. Leaving this variable unset would prevent the bulk
        !           158:          build system from even trying to build them, so possible
        !           159:          building errors would not show up.</p></li>
        !           160: <li><p><code class="varname">CHECK_FILES</code>
        !           161:          (<code class="filename">pkgsrc/mk/check/check-files.mk</code>) can be set to
        !           162:          &#8220;<span class="quote">yes</span>&#8221; to check that the installed set of files
        !           163:          matches the <code class="filename">PLIST</code>.</p></li>
        !           164: <li><p><code class="varname">CHECK_INTERPRETER</code>
        !           165:          (<code class="filename">pkgsrc/mk/check/check-interpreter.mk</code>) can be set to
        !           166:          &#8220;<span class="quote">yes</span>&#8221; to check that the installed
        !           167:          &#8220;<span class="quote">#!</span>&#8221;-scripts will find their
        !           168:          interpreter.</p></li>
        !           169: <li><p><code class="varname">PKGSRC_RUN_TEST</code> can be
        !           170:          set to &#8220;<span class="quote"><code class="literal">yes</code></span>&#8221; to run each
        !           171:          package's self-test before installing it. Note that some
        !           172:          packages make heavy use of &#8220;<span class="quote">good</span>&#8221; random
        !           173:          numbers, so you need to assure that the machine on which you
        !           174:          are doing the bulk builds is not completely idle. Otherwise
        !           175:          some test programs will seem to hang, while they are just
        !           176:          waiting for new random data to be
        !           177:          available.</p></li>
        !           178: </ul></div>
        !           179: </div>
        !           180: <div class="sect3" lang="en">
        !           181: <div class="titlepage"><div><div><h4 class="title">
        !           182: <a name="pre-build.local"></a>6.3.1.3. <code class="filename">pre-build.local</code>
        !           183: </h4></div></div></div>
        !           184: <p>It is possible to configure the bulk build to perform
        !           185:        certain site-specific tasks at the end of the pre-build
        !           186:        stage.  If the file
        !           187:        <code class="filename">pre-build.local</code> exists in
        !           188:        <code class="filename">/usr/pkgsrc/mk/bulk</code>, it will be executed
        !           189:        (as a <a href="http://netbsd.gw.com/cgi-bin/man-cgi?sh+1+NetBSD-current"><span class="citerefentry"><span class="refentrytitle">sh</span>(1)</span></a> script) at the end of the usual pre-build
        !           190:        stage.  An example use of
        !           191:        <code class="filename">pre-build.local</code> is to have the line:</p>
        !           192: <pre class="screen">echo "I do not have enough disk space to build this pig." \
        !           193:        &gt; misc/openoffice/$BROKENF</pre>
        !           194: <p>to prevent the system from trying to build a particular package
        !           195:        which requires nearly 3 GB of disk space.</p>
        !           196: </div>
        !           197: </div>
        !           198: <div class="sect2" lang="en">
        !           199: <div class="titlepage"><div><div><h3 class="title">
        !           200: <a name="other-environmental-considerations"></a>6.3.2. Other environmental considerations</h3></div></div></div>
        !           201: <p>As <code class="filename">/usr/pkg</code> will be completely
        !           202:       deleted at the start of bulk builds, make sure your login
        !           203:       shell is placed somewhere else. Either drop it into
        !           204:       <code class="filename">/usr/local/bin</code> (and adjust your login
        !           205:       shell in the passwd file), or (re-)install it via
        !           206:       <a href="http://netbsd.gw.com/cgi-bin/man-cgi?pkg_add+1+NetBSD-current"><span class="citerefentry"><span class="refentrytitle">pkg_add</span>(1)</span></a> from <code class="filename">/etc/rc.local</code>, so
        !           207:       you can login after a reboot (remember that your current
        !           208:       process won't die if the package is removed, you just can't
        !           209:       start any new instances of the shell any more).  Also, if you
        !           210:       use NetBSD earlier than 1.5, or you still want to use the pkgsrc
        !           211:       version of ssh for some reason, be sure to install ssh before
        !           212:       starting it from <code class="filename">rc.local</code>:</p>
        !           213: <pre class="programlisting">
        !           214: (cd /usr/pkgsrc/security/ssh &amp;&amp; make bulk-install)
        !           215: if [ -f /usr/pkg/etc/rc.d/sshd ]; then
        !           216:   /usr/pkg/etc/rc.d/sshd
        !           217: fi
        !           218: </pre>
        !           219: <p>Not doing so will result in you being not able to log in
        !           220:       via ssh after the bulk build is finished or if the machine
        !           221:       gets rebooted or crashes. You have been warned! :)</p>
        !           222: </div>
        !           223: <div class="sect2" lang="en">
        !           224: <div class="titlepage"><div><div><h3 class="title">
        !           225: <a name="operation"></a>6.3.3. Operation</h3></div></div></div>
        !           226: <p>Make sure you don't need any of the packages still
        !           227:       installed.</p>
        !           228: <div class="warning" style="margin-left: 0.5in; margin-right: 0.5in;">
        !           229: <h3 class="title">Warning</h3>
        !           230: <p>During the bulk build, <span class="emphasis"><em>all packages, their
        !           231:        configuration files and some more files from
        !           232:        <code class="filename">/var</code>, <code class="filename">/home</code> and
        !           233:        possibly other locations will be removed! So don't run a bulk
        !           234:        build with privileges that might harm your
        !           235:        system.</em></span></p>
        !           236: </div>
        !           237: <p>Be sure to remove all other things that might
        !           238:       interfere with builds, like some libs installed in
        !           239:       <code class="filename">/usr/local</code>, etc. then become root and type:</p>
        !           240: <pre class="screen">
        !           241: <code class="prompt">#</code> <strong class="userinput"><code>cd /usr/pkgsrc</code></strong>
        !           242: <code class="prompt">#</code> <strong class="userinput"><code>sh mk/bulk/build</code></strong>
        !           243:       </pre>
        !           244: <p>If for some reason your last build didn't complete (power
        !           245:       failure, system panic, ...), you can continue it by
        !           246:       running:</p>
        !           247: <pre class="screen"><code class="prompt">#</code> <strong class="userinput"><code>sh mk/bulk/build restart</code></strong></pre>
        !           248: <p>At the end of the bulk build, you will get a summary via mail,
        !           249:       and find build logs in the directory specified by
        !           250:       <code class="varname">FTP</code> in the <code class="filename">build.conf</code>
        !           251:       file.</p>
        !           252: </div>
        !           253: <div class="sect2" lang="en">
        !           254: <div class="titlepage"><div><div><h3 class="title">
        !           255: <a name="what-it-does"></a>6.3.4. What it does</h3></div></div></div>
        !           256: <p>The bulk builds consist of three steps:</p>
        !           257: <div class="variablelist"><dl>
        !           258: <dt><span class="term">1. pre-build</span></dt>
        !           259: <dd><p>The script updates your pkgsrc tree via (anon)cvs, then
        !           260:            cleans out any broken distfiles, and removes all
        !           261:            packages installed.</p></dd>
        !           262: <dt><span class="term">2. the bulk build</span></dt>
        !           263: <dd><p>This is basically &#8220;<span class="quote">make bulk-package</span>&#8221; with
        !           264:            an optimised order in which packages will be
        !           265:            built. Packages that don't require other packages will
        !           266:            be built first, and packages with many dependencies will
        !           267:            be built later.</p></dd>
        !           268: <dt><span class="term">3. post-build</span></dt>
        !           269: <dd><p>Generates a report that's placed in the directory
        !           270:            specified in the <code class="filename">build.conf</code> file
        !           271:            named <code class="filename">broken.html</code>, a short version
        !           272:            of that report will also be mailed to the build's
        !           273:            admin.</p></dd>
        !           274: </dl></div>
        !           275: <p>During the build, a list of broken packages will be compiled
        !           276:       in <code class="filename">/usr/pkgsrc/.broken</code> (or
        !           277:       <code class="filename">.../.broken.${MACHINE}</code> if
        !           278:       <code class="varname">OBJMACHINE</code> is set), individual build logs
        !           279:       of broken builds can be found in the package's
        !           280:       directory. These files are used by the bulk-targets to mark
        !           281:       broken builds to not waste time trying to rebuild them, and
        !           282:       they can be used to debug these broken package builds
        !           283:       later.</p>
        !           284: </div>
        !           285: <div class="sect2" lang="en">
        !           286: <div class="titlepage"><div><div><h3 class="title">
        !           287: <a name="disk-space-requirements"></a>6.3.5. Disk space requirements</h3></div></div></div>
        !           288: <p>Currently, roughly the following requirements are valid for
        !           289:       NetBSD 2.0/i386:</p>
        !           290: <div class="itemizedlist"><ul type="disc">
        !           291: <li><p>10 GB - distfiles (NFS ok)</p></li>
        !           292: <li><p>8 GB - full set of all binaries (NFS ok)</p></li>
        !           293: <li><p>5 GB - temp space for compiling (local disk recommended)</p></li>
        !           294: </ul></div>
        !           295: <p>Note that all pkgs will be de-installed as soon as they are
        !           296:       turned into a binary package, and that sources are removed,
        !           297:       so there is no excessively huge demand to disk
        !           298:       space. Afterwards, if the package is needed again, it will
        !           299:       be installed via <a href="http://netbsd.gw.com/cgi-bin/man-cgi?pkg_add+1+NetBSD-current"><span class="citerefentry"><span class="refentrytitle">pkg_add</span>(1)</span></a> instead of building again, so
        !           300:       there are no cycles wasted by recompiling.</p>
        !           301: </div>
        !           302: <div class="sect2" lang="en">
        !           303: <div class="titlepage"><div><div><h3 class="title">
        !           304: <a name="setting-up-a-sandbox"></a>6.3.6. Setting up a sandbox for chrooted builds</h3></div></div></div>
        !           305: <p>If you don't want all the packages nuked from a machine
        !           306:       (rendering it useless for anything but pkg compiling), there
        !           307:       is the possibility of doing the package bulk build inside a
        !           308:       chroot environment.</p>
        !           309: <p>The first step is to set up a chroot sandbox,
        !           310:       e.g. <code class="filename">/usr/sandbox</code>.  This can be done by
        !           311:       using null mounts, or manually.</p>
        !           312: <p>There is a shell script called
        !           313:       <code class="filename">pkgsrc/mk/bulk/mksandbox</code> which will set
        !           314:       up the sandbox environment using null mounts. It will also
        !           315:       create a script called <code class="filename">sandbox</code> in the
        !           316:       root of the sandbox environment, which will allow the null
        !           317:       mounts to be activated using the <span><strong class="command">sandbox
        !           318:       mount</strong></span> command and deactivated using the
        !           319:       <span><strong class="command">sandbox umount</strong></span> command.</p>
        !           320: <p>To set up a sandbox environment by hand, after extracting all
        !           321:       the sets from a NetBSD installation or doing a <span><strong class="command">make
        !           322:       distribution DESTDIR=/usr/sandbox</strong></span> in
        !           323:       <code class="filename">/usr/src/etc</code>, be sure the following items
        !           324:       are present and properly configured:</p>
        !           325: <div class="procedure"><ol type="1">
        !           326: <li>
        !           327: <p>Kernel</p>
        !           328: <pre class="screen"><code class="prompt">#</code> <strong class="userinput"><code>cp /netbsd /usr/sandbox</code></strong></pre>
        !           329: </li>
        !           330: <li>
        !           331: <p><code class="filename">/dev/*</code></p>
        !           332: <pre class="screen"><code class="prompt">#</code> <strong class="userinput"><code>cd /usr/sandbox/dev ; sh MAKEDEV all</code></strong></pre>
        !           333: </li>
        !           334: <li>
        !           335: <p><code class="filename">/etc/resolv.conf</code> (for <a href="ftp://ftp.NetBSD.org/pub/pkgsrc/current/pkgsrc/security/smtpd/README.html" target="_top"><code class="filename">security/smtpd</code></a> and mail):</p>
        !           336: <pre class="screen"><code class="prompt">#</code> <strong class="userinput"><code>cp /etc/resolv.conf /usr/sandbox/etc</code></strong></pre>
        !           337: </li>
        !           338: <li>
        !           339: <p>Working(!) mail config (hostname, sendmail.cf):</p>
        !           340: <pre class="screen"><code class="prompt">#</code> <strong class="userinput"><code>cp /etc/mail/sendmail.cf /usr/sandbox/etc/mail</code></strong></pre>
        !           341: </li>
        !           342: <li>
        !           343: <p><code class="filename">/etc/localtime</code> (for <a href="ftp://ftp.NetBSD.org/pub/pkgsrc/current/pkgsrc/security/smtpd/README.html" target="_top"><code class="filename">security/smtpd</code></a>):</p>
        !           344: <pre class="screen"><code class="prompt">#</code> <strong class="userinput"><code>ln -sf /usr/share/zoneinfo/UTC /usr/sandbox/etc/localtime</code></strong></pre>
        !           345: </li>
        !           346: <li>
        !           347: <p><code class="filename">/usr/src</code> (system sources,
        !           348:          e.&nbsp;g. for <a href="ftp://ftp.NetBSD.org/pub/pkgsrc/current/pkgsrc/sysutils/aperture/README.html" target="_top"><code class="filename">sysutils/aperture</code></a>):</p>
        !           349: <pre class="screen"><code class="prompt">#</code> <strong class="userinput"><code>ln -s ../disk1/cvs .</code></strong>
        !           350:          <code class="prompt">#</code> <strong class="userinput"><code>ln -s cvs/src-2.0 src</code></strong></pre>
        !           351: </li>
        !           352: <li>
        !           353: <p>Create <code class="filename">/var/db/pkg</code> (not part of default install):</p>
        !           354: <pre class="screen"><code class="prompt">#</code> <strong class="userinput"><code>mkdir /usr/sandbox/var/db/pkg</code></strong></pre>
        !           355: </li>
        !           356: <li>
        !           357: <p>Create <code class="filename">/usr/pkg</code> (not part of default install):</p>
        !           358: <pre class="screen"><code class="prompt">#</code> <strong class="userinput"><code>mkdir /usr/sandbox/usr/pkg</code></strong></pre>
        !           359: </li>
        !           360: <li>
        !           361: <p>Checkout pkgsrc via cvs into
        !           362:          <code class="filename">/usr/sandbox/usr/pkgsrc</code>:</p>
        !           363: <pre class="screen">
        !           364: <code class="prompt">#</code> <strong class="userinput"><code>cd /usr/sandbox/usr</code></strong>
        !           365: <code class="prompt">#</code> <strong class="userinput"><code>cvs -d anoncvs@anoncvs.NetBSD.org:/cvsroot checkout -d -P pkgsrc</code></strong>
        !           366:          </pre>
        !           367: <p>Do not mount/link this to the copy of your pkgsrc tree
        !           368:          you do development in, as this will likely cause problems!</p>
        !           369: </li>
        !           370: <li><p>Make
        !           371:          <code class="filename">/usr/sandbox/usr/pkgsrc/packages</code> and
        !           372:          <code class="filename">.../distfiles</code> point somewhere
        !           373:          appropriate. NFS- and/or nullfs-mounts may come in handy!</p></li>
        !           374: <li><p>Edit <code class="filename">/etc/mk.conf</code>, see <a href="binary.html#binary.mk.conf" title="6.3.1.2. /etc/mk.conf">Section 6.3.1.2, &#8220;/etc/mk.conf&#8221;</a>.</p></li>
        !           375: <li><p>Adjust <code class="filename">mk/bulk/build.conf</code> to suit your needs.</p></li>
        !           376: </ol></div>
        !           377: <p>When the chroot sandbox is set up, you can start
        !           378:       the build with the following steps:</p>
        !           379: <pre class="screen">
        !           380: <code class="prompt">#</code> <strong class="userinput"><code>cd /usr/sandbox/usr/pkgsrc</code></strong>
        !           381: <code class="prompt">#</code> <strong class="userinput"><code>sh mk/bulk/do-sandbox-build</code></strong>
        !           382:       </pre>
        !           383: <p>This will just jump inside the sandbox and start building.  At
        !           384:        the end of the build, mail will be sent with the results of
        !           385:        the build. Created binary pkgs will be in
        !           386:        <code class="filename">/usr/sandbox/usr/pkgsrc/packages</code>
        !           387:       (wherever that points/mounts to/from).</p>
        !           388: </div>
        !           389: <div class="sect2" lang="en">
        !           390: <div class="titlepage"><div><div><h3 class="title">
        !           391: <a name="building-a-partial-set"></a>6.3.7. Building a partial set of packages</h3></div></div></div>
        !           392: <p>In addition to building a complete set of all packages in
        !           393:       pkgsrc, the <code class="filename">pkgsrc/mk/bulk/build</code> script
        !           394:       may be used to build a subset of the packages contained in
        !           395:       pkgsrc.  By setting <code class="varname">SPECIFIC_PKGS</code>
        !           396:       in <code class="filename">/etc/mk.conf</code>, the variables</p>
        !           397: <div class="itemizedlist"><ul type="disc">
        !           398: <li><p>SITE_SPECIFIC_PKGS</p></li>
        !           399: <li><p>HOST_SPECIFIC_PKGS</p></li>
        !           400: <li><p>GROUP_SPECIFIC_PKGS</p></li>
        !           401: <li><p>USER_SPECIFIC_PKGS</p></li>
        !           402: </ul></div>
        !           403: <p>will define the set of packages which should be built.
        !           404:       The bulk build code will also include any packages which are
        !           405:       needed as dependencies for the explicitly listed packages.</p>
        !           406: <p>One use of this is to do a bulk build with
        !           407:       <code class="varname">SPECIFIC_PKGS</code> in a chroot sandbox
        !           408:       periodically to have a complete set of the binary packages
        !           409:       needed for your site available without the overhead of
        !           410:       building extra packages that are not needed.</p>
        !           411: </div>
        !           412: <div class="sect2" lang="en">
        !           413: <div class="titlepage"><div><div><h3 class="title">
        !           414: <a name="bulk-upload"></a>6.3.8. Uploading results of a bulk build</h3></div></div></div>
        !           415: <p>This section describes how pkgsrc developers can upload binary
        !           416:       pkgs built by bulk builds to ftp.NetBSD.org.</p>
        !           417: <p>If you would like to automatically create checksum files for the
        !           418:       binary packages you intend to upload, remember to set
        !           419:       <code class="varname">MKSUMS=yes</code> in your
        !           420:       <code class="filename">mk/bulk/build.conf</code>.</p>
        !           421: <p>If you would like to PGP sign the checksum files (highly
        !           422:       recommended!), remember to set
        !           423:       <code class="varname">SIGN_AS=username@NetBSD.org</code> in your
        !           424:       <code class="filename">mk/bulk/build.conf</code>.  This will prompt you for
        !           425:       your GPG password to sign the files before uploading everything.</p>
        !           426: <p>Then, make sure that you have <code class="varname">RSYNC_DST</code>
        !           427:       set properly in your <code class="filename">mk/bulk/build.conf</code>
        !           428:       file, i.e. adjust it to something like one of the following:</p>
        !           429: <pre class="screen">RSYNC_DST=ftp.NetBSD.org:/pub/NetBSD/packages/packages-200xQy/NetBSD-a.b.c/arch/upload </pre>
        !           430: <p>Please use appropriate values for "packages-200xQy",
        !           431:       "NetBSD-a.b.c" and "arch" here. If your login on ftp.NetBSD.org
        !           432:       is different from your local login, write your login directly
        !           433:       into the variable, e.g. my local account is "feyrer", but for my
        !           434:       login "hubertf", I use:</p>
        !           435: <pre class="screen">RSYNC_DST=hubertf@ftp.NetBSD.org:/pub/NetBSD/packages/packages-200xQy/NetBSD-a.b.c/arch/upload</pre>
        !           436: <p>A separate <code class="filename">upload</code> directory is used
        !           437:       here to allow "closing" the directory during upload. To do
        !           438:       so, run the following command on ftp.NetBSD.org next:</p>
        !           439: <pre class="screen">nbftp% <strong class="userinput"><code>mkdir -p -m 750 /pub/NetBSD/packages/packages-200xQy/NetBSD-a.b.c/arch/upload</code></strong></pre>
        !           440: <p>Please note that <code class="filename">/pub/NetBSD/packages</code> is
        !           441:       only appropriate for packages for the NetBSD operating
        !           442:       system. Binary packages for other operating systems should go
        !           443:       into <code class="filename">/pub/pkgsrc</code>.</p>
        !           444: <p>Before uploading the binary pkgs, ssh authentication needs
        !           445:       to be set up. This example shows how to set up temporary keys
        !           446:       for the root account <span class="emphasis"><em>inside the sandbox</em></span>
        !           447:       (assuming that no keys should be present there usually):</p>
        !           448: <pre class="screen">
        !           449: <code class="prompt">#</code> <strong class="userinput"><code>chroot /usr/sandbox</code></strong>
        !           450: chroot-<code class="prompt">#</code> <strong class="userinput"><code>rm $HOME/.ssh/id-dsa*</code></strong>
        !           451: chroot-<code class="prompt">#</code> <strong class="userinput"><code>ssh-keygen -t dsa</code></strong>
        !           452: chroot-<code class="prompt">#</code> <strong class="userinput"><code>cat $HOME/.ssh/id-dsa.pub</code></strong>
        !           453:       </pre>
        !           454: <p>Now take the output of <code class="filename">id-dsa.pub</code> and
        !           455:       append it to your <code class="filename">~/.ssh/authorized_keys</code>
        !           456:       file on ftp.NetBSD.org. You can remove the key after the
        !           457:       upload is done!</p>
        !           458: <p>Next, test if your ssh connection really works:</p>
        !           459: <pre class="screen">chroot-<code class="prompt">#</code> <strong class="userinput"><code>ssh ftp.NetBSD.org date</code></strong> </pre>
        !           460: <p>Use "-l yourNetBSDlogin" here as appropriate!</p>
        !           461: <p>Now after all this works, you can exit the sandbox and start
        !           462:       the upload:</p>
        !           463: <pre class="screen">
        !           464: chroot-<code class="prompt">#</code> <strong class="userinput"><code>exit</code></strong>
        !           465: <code class="prompt">#</code> <strong class="userinput"><code>cd /usr/sandbox/usr/pkgsrc</code></strong>
        !           466: <code class="prompt">#</code> <strong class="userinput"><code>sh mk/bulk/do-sandbox-upload</code></strong>
        !           467:       </pre>
        !           468: <p>The upload process may take quite some time. Use <a href="http://netbsd.gw.com/cgi-bin/man-cgi?ls+1+NetBSD-current"><span class="citerefentry"><span class="refentrytitle">ls</span>(1)</span></a> or
        !           469:       <a href="http://netbsd.gw.com/cgi-bin/man-cgi?du+1+NetBSD-current"><span class="citerefentry"><span class="refentrytitle">du</span>(1)</span></a> on the FTP server to monitor progress of the
        !           470:       upload. The upload script will take care of not uploading
        !           471:       restricted packages and putting vulnerable packages into the
        !           472:       <code class="filename">vulnerable</code> subdirectory.</p>
        !           473: <p>After the upload has ended, first thing is to revoke ssh access:</p>
        !           474: <pre class="screen">nbftp% <strong class="userinput"><code>vi ~/.ssh/authorized_keys</code></strong>
        !           475:       Gdd:x! </pre>
        !           476: <p>Use whatever is needed to remove the key you've entered
        !           477:       before! Last, move the uploaded packages out of the
        !           478:       <code class="filename">upload</code> directory to have them accessible
        !           479:       to everyone:</p>
        !           480: <pre class="screen">
        !           481: nbftp% <strong class="userinput"><code>cd /pub/NetBSD/packages/packages-200xQy/NetBSD-a.b.c/arch</code></strong>
        !           482: nbftp% <strong class="userinput"><code>mv upload/* .</code></strong>
        !           483: nbftp% <strong class="userinput"><code>rmdir upload</code></strong>
        !           484: nbftp% <strong class="userinput"><code>chmod 755 .</code></strong>
        !           485:       </pre>
        !           486: </div>
        !           487: </div>
        !           488: <div class="sect1" lang="en">
        !           489: <div class="titlepage"><div><div><h2 class="title" style="clear: both">
        !           490: <a name="creating-cdroms"></a>6.4. Creating a multiple CD-ROM packages collection</h2></div></div></div>
        !           491: <p>After your pkgsrc bulk-build has completed, you may wish to
        !           492:     create a CD-ROM set of the resulting binary packages to assist
        !           493:     in installing packages on other machines.  The
        !           494:     <a href="ftp://ftp.NetBSD.org/pub/pkgsrc/current/pkgsrc/pkgtools/cdpack/README.html" target="_top"><code class="filename">pkgtools/cdpack</code></a> package provides
        !           495:     a simple tool for creating the ISO 9660 images.
        !           496:     <span><strong class="command">cdpack</strong></span> arranges the packages on the CD-ROMs in a
        !           497:     way that keeps all the dependencies for a given package on the same
        !           498:     CD as that package.</p>
        !           499: <div class="sect2" lang="en">
        !           500: <div class="titlepage"><div><div><h3 class="title">
        !           501: <a name="cdpack-example"></a>6.4.1. Example of cdpack</h3></div></div></div>
        !           502: <p>Complete documentation for cdpack is found in the cdpack(1)
        !           503:       man page. The following short example assumes that the binary
        !           504:       packages are left in
        !           505:       <code class="filename">/usr/pkgsrc/packages/All</code> and that
        !           506:       sufficient disk space exists in <code class="filename">/u2</code> to
        !           507:       hold the ISO 9660 images.</p>
        !           508: <pre class="screen">
        !           509: <code class="prompt">#</code> <strong class="userinput"><code>mkdir /u2/images</code></strong>
        !           510: <code class="prompt">#</code> <strong class="userinput"><code>pkg_add /usr/pkgsrc/packages/All/cdpack</code></strong>
        !           511: <code class="prompt">#</code> <strong class="userinput"><code>cdpack /usr/pkgsrc/packages/All /u2/images</code></strong>
        !           512:       </pre>
        !           513: <p>If you wish to include a common set of files
        !           514:       (<code class="filename">COPYRIGHT</code>, <code class="filename">README</code>,
        !           515:       etc.) on each CD in the collection, then you need to create a
        !           516:       directory which contains these files. e.g.</p>
        !           517: <pre class="screen">
        !           518: <code class="prompt">#</code> <strong class="userinput"><code>mkdir /tmp/common</code></strong>
        !           519: <code class="prompt">#</code> <strong class="userinput"><code>echo "This is a README" &gt; /tmp/common/README</code></strong>
        !           520: <code class="prompt">#</code> <strong class="userinput"><code>echo "Another file" &gt; /tmp/common/COPYING</code></strong>
        !           521: <code class="prompt">#</code> <strong class="userinput"><code>mkdir /tmp/common/bin</code></strong>
        !           522: <code class="prompt">#</code> <strong class="userinput"><code>echo "#!/bin/sh" &gt; /tmp/common/bin/myscript</code></strong>
        !           523: <code class="prompt">#</code> <strong class="userinput"><code>echo "echo Hello world" &gt;&gt; /tmp/common/bin/myscript</code></strong>
        !           524: <code class="prompt">#</code> <strong class="userinput"><code>chmod 755 /tmp/common/bin/myscript</code></strong>
        !           525:       </pre>
        !           526: <p>Now create the images:</p>
        !           527: <pre class="screen"><code class="prompt">#</code> <strong class="userinput"><code>cdpack -x /tmp/common /usr/pkgsrc/packages/All /u2/images</code></strong></pre>
        !           528: <p>Each image will contain <code class="filename">README</code>,
        !           529:       <code class="filename">COPYING</code>, and <code class="filename">bin/myscript</code>
        !           530:       in their root directories.</p>
        !           531: </div>
        !           532: </div>
        !           533: </div>
        !           534: <div class="navfooter">
        !           535: <hr>
        !           536: <table width="100%" summary="Navigation footer">
        !           537: <tr>
        !           538: <td width="40%" align="left">
        !           539: <a accesskey="p" href="configuring.html">Prev</a> </td>
        !           540: <td width="20%" align="center"><a accesskey="u" href="users-guide.html">Up</a></td>
        !           541: <td width="40%" align="right"> <a accesskey="n" href="files.html">Next</a>
        !           542: </td>
        !           543: </tr>
        !           544: <tr>
        !           545: <td width="40%" align="left" valign="top">Chapter 5. Configuring pkgsrc </td>
        !           546: <td width="20%" align="center"><a accesskey="h" href="index.html">Home</a></td>
        !           547: <td width="40%" align="right" valign="top"> Chapter 7. Directory layout of the installed files</td>
        !           548: </tr>
        !           549: </table>
        !           550: </div>
        !           551: </body>
        !           552: </html>

CVSweb <webmaster@jp.NetBSD.org>