Annotation of pkgsrc/doc/guide/files/binary.xml, revision 1.32

1.32    ! rillig      1: <!-- $NetBSD: binary.xml,v 1.31 2007/05/23 12:36:56 rillig Exp $ -->
1.1       grant       2: 
                      3: <chapter id="binary">
                      4:   <title>Creating binary packages</title>
                      5: 
1.13      wiz         6:   <sect1 id="building-a-single-binary-package">
1.1       grant       7:     <title>Building a single binary package</title>
                      8: 
1.25      wiz         9:     <para>Once you have built and installed a package, you can create
                     10:     a <emphasis>binary package</emphasis> which can be installed on
                     11:     another system with &man.pkg.add.1;. This saves having to build
                     12:     the same package on a group of hosts and wasting CPU time. It also
                     13:     provides a simple means for others to install your package, should
                     14:     you distribute it.</para>
                     15: 
                     16:     <para>To create a binary package, change into the appropriate
                     17:     directory in pkgsrc, and run <command>make
                     18:     package</command>:</para>
1.1       grant      19: 
1.26      wiz        20:     <screen>
                     21: &rprompt; <userinput>cd misc/figlet</userinput>
                     22: &rprompt; <userinput>make package</userinput>
                     23:     </screen>
1.1       grant      24: 
1.25      wiz        25:     <para>This will build and install your package (if not already done),
                     26:     and then build a binary package from what was installed. You can
                     27:     then use the <command>pkg_*</command> tools to manipulate
                     28:     it. Binary packages are created by default in
                     29:     <filename>/usr/pkgsrc/packages</filename>, in the form of a
                     30:     gzipped tar file. See <xref linkend="logs.package"/> for a
                     31:     continuation of the above <filename
                     32:     role="pkg">misc/figlet</filename> example.</para>
                     33: 
                     34:     <para>See <xref linkend="submit"/> for information on how to submit
                     35:     such a binary package.</para>
1.1       grant      36:   </sect1>
                     37: 
1.13      wiz        38:   <sect1 id="settings-for-creationg-of-binary-packages">
1.1       grant      39:     <title>Settings for creation of binary packages</title>
                     40: 
                     41:     <para>See <xref linkend="build.helpful-targets"/>.</para>
                     42:   </sect1>
                     43: 
                     44:   <sect1 id="bulkbuild">
                     45:     <title>Doing a bulk build of all packages</title>
                     46: 
1.9       wiz        47:     <para>If you want to get a full set of precompiled binary
1.25      wiz        48:     packages, this section describes how to get them. Beware that
                     49:     the bulk build will remove all currently installed packages from
                     50:     your system!</para>
                     51: 
                     52:     <para>Having an FTP server configured either on the
                     53:     machine doing the bulk builds or on a nearby NFS server can help
                     54:     to make the packages available to other machines that can then
                     55:     save time by installing only the binary packages. See &man.ftpd.8; for
                     56:     more information. If you use a remote NFS server's storage, be
                     57:     sure to not actually compile on NFS storage, as this slows
                     58:     things down a lot.</para>
1.1       grant      59: 
                     60:     <sect2 id="binary.configuration">
                     61:       <title>Configuration</title>
                     62: 
1.17      rillig     63:       <sect3 id="binary.bulk.build.conf">
                     64:        <title><filename>build.conf</filename></title>
                     65: 
1.25      wiz        66:        <para>The <filename>build.conf</filename> file is the main
                     67:        configuration file for bulk builds. You can configure how your
                     68:        copy of pkgsrc is kept up to date, how the distfiles are
                     69:        downloaded, how the packages are built and how the report is
                     70:        generated. You can find an annotated example file in
                     71:        <filename>pkgsrc/mk/bulk/build.conf-example</filename>. To use
                     72:        it, copy <filename>build.conf-example</filename> to
                     73:        <filename>build.conf</filename> and edit it, following the
                     74:        comments in that file.</para>
1.17      rillig     75:       </sect3>
                     76: 
1.1       grant      77:       <sect3 id="binary.mk.conf">
                     78:        <title>/etc/mk.conf</title>
                     79: 
1.9       wiz        80:        <para>You may want to set variables in
1.25      wiz        81:        <filename>/etc/mk.conf</filename>.
                     82:        Look at <filename>pkgsrc/mk/defaults/mk.conf</filename> for
                     83:        details of the default settings. You will want to ensure that
                     84:        <varname>ACCEPTABLE_LICENSES</varname> meet your local policy.
                     85:        As used in this example, <varname>_ACCEPTABLE=yes</varname>
                     86:        accepts <emphasis>all</emphasis> licenses.</para>
                     87: 
1.32    ! rillig     88: <programlisting>
        !            89: PACKAGES?=      ${_PKGSRCDIR}/packages/${MACHINE_ARCH}
        !            90: WRKOBJDIR?=     /usr/tmp/pkgsrc   # build here instead of in pkgsrc
        !            91: BSDSRCDIR=      /usr/src
        !            92: BSDXSRCDIR=     /usr/xsrc         # for x11/xservers
        !            93: OBJHOSTNAME?=   yes               # use work.`hostname`
        !            94: FAILOVER_FETCH= yes               # insist on the correct checksum
        !            95: PKG_DEVELOPER?= yes
        !            96: _ACCEPTABLE=    yes
        !            97: </programlisting>
1.18      rillig     98: 
                     99:        <para>Some options that are especially useful for bulk builds
                    100:        can be found at the top lines of the file
                    101:        <filename>mk/bulk/bsd.bulk-pkg.mk</filename>. The most useful
                    102:        options of these are briefly described here.</para>
                    103: 
                    104:        <itemizedlist>
                    105: 
1.25      wiz       106:          <listitem><para>If you are on a slow machine, you may want to
                    107:          set <varname>USE_BULK_BROKEN_CHECK</varname> to
                    108:          <quote>no</quote>.</para></listitem>
                    109: 
                    110:          <listitem><para>If you are doing bulk builds from a read-only
                    111:          copy of pkgsrc, you have to set <varname>BULKFILESDIR</varname>
                    112:          to the directory where all log files are created. Otherwise the
                    113:          log files are created in the pkgsrc directory.</para></listitem>
                    114: 
                    115:          <listitem><para>Another important variable is
                    116:          <varname>BULK_PREREQ</varname>, which is a list of packages that
                    117:          should be always available while building other
                    118:          packages.</para></listitem>
1.18      rillig    119: 
                    120:        </itemizedlist>
1.19      rillig    121: 
                    122:        <para>Some other options are scattered in the pkgsrc
                    123:        infrastructure:</para>
                    124: 
                    125:        <itemizedlist>
                    126: 
1.25      wiz       127:          <listitem><para><varname>ALLOW_VULNERABLE_PACKAGES</varname>
                    128:          should be set to <literal>yes</literal>. The purpose of the bulk
                    129:          builds is creating binary packages, no matter if they are
                    130:          vulnerable or not. When uploading the packages to a public
                    131:          server, the vulnerable packages will be put into a directory of
                    132:          their own. Leaving this variable unset would prevent the bulk
                    133:          build system from even trying to build them, so possible
                    134:          building errors would not show up.</para></listitem>
                    135: 
                    136:          <listitem><para><varname>CHECK_FILES</varname>
1.30      yyamano   137:          (<filename>pkgsrc/mk/check/check-files.mk</filename>) can be set to
1.25      wiz       138:          <quote>yes</quote> to check that the installed set of files
                    139:          matches the <filename>PLIST</filename>.</para></listitem>
                    140: 
                    141:          <listitem><para><varname>CHECK_INTERPRETER</varname>
1.30      yyamano   142:          (<filename>pkgsrc/mk/check/check-interpreter.mk</filename>) can be set to
1.25      wiz       143:          <quote>yes</quote> to check that the installed
                    144:          <quote>#!</quote>-scripts will find their
                    145:          interpreter.</para></listitem>
1.19      rillig    146: 
1.28      wiz       147:          <listitem><para><varname>PKGSRC_RUN_TEST</varname> can be
                    148:          set to <quote><literal>yes</literal></quote> to run each
                    149:          package's self-test before installing it. Note that some
                    150:          packages make heavy use of <quote>good</quote> random
                    151:          numbers, so you need to assure that the machine on which you
                    152:          are doing the bulk builds is not completely idle. Otherwise
                    153:          some test programs will seem to hang, while they are just
                    154:          waiting for new random data to be
1.27      rillig    155:          available.</para></listitem>
                    156: 
1.19      rillig    157:        </itemizedlist>
                    158: 
1.1       grant     159:       </sect3>
                    160: 
1.13      wiz       161:       <sect3 id="pre-build.local">
1.1       grant     162:        <title><filename>pre-build.local</filename></title>
                    163: 
1.9       wiz       164:        <para>It is possible to configure the bulk build to perform
1.25      wiz       165:        certain site-specific tasks at the end of the pre-build
                    166:        stage.  If the file
                    167:        <filename>pre-build.local</filename> exists in
                    168:        <filename>/usr/pkgsrc/mk/bulk</filename>, it will be executed
                    169:        (as a &man.sh.1; script) at the end of the usual pre-build
                    170:        stage.  An example use of
                    171:        <filename>pre-build.local</filename> is to have the line:</para>
1.1       grant     172: 
1.23      wiz       173:        <screen>echo "I do not have enough disk space to build this pig." \
1.25      wiz       174:        &gt; misc/openoffice/$BROKENF</screen>
1.1       grant     175: 
                    176:        <para>to prevent the system from trying to build a particular package
1.25      wiz       177:        which requires nearly 3 GB of disk space.</para>
1.1       grant     178:       </sect3>
                    179:     </sect2>
                    180: 
1.13      wiz       181:     <sect2 id="other-environmental-considerations">
1.1       grant     182:       <title>Other environmental considerations</title>
                    183: 
1.6       hubertf   184:       <para>As <filename>/usr/pkg</filename> will be completely
1.25      wiz       185:       deleted at the start of bulk builds, make sure your login
                    186:       shell is placed somewhere else. Either drop it into
                    187:       <filename>/usr/local/bin</filename> (and adjust your login
                    188:       shell in the passwd file), or (re-)install it via
                    189:       &man.pkg.add.1; from <filename>/etc/rc.local</filename>, so
                    190:       you can login after a reboot (remember that your current
                    191:       process won't die if the package is removed, you just can't
                    192:       start any new instances of the shell any more).  Also, if you
                    193:       use &os; earlier than 1.5, or you still want to use the pkgsrc
                    194:       version of ssh for some reason, be sure to install ssh before
                    195:       starting it from <filename>rc.local</filename>:</para>
                    196: 
1.32    ! rillig    197: <programlisting>
        !           198: (cd /usr/pkgsrc/security/ssh && make bulk-install)
        !           199: if [ -f /usr/pkg/etc/rc.d/sshd ]; then
        !           200:   /usr/pkg/etc/rc.d/sshd
        !           201: fi
        !           202: </programlisting>
1.1       grant     203: 
1.6       hubertf   204:       <para>Not doing so will result in you being not able to log in
1.25      wiz       205:       via ssh after the bulk build is finished or if the machine
                    206:       gets rebooted or crashes. You have been warned! :)</para>
1.1       grant     207:     </sect2>
                    208: 
1.13      wiz       209:     <sect2 id="operation">
1.1       grant     210:       <title>Operation</title>
                    211: 
1.6       hubertf   212:       <para>Make sure you don't need any of the packages still
1.25      wiz       213:       installed.</para>
1.1       grant     214: 
                    215:       <warning>
1.31      rillig    216:        <para>During the bulk build, <emphasis>all packages, their
                    217:        configuration files and some more files from
                    218:        <filename>/var</filename>, <filename>/home</filename> and
                    219:        possibly other locations will be removed! So don't run a bulk
                    220:        build with privileges that might harm your
                    221:        system.</emphasis></para>
1.1       grant     222:       </warning>
                    223: 
                    224:       <para>Be sure to remove all other things that might
1.25      wiz       225:       interfere with builds, like some libs installed in
                    226:       <filename>/usr/local</filename>, etc. then become root and type:</para>
1.1       grant     227: 
1.26      wiz       228:       <screen>
                    229: &rprompt; <userinput>cd /usr/pkgsrc</userinput>
                    230: &rprompt; <userinput>sh mk/bulk/build</userinput>
                    231:       </screen>
1.1       grant     232: 
1.6       hubertf   233:       <para>If for some reason your last build didn't complete (power
1.25      wiz       234:       failure, system panic, ...), you can continue it by
                    235:       running:</para>
1.1       grant     236: 
                    237:       <screen>&rprompt; <userinput>sh mk/bulk/build restart</userinput></screen>
                    238: 
1.25      wiz       239:       <para>At the end of the bulk build, you will get a summary via mail,
                    240:       and find build logs in the directory specified by
                    241:       <varname>FTP</varname> in the <filename>build.conf</filename>
                    242:       file.</para>
1.1       grant     243:     </sect2>
                    244: 
1.13      wiz       245:     <sect2 id="what-it-does">
1.1       grant     246:       <title>What it does</title>
                    247: 
                    248:       <para>The bulk builds consist of three steps:</para>
                    249: 
                    250:       <variablelist>
                    251:        <varlistentry>
                    252:          <term>1. pre-build</term>
                    253: 
                    254:          <listitem>
1.25      wiz       255:            <para>The script updates your pkgsrc tree via (anon)cvs, then
                    256:            cleans out any broken distfiles, and removes all
                    257:            packages installed.</para>
1.1       grant     258:          </listitem>
                    259:        </varlistentry>
                    260: 
                    261:        <varlistentry>
                    262:          <term>2. the bulk build</term>
                    263: 
                    264:          <listitem>
1.25      wiz       265:            <para>This is basically <quote>make bulk-package</quote> with
                    266:            an optimised order in which packages will be
                    267:            built. Packages that don't require other packages will
                    268:            be built first, and packages with many dependencies will
                    269:            be built later.</para>
1.1       grant     270:          </listitem>
                    271:        </varlistentry>
                    272: 
                    273:        <varlistentry>
                    274:          <term>3. post-build</term>
                    275: 
                    276:          <listitem>
1.25      wiz       277:            <para>Generates a report that's placed in the directory
                    278:            specified in the <filename>build.conf</filename> file
                    279:            named <filename>broken.html</filename>, a short version
                    280:            of that report will also be mailed to the build's
                    281:            admin.</para>
1.1       grant     282:          </listitem>
                    283:        </varlistentry>
                    284:       </variablelist>
                    285: 
1.25      wiz       286:       <para>During the build, a list of broken packages will be compiled
                    287:       in <filename>/usr/pkgsrc/.broken</filename> (or
                    288:       <filename>.../.broken.${MACHINE}</filename> if
                    289:       <varname>OBJMACHINE</varname> is set), individual build logs
                    290:       of broken builds can be found in the package's
                    291:       directory. These files are used by the bulk-targets to mark
                    292:       broken builds to not waste time trying to rebuild them, and
                    293:       they can be used to debug these broken package builds
                    294:       later.</para>
1.1       grant     295:     </sect2>
                    296: 
1.13      wiz       297:     <sect2 id="disk-space-requirements">
1.1       grant     298:       <title>Disk space requirements</title>
                    299: 
                    300:       <para>Currently, roughly the following requirements are valid for
1.25      wiz       301:       NetBSD 2.0/i386:</para>
1.1       grant     302: 
                    303:       <itemizedlist>
                    304:        <listitem>
1.25      wiz       305:          <para>10 GB - distfiles (NFS ok)</para>
1.1       grant     306:        </listitem>
                    307: 
                    308:        <listitem>
1.25      wiz       309:          <para>8 GB - full set of all binaries (NFS ok)</para>
1.1       grant     310:        </listitem>
                    311: 
                    312:        <listitem>
1.25      wiz       313:          <para>5 GB - temp space for compiling (local disk recommended)</para>
1.1       grant     314:        </listitem>
                    315:       </itemizedlist>
                    316: 
1.25      wiz       317:       <para>Note that all pkgs will be de-installed as soon as they are
                    318:       turned into a binary package, and that sources are removed,
                    319:       so there is no excessively huge demand to disk
                    320:       space. Afterwards, if the package is needed again, it will
                    321:       be installed via &man.pkg.add.1; instead of building again, so
                    322:       there are no cycles wasted by recompiling.</para>
1.1       grant     323:     </sect2>
                    324: 
1.13      wiz       325:     <sect2 id="setting-up-a-sandbox">
1.15      rillig    326:       <title>Setting up a sandbox for chrooted builds</title>
1.1       grant     327: 
1.25      wiz       328:       <para>If you don't want all the packages nuked from a machine
                    329:       (rendering it useless for anything but pkg compiling), there
                    330:       is the possibility of doing the package bulk build inside a
                    331:       chroot environment.</para>
                    332: 
                    333:       <para>The first step is to set up a chroot sandbox,
                    334:       e.g. <filename>/usr/sandbox</filename>.  This can be done by
                    335:       using null mounts, or manually.</para>
                    336: 
                    337:       <para>There is a shell script called
                    338:       <filename>pkgsrc/mk/bulk/mksandbox</filename> which will set
                    339:       up the sandbox environment using null mounts. It will also
                    340:       create a script called <filename>sandbox</filename> in the
                    341:       root of the sandbox environment, which will allow the null
                    342:       mounts to be activated using the <command>sandbox
                    343:       mount</command> command and deactivated using the
                    344:       <command>sandbox umount</command> command.</para>
                    345: 
                    346:       <para>To set up a sandbox environment by hand, after extracting all
                    347:       the sets from a &os; installation or doing a <command>make
                    348:       distribution DESTDIR=/usr/sandbox</command> in
                    349:       <filename>/usr/src/etc</filename>, be sure the following items
                    350:       are present and properly configured:</para>
1.1       grant     351: 
                    352:       <procedure>
                    353:        <step>
                    354:          <para>Kernel</para>
                    355: 
                    356:          <screen>&rprompt; <userinput>cp /netbsd /usr/sandbox</userinput></screen>
                    357:        </step>
                    358: 
                    359:        <step>
                    360:          <para><filename>/dev/*</filename></para>
                    361: 
                    362:          <screen>&rprompt; <userinput>cd /usr/sandbox/dev ; sh MAKEDEV all</userinput></screen>
                    363:        </step>
                    364: 
                    365:        <step>
1.21      rillig    366:          <para><filename>/etc/resolv.conf</filename> (for <filename
1.25      wiz       367:          role="pkg">security/smtpd</filename> and mail):</para>
1.1       grant     368: 
                    369:          <screen>&rprompt; <userinput>cp /etc/resolv.conf /usr/sandbox/etc</userinput></screen>
                    370:        </step>
                    371: 
                    372:        <step>
                    373:          <para>Working(!) mail config (hostname, sendmail.cf):</para>
                    374: 
                    375:          <screen>&rprompt; <userinput>cp /etc/mail/sendmail.cf /usr/sandbox/etc/mail</userinput></screen>
                    376:        </step>
                    377: 
                    378:        <step>
1.21      rillig    379:          <para><filename>/etc/localtime</filename> (for <filename
1.25      wiz       380:          role="pkg">security/smtpd</filename>):</para>
1.1       grant     381: 
                    382:          <screen>&rprompt; <userinput>ln -sf /usr/share/zoneinfo/UTC /usr/sandbox/etc/localtime</userinput></screen>
                    383:        </step>
                    384: 
                    385:        <step>
1.24      rillig    386: 
                    387:          <para><filename>/usr/src</filename> (system sources,
                    388:          e.&nbsp;g. for <filename
                    389:          role="pkg">sysutils/aperture</filename>):</para>
1.1       grant     390: 
                    391:          <screen>&rprompt; <userinput>ln -s ../disk1/cvs .</userinput>
1.25      wiz       392:          &rprompt; <userinput>ln -s cvs/src-2.0 src</userinput></screen>
1.1       grant     393:        </step>
                    394: 
                    395:        <step>
                    396:          <para>Create <filename>/var/db/pkg</filename> (not part of default install):</para>
                    397: 
                    398:          <screen>&rprompt; <userinput>mkdir /usr/sandbox/var/db/pkg</userinput></screen>
                    399:        </step>
                    400: 
                    401:        <step>
                    402:          <para>Create <filename>/usr/pkg</filename> (not part of default install):</para>
                    403: 
                    404:          <screen>&rprompt; <userinput>mkdir /usr/sandbox/usr/pkg</userinput></screen>
                    405:        </step>
                    406: 
                    407:        <step>
                    408:          <para>Checkout pkgsrc via cvs into
1.25      wiz       409:          <filename>/usr/sandbox/usr/pkgsrc</filename>:</para>
1.1       grant     410: 
1.26      wiz       411:          <screen>
                    412: &rprompt; <userinput>cd /usr/sandbox/usr</userinput>
                    413: &rprompt; <userinput>cvs -d anoncvs@anoncvs.NetBSD.org:/cvsroot checkout -d -P pkgsrc</userinput>
                    414:          </screen>
1.1       grant     415: 
1.25      wiz       416:          <para>Do not mount/link this to the copy of your pkgsrc tree
                    417:          you do development in, as this will likely cause problems!</para>
1.1       grant     418:        </step>
                    419: 
                    420:        <step>
                    421:          <para>Make
1.25      wiz       422:          <filename>/usr/sandbox/usr/pkgsrc/packages</filename> and
                    423:          <filename>.../distfiles</filename> point somewhere
                    424:          appropriate. NFS- and/or nullfs-mounts may come in handy!</para>
1.1       grant     425:        </step>
                    426: 
                    427:        <step>
                    428:          <para>Edit <filename>/etc/mk.conf</filename>, see <xref linkend="binary.mk.conf"/>.</para>
                    429:        </step>
                    430: 
                    431:        <step>
                    432:          <para>Adjust <filename>mk/bulk/build.conf</filename> to suit your needs.</para>
                    433:        </step>
                    434:       </procedure>
                    435: 
1.15      rillig    436:       <para>When the chroot sandbox is set up, you can start
1.25      wiz       437:       the build with the following steps:</para>
1.1       grant     438: 
1.26      wiz       439:       <screen>
                    440: &rprompt; <userinput>cd /usr/sandbox/usr/pkgsrc</userinput>
                    441: &rprompt; <userinput>sh mk/bulk/do-sandbox-build</userinput>
                    442:       </screen>
1.1       grant     443: 
1.25      wiz       444:       <para>This will just jump inside the sandbox and start building.  At
                    445:        the end of the build, mail will be sent with the results of
                    446:        the build. Created binary pkgs will be in
                    447:        <filename>/usr/sandbox/usr/pkgsrc/packages</filename>
                    448:       (wherever that points/mounts to/from).</para>
1.1       grant     449:     </sect2>
                    450: 
1.13      wiz       451:     <sect2 id="building-a-partial-set">
1.1       grant     452:       <title>Building a partial set of packages</title>
                    453: 
1.25      wiz       454:       <para>In addition to building a complete set of all packages in
                    455:       pkgsrc, the <filename>pkgsrc/mk/bulk/build</filename> script
                    456:       may be used to build a subset of the packages contained in
                    457:       pkgsrc.  By setting <varname>SPECIFIC_PKGS</varname>
                    458:       in <filename>/etc/mk.conf</filename>, the variables</para>
1.1       grant     459: 
                    460:       <itemizedlist>
                    461:        <listitem><para>SITE_SPECIFIC_PKGS</para></listitem>
                    462:        <listitem><para>HOST_SPECIFIC_PKGS</para></listitem>
                    463:        <listitem><para>GROUP_SPECIFIC_PKGS</para></listitem>
                    464:        <listitem><para>USER_SPECIFIC_PKGS</para></listitem>
                    465:       </itemizedlist>
                    466: 
1.25      wiz       467:       <para>will define the set of packages which should be built.
                    468:       The bulk build code will also include any packages which are
                    469:       needed as dependencies for the explicitly listed packages.</para>
                    470: 
                    471:       <para>One use of this is to do a bulk build with
                    472:       <varname>SPECIFIC_PKGS</varname> in a chroot sandbox
                    473:       periodically to have a complete set of the binary packages
                    474:       needed for your site available without the overhead of
                    475:       building extra packages that are not needed.</para>
1.1       grant     476: 
                    477:     </sect2>
1.6       hubertf   478: 
                    479:     <sect2 id="bulk-upload">
                    480:       <title>Uploading results of a bulk build</title>
                    481: 
1.25      wiz       482:       <para>This section describes how pkgsrc developers can upload binary
                    483:       pkgs built by bulk builds to ftp.NetBSD.org.</para>
                    484: 
                    485:       <para>If you would like to automatically create checksum files for the
                    486:       binary packages you intend to upload, remember to set
                    487:       <varname>MKSUMS=yes</varname> in your
                    488:       <filename>mk/bulk/build.conf</filename>.</para>
                    489: 
                    490:       <para>If you would like to PGP sign the checksum files (highly
                    491:       recommended!), remember to set
                    492:       <varname>SIGN_AS=username@NetBSD.org</varname> in your
                    493:       <filename>mk/bulk/build.conf</filename>.  This will prompt you for
                    494:       your GPG password to sign the files before uploading everything.</para>
                    495: 
                    496:       <para>Then, make sure that you have <varname>RSYNC_DST</varname>
                    497:       set properly in your <filename>mk/bulk/build.conf</filename>
                    498:       file, i.e. adjust it to something like one of the following:</para>
1.6       hubertf   499: 
1.29      yyamano   500:       <screen>RSYNC_DST=ftp.NetBSD.org:/pub/NetBSD/packages/packages-200xQy/NetBSD-a.b.c/arch/upload </screen>
1.6       hubertf   501: 
1.29      yyamano   502:       <para>Please use appropriate values for "packages-200xQy",
1.25      wiz       503:       "NetBSD-a.b.c" and "arch" here. If your login on ftp.NetBSD.org
                    504:       is different from your local login, write your login directly
                    505:       into the variable, e.g. my local account is "feyrer", but for my
                    506:       login "hubertf", I use:</para>
1.6       hubertf   507: 
1.29      yyamano   508:       <screen>RSYNC_DST=hubertf@ftp.NetBSD.org:/pub/NetBSD/packages/packages-200xQy/NetBSD-a.b.c/arch/upload</screen>
1.6       hubertf   509: 
1.25      wiz       510:       <para>A separate <filename>upload</filename> directory is used
                    511:       here to allow "closing" the directory during upload. To do
                    512:       so, run the following command on ftp.NetBSD.org next:</para>
1.6       hubertf   513: 
1.29      yyamano   514:       <screen>nbftp% <userinput>mkdir -p -m 750 /pub/NetBSD/packages/packages-200xQy/NetBSD-a.b.c/arch/upload</userinput></screen>
1.6       hubertf   515: 
1.25      wiz       516:       <para>Please note that <filename>/pub/NetBSD/packages</filename> is
                    517:       only appropriate for packages for the NetBSD operating
                    518:       system. Binary packages for other operating systems should go
                    519:       into <filename>/pub/pkgsrc</filename>.</para>
                    520: 
                    521:       <para>Before uploading the binary pkgs, ssh authentication needs
                    522:       to be set up. This example shows how to set up temporary keys
                    523:       for the root account <emphasis>inside the sandbox</emphasis>
                    524:       (assuming that no keys should be present there usually):</para>
1.6       hubertf   525: 
1.26      wiz       526:       <screen>
                    527: &rprompt; <userinput>chroot /usr/sandbox</userinput>
                    528: chroot-&rprompt; <userinput>rm $HOME/.ssh/id-dsa*</userinput>
                    529: chroot-&rprompt; <userinput>ssh-keygen -t dsa</userinput>
                    530: chroot-&rprompt; <userinput>cat $HOME/.ssh/id-dsa.pub</userinput>
                    531:       </screen>
1.25      wiz       532: 
                    533:       <para>Now take the output of <filename>id-dsa.pub</filename> and
                    534:       append it to your <filename>~/.ssh/authorized_keys</filename>
                    535:       file on ftp.NetBSD.org. You can remove the key after the
                    536:       upload is done!</para>
                    537: 
                    538:       <para>Next, test if your ssh connection really works:</para>
1.6       hubertf   539: 
                    540:       <screen>chroot-&rprompt; <userinput>ssh ftp.NetBSD.org date</userinput> </screen>
                    541: 
1.25      wiz       542:       <para>Use "-l yourNetBSDlogin" here as appropriate!</para>
                    543: 
                    544:       <para>Now after all this works, you can exit the sandbox and start
                    545:       the upload:</para>
1.6       hubertf   546: 
1.26      wiz       547:       <screen>
                    548: chroot-&rprompt; <userinput>exit</userinput>
                    549: &rprompt; <userinput>cd /usr/sandbox/usr/pkgsrc</userinput>
                    550: &rprompt; <userinput>sh mk/bulk/do-sandbox-upload</userinput>
                    551:       </screen>
1.25      wiz       552: 
                    553:       <para>The upload process may take quite some time. Use &man.ls.1; or
                    554:       &man.du.1; on the FTP server to monitor progress of the
                    555:       upload. The upload script will take care of not uploading
                    556:       restricted packages and putting vulnerable packages into the
                    557:       <filename>vulnerable</filename> subdirectory.</para>
1.6       hubertf   558: 
1.25      wiz       559:       <para>After the upload has ended, first thing is to revoke ssh access:</para>
1.6       hubertf   560: 
                    561:       <screen>nbftp% <userinput>vi ~/.ssh/authorized_keys</userinput>
1.25      wiz       562:       Gdd:x! </screen>
1.6       hubertf   563: 
1.25      wiz       564:       <para>Use whatever is needed to remove the key you've entered
                    565:       before! Last, move the uploaded packages out of the
                    566:       <filename>upload</filename> directory to have them accessible
                    567:       to everyone:</para>
1.6       hubertf   568: 
1.26      wiz       569:       <screen>
1.29      yyamano   570: nbftp% <userinput>cd /pub/NetBSD/packages/packages-200xQy/NetBSD-a.b.c/arch</userinput>
1.26      wiz       571: nbftp% <userinput>mv upload/* .</userinput>
                    572: nbftp% <userinput>rmdir upload</userinput>
                    573: nbftp% <userinput>chmod 755 .</userinput>
                    574:       </screen>
1.6       hubertf   575:     </sect2>
1.1       grant     576:   </sect1>
                    577: 
1.13      wiz       578:   <sect1 id="creating-cdroms">
1.1       grant     579:     <title>Creating a multiple CD-ROM packages collection</title>
                    580: 
1.25      wiz       581:     <para>After your pkgsrc bulk-build has completed, you may wish to
                    582:     create a CD-ROM set of the resulting binary packages to assist
                    583:     in installing packages on other machines.  The
                    584:     <filename role="pkg">pkgtools/cdpack</filename> package provides
                    585:     a simple tool for creating the ISO 9660 images.
                    586:     <command>cdpack</command> arranges the packages on the CD-ROMs in a
                    587:     way that keeps all the dependencies for a given package on the same
                    588:     CD as that package.</para>
1.1       grant     589: 
1.13      wiz       590:     <sect2 id="cdpack-example">
1.1       grant     591:       <title>Example of cdpack</title>
                    592: 
1.25      wiz       593:       <para>Complete documentation for cdpack is found in the cdpack(1)
                    594:       man page. The following short example assumes that the binary
                    595:       packages are left in
                    596:       <filename>/usr/pkgsrc/packages/All</filename> and that
                    597:       sufficient disk space exists in <filename>/u2</filename> to
                    598:       hold the ISO 9660 images.</para>
1.1       grant     599: 
1.26      wiz       600:       <screen>
                    601: &rprompt; <userinput>mkdir /u2/images</userinput>
                    602: &rprompt; <userinput>pkg_add /usr/pkgsrc/packages/All/cdpack</userinput>
                    603: &rprompt; <userinput>cdpack /usr/pkgsrc/packages/All /u2/images</userinput>
                    604:       </screen>
1.1       grant     605: 
1.25      wiz       606:       <para>If you wish to include a common set of files
                    607:       (<filename>COPYRIGHT</filename>, <filename>README</filename>,
                    608:       etc.) on each CD in the collection, then you need to create a
                    609:       directory which contains these files. e.g.</para>
1.1       grant     610: 
1.26      wiz       611:       <screen>
                    612: &rprompt; <userinput>mkdir /tmp/common</userinput>
                    613: &rprompt; <userinput>echo "This is a README" &gt; /tmp/common/README</userinput>
                    614: &rprompt; <userinput>echo "Another file" &gt; /tmp/common/COPYING</userinput>
                    615: &rprompt; <userinput>mkdir /tmp/common/bin</userinput>
                    616: &rprompt; <userinput>echo "#!/bin/sh" &gt; /tmp/common/bin/myscript</userinput>
                    617: &rprompt; <userinput>echo "echo Hello world" &gt;&gt; /tmp/common/bin/myscript</userinput>
                    618: &rprompt; <userinput>chmod 755 /tmp/common/bin/myscript</userinput>
                    619:       </screen>
1.1       grant     620: 
                    621:       <para>Now create the images:</para>
                    622: 
                    623:       <screen>&rprompt; <userinput>cdpack -x /tmp/common /usr/pkgsrc/packages/All /u2/images</userinput></screen>
                    624: 
                    625:       <para>Each image will contain <filename>README</filename>,
1.25      wiz       626:       <filename>COPYING</filename>, and <filename>bin/myscript</filename>
                    627:       in their root directories.</para>
1.1       grant     628:     </sect2>
                    629:   </sect1>
                    630: </chapter>

CVSweb <webmaster@jp.NetBSD.org>