Annotation of pkgsrc/doc/guide/files/binary.xml, revision 1.1
1.1 ! grant 1: <!-- $NetBSD: binary.xml,v 1.3 2004/09/29 13:08:36 hubertf Exp $ -->
! 2:
! 3: <chapter id="binary">
! 4: <title>Creating binary packages</title>
! 5:
! 6: <sect1>
! 7: <title>Building a single binary package</title>
! 8:
! 9: <para>Once you have built and installed a package, you can create a
! 10: <emphasis>binary package</emphasis> which can be installed on another
! 11: system with &man.pkg.add.1; This saves having to build the same package on
! 12: a group of hosts and wasting CPU time. It also provides a simple means
! 13: for others to install your package, should you distribute it.</para>
! 14:
! 15: <para>To create a binary package, change into the appropriate
! 16: directory in pkgsrc, and run <command>make package</command>:</para>
! 17:
! 18: <screen>&rprompt; <userinput>cd misc/figlet</userinput>
! 19: &rprompt; <userinput>make package</userinput></screen>
! 20:
! 21: <para>This will build and install your package (if not already done), and then
! 22: build a binary package from what was installed. You can then use the
! 23: <command>pkg_*</command> tools to manipulate it. Binary packages are
! 24: created by default in <filename>/usr/pkgsrc/packages</filename>, in the
! 25: form of a gzipped tar file. See <xref linkend="logs.package"/> for
! 26: a continuation of the above <pkg>misc/figlet</pkg> example.</para>
! 27:
! 28: <para>See <xref linkend="submit"/> for information on how to submit such a
! 29: binary package.</para>
! 30: </sect1>
! 31:
! 32: <sect1>
! 33: <title>Settings for creation of binary packages</title>
! 34:
! 35: <para>See <xref linkend="build.helpful-targets"/>.</para>
! 36: </sect1>
! 37:
! 38: <sect1 id="bulkbuild">
! 39: <title>Doing a bulk build of all packages</title>
! 40:
! 41: <para>If you want to get a full set of precompiled binary packages, this section
! 42: describes how to get them. Beware that the bulk build will remove all
! 43: currently installed packages from your system! Having a FTP server
! 44: configured either on the machine doing the bulk builds or on a nearby NFS
! 45: server can help to make the packages available to everyone. See
! 46: &man.ftpd.8; for
! 47: more information. If you use a remote NFS server's storage, be sure to not
! 48: actually compile on NFS storage, as this slows things down a lot.</para>
! 49:
! 50: <sect2 id="binary.configuration">
! 51: <title>Configuration</title>
! 52:
! 53: <sect3 id="binary.mk.conf">
! 54: <title>/etc/mk.conf</title>
! 55:
! 56: <para>You may want to set things in <filename>/etc/mk.conf</filename>.
! 57: Look at <filename>pkgsrc/mk/bsd.pkg.defaults.mk</filename> for
! 58: details of the default settings. You will want to ensure that
! 59: <varname>ACCEPTABLE_LICENSES</varname> meet your local policy.
! 60: As used in this example, <varname>_ACCEPTABLE=yes</varname>
! 61: accepts <emphasis>all</emphasis> licenses.</para>
! 62:
! 63: <programlisting>PACKAGES?= ${_PKGSRCDIR}/packages/${MACHINE_ARCH}
! 64: WRKOBJDIR?= /usr/tmp/pkgsrc # build here instead of in pkgsrc
! 65: BSDSRCDIR= /usr/src
! 66: BSDXSRCDIR= /usr/xsrc # for x11/xservers
! 67: OBJHOSTNAME?= yes # use work.`hostname`
! 68: FAILOVER_FETCH= yes # insist on the correct checksum
! 69: PKG_DEVELOPER?= yes
! 70: _ACCEPTABLE= yes</programlisting>
! 71: </sect3>
! 72:
! 73: <sect3>
! 74: <title><filename>build.conf</filename></title>
! 75:
! 76: <para>In <filename>pkgsrc/mk/bulk</filename>, copy
! 77: <filename>build.conf-example</filename> to
! 78: <filename>build.conf</filename> and
! 79: edit it, following the comments in that file. This is the config
! 80: file that determines where log files are generated after the build,
! 81: where to mail the build report to, where your pkgsrc tree is located and
! 82: which user to &man.su.8; to to do a <command>cvs update</command>.</para>
! 83: </sect3>
! 84:
! 85: <sect3>
! 86: <title><filename>pre-build.local</filename></title>
! 87:
! 88: <para>It is possible to configure the bulk build to perform certain site
! 89: specific tasks at the end of the pre-build stage. If the file
! 90: <filename>pre-build.local</filename> exists in
! 91: <filename>/usr/pkgsrc/mk/bulk</filename> it will be executed
! 92: (as a sh(1) script) at the end of the usual pre-build stage. An
! 93: example use of <filename>pre-build.local</filename> is to have the
! 94: line:</para>
! 95:
! 96: <screen>&rprompt; <userinput>echo "I do not have enough disk space to build this pig." \
! 97: > pkgsrc/games/crafty-book-enormous/$BROKENF</userinput></screen>
! 98:
! 99: <para>to prevent the system from trying to build a particular package
! 100: which requires nearly 3 GB of disk space.</para>
! 101: </sect3>
! 102: </sect2>
! 103:
! 104: <sect2>
! 105: <title>Other environmental considerations</title>
! 106:
! 107: <para>As <filename>/usr/pkg</filename> will be completely deleted at the
! 108: start of bulk builds, make sure your login shell is placed somewhere
! 109: else. Either drop it into <filename>/usr/local/bin</filename>
! 110: (and adjust your login shell in the passwd file), or (re-)install
! 111: it via &man.pkg.add.1; from
! 112: <filename>/etc/rc.local</filename>, so you can login after a reboot
! 113: (remember that your current process won't die if the package is
! 114: removed, you just can't start any new instances of the shell any more).
! 115: Also, if you use &os; earlier than 1.5, or you still want to use the
! 116: pkgsrc version of ssh for some reason, be sure to install ssh before
! 117: starting it from <filename>rc.local</filename>:</para>
! 118:
! 119: <programlisting>( cd /usr/pkgsrc/security/ssh ; make bulk-install )
! 120: if [ -f /usr/pkg/etc/rc.d/sshd ]; then
! 121: /usr/pkg/etc/rc.d/sshd
! 122: fi</programlisting>
! 123:
! 124: <para>Not doing so will result in you being not able to log in via ssh
! 125: after the bulk build is finished or if the machine gets rebooted
! 126: or crashes. You have been warned! :)</para>
! 127: </sect2>
! 128:
! 129: <sect2>
! 130: <title>Operation</title>
! 131:
! 132: <para>Make sure you don't need any of the packages still installed.</para>
! 133:
! 134: <warning>
! 135: <para>During the bulk build, <emphasis>all packages will be
! 136: removed!</emphasis></para>
! 137: </warning>
! 138:
! 139: <para>Be sure to remove all other things that might
! 140: interfere with builds, like some libs installed in
! 141: <filename>/usr/local</filename>, etc. then become root and type:</para>
! 142:
! 143: <screen>&rprompt; <userinput>cd /usr/pkgsrc</userinput>
! 144: &rprompt; <userinput>sh mk/bulk/build</userinput></screen>
! 145:
! 146: <para>If for some reason your last build didn't complete (power failure,
! 147: system panic, ...), you can continue it by running:</para>
! 148:
! 149: <screen>&rprompt; <userinput>sh mk/bulk/build restart</userinput></screen>
! 150:
! 151: <para>At the end of the bulk build, you will get a summary via mail, and find
! 152: build logs in the directory specified by <varname>FTP</varname> in the
! 153: <filename>build.conf</filename> file.</para>
! 154: </sect2>
! 155:
! 156: <sect2>
! 157: <title>What it does</title>
! 158:
! 159: <para>The bulk builds consist of three steps:</para>
! 160:
! 161: <variablelist>
! 162: <varlistentry>
! 163: <term>1. pre-build</term>
! 164:
! 165: <listitem>
! 166: <para>The script updates your pkgsrc tree via (anon)cvs, then cleans
! 167: out any broken distfiles, and removes all packages installed.</para>
! 168: </listitem>
! 169: </varlistentry>
! 170:
! 171: <varlistentry>
! 172: <term>2. the bulk build</term>
! 173:
! 174: <listitem>
! 175: <para>This is basically
! 176: <quote>make bulk-package</quote> with an optimised
! 177: order in which packages will be built. Packages that don't require
! 178: other packages will be built first, and packages with many dependencies
! 179: will be built later.</para>
! 180: </listitem>
! 181: </varlistentry>
! 182:
! 183: <varlistentry>
! 184: <term>3. post-build</term>
! 185:
! 186: <listitem>
! 187: <para>Generates a report that's placed in the directory specified
! 188: in the <filename>build.conf</filename> file named
! 189: <filename>broken.html</filename>, a short version of
! 190: that report will also be mailed to the build's admin.</para>
! 191: </listitem>
! 192: </varlistentry>
! 193: </variablelist>
! 194:
! 195: <para>During the build, a list of broken packages will be compiled in
! 196: <filename>/usr/pkgsrc/.broken</filename> (or
! 197: <filename>.../.broken.${MACHINE}</filename> if
! 198: <varname>OBJMACHINE</varname> is set),
! 199: individual build logs of broken builds can be found in the package's
! 200: directory. These files are used by the bulk-targets to mark broken builds
! 201: to not waste time trying to rebuild them, and they can be used to debug
! 202: these broken package builds later.</para>
! 203: </sect2>
! 204:
! 205: <sect2>
! 206: <title>Disk space requirements</title>
! 207:
! 208: <para>Currently, roughly the following requirements are valid for
! 209: NetBSD 2.0/i386:</para>
! 210:
! 211: <itemizedlist>
! 212: <listitem>
! 213: <para>10 GB - distfiles (NFS ok)</para>
! 214: </listitem>
! 215:
! 216: <listitem>
! 217: <para>8 GB - full set of all binaries (NFS ok)</para>
! 218: </listitem>
! 219:
! 220: <listitem>
! 221: <para>5 GB - temp space for compiling (local disk recommended)</para>
! 222: </listitem>
! 223: </itemizedlist>
! 224:
! 225: <para>Note that all pkgs will be de-installed as soon as they are turned into a
! 226: binary package, and that sources are removed, so there is no excessively huge
! 227: demand to disk space. Afterwards, if the package is needed again, it will
! 228: be installed via &man.pkg.add.1; instead of building again, so
! 229: there are no cycles wasted by recompiling.</para>
! 230: </sect2>
! 231:
! 232: <sect2>
! 233: <title>Setting up a sandbox for chroot'ed builds</title>
! 234:
! 235: <para>If you don't want all the pkgs nuked from a machine (rendering it useless
! 236: for anything but pkg compiling), there is the possibility of doing the pkg
! 237: bulk build inside a chroot environment.</para>
! 238:
! 239: <para>The first step to do so is setting up a chroot
! 240: sandbox, e.g. <filename>/usr/sandbox</filename>.
! 241: After extracting all the sets from a &os; installation or doing a
! 242: <command>make distribution DESTDIR=/usr/sandbox</command> in
! 243: <filename>/usr/src/etc</filename>, be sure the following
! 244: items are present and properly configured:</para>
! 245:
! 246: <procedure>
! 247: <step>
! 248: <para>Kernel</para>
! 249:
! 250: <screen>&rprompt; <userinput>cp /netbsd /usr/sandbox</userinput></screen>
! 251: </step>
! 252:
! 253: <step>
! 254: <para><filename>/dev/*</filename></para>
! 255:
! 256: <screen>&rprompt; <userinput>cd /usr/sandbox/dev ; sh MAKEDEV all</userinput></screen>
! 257: </step>
! 258:
! 259: <step>
! 260: <para><filename>/etc/resolv.conf</filename> (for <pkg>security/smtpd</pkg> and mail):</para>
! 261:
! 262: <screen>&rprompt; <userinput>cp /etc/resolv.conf /usr/sandbox/etc</userinput></screen>
! 263: </step>
! 264:
! 265: <step>
! 266: <para>Working(!) mail config (hostname, sendmail.cf):</para>
! 267:
! 268: <screen>&rprompt; <userinput>cp /etc/mail/sendmail.cf /usr/sandbox/etc/mail</userinput></screen>
! 269: </step>
! 270:
! 271: <step>
! 272: <para><filename>/etc/localtime</filename> (for <pkg>security/smtpd</pkg>):</para>
! 273:
! 274: <screen>&rprompt; <userinput>ln -sf /usr/share/zoneinfo/UTC /usr/sandbox/etc/localtime</userinput></screen>
! 275: </step>
! 276:
! 277: <step>
! 278: <para><filename>/usr/src</filename> (system sources, for <pkg>sysutils/aperture</pkg>,
! 279: <pkg>net/ppp-mppe</pkg>):</para>
! 280:
! 281: <screen>&rprompt; <userinput>ln -s ../disk1/cvs .</userinput>
! 282: &rprompt; <userinput>ln -s cvs/src-1.6 src</userinput></screen>
! 283: </step>
! 284:
! 285: <step>
! 286: <para>Create <filename>/var/db/pkg</filename> (not part of default install):</para>
! 287:
! 288: <screen>&rprompt; <userinput>mkdir /usr/sandbox/var/db/pkg</userinput></screen>
! 289: </step>
! 290:
! 291: <step>
! 292: <para>Create <filename>/usr/pkg</filename> (not part of default install):</para>
! 293:
! 294: <screen>&rprompt; <userinput>mkdir /usr/sandbox/usr/pkg</userinput></screen>
! 295: </step>
! 296:
! 297: <step>
! 298: <para>Checkout pkgsrc via cvs into
! 299: <filename>/usr/sandbox/usr/pkgsrc</filename>:</para>
! 300:
! 301: <screen>&rprompt; <userinput>cd /usr/sandbox/usr</userinput>
! 302: &rprompt; <userinput>cvs -d anoncvs@anoncvs.NetBSD.org:/cvsroot checkout -d -P pkgsrc</userinput></screen>
! 303:
! 304: <para>Do not mount/link this to the copy of your pkgsrc tree
! 305: you do development in, as this will likely cause problems!
! 306: </para>
! 307: </step>
! 308:
! 309: <step>
! 310: <para>Make
! 311: <filename>/usr/sandbox/usr/pkgsrc/packages</filename> and
! 312: <filename>.../distfiles</filename> point somewhere
! 313: appropriate. NFS- and/or nullfs-mounts may come in handy!
! 314: </para>
! 315: </step>
! 316:
! 317: <step>
! 318: <para>Edit <filename>/etc/mk.conf</filename>, see <xref linkend="binary.mk.conf"/>.</para>
! 319: </step>
! 320:
! 321: <step>
! 322: <para>Adjust <filename>mk/bulk/build.conf</filename> to suit your needs.</para>
! 323: </step>
! 324:
! 325: <step>
! 326: <para>If you have set <varname>CVS_USER</varname> in
! 327: <filename>build.conf</filename>, make sure that
! 328: account exists and can do a <command>cvs ${CVS_FLAGS}
! 329: update</command> properly!</para>
! 330: </step>
! 331: </procedure>
! 332:
! 333: <para>When the chroot sandbox is setup, you can start
! 334: the build with the following steps:</para>
! 335:
! 336: <screen>&rprompt; <userinput>cd /usr/sandbox/usr/pkgsrc</userinput>
! 337: &rprompt; <userinput>sh mk/bulk/do-sandbox-build</userinput></screen>
! 338:
! 339: <para>This will just jump inside the sandbox and start building.
! 340: At the end of the build, mail will be sent with the results of the build.
! 341: Created binary pkgs will be in
! 342: <filename>/usr/sandbox/usr/pkgsrc/packages</filename> (wherever
! 343: that points/mounts to/from).</para>
! 344: </sect2>
! 345:
! 346: <sect2>
! 347: <title>Building a partial set of packages</title>
! 348:
! 349: <para> In addition to building a complete set of all packages in
! 350: pkgsrc, the <filename>pkgsrc/mk/bulk/build</filename> script
! 351: may be used to build a subset of the packages contained in
! 352: pkgsrc. By setting defining <varname>SPECIFIC_PKGS</varname>
! 353: in <filename>/etc/mk.conf</filename>, the variables</para>
! 354:
! 355: <itemizedlist>
! 356: <listitem><para>SITE_SPECIFIC_PKGS</para></listitem>
! 357: <listitem><para>HOST_SPECIFIC_PKGS</para></listitem>
! 358: <listitem><para>GROUP_SPECIFIC_PKGS</para></listitem>
! 359: <listitem><para>USER_SPECIFIC_PKGS</para></listitem>
! 360: </itemizedlist>
! 361:
! 362: <para> will define the set of packages which should be built.
! 363: The bulk build code will also include any packages which are
! 364: needed as dependencies for the explicitly listed packages.
! 365: </para>
! 366:
! 367: <para> One use of this is to do a bulk build with
! 368: <varname>SPECIFIC_PKGS</varname> in a chroot sandbox
! 369: periodically to have a complete set of the binary packages
! 370: needed for your site available without the overhead of
! 371: building extra packages that are not needed. </para>
! 372:
! 373: </sect2>
! 374: </sect1>
! 375:
! 376: <sect1>
! 377: <title>Creating a multiple CD-ROM packages collection</title>
! 378:
! 379: <para>After your pkgsrc bulk-build has completed, you may wish to create a CD-ROM
! 380: set of the resulting binary packages to assist in installing packages on
! 381: other machines. The <pkg>pkgtools/cdpack</pkg> package provides a simple
! 382: tool for creating the ISO 9660 images. <command>cdpack</command> arranges
! 383: the packages on the CD-ROMs in a way that keeps all the dependencies for
! 384: given package on the same CD as that package.</para>
! 385:
! 386: <sect2>
! 387: <title>Example of cdpack</title>
! 388:
! 389: <para>Complete documentation for cdpack is found in the
! 390: cdpack(1) manpage. The following
! 391: short example assumes that the binary packages are left in
! 392: <filename>/usr/pkgsrc/packages/All</filename> and that sufficient disk
! 393: space exists in <filename>/u2</filename> to hold the ISO 9660 images.</para>
! 394:
! 395: <screen>&rprompt; <userinput>mkdir /u2/images</userinput>
! 396: &rprompt; <userinput>pkg_add /usr/pkgsrc/packages/All/cdpack</userinput>
! 397: &rprompt; <userinput>cdpack /usr/pkgsrc/packages/All /u2/images</userinput></screen>
! 398:
! 399: <para>If you wish to include a common set of files
! 400: (<filename>COPYRIGHT</filename>, <filename>README</filename>, etc.)
! 401: on each CD in the collection, then you need to create a directory which
! 402: contains these files. e.g.</para>
! 403:
! 404: <screen>&rprompt; <userinput>mkdir /tmp/common</userinput>
! 405: &rprompt; <userinput>echo "This is a README" > /tmp/common/README</userinput>
! 406: &rprompt; <userinput>echo "Another file" > /tmp/common/COPYING</userinput>
! 407: &rprompt; <userinput>mkdir /tmp/common/bin</userinput>
! 408: &rprompt; <userinput>echo "#!/bin/sh" > /tmp/common/bin/myscript</userinput>
! 409: &rprompt; <userinput>echo "echo Hello world" >> /tmp/common/bin/myscript</userinput>
! 410: &rprompt; <userinput>chmod 755 /tmp/common/bin/myscript</userinput></screen>
! 411:
! 412: <para>Now create the images:</para>
! 413:
! 414: <screen>&rprompt; <userinput>cdpack -x /tmp/common /usr/pkgsrc/packages/All /u2/images</userinput></screen>
! 415:
! 416: <para>Each image will contain <filename>README</filename>,
! 417: <filename>COPYING</filename>, and <filename>bin/myscript</filename>
! 418: in their root directories.</para>
! 419: </sect2>
! 420: </sect1>
! 421: </chapter>
CVSweb <webmaster@jp.NetBSD.org>