Annotation of src/BUILDING, Revision 1.20
1.1 tv 1: BUILDING(8) NetBSD System Manager's Manual BUILDING(8)
2:
3: NAME
4: BUILDING - Procedure for building NetBSD from source code.
5:
6: STATUS
7: This document is a work-in-progress. As such, the information described
8: here may not match the reality of the build system as of this writing.
9: Once this document is completely in sync with reality, this paragraph
10: will be removed.
11:
12: Discrepancies between this documentation and the current reality of im-
13: plementation are noted specially, as with the note below:
14:
15: Note: This document applies only to platforms which use the new toolchain
1.15 lukem 16: as indicated by the default setting of TOOLCHAIN_MISSING in <bsd.own.mk>.
1.5 tv 17: Platforms which have not yet been switched to the new toolchain should
18: continue building traditionally, using the notes specified in the file
19: UPDATING.
1.1 tv 20:
21: REQUIREMENTS
22: NetBSD is designed to be buildable on most POSIX-compliant host systems.
23: The basic build procedure is the same whether compiling natively (on the
24: same NetBSD architecture) or cross compiling (on another architecture or
25: OS).
26:
27: This source tree contains a special subtree, ``tools'', which uses the
28: host system to create a build toolchain for the target architecture. The
29: host system must have at least C and C++ compilers in order to create the
30: toolchain (make is not required); all other tools are created as part of
31: the NetBSD build process.
32:
33: Note: A couple host toolchain components are not yet available in
34: the tools directory. Also, some tools use non-POSIX, non-ANSI C
35: extensions and need to be standardized. As a result, cross-compil-
36: ing from systems other than NetBSD is not currently supported.
37:
38: FILES
39: Source tree layout
40:
1.17 wiz 41: doc/BUILDING.mdoc
42: This document (in -mdoc troff format; the original copy).
1.1 tv 43:
1.2 wiz 44: BUILDING This document (in plaintext).
1.1 tv 45:
46: Makefile The main Makefile for NetBSD; should only be run for na-
47: tive builds with an appropriately up-to-date version of
48: NetBSD make(1). (For building from out-of-date systems or
49: on a non-native host, see the build.sh shell script.)
50:
51: UPDATING Special notes for updating from an earlier revision of
52: NetBSD. It is important to read this file before every
53: build of an updated source tree.
54:
55: build.sh Bourne-compatible shell script used for building the host
56: build tools and the NetBSD system from scratch. Can be
57: used for both native and cross builds, and should be used
58: instead of make(1) for any source tree that is updated and
59: recompiled regularly.
60:
61: crypto/dist/, dist/, gnu/dist/
62: Sources imported verbatim from third parties, without man-
63: gling the existing build structure. Other source trees in
64: bin through usr.sbin use the NetBSD make(1) ``reachover''
65: Makefile semantics when building these programs for a na-
66: tive host.
67:
68: distrib/, etc/
69: Sources for items used when making a full release snap-
70: shot, such as files installed in /etc on the destination
71: system, boot media, and release notes.
72:
73: regress/ Regression test harness. Can be cross-compiled, but only
74: run natively.
75:
76: sys/ NetBSD kernel sources.
77:
78: tools/ ``Reachover'' build structure for the host build tools.
79: This has a special method of determining out-of-date sta-
80: tus.
81:
82: bin/ ... usr.sbin/
83: Sources to the NetBSD userland (non-kernel) programs. If
84: any of these directories are missing, they will be skipped
85: during the build.
86:
87: Build tree layout
88: The NetBSD build tree is described in hier(7), and the release layout is
89: described in release(7).
90:
91: CONFIGURATION
1.8 lukem 92: Environment variables
93: Several environment variables control the behaviour of NetBSD builds.
94:
95: MACHINE Machine type.
96:
97: MACHINE_ARCH Machine architecture.
98:
99: MAKE Path name to invoke make(1) as.
100:
101: MAKEFLAGS Flags to invoke make(1) with.
102:
103: MAKEOBJDIR Directory to use as the .OBJDIR for the current direc-
104: tory. Used only if MAKEOBJDIRPREFIX is not defined.
105: MAKEOBJDIR can only be provided in the environment.
106:
107: MAKEOBJDIRPREFIX Top level directory of the object directory tree. If
108: this is defined, ${MAKEOBJDIRPREFIX}/${.CURDIR} is used
109: as the .OBJDIR for the current directory. The current
110: directory may be read only. MAKEOBJDIRPREFIX can only
111: be provided in the environment.
112:
1.1 tv 113: "make" variables
114: Several variables control the behavior of NetBSD builds. Unless other-
115: wise specified, these variables may be set in either the process environ-
116: ment or the make(1) configuration file specified by MAKECONF.
117:
1.9 thorpej 118: BUILDID Identifier for the build. The identifier will be appended to
119: object directory names, and can be consulted in the make(1)
120: configuration file in order to set additional build parame-
121: ters, such as compiler flags.
122:
1.1 tv 123: DESTDIR Directory to contain the built NetBSD system. If set, spe-
124: cial options are passed to the compilation tools to prevent
125: their default use of the host system's /usr/include,
126: /usr/lib, and so forth. This pathname should not end with a
127: slash (/) character (for installation into the system's root
1.7 lukem 128: directory, set DESTDIR to an empty string). The directory
129: must reside on a file system which supports long file names
130: and hard links.
1.1 tv 131:
132: Default: Empty string if USETOOLS is ``yes''; unset other-
133: wise.
134:
135: MAKECONF The name of the make(1) configuration file. Only settable in
136: the process environment.
137:
138: Default: ``/etc/mk.conf''
139:
140: MKCATPAGES Can be set to ``yes'' or ``no''. Indicates whether prefor-
141: matted plaintext manual pages will be created during a build.
142:
143: Default: ``yes''
144:
145: MKCRYPTO Can be set to ``yes'' or ``no''. Indicates whether crypto-
146: graphic code will be included in a build; provided for the
147: benefit of countries that do not allow strong cryptography.
148: Will not affect use of the standard low-security password en-
149: cryption system, crypt(3).
150:
151: Default: ``yes''
152:
153: MKDOC Can be set to ``yes'' or ``no''. Indicates whether system
154: documentation destined for /usr/share/doc will be installed
155: during a build.
156:
157: Default: ``yes''
158:
1.4 tv 159: MKHOSTOBJ Can be set to ``yes'' or ``no''. If set to ``yes'', then for
160: programs intended to be run on the compile host, the name,
161: release, and architecture of the host operating system will
162: be suffixed to the name of the object directory created by
163: ``make obj''. (This allows multiple host systems to compile
164: NetBSD for a single target.) If set to ``no'', then programs
165: built to be run on the compile host will use the same object
166: directory names as programs built to be run on the target.
167:
168: Default: ``no''
169:
1.1 tv 170: MKINFO Can be set to ``yes'' or ``no''. Indicates whether GNU Info
171: files, used for the documentation for most of the compilation
172: tools, will be created and installed during a build.
173:
174: Default: ``yes''
175:
176: MKLINT Can be set to ``yes'' or ``no''. Indicates whether lint(1)
177: will be run against portions of the NetBSD source code during
178: the build, and whether lint libraries will be installed into
179: /usr/libdata/lint.
180:
181: Default: ``yes''
182:
183: MKMAN Can be set to ``yes'' or ``no''. Indicates whether manual
184: pages will be installed during a build.
185:
186: Default: ``yes''
187:
188: MKNLS Can be set to ``yes'' or ``no''. Indicates whether Native
189: Language System locale zone files will be compiled and in-
190: stalled during a build.
191:
192: Default: ``yes''
193:
194: MKOBJ Can be set to ``yes'' or ``no''. Indicates whether object
195: directories will be created when running ``make obj''. If
196: set to ``no'', then all built files will be located inside
197: the regular source tree.
198:
199: Default: ``yes''
200:
201: MKPIC Can be set to ``yes'' or ``no''. Indicates whether shared
202: objects and libraries will be created and installed during a
203: build. If set to ``no'', the entire built system will be
204: statically linked.
205:
206: Default: Platform dependent. As of this writing, all plat-
207: forms except sh3 default to ``yes''.
208:
209: MKPICINSTALL
210: Can be set to ``yes'' or ``no''. Indicates whether the ar(1)
211: format libraries (lib*_pic.a), used to generate shared li-
212: braries, are installed during a build.
213:
214: Default: ``yes''
215:
216: MKPROFILE Can be set to ``yes'' or ``no''. Indicates whether profiled
217: libraries (lib*_p.a) will be built and installed during a
218: build.
219:
220: Default: ``yes''; however, some platforms turn off MKPROFILE
221: by default at times due to toolchain problems with profiled
222: code.
223:
224: MKSHARE Can be set to ``yes'' or ``no''. Indicates whether files
225: destined to reside in /usr/share will be built and installed
226: during a build. If set to ``no'', then all of MKCATPAGES,
227: MKDOC, MKINFO, MKMAN, and MKNLS will be set to ``no'' uncon-
228: ditionally.
229:
230: Default: ``yes''
231:
232: TOOLDIR Directory to hold the host tools, once built. This directory
233: should be unique to a given host system and NetBSD source
234: tree. (However, multiple targets may share the same TOOLDIR;
235: the target-dependent files have unique names.) If unset, a
236: default based on the uname(1) information of the host plat-
237: form will be created in the .OBJDIR of src/tools.
238:
239: Default: Unset.
240:
1.7 lukem 241: UNPRIVED If set, then an unprivileged install will occur. The user,
242: group, permissions, and file flags, will not be set on the
243: installed item; instead the information will be appended to a
1.11 lukem 244: file called METALOG in DESTDIR. The contents of METALOG is
245: used during the generation of the distribution tar files to
246: ensure that the appropriate file ownership is stored.
1.7 lukem 247:
248: Default: Unset.
249:
1.1 tv 250: UPDATE If set, then all install operations intended to write to
251: DESTDIR will compare file timestamps before installing, and
252: skip the install phase if the destination files are up-to-
253: date. This also has implications on full builds (see next
254: subsection).
255:
256: Default: Unset.
257:
258: USETOOLS Indicates whether the tools specified by TOOLDIR should be
259: used as part of a build in progress. Must be set to ``yes''
260: if cross-compiling.
261:
262: yes Use the tools from TOOLDIR.
263:
264: no Do not use the tools from TOOLDIR, but refuse to build
265: native compilation tool components that are version-
266: specific for that tool.
267:
268: never Do not use the tools from TOOLDIR, even when building
269: native tool components. This is similar to the tradi-
270: tional NetBSD build method, but does not verify that
271: the compilation tools in use are up-to-date enough in
272: order to build the tree successfully. This may cause
273: build or runtime problems when building the whole
274: NetBSD source tree.
275:
276: Default: ``yes'' if building all or part of a whole NetBSD
277: source tree (detected automatically); ``no'' otherwise (to
278: preserve traditional semantics of the <bsd.*.mk> make(1) in-
279: clude files).
280:
281: "make" variables for full builds
1.7 lukem 282: These variables only affect the top level ``Makefile'' and do not affect
283: manually building subtrees of the NetBSD source code.
1.1 tv 284:
1.20 ! lukem 285: INSTALLWORLDDIR Location for the ``make installworld'' target to install
! 286: to.
1.1 tv 287:
1.20 ! lukem 288: Default: ``/''
1.1 tv 289:
1.20 ! lukem 290: MKOBJDIRS Can be set to ``yes'' or ``no''. Indicates whether ob-
! 291: ject directories will be created automatically (via a
! 292: ``make obj'' pass) at the start of a build.
1.1 tv 293:
1.20 ! lukem 294: Default: ``yes''
1.1 tv 295:
1.20 ! lukem 296: NBUILDJOBS Now obsolete. Use the make(1) option -j, instead (see
! 297: below)
1.1 tv 298:
1.20 ! lukem 299: Default: Unset.
1.1 tv 300:
1.20 ! lukem 301: NOCLEANDIR If set, avoids the ``make cleandir'' phase of a full
! 302: build. This has the effect of allowing only changed
! 303: files in a source tree to be recompiled. This can speed
! 304: up builds when updating only a few files in the tree.
1.1 tv 305:
1.20 ! lukem 306: Default: Unset.
1.1 tv 307:
1.20 ! lukem 308: NODISTRIBDIRS If set, avoids the ``make distrib-dirs'' phase of a full
! 309: build. This skips running mtree(8) on DESTDIR, useful
! 310: on systems where building as an unprivileged user, or
! 311: where it is known that the system-wide mtree files have
! 312: not changed.
1.1 tv 313:
1.20 ! lukem 314: Default: Unset.
1.1 tv 315:
1.20 ! lukem 316: NOINCLUDES If set, avoids the ``make includes'' phase of a full
! 317: build. This has the effect of preventing make(1) from
! 318: thinking that some programs are out-of-date simply be-
! 319: cause the system include files have changed. However,
! 320: this option should not be used when updating the entire
! 321: NetBSD source tree arbitrarily; it is suggested to use
! 322: UPDATE in that case.
1.1 tv 323:
1.20 ! lukem 324: Default: Unset.
1.1 tv 325:
1.20 ! lukem 326: RELEASEDIR If set, specifies the directory to which a release(7)
! 327: layout will be written at the end of a ``make release''.
! 328:
! 329: Default: Unset.
! 330:
! 331: UPDATE If set, then in addition to the effects described for
! 332: UPDATE above, this implies the effects of NOCLEANDIR.
1.1 tv 333:
334: BUILDING
335: "make" command line options
336: This is only a summary of options available to make(1); only the options
337: used most frequently with NetBSD builds are listed here.
338:
1.10 sommerfe 339: -j njob Run up to njob make(1) subjobs in parallel. Makefiles should
340: use .WAIT or have explicit dependancies as necessary to en-
341: force build ordering. If you see build failures with -j,
342: please save complete build logs so the failures can be ana-
343: lyzed.
344:
1.1 tv 345: -m dir Specify the default directory for searching for system Make-
346: file segments, mainly the <bsd.*.mk> files. When building any
347: full NetBSD source tree, this should be set to the
348: ``share/mk'' directory in the source tree. (This is set auto-
349: matically when building from the top level.)
350:
351: -n Display the commands that would have been executed, but do not
352: actually execute them. This will still cause recursion to
353: take place.
354:
355: -v var Print make(1)'s idea of the value of var. Does not build any
356: targets.
357:
358: var=value Set the variable var to value, overriding any setting speci-
359: fied by the process environment, the MAKECONF configuration
360: file, or the system Makefile segments.
361:
362: "make" targets
363: These default targets may be built by running make(1) in any subtree of
364: the NetBSD source code. It is recommended that none of these be used
365: from the top level Makefile; as a specific exception, ``make obj'' and
366: ``make cleandir'' are useful in that context.
367:
368: all Build programs, libraries, and preformatted documentation.
369:
370: clean Remove program and library object code files.
371:
372: cleandir Same as clean, but also remove preformatted documentation, de-
373: pendency files generated by ``make depend'', and any other
374: files known to be created at build time. ``make distclean''
375: may be used as a synonym, for familiarity with a similar well-
376: known convention.
377:
378: depend Create dependency files (.depend) containing more detailed in-
379: formation about the dependencies of source code on header
380: files. Allows programs to be recompiled automatically when a
381: dependency changes.
382:
383: dependall Does a ``make depend'' immediately followed by a ``make all''.
1.10 sommerfe 384: This improves cache locality of the build since both passes
385: read the source files in their entirety.
1.1 tv 386:
387: includes Build and install system header files. Typically needed be-
388: fore any system libraries or programs can be built.
389:
390: install Install programs, libraries, and documentation into DESTDIR.
1.14 fredb 391: Few files will be installed to /dev, /etc, /root or /var in
392: order to prevent user supplied configuration data from being
393: overwritten.
1.1 tv 394:
395: lint Run lint(1) against the C source code, where appropriate, and
396: generate system-installed lint libraries.
397:
398: obj Create object directories to be used for built files, instead
399: of building directly in the source tree.
400:
401: tags Create ctags(1) searchable function lists usable by the ex(1)
402: and vi(1) text editors.
403:
404: "make" targets for the top level
405: Additional make(1) targets are usable specifically from the top source
406: level to facilitate building the entire NetBSD source tree.
407:
1.20 ! lukem 408: build Build the entire NetBSD system. This orders portions of
! 409: the source tree such that prerequisites will be built in
! 410: the proper order.
! 411:
! 412: distribution Do a ``make build'', and then install a full distribution
! 413: into DESTDIR.
! 414:
! 415: buildworld As per ``make distribution'', except that it ensures that
! 416: DESTDIR is not the root directory.
! 417:
! 418: installworld Install the distribution from DESTDIR to INSTALLWORLDDIR
! 419: (which defaults to the root directory). Ensures that
! 420: INSTALLWORLDDIR is the not root directory if cross compil-
! 421: ing.
! 422:
! 423: Note: It is highly recommended that you upgrade your kernel
! 424: and reboot before performing this operation.
! 425:
! 426: release Do a ``make build'', then package the system into a stan-
! 427: dard release layout as described by release(7). This re-
! 428: quires that RELEASEDIR be set (see above).
1.1 tv 429:
430: regression-tests
1.20 ! lukem 431: Can only be run after building the regression tests in the
! 432: directory ``regress''. Runs the compiled regression tests
! 433: on the local host.
1.1 tv 434:
435: The "build.sh" script
436: This script file is a Bourne shell script designed to build the entire
437: NetBSD system on any host with a Bourne shell in /bin/sh, including many
438: that are not POSIX compliant. Note that if a host system's /bin/sh is
439: unusually old and broken, the Korn Shell (/bin/ksh), if available, may be
440: a usable alternative.
441:
442: All cross-compile builds, and most native builds, of the entire system
443: should make use of build.sh rather than just running ``make''. This way,
444: the make(1) program will be bootstrapped properly, in case the host sys-
445: tem has an older or incompatible ``make'' program.
446:
447: When compiling the entire system via build.sh, many make(1) variables are
448: set for you in order to help encapsulate the build process. In the list
449: of options below, variables that are automatically set by build.sh are
450: noted where applicable.
451:
452: The following are available command line options that may be supplied to
453: build.sh:
454:
455: -a arch Set the value of MACHINE_ARCH to arch.
1.9 thorpej 456:
457: -B buildid
458: Set the value of BUILDID to buildid. This will also append the
459: build idenfitier to the name of the ``make'' wrapper script so
460: that the resulting name is of the form ``nbmake-MACHINE-
461: BUILDID''.
1.1 tv 462:
463: -b Bootstrap ``make'' and create a nbmake-MACHINE script (see be-
464: low).
465:
1.18 lukem 466: -D dest Set the value of DESTDIR to dest.
467:
1.14 fredb 468: -d Build a full distribution. This differs from a default build
469: in that files will also be installed to /dev, /etc, /root and
470: /var. Note this does not build a ``release''; no release sets
471: are placed in ${RELEASEDIR}. -d is implied by -R.
1.6 thorpej 472:
1.19 lukem 473: -E Set `expert' mode; DESTDIR does not have to be set to a non-
474: root path for builds when this is set.
475:
1.20 ! lukem 476: -i installworlddir
! 477: Install the contents of DESTDIR to installworlddir after all
! 478: other operations have completed, using the top level
! 479: ``installworld'' target.
! 480:
1.10 sommerfe 481: -j njob Passed through to make(1). Makefiles should use .WAIT or have
482: explicit dependancies as necessary to enforce build ordering.
483: If you see build failures with -j, please save complete build
484: logs so the failures can be analyzed.
1.1 tv 485:
1.16 lukem 486: -k kernel
487: Build a new kernel. The kernel argument is the name of a con-
488: figuration file suitable for use by config(8). If kernel does
489: not contain any `/' characters, the configuration file is ex-
490: pected to be found in the KERNCONFDIR directory, which is typi-
491: cally sys/arch/MACHINE/conf. The new kernel will be built in a
492: subdirectory of KERNOBJDIR, which is typically
493: sys/arch/MACHINE/compile or an associated object directory. In
494: order to ensure that the kernel is built using up-to-date
495: tools, it is strongly recommended that the tools be rebuilt
496: (using the -t option) in a separate invocation of build.sh pri-
497: or to using the -k option, or that the -t and -k options be
498: used together in a single invocation of build.sh.
499:
1.18 lukem 500: -M obj Set MAKEOBJDIRPREFIX to obj.
501:
1.1 tv 502: -m mach Set the value of MACHINE to mach. This will also override any
503: value of MACHINE_ARCH in the process environment with a value
504: deduced from mach, unless -a is specified. All cross builds
505: require -m, but if unset on a NetBSD host, the host's value of
506: MACHINE will be detected and used automatically.
507:
508: -n Show the commands that would be executed by build.sh, but do
509: not make any changes. This is similar in concept to ``make
510: -n''.
511:
1.18 lukem 512: -O obj Create an appropriate transform macro for MAKEOBJDIR that will
513: place the built object files under obj. For instance, a set-
514: ting of /usr/obj will place build-time files under
515: /usr/obj/bin, /usr/obj/lib, and so forth.
516:
1.1 tv 517: -o Set the value of MKOBJDIRS to ``no''.
518:
1.18 lukem 519: -R rel Set the value of RELEASEDIR to rel. Setting this option will
520: cause build.sh to run ``make release'' instead of ``make
521: build''.
522:
1.1 tv 523: -r Remove the contents of DESTDIR and TOOLDIR before building
524: (provides a clean starting point). This will skip deleting
525: DESTDIR if building on a native system to the root directory.
526:
1.18 lukem 527: -T tools Set the value of TOOLDIR to tools. If set, the bootstrap
528: ``make'' will only be rebuilt as needed (when the source files
529: for make(1) change).
530:
1.1 tv 531: -t Build and install the host tools from src/tools only. This op-
532: tion implies -b.
533:
1.18 lukem 534: -U Set the UNPRIVED variable.
535:
1.1 tv 536: -u Set the UPDATE variable.
537:
538: -w wrapper
539: Create the nbmake wrapper script (see below) in a custom loca-
540: tion, specified by wrapper. This allows, for instance, to
541: place the wrapper in PATH automatically. Note that wrapper is
542: the full name of the file, not just a directory name.
543:
544: The "nbmake-MACHINE" wrapper script
545: If using the build.sh script to build NetBSD, a nbmake-MACHINE script
546: will be created in TOOLDIR/bin upon the first build to assist in building
547: subtrees on a cross-compile host.
548:
549: nbmake-MACHINE can be invoked in lieu of make(1), and will instead call
550: the up-to-date version of ``nbmake'' installed into TOOLDIR/bin with sev-
551: eral key variables pre-set, including MACHINE, MACHINE_ARCH, and TOOLDIR.
552: This script can be symlinked into a directory listed in PATH, or called
553: with an absolute path.
554:
1.16 lukem 555: EXAMPLES
556: ./build.sh -t
557: Build a new toolchain.
558:
559: cd ${KERNCONFDIR} ; ${TOOLDIR}/bin/nbconfig GENERIC
560: Use the new version of config(8) to prepare to build a new
561: GENERIC kernel.
562:
563: cd ${KERNOBJDIR}/GENERIC ; ${TOOLDIR}/bin/nbmake-${MACHINE} dependall
564: Use the new toolchain to build a new GENERIC kernel.
565:
566: ./build.sh -t -k GENERIC
567: Build a new toolchain, and use the new toolchain to configure
568: and build a new GENERIC kernel.
569:
570: ./build.sh -U -d
571: Using unprivileged mode, build a complete distribution in
572: DESTDIR.
573:
574: ./build.sh -U -R /some/dir/RELEASE
575: Using unprivileged mode, build a complete release in the speci-
576: fied release directory.
577:
1.10 sommerfe 578: OBSOLETE VARIABLES
1.15 lukem 579: NBUILDJOBS Use the make(1) option -j, instead.
580:
581: USE_NEW_TOOLCHAIN
582: The new toolchain is now the default. To disable, use
583: TOOLCHAIN_MISSING=yes.
1.10 sommerfe 584:
1.1 tv 585: SEE ALSO
586: make(1), hier(7), release(7)
587:
588: HISTORY
1.15 lukem 589: The build.sh based build scheme was introduced for NetBSD 1.6 as
590: USE_NEW_TOOLCHAIN, and re-worked to TOOLCHAIN_MISSING after that.
1.1 tv 591:
592: BUGS
1.15 lukem 593: A few platforms are not yet using this build system.
1.1 tv 594:
1.20 ! lukem 595: NetBSD December 25, 2002 9
CVSweb <webmaster@jp.NetBSD.org>