src/BUILDING - annotate

Return to BUILDING CVS log

Up to [cvs.NetBSD.org] / src

Annotation of src/BUILDING, Revision 1.17

1.1 tv 1: BUILDING(8) NetBSD System Manager's Manual BUILDING(8)
2:
3: NAME
4: BUILDING - Procedure for building NetBSD from source code.
5:
6: STATUS
7: This document is a work-in-progress. As such, the information described
8: here may not match the reality of the build system as of this writing.
9: Once this document is completely in sync with reality, this paragraph
10: will be removed.
11:
12: Discrepancies between this documentation and the current reality of im-
13: plementation are noted specially, as with the note below:
14:
15: Note: This document applies only to platforms which use the new toolchain
1.15 lukem 16: as indicated by the default setting of TOOLCHAIN_MISSING in <bsd.own.mk>.
1.5 tv 17: Platforms which have not yet been switched to the new toolchain should
18: continue building traditionally, using the notes specified in the file
19: UPDATING.
1.1 tv 20:
21: REQUIREMENTS
22: NetBSD is designed to be buildable on most POSIX-compliant host systems.
23: The basic build procedure is the same whether compiling natively (on the
24: same NetBSD architecture) or cross compiling (on another architecture or
25: OS).
26:
27: This source tree contains a special subtree, ``tools'', which uses the
28: host system to create a build toolchain for the target architecture. The
29: host system must have at least C and C++ compilers in order to create the
30: toolchain (make is not required); all other tools are created as part of
31: the NetBSD build process.
32:
33: Note: A couple host toolchain components are not yet available in
34: the tools directory. Also, some tools use non-POSIX, non-ANSI C
35: extensions and need to be standardized. As a result, cross-compil-
36: ing from systems other than NetBSD is not currently supported.
37:
38: FILES
39: Source tree layout
40:
1.17 ! wiz 41: doc/BUILDING.mdoc
! 42: This document (in -mdoc troff format; the original copy).
1.1 tv 43:
1.2 wiz 44: BUILDING This document (in plaintext).
1.1 tv 45:
46: Makefile The main Makefile for NetBSD; should only be run for na-
47: tive builds with an appropriately up-to-date version of
48: NetBSD make(1). (For building from out-of-date systems or
49: on a non-native host, see the build.sh shell script.)
50:
51: UPDATING Special notes for updating from an earlier revision of
52: NetBSD. It is important to read this file before every
53: build of an updated source tree.
54:
55: build.sh Bourne-compatible shell script used for building the host
56: build tools and the NetBSD system from scratch. Can be
57: used for both native and cross builds, and should be used
58: instead of make(1) for any source tree that is updated and
59: recompiled regularly.
60:
61: crypto/dist/, dist/, gnu/dist/
62: Sources imported verbatim from third parties, without man-
63: gling the existing build structure. Other source trees in
64: bin through usr.sbin use the NetBSD make(1) ``reachover''
65: Makefile semantics when building these programs for a na-
66: tive host.
67:
68: distrib/, etc/
69: Sources for items used when making a full release snap-
70: shot, such as files installed in /etc on the destination
71: system, boot media, and release notes.
72:
73: regress/ Regression test harness. Can be cross-compiled, but only
74: run natively.
75:
76: sys/ NetBSD kernel sources.
77:
78: tools/ ``Reachover'' build structure for the host build tools.
79: This has a special method of determining out-of-date sta-
80: tus.
81:
82: bin/ ... usr.sbin/
83: Sources to the NetBSD userland (non-kernel) programs. If
84: any of these directories are missing, they will be skipped
85: during the build.
86:
87: Build tree layout
88: The NetBSD build tree is described in hier(7), and the release layout is
89: described in release(7).
90:
91: CONFIGURATION
1.8 lukem 92: Environment variables
93: Several environment variables control the behaviour of NetBSD builds.
94:
95: MACHINE Machine type.
96:
97: MACHINE_ARCH Machine architecture.
98:
99: MAKE Path name to invoke make(1) as.
100:
101: MAKEFLAGS Flags to invoke make(1) with.
102:
103: MAKEOBJDIR Directory to use as the .OBJDIR for the current direc-
104: tory. Used only if MAKEOBJDIRPREFIX is not defined.
105: MAKEOBJDIR can only be provided in the environment.
106:
107: MAKEOBJDIRPREFIX Top level directory of the object directory tree. If
108: this is defined, ${MAKEOBJDIRPREFIX}/${.CURDIR} is used
109: as the .OBJDIR for the current directory. The current
110: directory may be read only. MAKEOBJDIRPREFIX can only
111: be provided in the environment.
112:
1.1 tv 113: "make" variables
114: Several variables control the behavior of NetBSD builds. Unless other-
115: wise specified, these variables may be set in either the process environ-
116: ment or the make(1) configuration file specified by MAKECONF.
117:
1.9 thorpej 118: BUILDID Identifier for the build. The identifier will be appended to
119: object directory names, and can be consulted in the make(1)
120: configuration file in order to set additional build parame-
121: ters, such as compiler flags.
122:
1.1 tv 123: DESTDIR Directory to contain the built NetBSD system. If set, spe-
124: cial options are passed to the compilation tools to prevent
125: their default use of the host system's /usr/include,
126: /usr/lib, and so forth. This pathname should not end with a
127: slash (/) character (for installation into the system's root
1.7 lukem 128: directory, set DESTDIR to an empty string). The directory
129: must reside on a file system which supports long file names
130: and hard links.
1.1 tv 131:
132: Default: Empty string if USETOOLS is ``yes''; unset other-
133: wise.
134:
135: MAKECONF The name of the make(1) configuration file. Only settable in
136: the process environment.
137:
138: Default: ``/etc/mk.conf''
139:
140: MKCATPAGES Can be set to ``yes'' or ``no''. Indicates whether prefor-
141: matted plaintext manual pages will be created during a build.
142:
143: Default: ``yes''
144:
145: MKCRYPTO Can be set to ``yes'' or ``no''. Indicates whether crypto-
146: graphic code will be included in a build; provided for the
147: benefit of countries that do not allow strong cryptography.
148: Will not affect use of the standard low-security password en-
149: cryption system, crypt(3).
150:
151: Default: ``yes''
152:
153: MKDOC Can be set to ``yes'' or ``no''. Indicates whether system
154: documentation destined for /usr/share/doc will be installed
155: during a build.
156:
157: Default: ``yes''
158:
1.4 tv 159: MKHOSTOBJ Can be set to ``yes'' or ``no''. If set to ``yes'', then for
160: programs intended to be run on the compile host, the name,
161: release, and architecture of the host operating system will
162: be suffixed to the name of the object directory created by
163: ``make obj''. (This allows multiple host systems to compile
164: NetBSD for a single target.) If set to ``no'', then programs
165: built to be run on the compile host will use the same object
166: directory names as programs built to be run on the target.
167:
168: Default: ``no''
169:
1.1 tv 170: MKINFO Can be set to ``yes'' or ``no''. Indicates whether GNU Info
171: files, used for the documentation for most of the compilation
172: tools, will be created and installed during a build.
173:
174: Default: ``yes''
175:
176: MKLINT Can be set to ``yes'' or ``no''. Indicates whether lint(1)
177: will be run against portions of the NetBSD source code during
178: the build, and whether lint libraries will be installed into
179: /usr/libdata/lint.
180:
181: Default: ``yes''
182:
183: MKMAN Can be set to ``yes'' or ``no''. Indicates whether manual
184: pages will be installed during a build.
185:
186: Default: ``yes''
187:
188: MKNLS Can be set to ``yes'' or ``no''. Indicates whether Native
189: Language System locale zone files will be compiled and in-
190: stalled during a build.
191:
192: Default: ``yes''
193:
194: MKOBJ Can be set to ``yes'' or ``no''. Indicates whether object
195: directories will be created when running ``make obj''. If
196: set to ``no'', then all built files will be located inside
197: the regular source tree.
198:
199: Default: ``yes''
200:
201: MKPIC Can be set to ``yes'' or ``no''. Indicates whether shared
202: objects and libraries will be created and installed during a
203: build. If set to ``no'', the entire built system will be
204: statically linked.
205:
206: Default: Platform dependent. As of this writing, all plat-
207: forms except sh3 default to ``yes''.
208:
209: MKPICINSTALL
210: Can be set to ``yes'' or ``no''. Indicates whether the ar(1)
211: format libraries (lib*_pic.a), used to generate shared li-
212: braries, are installed during a build.
213:
214: Default: ``yes''
215:
216: MKPROFILE Can be set to ``yes'' or ``no''. Indicates whether profiled
217: libraries (lib*_p.a) will be built and installed during a
218: build.
219:
220: Default: ``yes''; however, some platforms turn off MKPROFILE
221: by default at times due to toolchain problems with profiled
222: code.
223:
224: MKSHARE Can be set to ``yes'' or ``no''. Indicates whether files
225: destined to reside in /usr/share will be built and installed
226: during a build. If set to ``no'', then all of MKCATPAGES,
227: MKDOC, MKINFO, MKMAN, and MKNLS will be set to ``no'' uncon-
228: ditionally.
229:
230: Default: ``yes''
231:
232: TOOLDIR Directory to hold the host tools, once built. This directory
233: should be unique to a given host system and NetBSD source
234: tree. (However, multiple targets may share the same TOOLDIR;
235: the target-dependent files have unique names.) If unset, a
236: default based on the uname(1) information of the host plat-
237: form will be created in the .OBJDIR of src/tools.
238:
239: Default: Unset.
240:
1.7 lukem 241: UNPRIVED If set, then an unprivileged install will occur. The user,
242: group, permissions, and file flags, will not be set on the
243: installed item; instead the information will be appended to a
1.11 lukem 244: file called METALOG in DESTDIR. The contents of METALOG is
245: used during the generation of the distribution tar files to
246: ensure that the appropriate file ownership is stored.
1.7 lukem 247:
248: Default: Unset.
249:
1.1 tv 250: UPDATE If set, then all install operations intended to write to
251: DESTDIR will compare file timestamps before installing, and
252: skip the install phase if the destination files are up-to-
253: date. This also has implications on full builds (see next
254: subsection).
255:
256: Default: Unset.
257:
258: USETOOLS Indicates whether the tools specified by TOOLDIR should be
259: used as part of a build in progress. Must be set to ``yes''
260: if cross-compiling.
261:
262: yes Use the tools from TOOLDIR.
263:
264: no Do not use the tools from TOOLDIR, but refuse to build
265: native compilation tool components that are version-
266: specific for that tool.
267:
268: never Do not use the tools from TOOLDIR, even when building
269: native tool components. This is similar to the tradi-
270: tional NetBSD build method, but does not verify that
271: the compilation tools in use are up-to-date enough in
272: order to build the tree successfully. This may cause
273: build or runtime problems when building the whole
274: NetBSD source tree.
275:
276: Default: ``yes'' if building all or part of a whole NetBSD
277: source tree (detected automatically); ``no'' otherwise (to
278: preserve traditional semantics of the <bsd.*.mk> make(1) in-
279: clude files).
280:
281: "make" variables for full builds
1.7 lukem 282: These variables only affect the top level ``Makefile'' and do not affect
283: manually building subtrees of the NetBSD source code.
1.1 tv 284:
285: MKOBJDIRS Can be set to ``yes'' or ``no''. Indicates whether object
286: directories will be created automatically (via a ``make
287: obj'' pass) at the start of a build.
288:
289: Default: ``yes''
290:
1.10 sommerfe 291: NBUILDJOBS Now obsolete. Use the make(1) option -j, instead (see be-
292: low)
1.1 tv 293:
294: Default: Unset.
295:
296: NOCLEANDIR If set, avoids the ``make cleandir'' phase of a full
297: build. This has the effect of allowing only changed files
298: in a source tree to be recompiled. This can speed up
299: builds when updating only a few files in the tree.
300:
301: Default: Unset.
302:
303: NODISTRIBDIRS If set, avoids the ``make distrib-dirs'' phase of a full
304: build. This skips running mtree(8) on DESTDIR, useful on
305: systems where building as an unprivileged user, or where
306: it is known that the system-wide mtree files have not
307: changed.
308:
309: Default: Unset.
310:
311: NOINCLUDES If set, avoids the ``make includes'' phase of a full
312: build. This has the effect of preventing make(1) from
313: thinking that some programs are out-of-date simply because
314: the system include files have changed. However, this op-
315: tion should not be used when updating the entire NetBSD
316: source tree arbitrarily; it is suggested to use UPDATE in
317: that case.
318:
319: Default: Unset.
320:
321: RELEASEDIR If set, specifies the directory to which a release(7) lay-
322: out will be written at the end of a ``make release''.
323:
324: Default: Unset.
325:
326: UPDATE If set, then in addition to the effects described for UP-
327: DATE above, this implies the effects of NOCLEANDIR.
328:
329: BUILDING
330: "make" command line options
331: This is only a summary of options available to make(1); only the options
332: used most frequently with NetBSD builds are listed here.
333:
1.10 sommerfe 334: -j njob Run up to njob make(1) subjobs in parallel. Makefiles should
335: use .WAIT or have explicit dependancies as necessary to en-
336: force build ordering. If you see build failures with -j,
337: please save complete build logs so the failures can be ana-
338: lyzed.
339:
1.1 tv 340: -m dir Specify the default directory for searching for system Make-
341: file segments, mainly the <bsd.*.mk> files. When building any
342: full NetBSD source tree, this should be set to the
343: ``share/mk'' directory in the source tree. (This is set auto-
344: matically when building from the top level.)
345:
346: -n Display the commands that would have been executed, but do not
347: actually execute them. This will still cause recursion to
348: take place.
349:
350: -v var Print make(1)'s idea of the value of var. Does not build any
351: targets.
352:
353: var=value Set the variable var to value, overriding any setting speci-
354: fied by the process environment, the MAKECONF configuration
355: file, or the system Makefile segments.
356:
357: "make" targets
358: These default targets may be built by running make(1) in any subtree of
359: the NetBSD source code. It is recommended that none of these be used
360: from the top level Makefile; as a specific exception, ``make obj'' and
361: ``make cleandir'' are useful in that context.
362:
363: all Build programs, libraries, and preformatted documentation.
364:
365: clean Remove program and library object code files.
366:
367: cleandir Same as clean, but also remove preformatted documentation, de-
368: pendency files generated by ``make depend'', and any other
369: files known to be created at build time. ``make distclean''
370: may be used as a synonym, for familiarity with a similar well-
371: known convention.
372:
373: depend Create dependency files (.depend) containing more detailed in-
374: formation about the dependencies of source code on header
375: files. Allows programs to be recompiled automatically when a
376: dependency changes.
377:
378: dependall Does a ``make depend'' immediately followed by a ``make all''.
1.10 sommerfe 379: This improves cache locality of the build since both passes
380: read the source files in their entirety.
1.1 tv 381:
382: includes Build and install system header files. Typically needed be-
383: fore any system libraries or programs can be built.
384:
385: install Install programs, libraries, and documentation into DESTDIR.
1.14 fredb 386: Few files will be installed to /dev, /etc, /root or /var in
387: order to prevent user supplied configuration data from being
388: overwritten.
1.1 tv 389:
390: lint Run lint(1) against the C source code, where appropriate, and
391: generate system-installed lint libraries.
392:
393: obj Create object directories to be used for built files, instead
394: of building directly in the source tree.
395:
396: tags Create ctags(1) searchable function lists usable by the ex(1)
397: and vi(1) text editors.
398:
399: "make" targets for the top level
400: Additional make(1) targets are usable specifically from the top source
401: level to facilitate building the entire NetBSD source tree.
402:
403: build Build the entire NetBSD system. This orders portions of the
404: source tree such that prerequisites will be built in the prop-
405: er order.
406:
407: release Do a ``make build'', then package the system into a standard
408: release layout as described by release(7). This requires that
409: RELEASEDIR be set (see above).
410:
411: regression-tests
412: Can only be run after building the regression tests in the di-
413: rectory ``regress''. Runs the compiled regression tests on
414: the local host.
415:
416: The "build.sh" script
417: This script file is a Bourne shell script designed to build the entire
418: NetBSD system on any host with a Bourne shell in /bin/sh, including many
419: that are not POSIX compliant. Note that if a host system's /bin/sh is
420: unusually old and broken, the Korn Shell (/bin/ksh), if available, may be
421: a usable alternative.
422:
423: All cross-compile builds, and most native builds, of the entire system
424: should make use of build.sh rather than just running ``make''. This way,
425: the make(1) program will be bootstrapped properly, in case the host sys-
426: tem has an older or incompatible ``make'' program.
427:
428: When compiling the entire system via build.sh, many make(1) variables are
429: set for you in order to help encapsulate the build process. In the list
430: of options below, variables that are automatically set by build.sh are
431: noted where applicable.
432:
433: The following are available command line options that may be supplied to
434: build.sh:
435:
436: -a arch Set the value of MACHINE_ARCH to arch.
1.9 thorpej 437:
438: -B buildid
439: Set the value of BUILDID to buildid. This will also append the
440: build idenfitier to the name of the ``make'' wrapper script so
441: that the resulting name is of the form ``nbmake-MACHINE-
442: BUILDID''.
1.1 tv 443:
444: -b Bootstrap ``make'' and create a nbmake-MACHINE script (see be-
445: low).
446:
1.14 fredb 447: -d Build a full distribution. This differs from a default build
448: in that files will also be installed to /dev, /etc, /root and
449: /var. Note this does not build a ``release''; no release sets
450: are placed in ${RELEASEDIR}. -d is implied by -R.
1.6 thorpej 451:
1.10 sommerfe 452: -j njob Passed through to make(1). Makefiles should use .WAIT or have
453: explicit dependancies as necessary to enforce build ordering.
454: If you see build failures with -j, please save complete build
455: logs so the failures can be analyzed.
1.1 tv 456:
1.16 lukem 457: -k kernel
458: Build a new kernel. The kernel argument is the name of a con-
459: figuration file suitable for use by config(8). If kernel does
460: not contain any `/' characters, the configuration file is ex-
461: pected to be found in the KERNCONFDIR directory, which is typi-
462: cally sys/arch/MACHINE/conf. The new kernel will be built in a
463: subdirectory of KERNOBJDIR, which is typically
464: sys/arch/MACHINE/compile or an associated object directory. In
465: order to ensure that the kernel is built using up-to-date
466: tools, it is strongly recommended that the tools be rebuilt
467: (using the -t option) in a separate invocation of build.sh pri-
468: or to using the -k option, or that the -t and -k options be
469: used together in a single invocation of build.sh.
470:
1.1 tv 471: -m mach Set the value of MACHINE to mach. This will also override any
472: value of MACHINE_ARCH in the process environment with a value
473: deduced from mach, unless -a is specified. All cross builds
474: require -m, but if unset on a NetBSD host, the host's value of
475: MACHINE will be detected and used automatically.
476:
477: -n Show the commands that would be executed by build.sh, but do
478: not make any changes. This is similar in concept to ``make
479: -n''.
480:
481: -o Set the value of MKOBJDIRS to ``no''.
482:
483: -r Remove the contents of DESTDIR and TOOLDIR before building
484: (provides a clean starting point). This will skip deleting
485: DESTDIR if building on a native system to the root directory.
486:
487: -t Build and install the host tools from src/tools only. This op-
488: tion implies -b.
489:
490: -u Set the UPDATE variable.
491:
492: -w wrapper
493: Create the nbmake wrapper script (see below) in a custom loca-
494: tion, specified by wrapper. This allows, for instance, to
495: place the wrapper in PATH automatically. Note that wrapper is
496: the full name of the file, not just a directory name.
497:
498: -D dest Set the value of DESTDIR to dest.
499:
1.12 lukem 500: -M obj Set MAKEOBJDIRPREFIX to obj.
501:
1.1 tv 502: -O obj Create an appropriate transform macro for MAKEOBJDIR that will
503: place the built object files under obj. For instance, a set-
1.13 wiz 504: ting of /usr/obj will place build-time files under
1.1 tv 505: /usr/obj/bin, /usr/obj/lib, and so forth.
506:
507: -R rel Set the value of RELEASEDIR to rel. Setting this option will
508: cause build.sh to run ``make release'' instead of ``make
509: build''.
510:
511: -T tools Set the value of TOOLDIR to tools. If set, the bootstrap
512: ``make'' will only be rebuilt as needed (when the source files
513: for make(1) change).
514:
1.7 lukem 515: -U Set the UNPRIVED variable.
516:
1.1 tv 517: The "nbmake-MACHINE" wrapper script
518: If using the build.sh script to build NetBSD, a nbmake-MACHINE script
519: will be created in TOOLDIR/bin upon the first build to assist in building
520: subtrees on a cross-compile host.
521:
522: nbmake-MACHINE can be invoked in lieu of make(1), and will instead call
523: the up-to-date version of ``nbmake'' installed into TOOLDIR/bin with sev-
524: eral key variables pre-set, including MACHINE, MACHINE_ARCH, and TOOLDIR.
525: This script can be symlinked into a directory listed in PATH, or called
526: with an absolute path.
527:
1.16 lukem 528: EXAMPLES
529: ./build.sh -t
530: Build a new toolchain.
531:
532: cd ${KERNCONFDIR} ; ${TOOLDIR}/bin/nbconfig GENERIC
533: Use the new version of config(8) to prepare to build a new
534: GENERIC kernel.
535:
536: cd ${KERNOBJDIR}/GENERIC ; ${TOOLDIR}/bin/nbmake-${MACHINE} dependall
537: Use the new toolchain to build a new GENERIC kernel.
538:
539: ./build.sh -t -k GENERIC
540: Build a new toolchain, and use the new toolchain to configure
541: and build a new GENERIC kernel.
542:
543: ./build.sh -U -d
544: Using unprivileged mode, build a complete distribution in
545: DESTDIR.
546:
547: ./build.sh -U -R /some/dir/RELEASE
548: Using unprivileged mode, build a complete release in the speci-
549: fied release directory.
550:
1.10 sommerfe 551: OBSOLETE VARIABLES
1.15 lukem 552: NBUILDJOBS Use the make(1) option -j, instead.
553:
554: USE_NEW_TOOLCHAIN
555: The new toolchain is now the default. To disable, use
556: TOOLCHAIN_MISSING=yes.
1.10 sommerfe 557:
1.1 tv 558: SEE ALSO
559: make(1), hier(7), release(7)
560:
561: HISTORY
1.15 lukem 562: The build.sh based build scheme was introduced for NetBSD 1.6 as
563: USE_NEW_TOOLCHAIN, and re-worked to TOOLCHAIN_MISSING after that.
1.1 tv 564:
565: BUGS
1.15 lukem 566: A few platforms are not yet using this build system.
1.1 tv 567:
1.16 lukem 568: NetBSD October 21, 2002 9

CVSweb <webmaster@jp.NetBSD.org>