[BACK]Return to chap-kernel.xml CVS log [TXT][DIR] Up to [cvs.NetBSD.org] / htdocs / docs / guide / en

File: [cvs.NetBSD.org] / htdocs / docs / guide / en / chap-kernel.xml (download)

Revision 1.6, Sun Apr 18 05:58:12 2021 UTC (2 years, 11 months ago) by nia
Branch: MAIN
CVS Tags: HEAD
Changes since 1.5: +23 -50 lines

Slightly modernize kernel compilation notes

<!-- $NetBSD: chap-kernel.xml,v 1.6 2021/04/18 05:58:12 nia Exp $ -->

<chapter id="chap-kernel">
  <title>Compiling the kernel</title>

  <para>There are several reasons you might want to compile a &os;
    kernel:</para>

  <itemizedlist>
    <listitem>
      <para>you can install bug-fixes, security updates, or new
        functionality by rebuilding the kernel from updated
        sources.</para>
    </listitem>

    <listitem>
      <para>by removing unused device drivers and kernel sub-systems
        from your configuration, you can dramatically reduce kernel size
        and, therefore, memory usage and attack surface. Care must be taken
	when doing this, since it can result in an unusable system.
	Always back up old working kernels.</para>
    </listitem>

    <listitem>
      <para>you can access additional features by enabling kernel
        options or sub-systems, some of which are experimental or
        disabled by default.</para>
    </listitem>

    <listitem>
      <para>you can get a deeper knowledge of the system.</para>
    </listitem>
  </itemizedlist>

  <!-- ============================================================= -->

  <sect1 id="chap-kernel-procedure">
    <title>Requirements and procedure</title>

    <para>To recompile the kernel you must have installed the compiler set
      (<filename>comp.tgz</filename>).</para>

    <para>The basic steps to an updated or customised kernel then are:
    </para>

    <orderedlist>
      <listitem>
        <para>Install or update the kernel sources</para>
      </listitem>

      <listitem>
        <para>Create or modify the kernel configuration file</para>
      </listitem>

      <listitem>
        <para>Building the kernel from the configuration file, either
          manually or using <command>build.sh</command></para>
      </listitem>

      <listitem>
	<para>Install the kernel</para>
      </listitem>
    </orderedlist>

  </sect1>

  <!-- ============================================================= -->

  <sect1 id="chap-kernel-installing-kernel-sources">
    <title>Installing the kernel sources</title>

    <sidebar>
      <para>You can get the kernel sources from AnonCVS (see
        <xref linkend="chap-fetch" />), or from the
	<filename>syssrc.tgz</filename>
	tarball that is located in the <filename>source/sets/</filename>
	directory of the release that you are using.</para>
    </sidebar>

    <para>If you chose to use AnonCVS to fetch the entire source tree,
      be patient, the operation can last many minutes, because the
      repository contains thousands of files.</para>

    <para>If you have a source tarball, you can extract it as root:</para>

    <screen>&rprompt; <userinput>cd /</userinput>
&rprompt; <userinput>tar zxf /path/to/syssrc.tgz</userinput></screen>


    <para>Even if you used the tarball from the release, you may wish to
      use AnonCVS to update the sources with changes that have been
      applied since the release.
      This might be especially relevant if you are updating the kernel
      to include the fix for a specific bug, including a vulnerability
      described in a NetBSD Security Advisory.
      You might want to get the latest sources on the relevant release
      or critical updates branch for your version, or Security
      Advisories will usually contain information on the dates or
      revisions of the files containing the specific fixes concerned.
      See <xref linkend="chap-fetch-cvs" /> for more details on the
      CVS commands used to update sources from these branches.
      </para>

    <para>Once you have the sources available, you can create a custom
      kernel: this is not as difficult as you might think.  
      In fact, a new kernel can be created in a few steps which will
      be described in the following sections.</para>
  </sect1>

  <!-- ============================================================= -->

  <sect1 id="chap-kernel-creating-kernel-configfile">
    <title>Creating the kernel configuration file</title>

    <note><para>The directories described in this section are amd64
      specific. Users of other architectures must substitute the
      appropriate directories, see the subdirectories of
      <filename>src/sys/arch</filename> for a list.</para></note>

    <para>The kernel configuration file defines the type, the number and
      the characteristics of the devices supported by the kernel as
      well as several kernel configuration options.  For the amd64
      port, kernel configuration files are located in the
      <filename>/usr/src/sys/arch/amd64/conf</filename> directory.</para>

    <para>Please note that the names of the kernel configuration files
      are historically in all uppercase, so they are easy to
      distinguish from other files in that directory:</para>

    <screen>&uprompt; <userinput>cd /usr/src/sys/arch/amd64/conf/</userinput>
&uprompt; <userinput>ls</userinput>
ALL                 INSTALL             XEN3_DOM0           kern.ldscript.kaslr
CVS                 INSTALL_XEN3_DOMU   XEN3_DOMU           majors.amd64
GENERIC             MODULAR             files.amd64         std.amd64
GENERIC_KASLR       Makefile.amd64      kern.ldscript       std.xen
GENERIC_USERMODE    NOCOMPAT            kern.ldscript.Xen</screen>

    <para>The easiest way to create a new file is to copy an existing one
      and modify it. Usually the best choice on most platforms is the
      GENERIC configuration, as it contains most drivers and options.
      In the configuration file there are comments describing the
      options; a more detailed description is found in the
      &man.options.4; man page. So, the usual procedure is:</para>

    <screen>&uprompt; <userinput>cp GENERIC <replaceable>MYKERNEL</replaceable></userinput>
&uprompt; <userinput>vi <replaceable>MYKERNEL</replaceable></userinput></screen>

    <para>The modification of a kernel configuration file basically
      involves three operations:</para>

    <orderedlist>
      <listitem>
        <para>support for hardware devices is included/excluded in the kernel
          (for example, SCSI support can be removed if it is not needed.)</para>
      </listitem>

      <listitem>
        <para>support for kernel features is enabled/disabled (for example,
          enable NFS client support, enable Linux compatibility, ...)</para>
      </listitem>

      <listitem>
	<para>tuning kernel parameters.</para>
      </listitem>
    </orderedlist>

    <para>Lines beginning with <quote>#</quote> are comments; lines are disabled by
      commenting them and enabled by removing the comment character.
      It is better to comment lines instead of deleting them; it is always
      possible uncomment them later.</para>

    <para>The output of the &man.dmesg.8; command can be used
      to determine which lines can be disabled.
      For each line of the type:</para>

    <programlisting><replaceable>XXX</replaceable> at <replaceable>YYY</replaceable></programlisting>

    <para>both <replaceable>XXX</replaceable> and
      <replaceable>YYY</replaceable> must be active in the kernel
      configuration file.
      You'll probably have to experiment a bit before achieving a minimal
      configuration but on a desktop system without SCSI and PCMCIA you can
      halve the kernel size.</para>

    <para>You could also examine the options in the configuration file and
      disable the ones that you don't need.
      Each option has a short comment describing it, which is normally
      sufficient to understand what the option does.
      Many options have a longer and more detailed description in the
      &man.options.4; man page.</para>
  </sect1>

  <!-- ============================================================= -->

  <sect1 id="chap-kernel-building-manually">
    <title>Building the kernel manually</title>

    <para>Based on your kernel configuration file, either one of the
      standard configurations or your customised configuration, a new
      kernel must be built.</para>

    <para>These steps can either be performed manually, or using the
      <command>build.sh</command> command that was introduced
      in section <xref linkend="chap-build" />. 
      This section will give instructions on how to build a native
      kernel using manual steps, the following section <xref
      linkend="chap-kernel-build.sh" /> describes how to use
      <command>build.sh</command> to do the same.</para>

    <itemizedlist>
      <listitem>
        <para>Configure the kernel</para>
      </listitem>

      <listitem>
        <para>Generate dependencies</para>
      </listitem>

      <listitem>
        <para>Compile the kernel</para>
      </listitem>
    </itemizedlist>

  <!-- ============================================================= -->

  <sect2 id="chap-kernel-configuring-kernel">
    <title>Configuring the kernel manually</title>

    <para>When you've finished modifying the kernel configuration file (which
      we'll call <filename>MYKERNEL</filename>), you should issue the
      following command:</para>

    <screen>&uprompt; <userinput>config MYKERNEL</userinput></screen>

    <para>If <filename>MYKERNEL</filename> contains no errors, the
      &man.config.1; program will create the necessary files for
      the compilation of the kernel, otherwise it will be necessary to correct
      the errors before running &man.config.1; again.</para>

    <note>
      <title>Notes for cross-compiling</title>

    <para>As the &man.config.1; program used to create header files and Makefile
    for a kernel build is platform specific, it is necessary to use the
    <command>nbconfig</command> program that's part of a newly created 
    toolchain (created for example with 
    <programlisting>/usr/src/build.sh -m sparc64 tools</programlisting>). 
    That aside, the procedure is just as like compiling a "native" NetBSD 
    kernel. The command is  for example:</para>

  <screen>&cprompt; <userinput>/usr/src/tooldir.NetBSD-9.0-amd64/bin/nbconfig <replaceable>MYKERNEL</replaceable></userinput></screen>

  <para>This command has created a directory
    <filename>../compile/<replaceable>MYKERNEL</replaceable></filename>
    with a number of header files defining information about devices
    to compile into the kernel, a Makefile that is setup to build
      all the needed files for the kernel, and link them together.</para>
  </note>

</sect2>

  <!-- ============================================================= -->

  <sect2 id="chap-kernel-generating-dependencies-n-recompiling">
    <title>Generating dependencies and recompiling manually</title>

    <para>Dependencies generation and kernel compilation is performed by the
      following commands:</para>

    <screen>&uprompt; <userinput>cd ../compile/MYKERNEL</userinput>
&uprompt; <userinput>make depend</userinput>
&uprompt; <userinput>make</userinput></screen>

    <para>It can happen that the compilation stops with errors; there can be
      a variety of reasons but the most common cause is an error in the
      configuration file which didn't get caught by &man.config.1;.
      Sometimes the failure is caused by a hardware problem (often faulty RAM
      chips): the compilation puts a higher stress on the system than most
      applications do.
      Another typical error is the following: option B, active, requires
      option A which is not active.
      A full compilation of the kernel can last from some minutes to several
      hours, depending on the hardware.</para>

    <para>The result of a successful make command is the
      <filename>netbsd</filename> file in the compile directory, ready
      to be installed.</para>

    <note>
      <title>Notes for cross-compiling</title>

      <para>For crosscompiling a sparc64 kernel, it is necessary to use the 
      crosscompiler toolchain's <filename>nbmake-sparc64</filename> shell 
      wrapper, which calls &man.make.1; with all the necessary settings for 
      crosscompiling for a sparc64 platform:</para>

    <screen>&cprompt; <userinput>cd ../compile/MYKERNEL/</userinput>
&cprompt; <userinput>/usr/src/tooldir.NetBSD-9.0-amd64/bin/nbmake-sparc64 depend</userinput>
&cprompt; <userinput>/usr/src/tooldir.NetBSD-9.0-amd64/bin/nbmake-sparc64</userinput></screen>

    <para>This will churn away a bit, then spit out a kernel:</para>
    <screen>...
text    data     bss     dec     hex filename
5016899  163728  628752 5809379  58a4e3 netbsd
&cprompt; <userinput>ls -l netbsd</userinput>
-rwxr-xr-x  1 feyrer  666  5874663 Dec  2 23:17 netbsd
&cprompt; <userinput>file netbsd</userinput>
netbsd: ELF 64-bit MSB executable, SPARC V9, version 1 (SYSV), statically linked, not stripped </screen>

    <para>Now the kernel in the file <filename>netbsd</filename> can
      either be transferred to an UltraSPARC machine (via NFS, FTP,
      scp, etc.) and booted from a possible harddisk, or directly from
      the cross-development machine using NFS.</para>
    </note>

  </sect2>

  </sect1>

  <!-- ============================================================= -->

  <sect1 id="chap-kernel-build.sh">
    <title>Building the kernel using <filename>build.sh</filename></title>

    <para>After creating and possibly editing the kernel config file, the
      manual steps of configuring the kernel, generating dependencies
      and recompiling can also be done using the
      <filename>src/build.sh</filename> script, all in one go:</para>

    <screen>&uprompt; <userinput>cd /usr/src</userinput>
&uprompt; <userinput>./build.sh kernel=MYKERNEL</userinput> </screen>

    <para>This will perform the same steps as above, with one small
      difference: before compiling, all old object files will be
      removed, to start with a fresh build. This is usually overkill,
      and it's fine to keep the old file and only rebuild the ones
      whose dependencies have changed. To do this, add the
      <option>-u</option> option to <filename>build.sh</filename>:</para>

    <screen>&uprompt; <userinput>cd /usr/src</userinput>
&uprompt; <userinput>./build.sh -u kernel=MYKERNEL</userinput> </screen>

    <para>At the end of its job, <filename>build.sh</filename> will print
      out the location where the new compiled kernel can be found. It
      can then be installed.</para>
  </sect1>


  <!-- ============================================================= -->

  <sect1 id="chap-kernel-install">
    <title>Installing the new kernel</title>

    <para>Whichever method was used to produce the new kernel file, it
      must now be installed.  
      The new kernel file should be copied to the root directory,
      after saving the previous version.</para>

    <screen>&rprompt; <userinput>mv /netbsd /netbsd.old</userinput>
&rprompt; <userinput>mv netbsd /</userinput></screen>

    <para>Customization can considerably reduce the kernel's size.
      In the following example <filename>netbsd.old</filename> is the
      install kernel and <filename>netbsd</filename> is the new kernel.</para>

    <screen>-rwxr-xr-x  3 root  wheel  3523098 Dec 10 00:13 /netbsd
-rwxr-xr-x  3 root  wheel  7566271 Dec 10 00:13 /netbsd.old</screen>

    <para>The new kernel is activated after rebooting:</para>

    <screen>&rprompt; <userinput>shutdown -r now</userinput></screen>

  </sect1>

  <!-- ============================================================= -->

  <sect1 id="chap-kernel-something-wrong">
    <title>If something went wrong</title>

    <para>When the computer is restarted it can happen that the new
      kernel doesn't work as expected or even doesn't boot at all.
      Don't worry: if this happens, just reboot with the previously
      saved kernel and remove the new one (it is better to reboot
      <quote>single user</quote>):</para>

    <itemizedlist>
      <listitem>
        <para>Reboot the machine</para>
      </listitem>

      <listitem>
        <para>Press the space bar at the boot prompt during the 5 seconds
	  countdown</para>

	<screen><prompt>boot:</prompt></screen>
      </listitem>

      <listitem>
        <para>Type</para>

	<screen><prompt>&gt;</prompt> <userinput>boot netbsd.old -s</userinput></screen>
      </listitem>

      <listitem>
        <para>Now issue the following commands to restore the previous version
          of the kernel:</para>

	<screen>&rprompt; <userinput>fsck /</userinput>
&rprompt; <userinput>mount /</userinput>
&rprompt; <userinput>mv netbsd.old netbsd</userinput>
&rprompt; <userinput>reboot</userinput></screen>
      </listitem>
    </itemizedlist>

    <para>This will give you back the working system you started with,
      and you can revise your custom kernel config file to resolve the
      problem. In general, it's wise to start with a GENERIC kernel
      first, and then make gradual changes.</para>
  </sect1>
</chapter>