Annotation of htdocs/docs/pkgsrc/makefile.html, revision 1.1

1.1     ! dsieger     1: <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
        !             2: <html>
        !             3: <head>
        !             4: <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
        !             5: <title>Chapter 11. Programming in Makefiles</title>
        !             6: <link rel="stylesheet" href="/NetBSD.css" type="text/css">
        !             7: <meta name="generator" content="DocBook XSL Stylesheets VX.X.X">
        !             8: <link rel="start" href="index.html" title="The pkgsrc guide">
        !             9: <link rel="up" href="developers-guide.html" title="Part II. The pkgsrc developer's guide">
        !            10: <link rel="prev" href="components.html" title="Chapter 10. Package components - files, directories and contents">
        !            11: <link rel="next" href="plist.html" title="Chapter 12. PLIST issues">
        !            12: </head>
        !            13: <body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
        !            14: <div class="navheader">
        !            15: <table width="100%" summary="Navigation header">
        !            16: <tr><th colspan="3" align="center">Chapter 11. Programming in <code class="filename">Makefile</code>s</th></tr>
        !            17: <tr>
        !            18: <td width="20%" align="left">
        !            19: <a accesskey="p" href="components.html">Prev</a> </td>
        !            20: <th width="60%" align="center">Part II. The pkgsrc developer's guide</th>
        !            21: <td width="20%" align="right"> <a accesskey="n" href="plist.html">Next</a>
        !            22: </td>
        !            23: </tr>
        !            24: </table>
        !            25: <hr>
        !            26: </div>
        !            27: <div class="chapter" lang="en">
        !            28: <div class="titlepage"><div><div><h2 class="title">
        !            29: <a name="makefile"></a>Chapter 11. Programming in <code class="filename">Makefile</code>s</h2></div></div></div>
        !            30: <div class="toc">
        !            31: <p><b>Table of Contents</b></p>
        !            32: <dl>
        !            33: <dt><span class="sect1"><a href="makefile.html#makefile.style">11.1. Caveats</a></span></dt>
        !            34: <dt><span class="sect1"><a href="makefile.html#makefile.variables">11.2. <code class="filename">Makefile</code> variables</a></span></dt>
        !            35: <dd><dl><dt><span class="sect2"><a href="makefile.html#makefile.variables.names">11.2.1. Naming conventions</a></span></dt></dl></dd>
        !            36: <dt><span class="sect1"><a href="makefile.html#makefile.code">11.3. Code snippets</a></span></dt>
        !            37: <dd><dl>
        !            38: <dt><span class="sect2"><a href="makefile.html#adding-to-list">11.3.1. Adding things to a list</a></span></dt>
        !            39: <dt><span class="sect2"><a href="makefile.html#converting-internal-to-external">11.3.2. Converting an internal list into an external list</a></span></dt>
        !            40: <dt><span class="sect2"><a href="makefile.html#passing-variable-to-shell">11.3.3. Passing variables to a shell command</a></span></dt>
        !            41: <dt><span class="sect2"><a href="makefile.html#quoting-guideline">11.3.4. Quoting guideline</a></span></dt>
        !            42: <dt><span class="sect2"><a href="makefile.html#bsd-make-bug-workaround">11.3.5. Workaround for a bug in BSD Make</a></span></dt>
        !            43: </dl></dd>
        !            44: </dl>
        !            45: </div>
        !            46: <p>Pkgsrc consists of many <code class="filename">Makefile</code> fragments,
        !            47:   each of which forms a well-defined part of the pkgsrc system. Using
        !            48:   the <a href="http://netbsd.gw.com/cgi-bin/man-cgi?make+1+NetBSD-current"><span class="citerefentry"><span class="refentrytitle">make</span>(1)</span></a> system as a programming language for a big system
        !            49:   like pkgsrc requires some discipline to keep the code correct and
        !            50:   understandable.</p>
        !            51: <p>The basic ingredients for <code class="filename">Makefile</code>
        !            52:   programming are variables (which are actually macros) and shell
        !            53:   commands. Among these shell commands may even be more complex ones
        !            54:   like <a href="http://netbsd.gw.com/cgi-bin/man-cgi?awk+1+NetBSD-current"><span class="citerefentry"><span class="refentrytitle">awk</span>(1)</span></a> programs. To make sure that every shell command runs
        !            55:   as intended it is necessary to quote all variables correctly when they
        !            56:   are used.</p>
        !            57: <p>This chapter describes some patterns, that appear quite often in
        !            58:   <code class="filename">Makefile</code>s, including the pitfalls that come along
        !            59:   with them.</p>
        !            60: <div class="sect1" lang="en">
        !            61: <div class="titlepage"><div><div><h2 class="title" style="clear: both">
        !            62: <a name="makefile.style"></a>11.1. Caveats</h2></div></div></div>
        !            63: <div class="itemizedlist"><ul type="disc"><li>
        !            64: <p>When you are creating a file as a
        !            65:     target of a rule, always write the data to a temporary file first
        !            66:     and finally rename that file. Otherwise there might occur an error
        !            67:     in the middle of generating the file, and when the user runs
        !            68:     <a href="http://netbsd.gw.com/cgi-bin/man-cgi?make+1+NetBSD-current"><span class="citerefentry"><span class="refentrytitle">make</span>(1)</span></a> for the second time, the file exists and will not be
        !            69:     regenerated properly. Example:</p>
        !            70: <pre class="programlisting">
        !            71: wrong:
        !            72:         @echo "line 1" &gt; ${.TARGET}
        !            73:         @echo "line 2" &gt;&gt; ${.TARGET}
        !            74:         @false
        !            75: 
        !            76: correct:
        !            77:         @echo "line 1" &gt; ${.TARGET}.tmp
        !            78:         @echo "line 2" &gt;&gt; ${.TARGET}.tmp
        !            79:         @false
        !            80:         @mv ${.TARGET}.tmp ${.TARGET}
        !            81: </pre>
        !            82: <p>When you run <span><strong class="command">make wrong</strong></span> twice, the file
        !            83:     <code class="filename">wrong</code> will exist, although there was an error
        !            84:     message in the first run. On the other hand, running <span><strong class="command">make
        !            85:     correct</strong></span> gives an error message twice, as expected.</p>
        !            86: <p>You might remember that <a href="http://netbsd.gw.com/cgi-bin/man-cgi?make+1+NetBSD-current"><span class="citerefentry"><span class="refentrytitle">make</span>(1)</span></a> sometimes removes
        !            87:     <code class="literal">${.TARGET}</code> in case of error, but this only
        !            88:     happens when it is interrupted, for example by pressing
        !            89:     <code class="literal">^C</code>. This does <span class="emphasis"><em>not</em></span> happen
        !            90:     when one of the commands fails (like <a href="http://netbsd.gw.com/cgi-bin/man-cgi?false+1+NetBSD-current"><span class="citerefentry"><span class="refentrytitle">false</span>(1)</span></a> above).</p>
        !            91: </li></ul></div>
        !            92: </div>
        !            93: <div class="sect1" lang="en">
        !            94: <div class="titlepage"><div><div><h2 class="title" style="clear: both">
        !            95: <a name="makefile.variables"></a>11.2. <code class="filename">Makefile</code> variables</h2></div></div></div>
        !            96: <p><code class="filename">Makefile</code> variables contain strings that
        !            97:     can be processed using the five operators ``='', ``+='', ``?='',
        !            98:     ``:='', and ``!='', which are described in the <a href="http://netbsd.gw.com/cgi-bin/man-cgi?make+1+NetBSD-current"><span class="citerefentry"><span class="refentrytitle">make</span>(1)</span></a> man
        !            99:     page.</p>
        !           100: <p>When a variable's value is parsed from a
        !           101:     <code class="filename">Makefile</code>, the hash character ``#'' and the
        !           102:     backslash character ``\'' are handled specially. If a backslash is
        !           103:     followed by a newline, any whitespace immediately in front of the
        !           104:     backslash, the backslash, the newline, and any whitespace
        !           105:     immediately behind the newline are replaced with a single space. A
        !           106:     backslash character and an immediately following hash character are
        !           107:     replaced with a single hash character. Otherwise, the backslash is
        !           108:     passed as is. In a variable assignment, any hash character that is
        !           109:     not preceded by a backslash starts a comment that continues upto the
        !           110:     end of the logical line.</p>
        !           111: <p><span class="emphasis"><em>Note:</em></span> Because of this parsing algorithm
        !           112:     the only way to create a variable consisting of a single backslash
        !           113:     is using the ``!='' operator, for example: <code class="varname">BACKSLASH!=echo "\\"</code>.</p>
        !           114: <p>So far for defining variables. The other thing you can do with
        !           115:     variables is evaluating them. A variable is evaluated when it is
        !           116:     part of the right side of the ``:='' or the ``!='' operator, or
        !           117:     directly before executing a shell command which the variable is part
        !           118:     of. In all other cases, <a href="http://netbsd.gw.com/cgi-bin/man-cgi?make+1+NetBSD-current"><span class="citerefentry"><span class="refentrytitle">make</span>(1)</span></a> performs lazy evaluation, that
        !           119:     is, variables are not evaluated until there's no other way. The
        !           120:     ``modifiers'' mentioned in the man page also evaluate the
        !           121:     variable.</p>
        !           122: <p>Some of the modifiers split the string into words and then
        !           123:     operate on the words, others operate on the string as a whole. When
        !           124:     a string is split into words, it is split as you would expect
        !           125:     it from <a href="http://netbsd.gw.com/cgi-bin/man-cgi?sh+1+NetBSD-current"><span class="citerefentry"><span class="refentrytitle">sh</span>(1)</span></a>.</p>
        !           126: <p>No rule without exception&mdash;the <span><strong class="command">.for</strong></span>
        !           127:     loop does not follow the shell quoting rules but splits at sequences
        !           128:     of whitespace.</p>
        !           129: <p>There are several types of variables that should be handled
        !           130:     differently. Strings and two types of lists.</p>
        !           131: <div class="itemizedlist"><ul type="disc">
        !           132: <li><p><span class="emphasis"><em>Strings</em></span> can contain arbitrary
        !           133:       characters. Nevertheless, you should restrict yourself to only
        !           134:       using printable characters. Examples are
        !           135:       <code class="varname">PREFIX</code> and
        !           136:       <code class="varname">COMMENT</code>.</p></li>
        !           137: <li><p><span class="emphasis"><em>Internal lists</em></span> are lists that
        !           138:       are never exported to any shell command. Their elements are
        !           139:       separated by whitespace. Therefore, the elements themselves cannot
        !           140:       have embedded whitespace. Any other characters are allowed.
        !           141:       Internal lists can be used in <span><strong class="command">.for</strong></span> loops.
        !           142:       Examples are <code class="varname">DEPENDS</code> and
        !           143:       <code class="varname">BUILD_DEPENDS</code>.</p></li>
        !           144: <li><p><span class="emphasis"><em>External lists</em></span> are lists that
        !           145:       may be exported to a shell command. Their elements can contain any
        !           146:       characters, including whitespace. That's why they cannot be used
        !           147:       in <span><strong class="command">.for</strong></span> loops. Examples are
        !           148:       <code class="varname">DISTFILES</code> and
        !           149:       <code class="varname">MASTER_SITES</code>.</p></li>
        !           150: </ul></div>
        !           151: <div class="sect2" lang="en">
        !           152: <div class="titlepage"><div><div><h3 class="title">
        !           153: <a name="makefile.variables.names"></a>11.2.1. Naming conventions</h3></div></div></div>
        !           154: <div class="itemizedlist"><ul type="disc">
        !           155: <li><p>All variable names starting with an underscore
        !           156:        are reserved for use by the pkgsrc infrastructure. They shall
        !           157:        not be used by package
        !           158:        <code class="filename">Makefile</code>s.</p></li>
        !           159: <li><p>In <span><strong class="command">.for</strong></span> loops you should use
        !           160:        lowercase variable names for the iteration
        !           161:        variables.</p></li>
        !           162: <li><p>All list variables should have a ``plural''
        !           163:        name, e.g. <code class="varname">PKG_OPTIONS</code> or
        !           164:        <code class="varname">DISTFILES</code>.</p></li>
        !           165: </ul></div>
        !           166: </div>
        !           167: </div>
        !           168: <div class="sect1" lang="en">
        !           169: <div class="titlepage"><div><div><h2 class="title" style="clear: both">
        !           170: <a name="makefile.code"></a>11.3. Code snippets</h2></div></div></div>
        !           171: <p>This section presents you with some code snippets you should
        !           172:     use in your own code. If you don't find anything appropriate here,
        !           173:     you should test your code and add it here.</p>
        !           174: <div class="sect2" lang="en">
        !           175: <div class="titlepage"><div><div><h3 class="title">
        !           176: <a name="adding-to-list"></a>11.3.1. Adding things to a list</h3></div></div></div>
        !           177: <pre class="programlisting">
        !           178: STRING=                 foo * bar `date`
        !           179: INT_LIST=               # empty
        !           180: ANOTHER_INT_LIST=       apache-[0-9]*:../../www/apache
        !           181: EXT_LIST=               # empty
        !           182: ANOTHER_EXT_LIST=       a=b c=d
        !           183: 
        !           184: INT_LIST+=              ${STRING}               # 1
        !           185: INT_LIST+=              ${ANOTHER_INT_LIST}     # 2
        !           186: EXT_LIST+=              ${STRING:Q}             # 3
        !           187: EXT_LIST+=              ${ANOTHER_EXT_LIST}     # 4
        !           188: </pre>
        !           189: <p>When you add a string to an external list (example 3), it
        !           190:       must be quoted. In all other cases, you must not add a quoting
        !           191:       level. You must not merge internal and external lists, unless you
        !           192:       are sure that all entries are correctly interpreted in both
        !           193:       lists.</p>
        !           194: </div>
        !           195: <div class="sect2" lang="en">
        !           196: <div class="titlepage"><div><div><h3 class="title">
        !           197: <a name="converting-internal-to-external"></a>11.3.2. Converting an internal list into an external list</h3></div></div></div>
        !           198: <pre class="programlisting">
        !           199: EXT_LIST=       # empty
        !           200: .for i in ${INT_LIST}
        !           201: EXT_LIST+=      ${i:Q}""
        !           202: .endfor
        !           203: </pre>
        !           204: <p>This code converts the internal list
        !           205:       <code class="varname">INT_LIST</code> into the external list
        !           206:       <code class="varname">EXT_LIST</code>. As the elements of an internal list
        !           207:       are unquoted they must be quoted here. The reason for appending
        !           208:       <code class="varname">""</code> is explained below.</p>
        !           209: </div>
        !           210: <div class="sect2" lang="en">
        !           211: <div class="titlepage"><div><div><h3 class="title">
        !           212: <a name="passing-variable-to-shell"></a>11.3.3. Passing variables to a shell command</h3></div></div></div>
        !           213: <p>Sometimes you may want to print an arbitrary string. There
        !           214:        are many ways to get it wrong and only few that can handle every
        !           215:        nastiness.</p>
        !           216: <pre class="programlisting">
        !           217: STRING=         foo bar &lt;    &gt; * `date` $$HOME ' "
        !           218: EXT_LIST=       string=${STRING:Q} x=second\ item
        !           219: 
        !           220: all:
        !           221:         echo ${STRING}                  # 1
        !           222:         echo "${STRING}"                # 2
        !           223:         echo "${STRING:Q}"              # 3
        !           224:         echo ${STRING:Q}                # 4
        !           225:         echo x${STRING:Q} | sed 1s,.,,  # 5
        !           226:         printf "%s\\n" ${STRING:Q}""    # 6
        !           227:         env ${EXT_LIST} /bin/sh -c 'echo "$$string"; echo "$$x"'
        !           228: </pre>
        !           229: <p>Example 1 leads to a syntax error in the shell, as the
        !           230:       characters are just copied.</p>
        !           231: <p>Example 2 leads to a syntax error too, and if you leave out
        !           232:       the last " character from <code class="varname">${STRING}</code>,
        !           233:       <a href="http://netbsd.gw.com/cgi-bin/man-cgi?date+1+NetBSD-current"><span class="citerefentry"><span class="refentrytitle">date</span>(1)</span></a> will be executed. The <code class="varname">$HOME</code> shell
        !           234:       variable would be evaluated, too.</p>
        !           235: <p>Example 3 outputs each space character preceded by a
        !           236:       backslash (or not), depending on the implementation of the
        !           237:       <a href="http://netbsd.gw.com/cgi-bin/man-cgi?echo+1+NetBSD-current"><span class="citerefentry"><span class="refentrytitle">echo</span>(1)</span></a> command.</p>
        !           238: <p>Example 4 handles correctly every string that does not start
        !           239:       with a dash. In that case, the result depends on the
        !           240:       implementation of the <a href="http://netbsd.gw.com/cgi-bin/man-cgi?echo+1+NetBSD-current"><span class="citerefentry"><span class="refentrytitle">echo</span>(1)</span></a> command. As long as you can
        !           241:       guarantee that your input does not start with a dash, this form is
        !           242:       appropriate.</p>
        !           243: <p>Example 5 handles even the case of a leading dash
        !           244:       correctly.</p>
        !           245: <p>Example 6 also works with every string and is the
        !           246:       light-weight solution, since it does not involve a pipe, which has
        !           247:       its own problems.</p>
        !           248: <p>The <code class="varname">EXT_LIST</code> does not need to be quoted
        !           249:       because the quoting has already been done when adding elements to
        !           250:       the list.</p>
        !           251: <p>As internal lists shall not be passed to the shell, there is
        !           252:       no example for it.</p>
        !           253: </div>
        !           254: <div class="sect2" lang="en">
        !           255: <div class="titlepage"><div><div><h3 class="title">
        !           256: <a name="quoting-guideline"></a>11.3.4. Quoting guideline</h3></div></div></div>
        !           257: <p>There are many possible sources of wrongly quoted variables.
        !           258:       This section lists some of the commonly known ones.</p>
        !           259: <div class="itemizedlist"><ul type="disc">
        !           260: <li>
        !           261: <p>Whenever you use the value of a list, think
        !           262:        about what happens to leading or trailing whitespace. If the
        !           263:        list is a well-formed shell expression, you can apply the
        !           264:        <code class="varname">:M*</code> modifier to strip leading and trailing
        !           265:        whitespace from each word. The <code class="varname">:M</code> operator
        !           266:        first splits its argument according to the rules of the shell,
        !           267:        and then creates a new list consisting of all words that match
        !           268:        the shell glob expression <code class="varname">*</code>, that is: all.
        !           269:        One class of situations where this is needed is when adding a
        !           270:        variable like <code class="varname">CPPFLAGS</code> to
        !           271:        <code class="varname">CONFIGURE_ARGS</code>. If the configure script
        !           272:        invokes other configure scripts, it strips the leading and
        !           273:        trailing whitespace from the variable and then passes it to the
        !           274:        other configure scripts. But these configure scripts expect the
        !           275:        (child) <code class="varname">CPPFLAGS</code> variable to be the same as
        !           276:        the parent <code class="varname">CPPFLAGS</code>. That's why we better
        !           277:        pass the <code class="varname">CPPFLAGS</code> value properly trimmed. And
        !           278:        here is how we do it:</p>
        !           279: <pre class="programlisting">
        !           280: CPPFLAGS=               # empty
        !           281: CPPFLAGS+=              -Wundef -DPREFIX=\"${PREFIX:Q}\"
        !           282: CPPFLAGS+=              ${MY_CPPFLAGS}
        !           283: 
        !           284: CONFIGURE_ARGS+=        CPPFLAGS=${CPPFLAGS:M*:Q}
        !           285: 
        !           286: all:
        !           287:         echo x${CPPFLAGS:Q}x            # leading and trailing whitespace
        !           288:         echo x${CONFIGURE_ARGS}x        # properly trimmed
        !           289: </pre>
        !           290: </li>
        !           291: <li><p>The example above contains one bug: The
        !           292:        <code class="varname">${PREFIX}</code> is a properly quoted shell
        !           293:        expression, but there is the C compiler after it, which also
        !           294:        expects a properly quoted string (this time in C syntax). The
        !           295:        version above is therefore only correct if
        !           296:        <code class="varname">${PREFIX}</code> does not have embedded backslashes
        !           297:        or double quotes. If you want to allow these, you have to add
        !           298:        another layer of quoting to each variable that is used as a C
        !           299:        string literal. You cannot use the <code class="varname">:Q</code>
        !           300:        operator for it, as this operator only works for the
        !           301:        shell.</p></li>
        !           302: <li>
        !           303: <p>Whenever a variable can be empty, the
        !           304:        <code class="varname">:Q</code> operator can have surprising results. Here
        !           305:        are two completely different cases which can be solved with the
        !           306:        same trick.</p>
        !           307: <pre class="programlisting">
        !           308: EMPTY=                  # empty
        !           309: empty_test:
        !           310:         for i in a ${EMPTY:Q} c; do \
        !           311:             echo "$$i"; \
        !           312:         done
        !           313: 
        !           314: for_test:
        !           315: .for i in a:\ a:\test.txt
        !           316:         echo ${i:Q}
        !           317:         echo "foo"
        !           318: .endfor
        !           319: </pre>
        !           320: <p>The first example will only print two of the three lines
        !           321:        we might have expected. This is because
        !           322:        <code class="varname">${EMPTY:Q}</code> expands to the empty string, which
        !           323:        the shell cannot see. The workaround is to write
        !           324:        <code class="varname">${EMPTY:Q}""</code>. This pattern can be often found
        !           325:        as <code class="varname">${TEST} -z ${VAR:Q}</code> or as <code class="varname">${TEST}
        !           326:        -f ${FNAME:Q}</code> (both of these are wrong).</p>
        !           327: <p>The second example will only print three lines instead of
        !           328:        four. The first line looks like <code class="varname">a:\ echo foo</code>.
        !           329:        This is because the backslash of the value
        !           330:        <code class="varname">a:\</code> is interpreted as a line-continuation by
        !           331:        <a href="http://netbsd.gw.com/cgi-bin/man-cgi?make+1+NetBSD-current"><span class="citerefentry"><span class="refentrytitle">make</span>(1)</span></a>, which makes the second line the arguments of the
        !           332:        <a href="http://netbsd.gw.com/cgi-bin/man-cgi?echo+1+NetBSD-current"><span class="citerefentry"><span class="refentrytitle">echo</span>(1)</span></a> command from the first line. To avoid this, write
        !           333:        <code class="varname">${i:Q}""</code>.</p>
        !           334: </li>
        !           335: </ul></div>
        !           336: </div>
        !           337: <div class="sect2" lang="en">
        !           338: <div class="titlepage"><div><div><h3 class="title">
        !           339: <a name="bsd-make-bug-workaround"></a>11.3.5. Workaround for a bug in BSD Make</h3></div></div></div>
        !           340: <p>The pkgsrc bmake program does not handle the following
        !           341:       assignment correctly. In case <code class="varname">_othervar_</code>
        !           342:       contains a ``-'' character, one of the closing braces is included
        !           343:       in <code class="varname">${VAR}</code> after this code executes.</p>
        !           344: <pre class="programlisting">
        !           345: VAR:=   ${VAR:N${_othervar_:C/-//}}
        !           346: </pre>
        !           347: <p>For a more complex code snippet and a workaround, see the
        !           348:       package <a href="ftp://ftp.NetBSD.org/pub/pkgsrc/current/pkgsrc/regress/make-quoting/README.html" target="_top"><code class="filename">regress/make-quoting</code></a>, testcase
        !           349:       <code class="varname">bug1</code>.</p>
        !           350: </div>
        !           351: </div>
        !           352: </div>
        !           353: <div class="navfooter">
        !           354: <hr>
        !           355: <table width="100%" summary="Navigation footer">
        !           356: <tr>
        !           357: <td width="40%" align="left">
        !           358: <a accesskey="p" href="components.html">Prev</a> </td>
        !           359: <td width="20%" align="center"><a accesskey="u" href="developers-guide.html">Up</a></td>
        !           360: <td width="40%" align="right"> <a accesskey="n" href="plist.html">Next</a>
        !           361: </td>
        !           362: </tr>
        !           363: <tr>
        !           364: <td width="40%" align="left" valign="top">Chapter 10. Package components - files, directories and contents </td>
        !           365: <td width="20%" align="center"><a accesskey="h" href="index.html">Home</a></td>
        !           366: <td width="40%" align="right" valign="top"> Chapter 12. PLIST issues</td>
        !           367: </tr>
        !           368: </table>
        !           369: </div>
        !           370: </body>
        !           371: </html>

CVSweb <webmaster@jp.NetBSD.org>