Annotation of htdocs/docs/pkgsrc/makefile.html, revision 1.1
1.1 ! dsieger 1: <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
! 2: <html>
! 3: <head>
! 4: <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
! 5: <title>Chapter 11. Programming in Makefiles</title>
! 6: <link rel="stylesheet" href="/NetBSD.css" type="text/css">
! 7: <meta name="generator" content="DocBook XSL Stylesheets VX.X.X">
! 8: <link rel="start" href="index.html" title="The pkgsrc guide">
! 9: <link rel="up" href="developers-guide.html" title="Part II. The pkgsrc developer's guide">
! 10: <link rel="prev" href="components.html" title="Chapter 10. Package components - files, directories and contents">
! 11: <link rel="next" href="plist.html" title="Chapter 12. PLIST issues">
! 12: </head>
! 13: <body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
! 14: <div class="navheader">
! 15: <table width="100%" summary="Navigation header">
! 16: <tr><th colspan="3" align="center">Chapter 11. Programming in <code class="filename">Makefile</code>s</th></tr>
! 17: <tr>
! 18: <td width="20%" align="left">
! 19: <a accesskey="p" href="components.html">Prev</a> </td>
! 20: <th width="60%" align="center">Part II. The pkgsrc developer's guide</th>
! 21: <td width="20%" align="right"> <a accesskey="n" href="plist.html">Next</a>
! 22: </td>
! 23: </tr>
! 24: </table>
! 25: <hr>
! 26: </div>
! 27: <div class="chapter" lang="en">
! 28: <div class="titlepage"><div><div><h2 class="title">
! 29: <a name="makefile"></a>Chapter 11. Programming in <code class="filename">Makefile</code>s</h2></div></div></div>
! 30: <div class="toc">
! 31: <p><b>Table of Contents</b></p>
! 32: <dl>
! 33: <dt><span class="sect1"><a href="makefile.html#makefile.style">11.1. Caveats</a></span></dt>
! 34: <dt><span class="sect1"><a href="makefile.html#makefile.variables">11.2. <code class="filename">Makefile</code> variables</a></span></dt>
! 35: <dd><dl><dt><span class="sect2"><a href="makefile.html#makefile.variables.names">11.2.1. Naming conventions</a></span></dt></dl></dd>
! 36: <dt><span class="sect1"><a href="makefile.html#makefile.code">11.3. Code snippets</a></span></dt>
! 37: <dd><dl>
! 38: <dt><span class="sect2"><a href="makefile.html#adding-to-list">11.3.1. Adding things to a list</a></span></dt>
! 39: <dt><span class="sect2"><a href="makefile.html#converting-internal-to-external">11.3.2. Converting an internal list into an external list</a></span></dt>
! 40: <dt><span class="sect2"><a href="makefile.html#passing-variable-to-shell">11.3.3. Passing variables to a shell command</a></span></dt>
! 41: <dt><span class="sect2"><a href="makefile.html#quoting-guideline">11.3.4. Quoting guideline</a></span></dt>
! 42: <dt><span class="sect2"><a href="makefile.html#bsd-make-bug-workaround">11.3.5. Workaround for a bug in BSD Make</a></span></dt>
! 43: </dl></dd>
! 44: </dl>
! 45: </div>
! 46: <p>Pkgsrc consists of many <code class="filename">Makefile</code> fragments,
! 47: each of which forms a well-defined part of the pkgsrc system. Using
! 48: the <a href="http://netbsd.gw.com/cgi-bin/man-cgi?make+1+NetBSD-current"><span class="citerefentry"><span class="refentrytitle">make</span>(1)</span></a> system as a programming language for a big system
! 49: like pkgsrc requires some discipline to keep the code correct and
! 50: understandable.</p>
! 51: <p>The basic ingredients for <code class="filename">Makefile</code>
! 52: programming are variables (which are actually macros) and shell
! 53: commands. Among these shell commands may even be more complex ones
! 54: like <a href="http://netbsd.gw.com/cgi-bin/man-cgi?awk+1+NetBSD-current"><span class="citerefentry"><span class="refentrytitle">awk</span>(1)</span></a> programs. To make sure that every shell command runs
! 55: as intended it is necessary to quote all variables correctly when they
! 56: are used.</p>
! 57: <p>This chapter describes some patterns, that appear quite often in
! 58: <code class="filename">Makefile</code>s, including the pitfalls that come along
! 59: with them.</p>
! 60: <div class="sect1" lang="en">
! 61: <div class="titlepage"><div><div><h2 class="title" style="clear: both">
! 62: <a name="makefile.style"></a>11.1. Caveats</h2></div></div></div>
! 63: <div class="itemizedlist"><ul type="disc"><li>
! 64: <p>When you are creating a file as a
! 65: target of a rule, always write the data to a temporary file first
! 66: and finally rename that file. Otherwise there might occur an error
! 67: in the middle of generating the file, and when the user runs
! 68: <a href="http://netbsd.gw.com/cgi-bin/man-cgi?make+1+NetBSD-current"><span class="citerefentry"><span class="refentrytitle">make</span>(1)</span></a> for the second time, the file exists and will not be
! 69: regenerated properly. Example:</p>
! 70: <pre class="programlisting">
! 71: wrong:
! 72: @echo "line 1" > ${.TARGET}
! 73: @echo "line 2" >> ${.TARGET}
! 74: @false
! 75:
! 76: correct:
! 77: @echo "line 1" > ${.TARGET}.tmp
! 78: @echo "line 2" >> ${.TARGET}.tmp
! 79: @false
! 80: @mv ${.TARGET}.tmp ${.TARGET}
! 81: </pre>
! 82: <p>When you run <span><strong class="command">make wrong</strong></span> twice, the file
! 83: <code class="filename">wrong</code> will exist, although there was an error
! 84: message in the first run. On the other hand, running <span><strong class="command">make
! 85: correct</strong></span> gives an error message twice, as expected.</p>
! 86: <p>You might remember that <a href="http://netbsd.gw.com/cgi-bin/man-cgi?make+1+NetBSD-current"><span class="citerefentry"><span class="refentrytitle">make</span>(1)</span></a> sometimes removes
! 87: <code class="literal">${.TARGET}</code> in case of error, but this only
! 88: happens when it is interrupted, for example by pressing
! 89: <code class="literal">^C</code>. This does <span class="emphasis"><em>not</em></span> happen
! 90: when one of the commands fails (like <a href="http://netbsd.gw.com/cgi-bin/man-cgi?false+1+NetBSD-current"><span class="citerefentry"><span class="refentrytitle">false</span>(1)</span></a> above).</p>
! 91: </li></ul></div>
! 92: </div>
! 93: <div class="sect1" lang="en">
! 94: <div class="titlepage"><div><div><h2 class="title" style="clear: both">
! 95: <a name="makefile.variables"></a>11.2. <code class="filename">Makefile</code> variables</h2></div></div></div>
! 96: <p><code class="filename">Makefile</code> variables contain strings that
! 97: can be processed using the five operators ``='', ``+='', ``?='',
! 98: ``:='', and ``!='', which are described in the <a href="http://netbsd.gw.com/cgi-bin/man-cgi?make+1+NetBSD-current"><span class="citerefentry"><span class="refentrytitle">make</span>(1)</span></a> man
! 99: page.</p>
! 100: <p>When a variable's value is parsed from a
! 101: <code class="filename">Makefile</code>, the hash character ``#'' and the
! 102: backslash character ``\'' are handled specially. If a backslash is
! 103: followed by a newline, any whitespace immediately in front of the
! 104: backslash, the backslash, the newline, and any whitespace
! 105: immediately behind the newline are replaced with a single space. A
! 106: backslash character and an immediately following hash character are
! 107: replaced with a single hash character. Otherwise, the backslash is
! 108: passed as is. In a variable assignment, any hash character that is
! 109: not preceded by a backslash starts a comment that continues upto the
! 110: end of the logical line.</p>
! 111: <p><span class="emphasis"><em>Note:</em></span> Because of this parsing algorithm
! 112: the only way to create a variable consisting of a single backslash
! 113: is using the ``!='' operator, for example: <code class="varname">BACKSLASH!=echo "\\"</code>.</p>
! 114: <p>So far for defining variables. The other thing you can do with
! 115: variables is evaluating them. A variable is evaluated when it is
! 116: part of the right side of the ``:='' or the ``!='' operator, or
! 117: directly before executing a shell command which the variable is part
! 118: of. In all other cases, <a href="http://netbsd.gw.com/cgi-bin/man-cgi?make+1+NetBSD-current"><span class="citerefentry"><span class="refentrytitle">make</span>(1)</span></a> performs lazy evaluation, that
! 119: is, variables are not evaluated until there's no other way. The
! 120: ``modifiers'' mentioned in the man page also evaluate the
! 121: variable.</p>
! 122: <p>Some of the modifiers split the string into words and then
! 123: operate on the words, others operate on the string as a whole. When
! 124: a string is split into words, it is split as you would expect
! 125: it from <a href="http://netbsd.gw.com/cgi-bin/man-cgi?sh+1+NetBSD-current"><span class="citerefentry"><span class="refentrytitle">sh</span>(1)</span></a>.</p>
! 126: <p>No rule without exception—the <span><strong class="command">.for</strong></span>
! 127: loop does not follow the shell quoting rules but splits at sequences
! 128: of whitespace.</p>
! 129: <p>There are several types of variables that should be handled
! 130: differently. Strings and two types of lists.</p>
! 131: <div class="itemizedlist"><ul type="disc">
! 132: <li><p><span class="emphasis"><em>Strings</em></span> can contain arbitrary
! 133: characters. Nevertheless, you should restrict yourself to only
! 134: using printable characters. Examples are
! 135: <code class="varname">PREFIX</code> and
! 136: <code class="varname">COMMENT</code>.</p></li>
! 137: <li><p><span class="emphasis"><em>Internal lists</em></span> are lists that
! 138: are never exported to any shell command. Their elements are
! 139: separated by whitespace. Therefore, the elements themselves cannot
! 140: have embedded whitespace. Any other characters are allowed.
! 141: Internal lists can be used in <span><strong class="command">.for</strong></span> loops.
! 142: Examples are <code class="varname">DEPENDS</code> and
! 143: <code class="varname">BUILD_DEPENDS</code>.</p></li>
! 144: <li><p><span class="emphasis"><em>External lists</em></span> are lists that
! 145: may be exported to a shell command. Their elements can contain any
! 146: characters, including whitespace. That's why they cannot be used
! 147: in <span><strong class="command">.for</strong></span> loops. Examples are
! 148: <code class="varname">DISTFILES</code> and
! 149: <code class="varname">MASTER_SITES</code>.</p></li>
! 150: </ul></div>
! 151: <div class="sect2" lang="en">
! 152: <div class="titlepage"><div><div><h3 class="title">
! 153: <a name="makefile.variables.names"></a>11.2.1. Naming conventions</h3></div></div></div>
! 154: <div class="itemizedlist"><ul type="disc">
! 155: <li><p>All variable names starting with an underscore
! 156: are reserved for use by the pkgsrc infrastructure. They shall
! 157: not be used by package
! 158: <code class="filename">Makefile</code>s.</p></li>
! 159: <li><p>In <span><strong class="command">.for</strong></span> loops you should use
! 160: lowercase variable names for the iteration
! 161: variables.</p></li>
! 162: <li><p>All list variables should have a ``plural''
! 163: name, e.g. <code class="varname">PKG_OPTIONS</code> or
! 164: <code class="varname">DISTFILES</code>.</p></li>
! 165: </ul></div>
! 166: </div>
! 167: </div>
! 168: <div class="sect1" lang="en">
! 169: <div class="titlepage"><div><div><h2 class="title" style="clear: both">
! 170: <a name="makefile.code"></a>11.3. Code snippets</h2></div></div></div>
! 171: <p>This section presents you with some code snippets you should
! 172: use in your own code. If you don't find anything appropriate here,
! 173: you should test your code and add it here.</p>
! 174: <div class="sect2" lang="en">
! 175: <div class="titlepage"><div><div><h3 class="title">
! 176: <a name="adding-to-list"></a>11.3.1. Adding things to a list</h3></div></div></div>
! 177: <pre class="programlisting">
! 178: STRING= foo * bar `date`
! 179: INT_LIST= # empty
! 180: ANOTHER_INT_LIST= apache-[0-9]*:../../www/apache
! 181: EXT_LIST= # empty
! 182: ANOTHER_EXT_LIST= a=b c=d
! 183:
! 184: INT_LIST+= ${STRING} # 1
! 185: INT_LIST+= ${ANOTHER_INT_LIST} # 2
! 186: EXT_LIST+= ${STRING:Q} # 3
! 187: EXT_LIST+= ${ANOTHER_EXT_LIST} # 4
! 188: </pre>
! 189: <p>When you add a string to an external list (example 3), it
! 190: must be quoted. In all other cases, you must not add a quoting
! 191: level. You must not merge internal and external lists, unless you
! 192: are sure that all entries are correctly interpreted in both
! 193: lists.</p>
! 194: </div>
! 195: <div class="sect2" lang="en">
! 196: <div class="titlepage"><div><div><h3 class="title">
! 197: <a name="converting-internal-to-external"></a>11.3.2. Converting an internal list into an external list</h3></div></div></div>
! 198: <pre class="programlisting">
! 199: EXT_LIST= # empty
! 200: .for i in ${INT_LIST}
! 201: EXT_LIST+= ${i:Q}""
! 202: .endfor
! 203: </pre>
! 204: <p>This code converts the internal list
! 205: <code class="varname">INT_LIST</code> into the external list
! 206: <code class="varname">EXT_LIST</code>. As the elements of an internal list
! 207: are unquoted they must be quoted here. The reason for appending
! 208: <code class="varname">""</code> is explained below.</p>
! 209: </div>
! 210: <div class="sect2" lang="en">
! 211: <div class="titlepage"><div><div><h3 class="title">
! 212: <a name="passing-variable-to-shell"></a>11.3.3. Passing variables to a shell command</h3></div></div></div>
! 213: <p>Sometimes you may want to print an arbitrary string. There
! 214: are many ways to get it wrong and only few that can handle every
! 215: nastiness.</p>
! 216: <pre class="programlisting">
! 217: STRING= foo bar < > * `date` $$HOME ' "
! 218: EXT_LIST= string=${STRING:Q} x=second\ item
! 219:
! 220: all:
! 221: echo ${STRING} # 1
! 222: echo "${STRING}" # 2
! 223: echo "${STRING:Q}" # 3
! 224: echo ${STRING:Q} # 4
! 225: echo x${STRING:Q} | sed 1s,.,, # 5
! 226: printf "%s\\n" ${STRING:Q}"" # 6
! 227: env ${EXT_LIST} /bin/sh -c 'echo "$$string"; echo "$$x"'
! 228: </pre>
! 229: <p>Example 1 leads to a syntax error in the shell, as the
! 230: characters are just copied.</p>
! 231: <p>Example 2 leads to a syntax error too, and if you leave out
! 232: the last " character from <code class="varname">${STRING}</code>,
! 233: <a href="http://netbsd.gw.com/cgi-bin/man-cgi?date+1+NetBSD-current"><span class="citerefentry"><span class="refentrytitle">date</span>(1)</span></a> will be executed. The <code class="varname">$HOME</code> shell
! 234: variable would be evaluated, too.</p>
! 235: <p>Example 3 outputs each space character preceded by a
! 236: backslash (or not), depending on the implementation of the
! 237: <a href="http://netbsd.gw.com/cgi-bin/man-cgi?echo+1+NetBSD-current"><span class="citerefentry"><span class="refentrytitle">echo</span>(1)</span></a> command.</p>
! 238: <p>Example 4 handles correctly every string that does not start
! 239: with a dash. In that case, the result depends on the
! 240: implementation of the <a href="http://netbsd.gw.com/cgi-bin/man-cgi?echo+1+NetBSD-current"><span class="citerefentry"><span class="refentrytitle">echo</span>(1)</span></a> command. As long as you can
! 241: guarantee that your input does not start with a dash, this form is
! 242: appropriate.</p>
! 243: <p>Example 5 handles even the case of a leading dash
! 244: correctly.</p>
! 245: <p>Example 6 also works with every string and is the
! 246: light-weight solution, since it does not involve a pipe, which has
! 247: its own problems.</p>
! 248: <p>The <code class="varname">EXT_LIST</code> does not need to be quoted
! 249: because the quoting has already been done when adding elements to
! 250: the list.</p>
! 251: <p>As internal lists shall not be passed to the shell, there is
! 252: no example for it.</p>
! 253: </div>
! 254: <div class="sect2" lang="en">
! 255: <div class="titlepage"><div><div><h3 class="title">
! 256: <a name="quoting-guideline"></a>11.3.4. Quoting guideline</h3></div></div></div>
! 257: <p>There are many possible sources of wrongly quoted variables.
! 258: This section lists some of the commonly known ones.</p>
! 259: <div class="itemizedlist"><ul type="disc">
! 260: <li>
! 261: <p>Whenever you use the value of a list, think
! 262: about what happens to leading or trailing whitespace. If the
! 263: list is a well-formed shell expression, you can apply the
! 264: <code class="varname">:M*</code> modifier to strip leading and trailing
! 265: whitespace from each word. The <code class="varname">:M</code> operator
! 266: first splits its argument according to the rules of the shell,
! 267: and then creates a new list consisting of all words that match
! 268: the shell glob expression <code class="varname">*</code>, that is: all.
! 269: One class of situations where this is needed is when adding a
! 270: variable like <code class="varname">CPPFLAGS</code> to
! 271: <code class="varname">CONFIGURE_ARGS</code>. If the configure script
! 272: invokes other configure scripts, it strips the leading and
! 273: trailing whitespace from the variable and then passes it to the
! 274: other configure scripts. But these configure scripts expect the
! 275: (child) <code class="varname">CPPFLAGS</code> variable to be the same as
! 276: the parent <code class="varname">CPPFLAGS</code>. That's why we better
! 277: pass the <code class="varname">CPPFLAGS</code> value properly trimmed. And
! 278: here is how we do it:</p>
! 279: <pre class="programlisting">
! 280: CPPFLAGS= # empty
! 281: CPPFLAGS+= -Wundef -DPREFIX=\"${PREFIX:Q}\"
! 282: CPPFLAGS+= ${MY_CPPFLAGS}
! 283:
! 284: CONFIGURE_ARGS+= CPPFLAGS=${CPPFLAGS:M*:Q}
! 285:
! 286: all:
! 287: echo x${CPPFLAGS:Q}x # leading and trailing whitespace
! 288: echo x${CONFIGURE_ARGS}x # properly trimmed
! 289: </pre>
! 290: </li>
! 291: <li><p>The example above contains one bug: The
! 292: <code class="varname">${PREFIX}</code> is a properly quoted shell
! 293: expression, but there is the C compiler after it, which also
! 294: expects a properly quoted string (this time in C syntax). The
! 295: version above is therefore only correct if
! 296: <code class="varname">${PREFIX}</code> does not have embedded backslashes
! 297: or double quotes. If you want to allow these, you have to add
! 298: another layer of quoting to each variable that is used as a C
! 299: string literal. You cannot use the <code class="varname">:Q</code>
! 300: operator for it, as this operator only works for the
! 301: shell.</p></li>
! 302: <li>
! 303: <p>Whenever a variable can be empty, the
! 304: <code class="varname">:Q</code> operator can have surprising results. Here
! 305: are two completely different cases which can be solved with the
! 306: same trick.</p>
! 307: <pre class="programlisting">
! 308: EMPTY= # empty
! 309: empty_test:
! 310: for i in a ${EMPTY:Q} c; do \
! 311: echo "$$i"; \
! 312: done
! 313:
! 314: for_test:
! 315: .for i in a:\ a:\test.txt
! 316: echo ${i:Q}
! 317: echo "foo"
! 318: .endfor
! 319: </pre>
! 320: <p>The first example will only print two of the three lines
! 321: we might have expected. This is because
! 322: <code class="varname">${EMPTY:Q}</code> expands to the empty string, which
! 323: the shell cannot see. The workaround is to write
! 324: <code class="varname">${EMPTY:Q}""</code>. This pattern can be often found
! 325: as <code class="varname">${TEST} -z ${VAR:Q}</code> or as <code class="varname">${TEST}
! 326: -f ${FNAME:Q}</code> (both of these are wrong).</p>
! 327: <p>The second example will only print three lines instead of
! 328: four. The first line looks like <code class="varname">a:\ echo foo</code>.
! 329: This is because the backslash of the value
! 330: <code class="varname">a:\</code> is interpreted as a line-continuation by
! 331: <a href="http://netbsd.gw.com/cgi-bin/man-cgi?make+1+NetBSD-current"><span class="citerefentry"><span class="refentrytitle">make</span>(1)</span></a>, which makes the second line the arguments of the
! 332: <a href="http://netbsd.gw.com/cgi-bin/man-cgi?echo+1+NetBSD-current"><span class="citerefentry"><span class="refentrytitle">echo</span>(1)</span></a> command from the first line. To avoid this, write
! 333: <code class="varname">${i:Q}""</code>.</p>
! 334: </li>
! 335: </ul></div>
! 336: </div>
! 337: <div class="sect2" lang="en">
! 338: <div class="titlepage"><div><div><h3 class="title">
! 339: <a name="bsd-make-bug-workaround"></a>11.3.5. Workaround for a bug in BSD Make</h3></div></div></div>
! 340: <p>The pkgsrc bmake program does not handle the following
! 341: assignment correctly. In case <code class="varname">_othervar_</code>
! 342: contains a ``-'' character, one of the closing braces is included
! 343: in <code class="varname">${VAR}</code> after this code executes.</p>
! 344: <pre class="programlisting">
! 345: VAR:= ${VAR:N${_othervar_:C/-//}}
! 346: </pre>
! 347: <p>For a more complex code snippet and a workaround, see the
! 348: package <a href="ftp://ftp.NetBSD.org/pub/pkgsrc/current/pkgsrc/regress/make-quoting/README.html" target="_top"><code class="filename">regress/make-quoting</code></a>, testcase
! 349: <code class="varname">bug1</code>.</p>
! 350: </div>
! 351: </div>
! 352: </div>
! 353: <div class="navfooter">
! 354: <hr>
! 355: <table width="100%" summary="Navigation footer">
! 356: <tr>
! 357: <td width="40%" align="left">
! 358: <a accesskey="p" href="components.html">Prev</a> </td>
! 359: <td width="20%" align="center"><a accesskey="u" href="developers-guide.html">Up</a></td>
! 360: <td width="40%" align="right"> <a accesskey="n" href="plist.html">Next</a>
! 361: </td>
! 362: </tr>
! 363: <tr>
! 364: <td width="40%" align="left" valign="top">Chapter 10. Package components - files, directories and contents </td>
! 365: <td width="20%" align="center"><a accesskey="h" href="index.html">Home</a></td>
! 366: <td width="40%" align="right" valign="top"> Chapter 12. PLIST issues</td>
! 367: </tr>
! 368: </table>
! 369: </div>
! 370: </body>
! 371: </html>
CVSweb <webmaster@jp.NetBSD.org>