Annotation of htdocs/docs/pkgsrc/makefile.html, revision 1.17
1.1 dsieger 1: <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
2: <html>
3: <head>
4: <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
1.4 rillig 5: <title>Chapter 12. Programming in Makefiles</title>
1.11 spz 6: <link rel="stylesheet" type="text/css" href="/global.css">
1.1 dsieger 7: <meta name="generator" content="DocBook XSL Stylesheets VX.X.X">
1.9 rillig 8: <link rel="home" href="index.html" title="The pkgsrc guide">
1.1 dsieger 9: <link rel="up" href="developers-guide.html" title="Part II. The pkgsrc developer's guide">
1.4 rillig 10: <link rel="prev" href="components.html" title="Chapter 11. Package components - files, directories and contents">
11: <link rel="next" href="plist.html" title="Chapter 13. PLIST issues">
1.1 dsieger 12: </head>
13: <body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
14: <div class="navheader">
15: <table width="100%" summary="Navigation header">
1.10 wiz 16: <tr><th colspan="3" align="center">Chapter 12. Programming in <code class="filename">Makefile</code>s</th></tr>
1.1 dsieger 17: <tr>
18: <td width="20%" align="left">
19: <a accesskey="p" href="components.html">Prev</a> </td>
20: <th width="60%" align="center">Part II. The pkgsrc developer's guide</th>
21: <td width="20%" align="right"> <a accesskey="n" href="plist.html">Next</a>
22: </td>
23: </tr>
24: </table>
25: <hr>
26: </div>
1.13 asau 27: <div class="chapter">
1.1 dsieger 28: <div class="titlepage"><div><div><h2 class="title">
1.4 rillig 29: <a name="makefile"></a>Chapter 12. Programming in <code class="filename">Makefile</code>s</h2></div></div></div>
1.1 dsieger 30: <div class="toc">
31: <p><b>Table of Contents</b></p>
1.17 ! sevan 32: <dl>
1.4 rillig 33: <dt><span class="sect1"><a href="makefile.html#makefile.style">12.1. Caveats</a></span></dt>
1.10 wiz 34: <dt><span class="sect1"><a href="makefile.html#makefile.variables">12.2. <code class="filename">Makefile</code> variables</a></span></dt>
1.4 rillig 35: <dd><dl><dt><span class="sect2"><a href="makefile.html#makefile.variables.names">12.2.1. Naming conventions</a></span></dt></dl></dd>
36: <dt><span class="sect1"><a href="makefile.html#makefile.code">12.3. Code snippets</a></span></dt>
1.1 dsieger 37: <dd><dl>
1.4 rillig 38: <dt><span class="sect2"><a href="makefile.html#adding-to-list">12.3.1. Adding things to a list</a></span></dt>
39: <dt><span class="sect2"><a href="makefile.html#converting-internal-to-external">12.3.2. Converting an internal list into an external list</a></span></dt>
40: <dt><span class="sect2"><a href="makefile.html#passing-variable-to-shell">12.3.3. Passing variables to a shell command</a></span></dt>
41: <dt><span class="sect2"><a href="makefile.html#quoting-guideline">12.3.4. Quoting guideline</a></span></dt>
42: <dt><span class="sect2"><a href="makefile.html#bsd-make-bug-workaround">12.3.5. Workaround for a bug in BSD Make</a></span></dt>
1.1 dsieger 43: </dl></dd>
44: </dl>
45: </div>
46: <p>Pkgsrc consists of many <code class="filename">Makefile</code> fragments,
47: each of which forms a well-defined part of the pkgsrc system. Using
1.12 dholland 48: the <a class="citerefentry" href="http://netbsd.gw.com/cgi-bin/man-cgi?make+1+NetBSD-5.0.1+i386"><span class="citerefentry"><span class="refentrytitle">make</span>(1)</span></a> system as a programming language for a big system
1.1 dsieger 49: like pkgsrc requires some discipline to keep the code correct and
50: understandable.</p>
51: <p>The basic ingredients for <code class="filename">Makefile</code>
52: programming are variables (which are actually macros) and shell
53: commands. Among these shell commands may even be more complex ones
1.12 dholland 54: like <a class="citerefentry" href="http://netbsd.gw.com/cgi-bin/man-cgi?awk+1+NetBSD-5.0.1+i386"><span class="citerefentry"><span class="refentrytitle">awk</span>(1)</span></a> programs. To make sure that every shell command runs
1.1 dsieger 55: as intended it is necessary to quote all variables correctly when they
56: are used.</p>
57: <p>This chapter describes some patterns, that appear quite often in
58: <code class="filename">Makefile</code>s, including the pitfalls that come along
59: with them.</p>
1.13 asau 60: <div class="sect1">
1.1 dsieger 61: <div class="titlepage"><div><div><h2 class="title" style="clear: both">
1.4 rillig 62: <a name="makefile.style"></a>12.1. Caveats</h2></div></div></div>
1.13 asau 63: <div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem">
1.1 dsieger 64: <p>When you are creating a file as a
65: target of a rule, always write the data to a temporary file first
66: and finally rename that file. Otherwise there might occur an error
67: in the middle of generating the file, and when the user runs
1.12 dholland 68: <a class="citerefentry" href="http://netbsd.gw.com/cgi-bin/man-cgi?make+1+NetBSD-5.0.1+i386"><span class="citerefentry"><span class="refentrytitle">make</span>(1)</span></a> for the second time, the file exists and will not be
1.1 dsieger 69: regenerated properly. Example:</p>
70: <pre class="programlisting">
71: wrong:
72: @echo "line 1" > ${.TARGET}
73: @echo "line 2" >> ${.TARGET}
74: @false
75:
76: correct:
77: @echo "line 1" > ${.TARGET}.tmp
78: @echo "line 2" >> ${.TARGET}.tmp
79: @false
80: @mv ${.TARGET}.tmp ${.TARGET}
81: </pre>
1.5 hubertf 82: <p>When you run <span class="command"><strong>make wrong</strong></span> twice, the file
1.1 dsieger 83: <code class="filename">wrong</code> will exist, although there was an error
1.5 hubertf 84: message in the first run. On the other hand, running <span class="command"><strong>make
1.1 dsieger 85: correct</strong></span> gives an error message twice, as expected.</p>
1.12 dholland 86: <p>You might remember that <a class="citerefentry" href="http://netbsd.gw.com/cgi-bin/man-cgi?make+1+NetBSD-5.0.1+i386"><span class="citerefentry"><span class="refentrytitle">make</span>(1)</span></a> sometimes removes
1.1 dsieger 87: <code class="literal">${.TARGET}</code> in case of error, but this only
88: happens when it is interrupted, for example by pressing
89: <code class="literal">^C</code>. This does <span class="emphasis"><em>not</em></span> happen
1.12 dholland 90: when one of the commands fails (like <a class="citerefentry" href="http://netbsd.gw.com/cgi-bin/man-cgi?false+1+NetBSD-5.0.1+i386"><span class="citerefentry"><span class="refentrytitle">false</span>(1)</span></a> above).</p>
1.1 dsieger 91: </li></ul></div>
92: </div>
1.13 asau 93: <div class="sect1">
1.1 dsieger 94: <div class="titlepage"><div><div><h2 class="title" style="clear: both">
1.4 rillig 95: <a name="makefile.variables"></a>12.2. <code class="filename">Makefile</code> variables</h2></div></div></div>
1.1 dsieger 96: <p><code class="filename">Makefile</code> variables contain strings that
97: can be processed using the five operators ``='', ``+='', ``?='',
1.12 dholland 98: ``:='', and ``!='', which are described in the <a class="citerefentry" href="http://netbsd.gw.com/cgi-bin/man-cgi?make+1+NetBSD-5.0.1+i386"><span class="citerefentry"><span class="refentrytitle">make</span>(1)</span></a> man
1.1 dsieger 99: page.</p>
100: <p>When a variable's value is parsed from a
101: <code class="filename">Makefile</code>, the hash character ``#'' and the
102: backslash character ``\'' are handled specially. If a backslash is
103: followed by a newline, any whitespace immediately in front of the
104: backslash, the backslash, the newline, and any whitespace
105: immediately behind the newline are replaced with a single space. A
106: backslash character and an immediately following hash character are
107: replaced with a single hash character. Otherwise, the backslash is
108: passed as is. In a variable assignment, any hash character that is
109: not preceded by a backslash starts a comment that continues upto the
110: end of the logical line.</p>
111: <p><span class="emphasis"><em>Note:</em></span> Because of this parsing algorithm
112: the only way to create a variable consisting of a single backslash
113: is using the ``!='' operator, for example: <code class="varname">BACKSLASH!=echo "\\"</code>.</p>
114: <p>So far for defining variables. The other thing you can do with
115: variables is evaluating them. A variable is evaluated when it is
116: part of the right side of the ``:='' or the ``!='' operator, or
117: directly before executing a shell command which the variable is part
1.12 dholland 118: of. In all other cases, <a class="citerefentry" href="http://netbsd.gw.com/cgi-bin/man-cgi?make+1+NetBSD-5.0.1+i386"><span class="citerefentry"><span class="refentrytitle">make</span>(1)</span></a> performs lazy evaluation, that
1.1 dsieger 119: is, variables are not evaluated until there's no other way. The
120: ``modifiers'' mentioned in the man page also evaluate the
121: variable.</p>
122: <p>Some of the modifiers split the string into words and then
123: operate on the words, others operate on the string as a whole. When
124: a string is split into words, it is split as you would expect
1.12 dholland 125: it from <a class="citerefentry" href="http://netbsd.gw.com/cgi-bin/man-cgi?sh+1+NetBSD-5.0.1+i386"><span class="citerefentry"><span class="refentrytitle">sh</span>(1)</span></a>.</p>
1.8 jmcneill 126: <p>No rule without exception—the <span class="command"><strong>.for</strong></span>
1.1 dsieger 127: loop does not follow the shell quoting rules but splits at sequences
128: of whitespace.</p>
129: <p>There are several types of variables that should be handled
130: differently. Strings and two types of lists.</p>
1.13 asau 131: <div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; ">
1.10 wiz 132: <li class="listitem"><p><span class="emphasis"><em>Strings</em></span> can contain arbitrary
1.1 dsieger 133: characters. Nevertheless, you should restrict yourself to only
134: using printable characters. Examples are
135: <code class="varname">PREFIX</code> and
136: <code class="varname">COMMENT</code>.</p></li>
1.10 wiz 137: <li class="listitem"><p><span class="emphasis"><em>Internal lists</em></span> are lists that
1.1 dsieger 138: are never exported to any shell command. Their elements are
139: separated by whitespace. Therefore, the elements themselves cannot
140: have embedded whitespace. Any other characters are allowed.
1.5 hubertf 141: Internal lists can be used in <span class="command"><strong>.for</strong></span> loops.
1.1 dsieger 142: Examples are <code class="varname">DEPENDS</code> and
143: <code class="varname">BUILD_DEPENDS</code>.</p></li>
1.10 wiz 144: <li class="listitem"><p><span class="emphasis"><em>External lists</em></span> are lists that
1.1 dsieger 145: may be exported to a shell command. Their elements can contain any
146: characters, including whitespace. That's why they cannot be used
1.5 hubertf 147: in <span class="command"><strong>.for</strong></span> loops. Examples are
1.1 dsieger 148: <code class="varname">DISTFILES</code> and
149: <code class="varname">MASTER_SITES</code>.</p></li>
150: </ul></div>
1.13 asau 151: <div class="sect2">
1.1 dsieger 152: <div class="titlepage"><div><div><h3 class="title">
1.4 rillig 153: <a name="makefile.variables.names"></a>12.2.1. Naming conventions</h3></div></div></div>
1.13 asau 154: <div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; ">
1.10 wiz 155: <li class="listitem"><p>All variable names starting with an underscore
1.1 dsieger 156: are reserved for use by the pkgsrc infrastructure. They shall
157: not be used by package
158: <code class="filename">Makefile</code>s.</p></li>
1.10 wiz 159: <li class="listitem"><p>In <span class="command"><strong>.for</strong></span> loops you should use
1.1 dsieger 160: lowercase variable names for the iteration
161: variables.</p></li>
1.10 wiz 162: <li class="listitem"><p>All list variables should have a ``plural''
1.1 dsieger 163: name, e.g. <code class="varname">PKG_OPTIONS</code> or
164: <code class="varname">DISTFILES</code>.</p></li>
165: </ul></div>
166: </div>
167: </div>
1.13 asau 168: <div class="sect1">
1.1 dsieger 169: <div class="titlepage"><div><div><h2 class="title" style="clear: both">
1.4 rillig 170: <a name="makefile.code"></a>12.3. Code snippets</h2></div></div></div>
1.1 dsieger 171: <p>This section presents you with some code snippets you should
172: use in your own code. If you don't find anything appropriate here,
173: you should test your code and add it here.</p>
1.13 asau 174: <div class="sect2">
1.1 dsieger 175: <div class="titlepage"><div><div><h3 class="title">
1.4 rillig 176: <a name="adding-to-list"></a>12.3.1. Adding things to a list</h3></div></div></div>
1.1 dsieger 177: <pre class="programlisting">
178: STRING= foo * bar `date`
179: INT_LIST= # empty
180: ANOTHER_INT_LIST= apache-[0-9]*:../../www/apache
181: EXT_LIST= # empty
182: ANOTHER_EXT_LIST= a=b c=d
183:
184: INT_LIST+= ${STRING} # 1
185: INT_LIST+= ${ANOTHER_INT_LIST} # 2
186: EXT_LIST+= ${STRING:Q} # 3
187: EXT_LIST+= ${ANOTHER_EXT_LIST} # 4
188: </pre>
189: <p>When you add a string to an external list (example 3), it
190: must be quoted. In all other cases, you must not add a quoting
191: level. You must not merge internal and external lists, unless you
192: are sure that all entries are correctly interpreted in both
193: lists.</p>
194: </div>
1.13 asau 195: <div class="sect2">
1.1 dsieger 196: <div class="titlepage"><div><div><h3 class="title">
1.4 rillig 197: <a name="converting-internal-to-external"></a>12.3.2. Converting an internal list into an external list</h3></div></div></div>
1.1 dsieger 198: <pre class="programlisting">
199: EXT_LIST= # empty
200: .for i in ${INT_LIST}
201: EXT_LIST+= ${i:Q}""
202: .endfor
203: </pre>
204: <p>This code converts the internal list
205: <code class="varname">INT_LIST</code> into the external list
206: <code class="varname">EXT_LIST</code>. As the elements of an internal list
207: are unquoted they must be quoted here. The reason for appending
208: <code class="varname">""</code> is explained below.</p>
209: </div>
1.13 asau 210: <div class="sect2">
1.1 dsieger 211: <div class="titlepage"><div><div><h3 class="title">
1.4 rillig 212: <a name="passing-variable-to-shell"></a>12.3.3. Passing variables to a shell command</h3></div></div></div>
1.1 dsieger 213: <p>Sometimes you may want to print an arbitrary string. There
214: are many ways to get it wrong and only few that can handle every
215: nastiness.</p>
216: <pre class="programlisting">
217: STRING= foo bar < > * `date` $$HOME ' "
218: EXT_LIST= string=${STRING:Q} x=second\ item
219:
220: all:
221: echo ${STRING} # 1
222: echo "${STRING}" # 2
223: echo "${STRING:Q}" # 3
224: echo ${STRING:Q} # 4
225: echo x${STRING:Q} | sed 1s,.,, # 5
226: printf "%s\\n" ${STRING:Q}"" # 6
227: env ${EXT_LIST} /bin/sh -c 'echo "$$string"; echo "$$x"'
228: </pre>
229: <p>Example 1 leads to a syntax error in the shell, as the
230: characters are just copied.</p>
231: <p>Example 2 leads to a syntax error too, and if you leave out
232: the last " character from <code class="varname">${STRING}</code>,
1.12 dholland 233: <a class="citerefentry" href="http://netbsd.gw.com/cgi-bin/man-cgi?date+1+NetBSD-5.0.1+i386"><span class="citerefentry"><span class="refentrytitle">date</span>(1)</span></a> will be executed. The <code class="varname">$HOME</code> shell
1.1 dsieger 234: variable would be evaluated, too.</p>
235: <p>Example 3 outputs each space character preceded by a
236: backslash (or not), depending on the implementation of the
1.12 dholland 237: <a class="citerefentry" href="http://netbsd.gw.com/cgi-bin/man-cgi?echo+1+NetBSD-5.0.1+i386"><span class="citerefentry"><span class="refentrytitle">echo</span>(1)</span></a> command.</p>
1.1 dsieger 238: <p>Example 4 handles correctly every string that does not start
239: with a dash. In that case, the result depends on the
1.12 dholland 240: implementation of the <a class="citerefentry" href="http://netbsd.gw.com/cgi-bin/man-cgi?echo+1+NetBSD-5.0.1+i386"><span class="citerefentry"><span class="refentrytitle">echo</span>(1)</span></a> command. As long as you can
1.1 dsieger 241: guarantee that your input does not start with a dash, this form is
242: appropriate.</p>
243: <p>Example 5 handles even the case of a leading dash
244: correctly.</p>
245: <p>Example 6 also works with every string and is the
246: light-weight solution, since it does not involve a pipe, which has
247: its own problems.</p>
248: <p>The <code class="varname">EXT_LIST</code> does not need to be quoted
249: because the quoting has already been done when adding elements to
250: the list.</p>
251: <p>As internal lists shall not be passed to the shell, there is
252: no example for it.</p>
253: </div>
1.13 asau 254: <div class="sect2">
1.1 dsieger 255: <div class="titlepage"><div><div><h3 class="title">
1.4 rillig 256: <a name="quoting-guideline"></a>12.3.4. Quoting guideline</h3></div></div></div>
1.1 dsieger 257: <p>There are many possible sources of wrongly quoted variables.
258: This section lists some of the commonly known ones.</p>
1.13 asau 259: <div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; ">
1.10 wiz 260: <li class="listitem">
1.1 dsieger 261: <p>Whenever you use the value of a list, think
262: about what happens to leading or trailing whitespace. If the
263: list is a well-formed shell expression, you can apply the
264: <code class="varname">:M*</code> modifier to strip leading and trailing
265: whitespace from each word. The <code class="varname">:M</code> operator
266: first splits its argument according to the rules of the shell,
267: and then creates a new list consisting of all words that match
268: the shell glob expression <code class="varname">*</code>, that is: all.
269: One class of situations where this is needed is when adding a
270: variable like <code class="varname">CPPFLAGS</code> to
271: <code class="varname">CONFIGURE_ARGS</code>. If the configure script
272: invokes other configure scripts, it strips the leading and
273: trailing whitespace from the variable and then passes it to the
274: other configure scripts. But these configure scripts expect the
275: (child) <code class="varname">CPPFLAGS</code> variable to be the same as
276: the parent <code class="varname">CPPFLAGS</code>. That's why we better
277: pass the <code class="varname">CPPFLAGS</code> value properly trimmed. And
278: here is how we do it:</p>
279: <pre class="programlisting">
280: CPPFLAGS= # empty
281: CPPFLAGS+= -Wundef -DPREFIX=\"${PREFIX:Q}\"
282: CPPFLAGS+= ${MY_CPPFLAGS}
283:
284: CONFIGURE_ARGS+= CPPFLAGS=${CPPFLAGS:M*:Q}
285:
286: all:
287: echo x${CPPFLAGS:Q}x # leading and trailing whitespace
288: echo x${CONFIGURE_ARGS}x # properly trimmed
289: </pre>
290: </li>
1.10 wiz 291: <li class="listitem"><p>The example above contains one bug: The
1.1 dsieger 292: <code class="varname">${PREFIX}</code> is a properly quoted shell
293: expression, but there is the C compiler after it, which also
294: expects a properly quoted string (this time in C syntax). The
295: version above is therefore only correct if
296: <code class="varname">${PREFIX}</code> does not have embedded backslashes
297: or double quotes. If you want to allow these, you have to add
298: another layer of quoting to each variable that is used as a C
299: string literal. You cannot use the <code class="varname">:Q</code>
300: operator for it, as this operator only works for the
301: shell.</p></li>
1.10 wiz 302: <li class="listitem">
1.1 dsieger 303: <p>Whenever a variable can be empty, the
304: <code class="varname">:Q</code> operator can have surprising results. Here
305: are two completely different cases which can be solved with the
306: same trick.</p>
307: <pre class="programlisting">
308: EMPTY= # empty
309: empty_test:
310: for i in a ${EMPTY:Q} c; do \
311: echo "$$i"; \
312: done
313:
314: for_test:
315: .for i in a:\ a:\test.txt
316: echo ${i:Q}
317: echo "foo"
318: .endfor
319: </pre>
320: <p>The first example will only print two of the three lines
321: we might have expected. This is because
322: <code class="varname">${EMPTY:Q}</code> expands to the empty string, which
323: the shell cannot see. The workaround is to write
324: <code class="varname">${EMPTY:Q}""</code>. This pattern can be often found
325: as <code class="varname">${TEST} -z ${VAR:Q}</code> or as <code class="varname">${TEST}
326: -f ${FNAME:Q}</code> (both of these are wrong).</p>
327: <p>The second example will only print three lines instead of
328: four. The first line looks like <code class="varname">a:\ echo foo</code>.
329: This is because the backslash of the value
330: <code class="varname">a:\</code> is interpreted as a line-continuation by
1.12 dholland 331: <a class="citerefentry" href="http://netbsd.gw.com/cgi-bin/man-cgi?make+1+NetBSD-5.0.1+i386"><span class="citerefentry"><span class="refentrytitle">make</span>(1)</span></a>, which makes the second line the arguments of the
332: <a class="citerefentry" href="http://netbsd.gw.com/cgi-bin/man-cgi?echo+1+NetBSD-5.0.1+i386"><span class="citerefentry"><span class="refentrytitle">echo</span>(1)</span></a> command from the first line. To avoid this, write
1.1 dsieger 333: <code class="varname">${i:Q}""</code>.</p>
334: </li>
335: </ul></div>
336: </div>
1.13 asau 337: <div class="sect2">
1.1 dsieger 338: <div class="titlepage"><div><div><h3 class="title">
1.4 rillig 339: <a name="bsd-make-bug-workaround"></a>12.3.5. Workaround for a bug in BSD Make</h3></div></div></div>
1.1 dsieger 340: <p>The pkgsrc bmake program does not handle the following
341: assignment correctly. In case <code class="varname">_othervar_</code>
342: contains a ``-'' character, one of the closing braces is included
343: in <code class="varname">${VAR}</code> after this code executes.</p>
344: <pre class="programlisting">
345: VAR:= ${VAR:N${_othervar_:C/-//}}
346: </pre>
347: <p>For a more complex code snippet and a workaround, see the
348: package <a href="ftp://ftp.NetBSD.org/pub/pkgsrc/current/pkgsrc/regress/make-quoting/README.html" target="_top"><code class="filename">regress/make-quoting</code></a>, testcase
349: <code class="varname">bug1</code>.</p>
350: </div>
351: </div>
352: </div>
353: <div class="navfooter">
354: <hr>
355: <table width="100%" summary="Navigation footer">
356: <tr>
357: <td width="40%" align="left">
358: <a accesskey="p" href="components.html">Prev</a> </td>
359: <td width="20%" align="center"><a accesskey="u" href="developers-guide.html">Up</a></td>
360: <td width="40%" align="right"> <a accesskey="n" href="plist.html">Next</a>
361: </td>
362: </tr>
363: <tr>
1.4 rillig 364: <td width="40%" align="left" valign="top">Chapter 11. Package components - files, directories and contents </td>
1.1 dsieger 365: <td width="20%" align="center"><a accesskey="h" href="index.html">Home</a></td>
1.4 rillig 366: <td width="40%" align="right" valign="top"> Chapter 13. PLIST issues</td>
1.1 dsieger 367: </tr>
368: </table>
369: </div>
370: </body>
371: </html>
CVSweb <webmaster@jp.NetBSD.org>