Annotation of htdocs/docs/pkgsrc/creating.html, revision 1.5

1.1       dsieger     1: <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
                      2: <html>
                      3: <head>
                      4: <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
                      5: <title>Chapter 9. Creating a new pkgsrc package from scratch</title>
1.2       rillig      6: <link rel="stylesheet" href="/global.css" type="text/css">
1.1       dsieger     7: <meta name="generator" content="DocBook XSL Stylesheets VX.X.X">
                      8: <link rel="start" href="index.html" title="The pkgsrc guide">
                      9: <link rel="up" href="developers-guide.html" title="Part II. The pkgsrc developer's guide">
                     10: <link rel="prev" href="developers-guide.html" title="Part II. The pkgsrc developer's guide">
                     11: <link rel="next" href="components.html" title="Chapter 10. Package components - files, directories and contents">
                     12: </head>
                     13: <body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
                     14: <div class="navheader">
                     15: <table width="100%" summary="Navigation header">
                     16: <tr><th colspan="3" align="center">Chapter 9. Creating a new pkgsrc package from scratch</th></tr>
                     17: <tr>
                     18: <td width="20%" align="left">
                     19: <a accesskey="p" href="developers-guide.html">Prev</a> </td>
                     20: <th width="60%" align="center">Part II. The pkgsrc developer's guide</th>
                     21: <td width="20%" align="right"> <a accesskey="n" href="components.html">Next</a>
                     22: </td>
                     23: </tr>
                     24: </table>
                     25: <hr>
                     26: </div>
                     27: <div class="chapter" lang="en">
                     28: <div class="titlepage"><div><div><h2 class="title">
                     29: <a name="creating"></a>Chapter 9. Creating a new pkgsrc package from scratch</h2></div></div></div>
                     30: <div class="toc">
                     31: <p><b>Table of Contents</b></p>
                     32: <dl>
                     33: <dt><span class="sect1"><a href="creating.html#creating.common">9.1. Common types of packages</a></span></dt>
                     34: <dd><dl>
                     35: <dt><span class="sect2"><a href="creating.html#creating.perl-module">9.1.1. Perl modules</a></span></dt>
                     36: <dt><span class="sect2"><a href="creating.html#creating.kde-app">9.1.2. KDE applications</a></span></dt>
                     37: </dl></dd>
                     38: <dt><span class="sect1"><a href="creating.html#creating.examples">9.2. Examples</a></span></dt>
                     39: <dd><dl><dt><span class="sect2"><a href="creating.html#creating.nvu">9.2.1. How the www/nvu package came into pkgsrc</a></span></dt></dl></dd>
                     40: </dl>
                     41: </div>
                     42: <p>When you find a package that is not yet in pkgsrc, you
                     43: most likely have a URL from where you can download the source
                     44: code. Starting with this URL, creating a package involves only a
                     45: few steps.</p>
                     46: <div class="procedure"><ol type="1">
                     47: <li><p>First, install the packages <a href="ftp://ftp.NetBSD.org/pub/pkgsrc/current/pkgsrc/pkgtools/url2pkg/README.html" target="_top"><code class="filename">pkgtools/url2pkg</code></a> and <a href="ftp://ftp.NetBSD.org/pub/pkgsrc/current/pkgsrc/pkgtools/pkglint/README.html" target="_top"><code class="filename">pkgtools/pkglint</code></a>.</p></li>
                     48: <li><p>Then, choose one of the top-level directories as the
                     49: category in which you want to place your package. You can also create a
                     50: directory of your own (maybe called <code class="filename">local</code>). In that
                     51: category directory, create another directory for your package and change
                     52: into it.</p></li>
                     53: <li><p>Run the program <span><strong class="command">url2pkg</strong></span>, which will ask
                     54: you for a URL. Enter the URL of the distribution file (in most cases a
                     55: <code class="filename">.tar.gz</code> file) and watch how the basic ingredients
                     56: of your package are created automatically. The distribution file is
                     57: extracted automatically to fill in some details in the
                     58: <code class="filename">Makefile</code> that would otherwise have to be done
                     59: manually.</p></li>
                     60: <li>
                     61: <p>Examine the extracted files to determine the dependencies of
                     62: your package. Ideally, this is mentioned in some
                     63: <code class="filename">README</code> file, but things may differ. For each of
                     64: these dependencies, look where it exists in pkgsrc, and if there is a
                     65: file called <code class="filename">buildlink3.mk</code> in that directory, add a
                     66: line to your package <code class="filename">Makefile</code> which includes that
                     67: file just before the last line. If the
                     68: <code class="filename">buildlink3.mk</code> file does not exist, add a
                     69: <code class="varname">DEPENDS</code> line to the Makefile, which specifies the
                     70: version of the dependency and where it can be found in pkgsrc. This line
                     71: should be placed in the third paragraph. If the dependency is only
                     72: needed for building the package, but not when using it, use
                     73: <code class="varname">BUILD_DEPENDS</code> instead of <code class="varname">DEPENDS</code>.
                     74: Your package may then look like this:</p>
                     75: <pre class="programlisting">
                     76: [...]
                     77: 
                     78: BUILD_DEPENDS+= lua&gt;=5.0:../../lang/lua
                     79: DEPENDS+=       screen-[0-9]*:../../misc/screen
                     80: DEPENDS+=       screen&gt;=4.0:../../misc/screen
                     81: 
                     82: [...]
                     83: 
                     84: .include "../../<em class="replaceable"><code>category</code></em>/<em class="replaceable"><code>package</code></em>/buildlink3.mk"
                     85: .include "../../devel/glib2/buildlink3.mk"
                     86: .include "../../mk/bsd.pkg.mk"
                     87: </pre>
                     88: </li>
                     89: <li><p>Run <span><strong class="command">pkglint</strong></span> to see what things still need
                     90: to be done to make your package a &#8220;<span class="quote">good</span>&#8221; one. If you don't
                     91: know what pkglint's warnings want to tell you, try <span><strong class="command">pkglint
                     92: --explain</strong></span> or <span><strong class="command">pkglint
                     93: -e</strong></span>, which outputs additional
                     94: explanations.</p></li>
                     95: <li><p>In many cases the package is not yet ready to build. You can
                     96: find instructions for the most common cases in the next section, <a href="creating.html#creating.common" title="9.1. Common types of packages">Section 9.1, &#8220;Common types of packages&#8221;</a>. After you have followed the instructions
                     97: over there, you can hopefully continue here.</p></li>
                     98: <li><p>Run <span><strong class="command">bmake clean</strong></span> to clean the working
                     99: directory from the extracted files. Besides these files, a lot of cache
                    100: files and other system information has been saved in the working
                    101: directory, which may become wrong after you edited the
                    102: <code class="filename">Makefile</code>.</p></li>
                    103: <li><p>Now, run <span><strong class="command">bmake</strong></span> to build the package. For
                    104: the various things that can go wrong in this phase, consult <a href="fixes.html" title="Chapter 18. Making your package work">Chapter 18, <i>Making your package work</i></a>.</p></li>
                    105: <li><p>When the package builds fine, the next step is to install
                    106: the package. Run <span><strong class="command">bmake install</strong></span> and hope that
                    107: everything works.</p></li>
                    108: <li><p>Up to now, the file <code class="filename">PLIST</code>, which
                    109: contains a list of the files that are installed by the package, is
                    110: nearly empty. Run <span><strong class="command">bmake print-PLIST
                    111: &gt;PLIST</strong></span> to generate a probably correct list. Check
                    112: the file using your preferred text editor to see if the list of
                    113: files looks plausible.</p></li>
                    114: <li><p>Run <span><strong class="command">pkglint</strong></span> again to see if the generated
                    115: <code class="filename">PLIST</code> contains garbage or not.</p></li>
                    116: <li><p>When you ran <span><strong class="command">bmake install</strong></span>, the package
                    117: has been registered in the database of installed files, but with an
                    118: empty list of files. To fix this, run <span><strong class="command">bmake deinstall</strong></span>
                    119: and <span><strong class="command">bmake install</strong></span> again. Now the package is
                    120: registered with the list of files from
                    121: <code class="filename">PLIST</code>.</p></li>
                    122: <li><p>Run <span><strong class="command">bmake package</strong></span> to create a binary
                    123: package from the set of installed files.</p></li>
                    124: </ol></div>
                    125: <div class="sect1" lang="en">
                    126: <div class="titlepage"><div><div><h2 class="title" style="clear: both">
                    127: <a name="creating.common"></a>9.1. Common types of packages</h2></div></div></div>
                    128: <div class="sect2" lang="en">
                    129: <div class="titlepage"><div><div><h3 class="title">
                    130: <a name="creating.perl-module"></a>9.1.1. Perl modules</h3></div></div></div>
                    131: <p>Simple Perl modules are handled automatically by
                    132: <span><strong class="command">url2pkg</strong></span>, including dependencies.</p>
                    133: </div>
                    134: <div class="sect2" lang="en">
                    135: <div class="titlepage"><div><div><h3 class="title">
                    136: <a name="creating.kde-app"></a>9.1.2. KDE applications</h3></div></div></div>
                    137: <p>KDE applications should always include
                    138: <code class="filename">meta-pkgs/kde3/kde3.mk</code>, which contains numerous
                    139: settings that are typical of KDE packages.</p>
                    140: </div>
                    141: </div>
                    142: <div class="sect1" lang="en">
                    143: <div class="titlepage"><div><div><h2 class="title" style="clear: both">
                    144: <a name="creating.examples"></a>9.2. Examples</h2></div></div></div>
                    145: <div class="sect2" lang="en">
                    146: <div class="titlepage"><div><div><h3 class="title">
                    147: <a name="creating.nvu"></a>9.2.1. How the www/nvu package came into pkgsrc</h3></div></div></div>
                    148: <div class="sect3" lang="en">
                    149: <div class="titlepage"><div><div><h4 class="title">
                    150: <a name="creating.nvu.init"></a>9.2.1.1. The initial package</h4></div></div></div>
                    151: <p>Looking at the file <code class="filename">pkgsrc/doc/TODO</code>, I saw
                    152: that the &#8220;<span class="quote">nvu</span>&#8221; package has not yet been imported into
                    153: pkgsrc. As the description says it has to do with the web, the obvious
                    154: choice for the category is &#8220;<span class="quote">www</span>&#8221;.</p>
                    155: <pre class="programlisting">
                    156: <code class="prompt">$</code> mkdir www/nvu
                    157: <code class="prompt">$</code> cd www/nvu
                    158: </pre>
                    159: <p>The web site says that the sources are available as a tar file, so
                    160: I fed that URL to the <span><strong class="command">url2pkg</strong></span> program:</p>
                    161: <pre class="programlisting">
                    162: <code class="prompt">$</code> url2pkg http://cvs.nvu.com/download/nvu-1.0-sources.tar.bz2
                    163: </pre>
                    164: <p>My editor popped up, and I added a <code class="varname">PKGNAME</code> line
                    165: below the <code class="varname">DISTNAME</code> line, as the package name should
                    166: not have the word &#8220;<span class="quote">sources</span>&#8221; in it. I also filled in the
                    167: <code class="varname">MAINTAINER</code>, <code class="varname">HOMEPAGE</code> and
                    168: <code class="varname">COMMENT</code> fields. Then the package
                    169: <code class="filename">Makefile</code> looked like that:</p>
                    170: <pre class="programlisting">
1.2       rillig    171: # $NetBSD$
1.1       dsieger   172: #
                    173: 
                    174: DISTNAME=       nvu-1.0-sources
                    175: PKGNAME=        nvu-1.0
                    176: CATEGORIES=     www
                    177: MASTER_SITES=   http://cvs.nvu.com/download/
                    178: EXTRACT_SUFX=   .tar.bz2
                    179: 
                    180: MAINTAINER=     rillig@NetBSD.org
                    181: HOMEPAGE=       http://cvs.nvu.com/
                    182: COMMENT=        Web Authoring System
                    183: 
                    184: # url2pkg-marker (please do not remove this line.)
                    185: .include "../../mk/bsd.pkg.mk"
                    186: </pre>
                    187: <p>Then, I quit the editor and watched pkgsrc downloading a large
                    188: source archive:</p>
                    189: <pre class="programlisting">
                    190: url2pkg&gt; Running "make makesum" ...
                    191: =&gt; Required installed package digest&gt;=20010302: digest-20060826 found
                    192: =&gt; Fetching nvu-1.0-sources.tar.bz2
                    193: Requesting http://cvs.nvu.com/download/nvu-1.0-sources.tar.bz2
                    194: 100% |*************************************| 28992 KB  150.77 KB/s00:00 ETA
                    195: 29687976 bytes retrieved in 03:12 (150.77 KB/s)
                    196: url2pkg&gt; Running "make extract" ...
                    197: =&gt; Required installed package digest&gt;=20010302: digest-20060826 found
                    198: =&gt; Checksum SHA1 OK for nvu-1.0-sources.tar.bz2
                    199: =&gt; Checksum RMD160 OK for nvu-1.0-sources.tar.bz2
                    200: work.bacc -&gt; /tmp/roland/pkgsrc/www/nvu/work.bacc
                    201: ===&gt; Installing dependencies for nvu-1.0
                    202: ===&gt; Overriding tools for nvu-1.0
                    203: ===&gt; Extracting for nvu-1.0
                    204: url2pkg&gt; Adjusting the Makefile.
                    205: 
                    206: Remember to correct CATEGORIES, HOMEPAGE, COMMENT, and DESCR when you're done!
                    207: 
                    208: Good luck! (See pkgsrc/doc/pkgsrc.txt for some more help :-)
                    209: </pre>
                    210: </div>
                    211: <div class="sect3" lang="en">
                    212: <div class="titlepage"><div><div><h4 class="title">
                    213: <a name="creating.nvu.problems"></a>9.2.1.2. Fixing all kinds of problems to make the package work</h4></div></div></div>
                    214: <p>Now that the package has been extracted, let's see what's inside
                    215: it. The package has a <code class="filename">README.txt</code>, but that only
                    216: says something about mozilla, so it's probably useless for seeing what
                    217: dependencies this package has. But since there is a GNU configure script
                    218: in the package, let's hope that it will complain about everything it
                    219: needs.</p>
                    220: <pre class="programlisting">
                    221: <code class="prompt">$</code> bmake
                    222: =&gt; Required installed package digest&gt;=20010302: digest-20060826 found
                    223: =&gt; Checksum SHA1 OK for nvu-1.0-sources.tar.bz2
                    224: =&gt; Checksum RMD160 OK for nvu-1.0-sources.tar.bz2
                    225: ===&gt; Patching for nvu-1.0
                    226: ===&gt; Creating toolchain wrappers for nvu-1.0
                    227: ===&gt; Configuring for nvu-1.0
                    228: [...]
                    229: configure: error: Perl 5.004 or higher is required.
                    230: [...]
                    231: WARNING: Please add USE_TOOLS+=perl to the package Makefile.
                    232: [...]
                    233: </pre>
                    234: <p>That worked quite well. So I opened the package Makefile in my
                    235: editor, and since it already has a <code class="varname">USE_TOOLS</code> line, I
                    236: just appended &#8220;<span class="quote">perl</span>&#8221; to it. Since the dependencies of the
                    237: package have changed now, and since a perl wrapper is automatically
                    238: installed in the &#8220;<span class="quote">tools</span>&#8221; phase, I need to build the package
                    239: from scratch.</p>
                    240: <pre class="programlisting">
                    241: <code class="prompt">$</code> bmake clean
                    242: ===&gt; Cleaning for nvu-1.0
                    243: <code class="prompt">$</code> bmake
                    244: [...]
                    245: *** /tmp/roland/pkgsrc/www/nvu/work.bacc/.tools/bin/make is not \
                    246: GNU Make.  You will not be able to build Mozilla without GNU Make.
                    247: [...]
                    248: </pre>
                    249: <p>So I added &#8220;<span class="quote">gmake</span>&#8221; to the
                    250: <code class="varname">USE_TOOLS</code> line and tried again (from scratch).</p>
                    251: <pre class="programlisting">
                    252: [...]
                    253: checking for GTK - version &gt;= 1.2.0... no
                    254: *** Could not run GTK test program, checking why...
                    255: [...]
                    256: </pre>
                    257: <p>Now to the other dependencies. The first question is: Where is the
                    258: GTK package hidden in pkgsrc?</p>
                    259: <pre class="programlisting">
                    260: <code class="prompt">$</code> echo ../../*/gtk*
                    261: [many packages ...]
                    262: <code class="prompt">$</code> echo ../../*/gtk
                    263: ../../x11/gtk
                    264: <code class="prompt">$</code> echo ../../*/gtk2
                    265: ../../x11/gtk2
                    266: <code class="prompt">$</code> echo ../../*/gtk2/bui*
                    267: ../../x11/gtk2/buildlink3.mk
                    268: </pre>
                    269: <p>The first try was definitely too broad. The second one had exactly
                    270: one result, which is very good. But there is one pitfall with GNOME
                    271: packages. Before GNOME 2 had been released, there were already many
                    272: GNOME 1 packages in pkgsrc. To be able to continue to use these
                    273: packages, the GNOME 2 packages were imported as separate packages, and
                    274: their names usually have a &#8220;<span class="quote">2</span>&#8221; appended. So I checked
                    275: whether this was the case here, and indeed it was.</p>
                    276: <p>Since the GTK2 package has a <code class="filename">buildlink3.mk</code>
                    277: file, adding the dependency is very easy. I just inserted an
                    278: <code class="literal">.include</code> line before the last line of the package
                    279: <code class="filename">Makefile</code>, so that it now looks like this:</p>
                    280: <pre class="programlisting">
                    281: [...]
                    282: .include "../../x11/gtk2/buildlink3.mk"
                    283: .include "../../mk/bsd.pkg.mk
                    284: </pre>
                    285: <p>After another <span><strong class="command">bmake clean &amp;&amp; bmake</strong></span>, the answer
                    286: was:</p>
                    287: <pre class="programlisting">
                    288: [...]
                    289: checking for gtk-config... /home/roland/pkg/bin/gtk-config
                    290: checking for GTK - version &gt;= 1.2.0... no
                    291: *** Could not run GTK test program, checking why...
                    292: *** The test program failed to compile or link. See the file config.log for the
                    293: *** exact error that occured. This usually means GTK was incorrectly installed
                    294: *** or that you have moved GTK since it was installed. In the latter case, you
                    295: *** may want to edit the gtk-config script: /home/roland/pkg/bin/gtk-config
                    296: configure: error: Test for GTK failed.
                    297: [...]
                    298: </pre>
                    299: <p>In this particular case, the assumption that &#8220;<span class="quote">every package
                    300: prefers GNOME 2</span>&#8221; had been wrong. The first of the lines above
                    301: told me that this package really wanted to have the GNOME 1 version of
                    302: GTK. If the package had looked for GTK2, it would have looked for
                    303: <span><strong class="command">pkg-config</strong></span> instead of <span><strong class="command">gtk-config</strong></span>.
                    304: So I changed the <code class="literal">x11/gtk2</code> to
                    305: <code class="literal">x11/gtk</code> in the package <code class="filename">Makefile</code>,
                    306: and tried again.</p>
                    307: <pre class="programlisting">
                    308: [...]
                    309: cc -o xpidl.o -c -DOSTYPE=\"NetBSD3\" -DOSARCH=\"NetBSD\"   -I../../../dist/include/xpcom -I../../../dist/include -I/tmp/roland/pkgsrc/www/nvu/work.bacc/mozilla/dist/include/nspr -I/usr/X11R6/include   -fPIC -DPIC -I/home/roland/pkg/include -I/usr/include  -I/usr/X11R6/include -Wall -W -Wno-unused -Wpointer-arith -Wcast-align -Wno-long-long -pedantic -O2 -I/home/roland/pkg/include -I/usr/include -Dunix -pthread -pipe  -DDEBUG -D_DEBUG -DDEBUG_roland -DTRACING -g -I/home/roland/pkg/include/glib/glib-1.2 -I/home/roland/pkg/lib/glib/include -I/usr/pkg/include/orbit-1.0   -I/home/roland/pkg/include -I/usr/include  -I/usr/X11R6/include -include ../../../mozilla-config.h -DMOZILLA_CLIENT -Wp,-MD,.deps/xpidl.pp xpidl.c
                    310: In file included from xpidl.c:42:
                    311: xpidl.h:53:24: libIDL/IDL.h: No such file or directory
                    312: In file included from xpidl.c:42:
                    313: xpidl.h:132: error: parse error before "IDL_ns"
                    314: [...]
                    315: </pre>
                    316: <p>The package still does not find all of its dependencies. Now the
                    317: question is: Which package provides the
                    318: <code class="filename">libIDL/IDL.h</code> header file?</p>
                    319: <pre class="programlisting">
                    320: <code class="prompt">$</code> echo ../../*/*idl*
                    321: ../../devel/py-idle ../../wip/idled ../../x11/acidlaunch
                    322: <code class="prompt">$</code> echo ../../*/*IDL*
                    323: ../../net/libIDL
                    324: </pre>
                    325: <p>Let's take the one from the second try. So I included the
                    326: <code class="filename">../../net/libIDL/buildlink3.mk</code> file and tried
                    327: again. But the error didn't change. After digging through some of the
                    328: code, I concluded that the build process of the package was broken and
                    329: couldn't have ever worked, but since the Mozilla source tree is quite
                    330: large, I didn't want to fix it. So I added the following to the package
                    331: <code class="filename">Makefile</code> and tried again:</p>
                    332: <pre class="programlisting">
                    333: CPPFLAGS+=              -I${BUILDLINK_PREFIX.libIDL}/include/libIDL-2.0
                    334: BUILDLINK_TRANSFORM+=   -l:IDL:IDL-2
                    335: </pre>
                    336: <p>The latter line is needed because the package expects the library
                    337: <code class="filename">libIDL.so</code>, but only
                    338: <code class="filename">libIDL-2.so</code> is available. So I told the compiler
                    339: wrapper to rewrite that on the fly.</p>
                    340: <p>The next problem was related to a recent change of the FreeType
                    341: interface. I looked up in <a href="ftp://ftp.NetBSD.org/pub/pkgsrc/current/pkgsrc/www/seamonkey/README.html" target="_top"><code class="filename">www/seamonkey</code></a>
                    342: which patch files were relevant for this issue and copied them to the
                    343: <code class="filename">patches</code> directory. Then I retried, fixed the
                    344: patches so that they applied cleanly and retried again. This time,
                    345: everything worked.</p>
                    346: </div>
                    347: <div class="sect3" lang="en">
                    348: <div class="titlepage"><div><div><h4 class="title">
                    349: <a name="creating.nvu.inst"></a>9.2.1.3. Installing the package</h4></div></div></div>
                    350: <pre class="programlisting">
                    351: <code class="prompt">$</code> bmake CHECK_FILES=no install
                    352: [...]
                    353: <code class="prompt">$</code> bmake print-PLIST &gt;PLIST
                    354: <code class="prompt">$</code> bmake deinstall
                    355: <code class="prompt">$</code> bmake install
                    356: </pre>
                    357: </div>
                    358: </div>
                    359: </div>
                    360: </div>
                    361: <div class="navfooter">
                    362: <hr>
                    363: <table width="100%" summary="Navigation footer">
                    364: <tr>
                    365: <td width="40%" align="left">
                    366: <a accesskey="p" href="developers-guide.html">Prev</a> </td>
                    367: <td width="20%" align="center"><a accesskey="u" href="developers-guide.html">Up</a></td>
                    368: <td width="40%" align="right"> <a accesskey="n" href="components.html">Next</a>
                    369: </td>
                    370: </tr>
                    371: <tr>
                    372: <td width="40%" align="left" valign="top">Part II. The pkgsrc developer's guide </td>
                    373: <td width="20%" align="center"><a accesskey="h" href="index.html">Home</a></td>
                    374: <td width="40%" align="right" valign="top"> Chapter 10. Package components - files, directories and contents</td>
                    375: </tr>
                    376: </table>
                    377: </div>
                    378: </body>
                    379: </html>

CVSweb <webmaster@jp.NetBSD.org>