[BACK]Return to mirror.html CVS log [TXT][DIR] Up to [cvs.NetBSD.org] / htdocs / docs

File: [cvs.NetBSD.org] / htdocs / docs / mirror.html (download) (as text)

Revision 1.79, Mon May 17 12:47:46 2021 UTC (5 months ago) by martin
Branch: MAIN
CVS Tags: HEAD
Changes since 1.78: +1 -1 lines

Regen for 9.2

<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
<meta name="generator" content="Website XSL Stylesheet V2.6.0">
<link rel="home" href="../." title="Welcome to The NetBSD Project: Of course it runs NetBSD.">
<link rel="up" href="../docs/." title="NetBSD Documentation">
<link rel="previous" href="../docs/kernel/." title="NetBSD Documentation: Kernel">
<link rel="next" href="../docs/misc/." title="NetBSD Documentation: Other FAQs and HOWTOs">
<link rel="first" href="../docs/Hardware/." title="Hardware Documentation">
<link rel="last" href="../docs/x/." title="NetBSD Documentation: The X Window System">
<link rel="stylesheet" href="../global.css" type="text/css">
<link rel="stylesheet" href="../donations/thermo/fundraiser.css" type="text/css">

    

    

    <title>NetBSD Mirror FAQ</title>
  </head>
<meta name="viewport" content="width=device-width, initial-scale=1.0, minimum-scale=1.0">
<body class="website"><div class="webpage">
<a name="docs-mirror"></a><div id="top"><a href="#mainContent" id="skiplink" tabindex="1">Skip to main content.</a></div>
<input id="hamburger" type="checkbox"><label class="menuicon" for="hamburger"><span></span><span></span><span></span></label><div id="navBar" role="navigation">
<div id="centralHeader"><div id="logo">
<a href="../"><img id="projectLogo" alt="" height="120" src="../images/NetBSD-smaller-tb.png"></a><a href="/"><div id="fundraiser">
<br><div id="fundraiser-amount"><div id="fundraiser-raised"></div></div>
</div></a>
</div></div>
<span class="doNotDisplay">
	  Navigation:
	</span><ul>
<li>
<a href="../">
	  Home</a><ul>
<li><a href="../changes/">
	    Recent changes</a></li>
<li><a href="//blog.NetBSD.org/">
	    NetBSD blog</a></li>
<li><a href="../gallery/presentations/">
	    Presentations</a></li>
</ul>
</li>
<li>
<a href="../about/">
	  About</a><ul>
<li><a href="../people/developers.html">
	    Developers</a></li>
<li><a href="../gallery/">
	    Gallery</a></li>
<li><a href="//wiki.NetBSD.org/ports/">
	    Ports</a></li>
<li><a href="//www.pkgsrc.org/">
	    Packages</a></li>
</ul>
</li>
<li>
<a href="../docs/">
	  Documentation</a><ul>
<li><a href="../docs/misc/index.html">
	    FAQ &amp; HOWTOs</a></li>
<li><a href="../docs/guide/en/">
	    The Guide</a></li>
<li><a href="//man.NetBSD.org/">
	    Manual pages</a></li>
<li><a href="//wiki.NetBSD.org/">
	    Wiki</a></li>
</ul>
</li>
<li>
<a href="../support/">
	  Support</a><ul>
<li><a href="/community/">
	    Community</a></li>
<li><a href="/mailinglists/">
	    Mailing lists</a></li>
<li><a href="../support/send-pr.html">
	    Bug reports</a></li>
<li><a href="../support/security/">
	    Security</a></li>
</ul>
</li>
<li>
<a href="../developers/">
	  Developers</a><ul>
<li><a href="http://cvsweb.NetBSD.org/">
	    CVSWeb</a></li>
<li><a href="//anonhg.NetBSD.org/">
	    Mercurial</a></li>
<li><a href="//nxr.NetBSD.org/">
	    Cross-reference</a></li>
<li><a href="//releng.NetBSD.org/">
	    Release engineering</a></li>
<li><a href="//wiki.NetBSD.org/projects/">
	    Projects list</a></li>
</ul>
</li>
</ul>
</div>
<div id="content"><div id="mainContent" class="fullWidth"><div class="rowOfBoxes">
<h1>NetBSD Mirror FAQ</h1>
<h3 class="title"><a name="ftp-mirror">FTP and ISO Mirror FAQ</a></h3>
<ul>
<li><a href="#ftp-overview">Overview for setting up a NetBSD FTP/ISO Mirror</a></li>
<li><a href="#ftp-diskspace">How much disk space is needed for an FTP/ISO mirror?</a></li>
<li><a href="#ftp-bandwidth">How much bandwidth does offering an FTP/ISO mirror require?</a></li>
<li><a href="#ftp-retrieve-sync">Retrieving/Syncing a copy of the NetBSD FTP site
	  and ISO images area</a></li>
<li><a href="#ftp-notes">Notes about a NetBSD FTP/ISO mirror</a></li>
<li><a href="#ftp-outage">Where should an outage be announced</a></li>
<li><a href="#ftp-contactinfo">Contact information format for FTP/ISO mirrors</a></li>
<li><a href="#ftp-cname">How to get (ftp|iso).country-code.NetBSD.org</a></li>
<li><a href="#ftp-teardown">If you can no longer provide a FTP/ISO mirror</a></li>
</ul>
<h3 class="title"><a name="bitorrent">BitTorrent Mirror FAQ</a></h3>
<ul>
<li><a href="#bittorrent-what">Bit-what?</a></li>
<li><a href="#bittorrent-overview">Overview for setting up a NetBSD Bittorrent Mirror</a></li>
<li><a href="#bittorrent-contactinfo">Contact information format for BitTorrent mirrors</a></li>
</ul>
<h3 class="title"><a name="www">WWW Mirror FAQ</a></h3>
<ul>
<li><a href="#www-overview">Overview for setting up a NetBSD WWW Mirror</a></li>
<li><a href="#www-diskspace">How much disk space is needed for a WWW mirror?</a></li>
<li><a href="#www-retrieve">Retrieving a copy of the NetBSD WWW site</a></li>
<li><a href="#www-sync">Syncing the NetBSD WWW site contents</a></li>
<li><a href="#www-fetch">What is the purpose of htdocs/mirrors/fetch.html</a></li>
<li><a href="#www-http-1.0">Ensuring HTTP/1.0 clients get the right page</a></li>
<li><a href="#www-contactinfo">Contact information format for WWW mirrors</a></li>
<li><a href="#www-cname">How to get www.country-code.NetBSD.org</a></li>
<li><a href="#www-teardown">If you can no longer provide a WWW mirror</a></li>
</ul>
<h3 class="title"><a name="cvsweb">CVSweb Mirror FAQ</a></h3>
<ul>
<li><a href="#cvsweb-overview">Overview for setting up a NetBSD CVSweb Mirror</a></li>
<li><a href="#cvsweb-diskspace">How much disk space is needed for a CVSweb mirror?</a></li>
<li><a href="#cvsweb-retrieve">Retrieving a copy of the NetBSD CVS tree</a></li>
<li><a href="#cvsweb-sync">Syncing the NetBSD CVS tree contents</a></li>
<li><a href="#cvsweb-cgi">How to setup the cvsweb CGI script</a></li>
<li><a href="#cvsweb-http-1.0">Ensuring HTTP/1.0 clients get the right page</a></li>
<li><a href="#cvsweb-contactinfo">Contact information format for CVSweb mirrors</a></li>
<li><a href="#cvsweb-cname">How to get cvsweb.country-code.NetBSD.org</a></li>
<li><a href="#cvsweb-teardown">If you can no longer provide a CVSweb mirror</a></li>
</ul>
<hr>
<h3 class="title">FTP and ISO Mirror FAQ</h3>
	<h4 class="title">
<a name="ftp-overview"></a>Overview for setting up a NetBSD FTP/ISO Mirror</h4>

	<p>The basic steps for setting up a NetBSD FTP or ISO Mirror
	  are:</p>

	<div class="orderedlist"><ol class="orderedlist" type="1">
<li class="listitem">Prepare disk space.  See "<a class="ulink" href="#ftp-diskspace" target="_top">How much disk space is needed
	      for an FTP/ISO mirror</a>"
	    entry below for information on amount of space
	    required.</li>
<li class="listitem">Retrieve a copy of the NetBSD FTP site
	    contents</li>
<li class="listitem">Set things up to synchronize the contents of your
	    mirror with <code class="code">ftp.NetBSD.org</code>.</li>
<li class="listitem">Send in the information about your mirror to
	    <code class="email">&lt;<a class="email" href="mailto:mirrors@NetBSD.org">mirrors@NetBSD.org</a>&gt;</code>. See "<a class="ulink" href="#ftp-contactinfo" target="_top">Contact information format for
	      FTP/ISO Mirrors</a>"
	    for details on that.</li>
</ol></div>
      
	<h4 class="title">
<a name="ftp-diskspace"></a>How much disk space is needed for an FTP/ISO mirror?</h4>

	<p>The amount of disk space needed fluctuates over time.
	  At this time (July 2018) a complete mirror of the supported
	  NetBSD releases needs approximately 250 GB (and an additional
          520 GB for unsupported historical releases). The daily builds of
          NetBSD amount to 1.3 TB and the whole of pkgsrc (source and binary
          packages for several branches) to 1.3 TB.</p>

	<p>Space requirements for ISO files are dependent on
	  how many binary releases are supported at the time. The
	  size may be severely decreased when old branches become
	  obsolete and will grow when new versions of NetBSD are
	  released. Each release takes from 20 to 30 GB, and
	  currently (July 2018) the complete iso area needs around
	  225 GB space.</p>
      
	<h4 class="title">
<a name="ftp-bandwidth"></a>How much bandwidth does offering an FTP/ISO mirror require?</h4>

	<p>At the moment, we do not have any detailed numbers on bandwidth
          usage from our mirrors.  The actual usage of any mirror depends too
          heavily on a variety of factors such as already available bandwidth,
          geographical location relative to other mirrors and amount of data
          mirrored for us to provide you with accurate numbers.</p>

        <p>If you have been running a full mirror for a while and have
          collected bandwidth statistics, please <a class="ulink" href="http://www.NetBSD.org/cgi-bin/feedback.cgi" target="_top">let us
          know</a>.</p>
      
	<h4 class="title">
<a name="ftp-retrieve-sync"></a>Retrieving/Syncing a copy of the NetBSD FTP site
	  and ISO images area</h4>

	<p>There are several ways that you can retrieve and sync
	  the files. Depending on your personal preference, your
	  network connection, the packages installed on your systems
	  etc. you may choose one over the other. Either way, it's
	  probably a good idea to first run the relevant command by
	  hand and then, when you have confirmed that everything is
	  working as it should, create a script to be run by
	  <a href="//man.NetBSD.org/NetBSD-9.2/i386/cron.8">cron(8)</a>.</p>

	<p>All ISO images of the NetBSD distributions are stored on
	  the FTP site inside <code class="filename">images/</code> subdirectory.
	  Thus, mirroring of ISO images is very similar to mirroring
	  the entire FTP site.</p>

	<p>Below you can find some examples of how to keep your
	  local copy in sync. Simply replace "&lt;FTPROOT&gt;" with
	  the actual root of your ftp server.</p>

	<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; ">
<li class="listitem">
<p>Using <a href="https://cdn.NetBSD.org/pub/pkgsrc/current/pkgsrc/net/rsync/index.html" target="_top"><code class="filename">net/rsync</code></a>:
	    </p>
<pre class="programlisting">$ rsync -v -Wartz --delete rsync://rsync.NetBSD.org/NetBSD &lt;FTPROOT&gt;/pub/NetBSD/</pre>
<p>
	    For ISO files only:
	    </p>
<pre class="programlisting">$ rsync -v -Wartz --delete rsync://rsync.NetBSD.org/NetBSD/images &lt;FTPROOT&gt;/pub/NetBSD/images</pre>
</li>
<li class="listitem">
<p>Using <a href="https://cdn.NetBSD.org/pub/pkgsrc/current/pkgsrc/net/wget/index.html" target="_top"><code class="filename">net/wget</code></a>:
	    </p>
<pre class="programlisting">$ cd &lt;FTPROOT&gt;
$ wget -v -xnH -m ftp://ftp.NetBSD.org/pub/NetBSD/</pre>
<p>
	    For ISO files only:
	    </p>
<pre class="programlisting">$ cd &lt;FTPROOT&gt;
$ wget -v -xnH -m ftp://ftp.NetBSD.org/pub/NetBSD/images/</pre>
</li>
<li class="listitem">
<p>Using <a href="https://cdn.NetBSD.org/pub/pkgsrc/current/pkgsrc/net/mirror/index.html" target="_top"><code class="filename">net/mirror</code></a>:
	    </p>
<pre class="programlisting">$ cat &gt; ftp.NetBSD.org &lt;&lt;EOF
package=netbsd
comment=NetBSD
site=ftp.NetBSD.org
remote_dir=/pub/NetBSD
local_dir=&lt;FTPROOT&gt;/pub/NetBSD
do_deletes=true
algorithm=1
EOF
$ mirror ftp.NetBSD.org</pre>
<p>
	    For ISO files only:
	    </p>
<pre class="programlisting">$ cat &gt; iso.NetBSD.org &lt;&lt;EOF
package=netbsd
comment=NetBSD
site=iso.NetBSD.org
remote_dir=/pub/NetBSD/mages
local_dir=&lt;FTPROOT&gt;/pub/NetBSD/images
do_deletes=true
algorithm=1
EOF
$ mirror iso.NetBSD.org</pre>
</li>
</ul></div>
      
	<h4 class="title">
<a name="ftp-notes"></a>Notes about a NetBSD FTP/ISO mirror</h4>

	<p>As a NetBSD FTP/ISO mirror, it's important that you
	  synchronize your copy of the NetBSD FTP tree and ISO
	  images area regularly. Otherwise, old and/or obsolete
	  files may confuse users.</p>

	<p>Also, it might happen that due to, e.g., a security
	  issue, file removal (or another administrative task) may be
	  requested from the NetBSD Project.</p>
      
	<h4 class="title">
<a name="ftp-outage"></a>Where should an outage be announced</h4>

	<p>If your FTP/ISO mirror experiences any significant system
	  outage, you should notify <code class="email">&lt;<a class="email" href="mailto:mirrors@NetBSD.org">mirrors@NetBSD.org</a>&gt;</code>
	  as soon as possible so that we can temporarily redirect your
	  CNAME (if there is one) to a nearby alternative. Also, you
	  may wish to send a note about the outage to the <a class="ulink" href="../mailinglists/#netbsd-users" target="_top">netbsd-users mailing list</a>.</p>

	<p>If the CNAME for your mirror has been redirected, be
	  sure to let us know when the situation is rectified, so we
	  can revert the change.</p>
      
	<h4 class="title">
<a name="ftp-contactinfo"></a>Contact information format for FTP/ISO mirrors</h4>

	<p>Contact information needs to be submitted to
	  <code class="email">&lt;<a class="email" href="mailto:mirrors@NetBSD.org">mirrors@NetBSD.org</a>&gt;</code>.</p>

	<p>This information gets added to an internal mirrors
	  database, from which the mirror-related documentation gets
	  generated. The "<code class="code">contact*</code>" fields are never made
	  public. Contact information for NetBSD FTP/ISO mirrors should be
	  sent in the following format:</p>

	<pre class="programlisting">mirror          *machine name*
country         *ISO 3166 country code of host, see /usr/share/zoneinfo/iso3166.tab*
location        *more specific location (e.g, ``University of Foo'')*
ftp		*url to NetBSD ftp mirror area*
ftp-host	*fqdn of the host providing ftp mirror service*
ftp-releng	*url to NetBSD release engineering files, aka NetBSD-daily*
iso		*url to NetBSD iso images area*
iso-host	*fqdn of the host providing iso images mirror service*
organisation    *organisation name*
contactperson   *contact person's name*
contactemail    *contact person's electronic mail address*
contactkeyid	*contact person's pgp key id*
contactphone    *contact person's telephone number*
contactaddress  *contact person's postal address*</pre>

	<p>An example submission might look like this.</p>
	<pre class="programlisting">mirror          netbsd.foo.com
country         jp
location        Department of Computer Science, Foo University
ftp             ftp://ftp.foo-univ.ac.jp/pub/NetBSD/
ftp-host        ftp.foo-univ.ac.jp
iso		ftp://iso.foo-univ.ac.jp/pub/NetBSD/images/
iso-host	iso.foo-univ.ac.jp
organisation    Foo University
contactperson   Foo Bar
contactemail    Bar@foo-univ.ac.jp
contactkeyid	GA23FG12
contactphone    +81 3 xxxx yyyy
contactaddress  9876 Foo City, Tokyo 1234567 Japan</pre>

	<p>Please use <code class="literal">ftp*</code> and
	  <code class="literal">iso*</code> fileds in case only if you
	  providing appropriate mirror services.  For example, omit
	  <code class="literal">ftp-releng</code> field if there are no releng
	  files on your server, or don't use any of <code class="literal">ftp*</code>
	  fields if you running ISO mirror only.</p>

	<p>If the contact information for your mirror ever needs to
	  be changed, please feel free to let us know at
	  <code class="email">&lt;<a class="email" href="mailto:mirrors@NetBSD.org">mirrors@NetBSD.org</a>&gt;</code>.</p>
      
	<h4 class="title">
<a name="ftp-cname"></a>How to get (ftp|iso).country-code.NetBSD.org</h4>

	<p>We have temporarily stopped assigning names in the
	  .NetBSD.org domain to mirrors as we are re-evaluating our
	  mirror policies.</p>
      
	<h4 class="title">
<a name="ftp-teardown"></a>If you can no longer provide a FTP/ISO mirror</h4>

	<p>Please let us know at <code class="email">&lt;<a class="email" href="mailto:mirrors@NetBSD.org">mirrors@NetBSD.org</a>&gt;</code>
	  so that we can remove your site from our list and redirect
	  the DNS name to an alternative site.</p>
      <hr>
<h3 class="title">BitTorrent Mirror FAQ</h3>
	<h4 class="title">
<a name="bittorrent-what"></a>Bit-what?</h4>

	<p><a class="ulink" href="http://bitconjurer.org/BitTorrent/" target="_top">BitTorrent</a>
	  is a tool for distributing files. It's extremely easy to
	  use - downloads are started by clicking on hyperlinks.
	  Whenever more than one person is downloading at once
	  they send pieces of the file(s) to each other, thus
	  relieving the central server's bandwidth burden.  Even with
	  many simultaneous downloads, the upload burden on the
	  central server remains quite small, since each new
	  downloader introduces new upload capacity.</p>
      
	<h4 class="title">
<a name="bittorrent-overview"></a>Overview for setting up a NetBSD Bittorrent Mirror</h4>

	<p>The basic steps for setting up a NetBSD BitTorrent
	  Mirror are:</p>

	<div class="orderedlist"><ol class="orderedlist" type="1">
<li class="listitem">Install <a href="https://cdn.NetBSD.org/pub/pkgsrc/current/pkgsrc/net/bittorrent/index.html" target="_top"><code class="filename">net/bittorrent</code></a>
</li>
<li class="listitem">Fetch the <code class="filename">.torrent</code> files from
	    our FTP site. The files are in the
	    <code class="filename">/pub/NetBSD/images/&lt;release&gt;</code>
	    subdirectory, so if you already are mirroring this
	    directory, you're all set. For more information see <a class="ulink" href="../mirrors/torrents/" target="_top">this page</a>.</li>
<li class="listitem">
<p>Start BitTorrent:
	    </p>
<pre class="programlisting">cd /ftp/pub/NetBSD/images/&lt;release&gt;; (btlaunchmany.py .) &gt;/dev/null 2&gt;/dev/null &amp;</pre>
<p>
	    If you didn't download the ISO images already, they will be
	    downloaded now. You can use the
	    <code class="code">--max_upload_rate arg</code>
	    and <code class="code">--max_download_rate arg</code>
	    options to limit the transfer rates.</p>
</li>
</ol></div>
      
	<h4 class="title">
<a name="bittorrent-contactinfo"></a>Contact information format for BitTorrent mirrors</h4>

	<p>Contact information needs to be submitted to
	  <code class="email">&lt;<a class="email" href="mailto:mirrors@NetBSD.org">mirrors@NetBSD.org</a>&gt;</code>.
	  This information gets added to an internal mirrors database,
	  from which the mirror-related documentation gets generated.
	  The "<code class="code">contact*</code>" fields are never made public.
	  Contact information for NetBSD BitTorrent mirrors should be
	  sent in the following format:</p>

	<pre class="programlisting">mirror          *machine name*
country         *ISO 3166 country code of host, see /usr/share/zoneinfo/iso3166.tab*
location        *more specific location (e.g, ``University of Foo'')*
bittorrent	*BitTorrent URL / URL with additional Information*
ipv6		*yes or no, if your server supports IPv6*
organisation    *organisation name*
contactperson   *contact person's name*
contactemail    *contact person's electronic mail address*
contactkeyid	*contact person's pgp key id*
contactphone    *contact person's telephone number*
contactaddress  *contact person's postal address*</pre>

	<p>An example submission might look like this.</p>

	<pre class="programlisting">mirror          ftp.NetBSD.org
country         us
location        Silicon Valley, California
bittorrent	http://www.NetBSD.org/mirrors/torrents/
ipv6            yes
organisation    The NetBSD Foundation, Inc.
contactperson   Admins
contactemail    &lt;email-here&gt;@NetBSD.org
contactkeyid	AF226A4C
contactphone    555-123-1234
contactaddress 	Somwhere in Silicon Valley, California</pre>
      <hr>
<h3 class="title">WWW Mirror FAQ</h3>
	<h4 class="title">
<a name="www-overview"></a>Overview for setting up a NetBSD WWW Mirror</h4>

	<p>The following is an overview of the recommended
	  procedure for setting up a NetBSD WWW mirror:</p>

	<div class="orderedlist"><ol class="orderedlist" type="1">
<li class="listitem">For setting up a mirror with this procedure,
	    you'll need cvs, ssh, and wget
            (see <a href="https://cdn.NetBSD.org/pub/pkgsrc/current/pkgsrc/net/wget/index.html" target="_top"><code class="filename">net/wget</code></a>).  Of course you'll
            also need a web server, such as apache.</li>
<li class="listitem">Retrieve a copy of the NetBSD WWW site.  See "<a class="ulink" href="#www-retrieve" target="_top">Retrieving a copy of the NetBSD WWW site</a>"
	    for instructions on how to do so with anoncvs.</li>
<li class="listitem">Set things up to synchronize the contents of your
	    mirror with <code class="code">www.NetBSD.org</code>.
	    See "<a class="ulink" href="#www-sync" target="_top">Syncing the NetBSD WWW site contents</a>"
	    for an example of how to do so using a cron job.</li>
<li class="listitem">Send in the information about your mirror to
	    <code class="email">&lt;<a class="email" href="mailto:mirrors@NetBSD.org">mirrors@NetBSD.org</a>&gt;</code>. See
	    "<a class="ulink" href="#www-contactinfo" target="_top">Contact information format for WWW Mirrors</a>"
	    for details on that.</li>
</ol></div>
      
	<h4 class="title">
<a name="www-diskspace"></a>How much disk space is needed for a WWW mirror?</h4>

	<p>The amount of diskspace needed for a complete copy of
	  the htdocs repository is currently (July 2018) around 310 MB.
	  The total amount should not vary too much, but will
	  obviously grow with time.</p>
      
	<h4 class="title">
<a name="www-retrieve"></a>Retrieving a copy of the NetBSD WWW site</h4>

	<p>To retrieve a copy of <code class="code">www.NetBSD.org</code> via
	  anoncvs, you should do:
	  </p>
<pre class="programlisting"># su - nbwww
# cd /path/to/htdocs
# env CVS_RSH=ssh CVSROOT=anoncvs@anoncvs.NetBSD.org:/cvsroot \
  cvs checkout -P htdocs</pre>
<p>

	  <span class="bold"><strong>Note:</strong></span>
	  "<code class="filename">/path/to/htdocs</code>"
	  should be where you want the mirror of
	  <code class="code">www.NetBSD.org</code> to reside.  The directory
	  should be owned by an e.g. "nbwww" user.</p>
      
	<h4 class="title">
<a name="www-sync"></a>Syncing the NetBSD WWW site contents</h4>

	<p>The best way to do this is to set up a cron job for
	  your e.g. nbwww user to update the web site at a regular
	  interval using anoncvs and wget (more on the wget usage
	  below):
	  </p>
<pre class="programlisting"># crontab -u nbwww -e
SHELL=/bin/sh
PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/pkg/bin
MAILTO=whoever@gets.the.mirror.mail
CVS_RSH=ssh
CVSROOT=anoncvs@anoncvs.NetBSD.org:/cvsroot
0 1 * * * cd /path/to/htdocs &amp;&amp; cvs update -dP &amp;&amp; wget -B http://www.NetBSD.org/ -nv -xnH -N -Fi mirrors/fetch.html</pre>
<p>
	  <span class="bold"><strong>Note:</strong></span> In the suggested
	  contents, you'll want to change "MAILTO" to reflect who
	  should get the mirror-related mail.
	  Also, "<code class="filename">/path/to/htdocs</code>" should be
	  changed to where your <code class="code">www.NetBSD.org</code> mirror
	  resides.</p>
      
	<h4 class="title">
<a name="www-fetch"></a>What is the purpose of <code class="filename">htdocs/mirrors/fetch.html</code>
</h4>

	<p>The <code class="filename">fetch.html</code> file links to the
	  handful of documents on <code class="code">www.NetBSD.org</code> that
	  are generated <span class="emphasis"><em>by</em></span>
	  <code class="code">www.NetBSD.org</code>.  These files get generated
	  daily from content in the <code class="code">doc</code> module of the
	  NetBSD CVS repository.</p>

	<p>In the example cron tab above, you'll see
	  "<code class="code">wget -nv -xnH -NFi mirrors/fetch.html</code>".
	  This means that daily, through wget, these
	  <code class="code">www.NetBSD.org</code> created files
	  will be updated in your copy of the web site
	  contents.</p>
      
	<h4 class="title">
<a name="www-http-1.0"></a>Ensuring HTTP/1.0 clients get the right page</h4>

	<p>It is much preferred that NetBSD WWW mirror sites run
	  a separate virtual web server for a
	  <code class="code">www.NetBSD.org</code> mirror on a separate IP
	  address.  This is so that clients using HTTP/1.0 still
	  get to the correct web site.</p>

	<p>If that's not possible in your set up, then the other
	  solution for HTTP/1.0 clients would be putting the NetBSD
	  mirror as the primary name-based virtual host.</p>
      
	<h4 class="title">
<a name="www-contactinfo"></a>Contact information format for WWW mirrors</h4>

	<p>Contact information needs to be submitted to
	  <code class="email">&lt;<a class="email" href="mailto:mirrors@NetBSD.org">mirrors@NetBSD.org</a>&gt;</code>.
	  This information gets added to an internal mirrors
	  database, from which the mirror-related documentation gets
	  generated.  The "<code class="code">contact*</code>" fields are never
	  made public.  Contact information for NetBSD WWW mirrors
	  should be sent in the following format:</p>
	<pre class="programlisting">mirror          *machine name*
country         *ISO 3166 country code of host, see /usr/share/zoneinfo/iso3166.tab*
location        *more specific location (e.g, ``University of Foo'')*
www             *WWW URL*
www-host        *external WWW name*
organisation    *organisation name*
contactperson   *contact person's name*
contactemail    *contact person's electronic mail address*
contactkeyid	*contact person's pgp key id*
contactphone    *contact person's telephone number*
contactaddress  *contact person's postal address*</pre>

	<p>An example submission might look like this.</p>

	<pre class="programlisting">mirror          netbsd.foo-univ.ac.jp
country         jp
location        Department of Computer Science, Foo University
www             http://netbsd.foo-univ.ac.jp/
www-host        netbsd.foo-univ.ac.jp
organisation    Foo University
contactperson   Foo Bar
contactemail    Bar@foo-univ.ac.jp
contactkeyid	GA23FG12
contactphone    +81 3 xxxx yyyy
contactaddress  9876 Foo City, Tokyo 1234567 Japan</pre>

	<p>If the contact information for your mirror ever needs
	  to be changed, please feel free to let us know at
	  <code class="email">&lt;<a class="email" href="mailto:mirrors@NetBSD.org">mirrors@NetBSD.org</a>&gt;</code>.</p>
      
	<h4 class="title">
<a name="www-cname"></a>How to get www.country-code.NetBSD.org</h4>

	<p>We have temporarily stopped assigning names in the
	  .NetBSD.org domain to mirrors as we are re-evaluating our
	  mirror policies.</p>
      
	<h4 class="title">
<a name="www-teardown"></a>If you can no longer provide a WWW mirror</h4>

	  <p>Please let us know at
	  <code class="email">&lt;<a class="email" href="mailto:mirrors@NetBSD.org">mirrors@NetBSD.org</a>&gt;</code> so that we can remove your
	  site from our list and redirect the DNS name to an
	  alternative site.</p>
      <hr>
<h3 class="title">CVSweb Mirror FAQ</h3>
	<h4 class="title">
<a name="cvsweb-overview"></a>Overview for setting up a NetBSD CVSweb Mirror</h4>

	<p>The following is an overview of the recommended
	  procedure for setting up a NetBSD CVSweb mirror:</p>

	<div class="orderedlist"><ol class="orderedlist" type="1">
<li class="listitem">For setting up a mirror with this procedure,
	    you'll need cvs, cvsup, and the cvsweb CGI script.  Both cvsup
            and the cvsweb CGI script are available from the <a class="ulink" href="software/packages.html" target="_top">NetBSD packages collection</a>
	    (see <a href="https://cdn.NetBSD.org/pub/pkgsrc/current/pkgsrc/devel/cvsup-bin/index.html" target="_top"><code class="filename">devel/cvsup-bin</code></a> and <a href="https://cdn.NetBSD.org/pub/pkgsrc/current/pkgsrc/www/cvsweb/index.html" target="_top"><code class="filename">www/cvsweb</code></a>).  Of course you'll also
	    need a web server, such as <a href="https://cdn.NetBSD.org/pub/pkgsrc/current/pkgsrc/www/apache/index.html" target="_top"><code class="filename">www/apache</code></a>.</li>
<li class="listitem">Retrieve a copy of the NetBSD CVS tree.  See
	    "<a class="ulink" href="#cvsweb-retrieve" target="_top">Retrieving a copy of the NetBSD CVS tree</a>"
	    for instructions on how to do so with cvsup.</li>
<li class="listitem">Set things up to synchronize the contents of your
	    mirror with the cvsup server. See "<a class="ulink" href="#cvsweb-sync" target="_top">Syncing the NetBSD CVS tree contents</a>"
	    for an example of how to do so using a cron job.</li>
<li class="listitem">Send in the information about your mirror to
	    <code class="email">&lt;<a class="email" href="mailto:mirrors@NetBSD.org">mirrors@NetBSD.org</a>&gt;</code>. See  "<a class="ulink" href="#cvsweb-contactinfo" target="_top">Contact information format for CVSweb Mirrors</a>"
	    for details on that.</li>
</ol></div>
      
	<h4 class="title">
<a name="cvsweb-diskspace"></a>How much disk space is needed for a CVSweb mirror?</h4>

	<p>The amount of disk space needed is rather large due to
	  the fact that you are providing an interface to the
	  <span class="emphasis"><em>entire CVS repository</em></span>. Currently
	  (2006-05), the total disk space required for all sources in
	  CVS is around 4.3 GB.</p>
      
	<h4 class="title">
<a name="cvsweb-retrieve"></a>Retrieving a copy of the NetBSD CVS tree</h4>

	<p>To retrieve a copy of the CVS tree via cvsup, you
	  should do:
	  </p>
<pre class="programlisting"># su - cvsweb
# mkdir -p /home/cvsweb/netbsd
# cd /home/cvsweb</pre>
<p>

	  <span class="bold"><strong>Note:</strong></span>
	  "<code class="filename">/home/cvsweb/netbsd</code>"
	  should be where you want the mirror of the tree to reside.
	  The directory should be owned by an e.g. "cvsweb"
	  user. Create an "supfile" using your editor (For examples,
	  see <a class="ulink" href="netbsd-supfile" target="_top">netbsd-supfile</a>).
	  Please take a look at the list of available public cvsup
	  servers, you'll find the list <a class="ulink" href="../mirrors/index.html#cvsup" target="_top">here</a>.
	  </p>
<pre class="programlisting"># cvsup -g -L 2 netbsd-supfile</pre>
<p>
	</p>
      
	<h4 class="title">
<a name="cvsweb-sync"></a>Syncing the NetBSD CVS tree contents</h4>

	<p>The best way to do this is to set up a cron job for your
	  e.g. cvsweb user to update the tree at a regular interval
	  using cvsup:
	  </p>
<pre class="programlisting"># crontab -u cvsweb -e
SHELL=/bin/sh
PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/pkg/bin
MAILTO=whoever@gets.the.mirror.mail
0 1 * * * cvsup -g /home/cvsweb/netbsd-supfile</pre>
<p>

	  <span class="bold"><strong>Note:</strong></span> In the suggested
	  contents, you'll want to change "MAILTO" to reflect who
	  should get the mirror-related mail.</p>
      
	<h4 class="title">
<a name="cvsweb-cgi"></a>How to setup the cvsweb CGI script</h4>

	<p>Take a look at the <code class="filename">cvsweb.conf</code> and
	  be sure to edit the following options:</p>
	  <pre class="programlisting">
@CVSrepositories = (
        'netbsd'  =&gt; ['NetBSD',           '/home/cvsweb/netbsd'],
);

$mancgi = "http://man.NetBSD.org/%s.%s";
</pre>
	  <p>Place cvsweb.cgi in your web servers' cgi-bin
	  directory and be sure to edit it to point to your
	  <code class="filename">cvsweb.conf</code>.</p>
      
	<h4 class="title">
<a name="cvsweb-http-1.0"></a>Ensuring HTTP/1.0 clients get the right page</h4>

	<p>It is much preferred that NetBSD CVSweb mirror sites run
	  a separate virtual web server for a CVSweb mirror on a
	  separate IP address.  This is so that clients using HTTP/1.0
	  still get to the correct web site.</p>

	<p>If that's not possible in your set up, then the other
	  solution for HTTP/1.0 clients would be putting the NetBSD
	  mirror as the primary name-based virtual host.</p>
      
	<h4 class="title">
<a name="cvsweb-contactinfo"></a>Contact information format for CVSweb mirrors</h4>

	<p>Contact information needs to be submitted to
	  <code class="email">&lt;<a class="email" href="mailto:mirrors@NetBSD.org">mirrors@NetBSD.org</a>&gt;</code>.

	  This information gets added to an internal mirrors database, from
	  which the mirror-related documentation gets generated. The
	  "<code class="code">contact*</code>" fields are never made public.
	  Contact information for NetBSD CVSweb mirrors should be sent
	  in the following format:</p>

	<pre class="programlisting">mirror          *machine name*
country         *ISO 3166 country code of host, see /usr/share/zoneinfo/iso3166.tab*
location        *more specific location (e.g, ``University of Foo'')*
cvsweb          *CVSweb URL*
cvsweb-host     *external CVSweb name*
organisation    *organisation name*
contactperson   *contact person's name*
contactemail    *contact person's electronic mail address*
contactkeyid	*contact person's pgp key id*
contactphone    *contact person's telephone number*
contactaddress  *contact person's postal address*</pre>

	<p>An example submission might look like this.</p>

	<pre class="programlisting">mirror          netbsd.foo-univ.ac.jp
country         jp
location        Department of Computer Science, Foo University
cvsweb          http://netbsd.foo-univ.ac.jp/cgi-bin/cvsweb.cgi/
cvsweb-host     cvsweb.netbsd.foo-univ.ac.jp
organisation    Foo University
contactperson   Foo Bar
contactemail    Bar@foo-univ.ac.jp
contactkeyid	GA23FG12
contactphone    +81 3 xxxx yyyy
contactaddress  9876 Foo City, Tokyo 1234567 Japan</pre>

	<p>If the contact information for your mirror ever needs to
	  be changed, please feel free to let us know at
	  <code class="email">&lt;<a class="email" href="mailto:mirrors@NetBSD.org">mirrors@NetBSD.org</a>&gt;</code>.</p>
      
	<h4 class="title">
<a name="cvsweb-cname"></a>How to get cvsweb.country-code.NetBSD.org</h4>

	<p>We have temporarily stopped assigning names in the
	  .NetBSD.org domain to mirrors as we are re-evaluating our
	  mirror policies.</p>
      
	<h4 class="title">
<a name="cvsweb-teardown"></a>If you can no longer provide a CVSweb mirror</h4>

	<p>Please let us know at <code class="email">&lt;<a class="email" href="mailto:mirrors@NetBSD.org">mirrors@NetBSD.org</a>&gt;</code>
	  so that we can remove your site from our list and redirect the
	  DNS name to an alternative site.</p>
      </div></div></div>
<div class="navfoot"></div>
<div id="footer"><div id="footerContent"><center>
<span class="footfeed"><a href="//www.NetBSD.org/cgi-bin/feedback.cgi">
	  Contact</a> |
      </span><span class="footcopy"><a href="../about/disclaimer.html">
      Disclaimer</a> |

      <span class="copyright">Copyright 1994-2021 The NetBSD Foundation, Inc. </span>ALL RIGHTS RESERVED.<br>NetBSD<sup>/sup> is a registered trademark of The NetBSD
	Foundation, Inc.</span>
</center></div></div>
</div></body>
</html>