[BACK]Return to index.html CVS log [TXT][DIR] Up to [cvs.NetBSD.org] / htdocs / docs / puffs

File: [cvs.NetBSD.org] / htdocs / docs / puffs / index.html (download) (as text)

Revision 1.69, Thu Jul 30 12:11:13 2020 UTC (3 years, 7 months ago) by maya
Branch: MAIN
Changes since 1.68: +12 -39 lines

Regen for menu changes

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
<meta name="generator" content="Website XSL Stylesheet V2.6.0">
<link rel="home" href="../../." title="Welcome to The NetBSD Project: Of course it runs NetBSD.">
<link rel="up" href="../../docs/." title="NetBSD Documentation">
<link rel="previous" href="../../docs/power-mgmt/." title="NetBSD Documentation: Power Management">
<link rel="next" href="../../docs/puffs/rump.html" title="Kernel File Systems as Userspace Servers">
<link rel="first" href="../../docs/Hardware/." title="Hardware Documentation">
<link rel="last" href="../../docs/x/." title="NetBSD Documentation: The X Window System">
<link rel="stylesheet" href="../../global.css" type="text/css">
<link rel="stylesheet" href="../../donations/thermo/fundraiser.css" type="text/css">

<title>Filesystems in userspace: puffs, refuse, FUSE, and more</title>
</head>
<meta name="viewport" content="width=device-width, initial-scale=1.0, minimum-scale=1.0">
<body class="website"><div class="webpage">
<a name="docs-puffs-index"></a><div id="top"><a href="#mainContent" id="skiplink" tabindex="1">Skip to main content.</a></div>
<input id="hamburger" type="checkbox"><label class="menuicon" for="hamburger"><span></span><span></span><span></span></label><div id="navBar" role="navigation">
<div id="centralHeader"><div id="logo">
<a href="../../"><img id="projectLogo" alt="" height="120" src="../../images/NetBSD-smaller-tb.png"></a><a href="/"><div id="fundraiser">
<br><div id="fundraiser-amount"><div id="fundraiser-raised"></div></div>
</div></a>
</div></div>
<span class="doNotDisplay">
	  Navigation:
	</span><ul>
<li>
<a href="../../">
	  Home</a><ul>
<li><a href="../../changes/">
	    Recent changes</a></li>
<li><a href="//blog.NetBSD.org/">
	    NetBSD blog</a></li>
<li><a href="../../gallery/presentations/">
	    Presentations</a></li>
</ul>
</li>
<li>
<a href="../../about/">
	  About</a><ul>
<li><a href="../../people/developers.html">
	    Developers</a></li>
<li><a href="../../gallery/">
	    Gallery</a></li>
<li><a href="//wiki.NetBSD.org/ports/">
	    Ports</a></li>
<li><a href="http://pkgsrc.se">
	    Packages</a></li>
</ul>
</li>
<li>
<a href="../../docs/">
	  Documentation</a><ul>
<li><a href="../../docs/misc/index.html">
	    FAQ &amp; HOWTOs</a></li>
<li><a href="../../docs/guide/en/">
	    The Guide</a></li>
<li><a href="https://man.NetBSD.org/">
	    Manual pages</a></li>
<li><a href="//wiki.NetBSD.org/">
	    Wiki</a></li>
</ul>
</li>
<li>
<a href="../../support/">
	  Support</a><ul>
<li><a href="//wiki.NetBSD.org/community/">
	    Community</a></li>
<li><a href="../../support/send-pr.html">
	    Bug reports</a></li>
<li><a href="../../support/security/">
	    Security</a></li>
</ul>
</li>
<li>
<a href="../../developers/">
	  Developers</a><ul>
<li><a href="http://cvsweb.NetBSD.org/">
	    CVSWeb</a></li>
<li><a href="//anonhg.NetBSD.org/">
	    Mercurial</a></li>
<li><a href="//nxr.NetBSD.org/">
	    Cross-reference</a></li>
<li><a href="//releng.NetBSD.org/">
	    Release engineering</a></li>
<li><a href="//wiki.NetBSD.org/projects/">
	    Projects list</a></li>
</ul>
</li>
</ul>
</div>
<div id="content"><div id="mainContent" class="fullWidth"><div class="rowOfBoxes">
<h1>Filesystems in userspace: puffs, refuse, FUSE, and more</h1>
<div class="sect1">
<div class="titlepage"></div>
</div>
<h3 class="title"><a name="puffs"></a></h3>
<ul>
<li><a href="#about">About</a></li>
<li><a href="#components">Components</a></li>
<li><a href="#enabling">Enabling puffs</a></li>
<li><a href="#examples">Base system examples</a></li>
<li><a href="#pkgsrc">pkgsrc &amp; FUSE</a></li>
<li><a href="#moreinfo">Further Information</a></li>
</ul>
<hr>
<h3 class="title"></h3>
    <h4 class="title">
<a name="about"></a>About</h4>
  
    <p>
      NetBSD now offers full support for running file systems in
      userspace. Major components are "puffs", which is the kernel
      subsystem that realizes the  pass-to-userspace framework file
      system, as well as the userland libraries that support
      constructing file system implementations.  These are libpuffs
      and the FUSE compatibility library, librefuse.
    </p>

    <p>
      This page describes the situation in NetBSD 5.0 and later.
    </p>
  
    <h4 class="title">
<a name="components"></a>Components</h4>

    <p>
      There are a number of components interacting to provide routines
      for userland file systems: puffs as the kernel part, and
      libpuffs and librefuse to provide functions for userland file
      systems to call. The following image gives an overview of the
      components and their connections:
    </p>

    <div class="figure">
<a name="puffs-overview"></a><p class="title"><b>Figure. Ïverview about puffs</b></p>
<div class="figure-contents">
      
      <div class="mediaobject"><img src="puffs.png" alt="Overview about puffs"></div>
    </div>
</div>
<br class="figure-break">

    <p>
      The components themselves are:
    </p>

    <div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; ">
<li class="listitem">
        <p>
          <span class="bold"><strong>puffs</strong></span> is the core component
          inside the kernel. It exposes a file 
          system interface towards the userland programs (accessible via
          <code class="filename">/dev/puffs</code> and libpuffs), and
          communicates with the kernel's own idea of files,  
          which are represented as vnodes. 
          The puffs kernel component and how to interface it is described
          further in the <a href="https://netbsd.gw.com/cgi-bin/man-cgi?puffs+4.i386+NetBSD-9.0">puffs(4)</a> manpage. 
        </p>
      </li>
<li class="listitem">
        <p>
          The "<span class="bold"><strong>libpuffs</strong></span>" library is the
          interface between userland 
          file systems and the kernel component. It provides a number of
          convenience routines that userland file systems can use, and is
          described in the <a href="https://netbsd.gw.com/cgi-bin/man-cgi?puffs+3.i386+NetBSD-9.0">puffs(3)</a> manpage. 
        </p>
      </li>
<li class="listitem">
        <p>
          To facilitate running the huge amount of file systems already
          available for the FUSE interface, but not dictate the
          capabilities of puffs by it, it was decided that FUSE support
          should be provided as a compatibility layer on top of the native
          puffs interface, which is offered by "<span class="bold"><strong>librefuse</strong></span>".  The re-implementation of the
          FUSE functionality is designed to be source code compatible with
          FUSE, and it is further described in the <a href="https://netbsd.gw.com/cgi-bin/man-cgi?refuse+3.i386+NetBSD-9.0">refuse(3)</a> manpage.
        </p>
      </li>
</ul></div>
  
    <p>
      Examples for puffs and refuse filesystems can be found in the
      following directory of the NetBSD source tree in
      <code class="filename">src/share/examples/{puffs,refuse}</code>.
    </p>
  
    <h4 class="title">
<a name="enabling"></a>Enabling puffs</h4>

      <p>
	puffs is enabled by default in GENERIC on major architectures
	starting from NetBSD 5.0.  For a custom kernel, the following
	are required in the kernel configuration.
    
      </p>
<pre class="programlisting">
file-system     PUFFS
pseudo-device   putter
      </pre>
<p>

      Alternatively, modules can be used.
    </p>
  
    <h4 class="title">
<a name="examples"></a>Base system examples</h4>
    
    <p>
      This section presents usage examples for file systems shipped with
      the NetBSD base system.
    </p>

    <div class="sect4">
<div class="titlepage"><div><div><h5 class="title">
<a name="example-psshfs"></a>psshfs, The NetBSD sshfs (<a href="https://netbsd.gw.com/cgi-bin/man-cgi?mount_psshfs+8.i386+NetBSD-9.0">mount_psshfs(8)</a>)</h5></div></div></div>
      

      <pre class="programlisting">
# mount_psshfs host.name.tld:/directory /puffs
# ls /puffs
AdobeFnt.lst    OS                  bin     public_html
Desktop         OpenOffice.org1.1.0 in      tmp
...
# cd /puffs
# ls -l .cshrc
-rw-r--r--  1 39068  2000  4706 Jun 16 01:01 .cshrc
# head -2 .cshrc
# Default .cshrc for Solaris, Irix, ...
#
# md5 .cshrc
MD5 (.cshrc) = 2ad1d2606a5678f312709a388376c2e5
# ls -l test
ls: test: No such file or directory
# date &gt;test
# ls -l test
-rw-r--r--  1 39068  2000  29 Nov 23 01:19 test
# cat test
Thu Nov 23 01:19:36 MET 2006
# umount /puffs
      </pre>
    </div>

    <div class="sect4">
<div class="titlepage"><div><div><h5 class="title">
<a name="example-9p"></a>
        <a class="ulink" href="http://www.cs.bell-labs.com/sys/man/5/INDEX.html" target="_top">9P</a>
	file servers (<a href="https://netbsd.gw.com/cgi-bin/man-cgi?mount_9p+8.i386+NetBSD-9.0">mount_9p(8)</a>)
      </h5></div></div></div>
      

      <pre class="programlisting">
# mount_9p nobody@192.168.1.2:/tmp /puffs
# cd /puffs
# echo 9puffs in action &gt; msg_from_earth
# ls -l msg_from_earth
-rw-r--r--  1 nobody  wheel  17 Apr 25 23:24 msg_from_earth
# rsh 192.168.1.2 cat /tmp/msg_from_earth
9puffs in action
# 
</pre>
      <p>
	Since there is currently no support in the implementation for
	access control or support for authentication, the account
	nobody was picked just to prove a point.  The NFS nobody
	user really does not have anything to do with 9P.  Support
	for access control and use of pre-established secure
	connections will be added to <span class="command"><strong>mount_9p</strong></span>
	on a later date.
      </p>
    </div>

  
    <h4 class="title">
<a name="pkgsrc"></a>pkgsrc &amp; FUSE</h4>
    
    <p>
      FUSE compatibility was added within pkgsrc, and besides the
      required infrastructure work a number of FUSE packages were
      added to pkgsrc in the new "filesystem" category. Example
      packages that are currently available include:
    </p>

    <div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; ">
<li class="listitem">
        <a class="ulink" href="ftp://ftp.NetBSD.org/pub/pkgsrc/current/pkgsrc/filesystems/fuse/README.html" target="_top">fuse</a>: Filesystem in Userspace (compat headers, pkg-config files,
         etc.), needed for pkgsrc on Linux 
      </li>
<li class="listitem">
        <a class="ulink" href="ftp://ftp.NetBSD.org/pub/pkgsrc/current/pkgsrc/filesystems/fuse-archivemount/README.html" target="_top">fuse-archivemount</a>:
        FUSE gateway to libarchive
      </li>
<li class="listitem">
        <a class="ulink" href="ftp://ftp.NetBSD.org/pub/pkgsrc/current/pkgsrc/filesystems/fuse-cddfs/README.html" target="_top">fuse-cddfs</a>:
        FUSE filesystem that uses libparanoia for audio CDs
      </li>
<li class="listitem">
        <a class="ulink" href="ftp://ftp.NetBSD.org/pub/pkgsrc/current/pkgsrc/filesystems/fuse-ntfs-3g/README.html" target="_top">fuse-ntfs-3g</a>:
        a NTFS driver with read and write support
      </li>
<li class="listitem">
	  ...
      </li>
</ul></div>

    <p>
      More packages are currently being worked on, please see 
      <a class="ulink" href="ftp://ftp.NetBSD.org/pub/pkgsrc/current/pkgsrc/filesystems/README.html" target="_top">pkgsrc/filesystems</a>
      for a full list.
    </p>

    <p>
      Here is an example of installing and using the FUSE-enabled
      NTFS-3g implementation: 
    </p>

    <pre class="programlisting">
# cd pkgsrc/filesystems/fuse-ntfs-3g
# make install
# ntfs-3g ntfs.img /ntfs
    </pre>

    <p>
      Using FUSE file systems on NetBSD 4.0 is possible, but in
      addition to adding puffs support support to the kernel, it
      requires fetching and manually installing a backport of the
      ReFUSE library.  The library is available
      <a class="ulink" href="http://www.cs.hut.fi/~pooka/NetBSD/librefuse_nb4-20080115.tar.gz" target="_top">here</a>
      and further instructions are available
      <a class="ulink" href="http://mail-index.NetBSD.org/netbsd-users/2008/01/15/msg000075.html" target="_top">here</a>.
    </p>
  
    <h4 class="title">
<a name="moreinfo"></a>Further Information</h4>

    <p>An in-depth technical description of puffs was presented at
      <a class="ulink" href="http://www.asiabsdcon.org/" target="_top">AsiaBSDCon</a> 2007
      in a paper entitled "puffs - Pass-to-Userspace Framework File System".
      The <a class="ulink" href="http://2007.asiabsdcon.org/papers/P04-paper.pdf" target="_top">paper</a> and
      <a class="ulink" href="http://2007.asiabsdcon.org/papers/P04-slides.pdf" target="_top">slides</a>
      are available.
    </p>

    <p>The ReFUSE emulation layer for FUSE file systems is described
      in "<a class="ulink" href="refuse.pdf" target="_top">ReFUSE: Userspace FUSE
      Reimplementation Using puffs</a>" presented at EuroBSDCon 2007.
    </p>

    <p>A
      <a class="ulink" href="http://2008.asiabsdcon.org/papers/P4B-paper.pdf" target="_top">paper</a>
      discussing the implementation of distributed file systems on top
      of puffs was presented at AsiaBSDCon 2008.
    </p>

    <p>The <a class="ulink" href="http://man.NetBSD.org/man/puffs+3" target="_top">puffs</a>
      manual pages provide further information from a development perspective.
    </p>

    <p>The source code in browsable form:
      </p>
<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; ">
<li class="listitem"><a class="ulink" href="http://cvsweb.NetBSD.org/bsdweb.cgi/src/sys/fs/puffs/" target="_top">kernel code</a></li>
<li class="listitem"><a class="ulink" href="http://cvsweb.NetBSD.org/bsdweb.cgi/src/lib/libpuffs/" target="_top">libpuffs</a></li>
<li class="listitem"><a class="ulink" href="http://cvsweb.NetBSD.org/bsdweb.cgi/src/lib/librefuse/" target="_top">librefuse</a></li>
<li class="listitem"><a class="ulink" href="http://cvsweb.NetBSD.org/bsdweb.cgi/src/usr.sbin/puffs/" target="_top">puffs file systems in the base system</a></li>
<li class="listitem"><a class="ulink" href="http://cvsweb.NetBSD.org/bsdweb.cgi/src/share/examples/puffs/" target="_top">source code examples of other puffs file systems</a></li>
<li class="listitem"><a class="ulink" href="http://cvsweb.NetBSD.org/bsdweb.cgi/src/share/examples/refuse/" target="_top">source code examples for refuse</a></li>
</ul></div>
<p>
    </p>

  </div></div></div>
<div class="navfoot"></div>
<div id="footer"><div id="footerContent"><center>
<span class="footfeed"><a href="//www.NetBSD.org/cgi-bin/feedback.cgi">
	  Contact</a> |
      </span><span class="footcopy"><a href="../../about/disclaimer.html">
      Disclaimer</a> |

      <span class="copyright">Copyright 1994-2020 The NetBSD Foundation, Inc. </span>ALL RIGHTS RESERVED.<br>NetBSD<sup>/sup> is a registered trademark of The NetBSD
	Foundation, Inc.</span>
</center></div></div>
</div></body>
</html>