Annotation of htdocs/developers/PR.xml, revision 1.2
1.1 hubertf 1: <?xml version="1.0"?>
2: <!DOCTYPE webpage
3: PUBLIC "-//NetBSD//DTD Website-based NetBSD Extension//EN"
4: "http://www.NetBSD.org/XML/htdocs/lang/share/xml/website-netbsd.dtd">
5:
6: <webpage id="developers-PR">
7: <config param="desc" value="Resolving Problem Reports"/>
1.2 ! kano 8: <config param="cvstag" value="$NetBSD: PR.xml,v 1.1 2007/04/05 17:50:59 hubertf Exp $"/>
! 9: <config param="rcsdate" value="$Date: 2007/04/05 17:50:59 $"/>
1.1 hubertf 10: <head>
11: <title>Resolving Problem Reports</title>
12: </head>
13:
14: <sect1 role="toc">
15:
16: <sect2 id="top">
17: <title>Resolving Problem Reports</title>
18:
19: <sect3 id="intro">
20: <title>Introduction</title>
21: <para>The NetBSD Project uses the GNATS <quote>Problem Report</quote>
22: database to accept and track bug/problem reports from all users of
23: NetBSD. When used properly, this facility allows us to make sure
24: that no problem with the NetBSD software goes unfixed.</para>
25:
26: <para>The GNATS database uses Internet E-mail as its principal
27: submission mechanism, and keeps the problem reports (commonly
28: abbreviated <quote>PR</quote>) in Internet E-mail format, with an
29: extended header format in the body of the message. The database uses
30: one file per PR, and each category is a directory, in a manner
31: similar to the MH mail system, and NetNews.</para>
32: </sect3>
33:
34: <sect3 id="access">
35: <title>Accessing the Problem Report Database</title>
36:
37: <sect4 id="webinterface">
38: <title>Web Interface</title>
39:
40: <para>There is a web-based interface to the GNATS database that has
41: both a <ulink url="../Gnats/">tree of database summary pages</ulink>, and a <ulink
42: url="../Misc/query-pr.html">search</ulink> facility. The web interface
43: has some limitations:</para>
44:
45: <itemizedlist>
46: <listitem>The web interface is several hours behind the actual
47: database because it operates on a copy of the database for
48: security.</listitem>
49: <listitem>The web interface <emphasis>does not</emphasis> provide
50: access to any PR marked <quote>confidential</quote>.</listitem>
51: <listitem>PRs displayed by a web browser have been through
52: sufficient transformation that <quote>cut & paste</quote> is
53: not likely to do what you expect (i.e. patches will not apply
54: properly because of white space substitution, uuencoded text
55: will not decode properly).</listitem>
56: </itemizedlist>
57: </sect4>
58:
59: <sect4 id="netbsd-bugs">
60: <title>The netbsd-bugs Mailing List</title>
61:
62: <para>All PRs, confidential or not, are sent to the <ulink
1.2 ! kano 63: url="/MailingLists/#netbsd-bugs">netbsd-bugs</ulink> mailing list,
1.1 hubertf 64: so that subscribers can see each PR as it is added to the
65: database.</para>
66: </sect4>
67:
68: <sect4 id="gnatscommands">
69: <title>The GNATS commands on gnats.NetBSD.org</title>
70:
71: <para>The GNATS database lives on the host gnats.NetBSD.org. All
72: developers are given an account on that host so that they can
73: directly manipulate PRs, and access confidential PRs. The commands
74: <command>edit-pr</command> and <command>query-pr</command> live in
75: <filename>/usr/pkg/bin</filename>; Make sure you have that directory
76: in your <varname>$PATH</varname>.</para>
77:
78: <para>There are no UNIX Manual pages for these commands. As with most
79: GNU software, there are <quote>info</quote> pages available through
80: the <command>info</command> command. Also, invoking
81: <command>query-pr</command> without any arguments will cause it to
82: give its usage message.</para>
83:
84: <para>The <command>query-pr</command> command is a proper database
85: query interface; it has a large number of options to search the
86: database. Once you know the PR number of the PR you wish to
87: manipulate, you can:</para>
88:
89: <itemizedlist>
90: <listitem>
91: <para><command>query-pr --full</command> <number></para>
92: <para>This will dump out the full PR without any transformation to
93: standard output.</para>
94: </listitem>
95: <listitem>
96: <para><command>edit-pr</command> <number></para>
97: <para>This command will start up a text editor
98: (<application>vi</application> by default; but this can be
99: overridden by the <varname>$EDITOR</varname> or
100: <varname>$VISUAL</varname> environment variables), so that
101: changes can be made to the PR.</para>
102: </listitem>
103: </itemizedlist>
104: </sect4>
105:
106: <sect4 id="edit-pr">
107: <title>The Common Reasons to <command>edit-pr</command></title>
108:
109: <para>The most common changes made to a PR are:</para>
110:
111: <orderedlist>
112: <listitem>
113: <para>Change the PR's <code>>State:</code> field to one of the values
1.2 ! kano 114: listed in <ulink url="../Misc/pr-states.html">state</ulink> as it progresses
1.1 hubertf 115: through the process of resolving it.</para>
116: <para>The person listed in the <code>>Responsible:</code> field of the
117: PR should be making these state changes, as it is necessary.</para>
118: </listitem>
119: <listitem>
120: <para>Change the PR's <code>>Responsible:</code> field to the account name
121: of a developer who will handle the PR.
122: This person becomes the PR Submitter's primary contact for
123: getting the problem resolved.</para>
124: <para>This field can have any username from /etc/passwd on
125: <code>gnats.NetBSD.org</code> and anyone listed in the
126: <filename>/usr/pkg/share/gnats/gnats-db/gnats-adm/responsible</filename>
127: file.</para>
128: <para>All PRs get a default Responsible Person when they are
129: initially filed, appropriate to the category in which the PR
130: was filed (e.g. <quote>security-officer</quote> for PRs in
131: the <quote>security</quote> category). The
132: <filename>/usr/pkg/share/gnats/gnats-db/gnats-adm/categories</filename>
133: file lists the default Responsible Person for each
134: category.</para>
135: <para>There is also a table of <ulink
136: url="../Gnats/#table-responsible">developers responsible for
137: current PRs</ulink>.</para>
138: </listitem>
139: <listitem>
140: <para>Change the PR's <code>>Category:</code> field.</para>
141: <para>It is not unusual for a PR Submitter to have made a poor
142: category choice. There is a list of <ulink
143: url="../Gnats/#category-descriptions">PR categories and their
144: definitions</ulink> and a table of <ulink
145: url="../Gnats/#table-category">current PRs by
146: category</ulink>.</para>
147: <para>The
148: <filename>/usr/pkg/share/gnats/gnats-db/gnats-adm/categories</filename>
149: file lists the valid categories and the default Responsible
150: Person. It is usually necessary to change the
151: <code>>Responsible:</code> field at the same time to a more
152: appropriate person. Most often, the correct Responsible Person
153: is the default Responsible Person for the new category.</para>
154: </listitem>
155: <listitem>
156: <para>Change other <ulink url="../Misc/pr-fields.html">PR
157: Fields</ulink> according to the analysis of the responsible
158: developer.</para>
159: </listitem>
160: </orderedlist>
161:
162: <para>Edit each field you think needs modifying, then save the file
163: and exit the editor. The <command>edit-pr</command> will then
164: prompt for a short explanation to be typed for each key field
165: change (mostly <code>>State:</code> and
166: <code>>Responsible:</code>). This text is entered one line at
167: time, ending with <command>^D</command>.</para>
168:
169: <para>This text is then sent via e-mail to the PR Submitter, the
170: Responsible developer, and <email>gnats-admin@NetBSD.org</email>.
171: It is also appended to the PR by <command>edit-pr</command> along
172: with the user ID of the developer making the change, a timestamp,
173: and the entered text.</para>
174:
175: <para>Unfortunately, no external editor can be invoked at this
176: point; if you make a mistake, you'll have to use
177: <command>edit-pr</command> to correct it.</para>
178: </sect4>
179: </sect3>
180:
181: <sect3 id="resolve">
182: <title>Resolving Problem Reports</title>
183:
184: <sect4 id="ideal">
185: <title>The Ideal Process</title>
186:
187: <para>In an ideal world, the process for a problem report is as
188: follows:</para>
189:
190: <orderedlist>
191: <listitem>
192: <para>A NetBSD user has a problem with NetBSD. He invokes the
193: <command>send-pr</command> command on his system (assuming
194: it's still stable enough to do that), and files a Problem
195: Report. Hopefully, he follows all the advice found in <ulink
196: url="../Misc/pr-hints.html">"What goes into a Problem
197: Report."</ulink></para>
198: <para>If the user's own system is not stable enough to use
199: <command>send-pr</command>, there is a <ulink
200: url="../Misc/send-pr.html">web interface</ulink> that can be
201: used to submit problem reports.</para>
202: </listitem>
203: <listitem>
204: <para>When the PR arrives at <code>gnats.NetBSD.org</code>,
205: the GNATS database system examines it, and files it. If the
206: PR is malformed, it will be filed in the
207: <code>pending</code> category, marked confidential, awaiting
208: manual intervention by the GNATS database
209: administrator.</para>
210: <para>If the format is OK, the PR is assigned a PR number,
211: filed into the requested category, and E-mailed out again to
212: the default responsible party for the category, and to the
213: <ulink
214: url="../MailingLists/#netbsd-bugs">netbsd-bugs</ulink>
215: mailing list. A notice of the PR number and default
216: Responsible Person is also E-mailed back to the PR
217: Submitter.</para>
218: </listitem>
219: <listitem>
220: <para>The Responsible Person should read and analyze the
221: PR. Any other person who has insight into the problem should
222: also <ulink url="../Misc/send-pr.html#appending">add
223: whatever information they can to the problem report</ulink>
224: (this is why the report is mailed out to a mailing list; a
225: wide audience increases the probability that a key insight
226: needed to solve the problem will be discovered).</para>
227:
228: <para>If the default Responsible Person determines that
229: another developer is a more appropriate Responsible Person,
230: the PR should be reassigned with
231: <command>edit-pr</command>. The new Responsible Person
232: should read and analyze the PR.</para>
233: </listitem>
234: <listitem>
235: <para>Once a cause and potential fix has been identified, a
236: description should be added to the PR, and the <ulink
237: url="../Misc/pr-states.html">state</ulink> should be changed
238: to <code>analyzed</code>. At this point, implementation of
239: the fix for the problem begins.</para>
240: <para>This part of the process should begin as quickly as
241: possible, since a user with a current problem is suffering,
242: but also has his attention engaged, and his hardware
243: available for testing potential fixes. If a PR is allowed to
244: languish, the opportunity to reproduce the problem and test
245: potential fixes may be lost.</para>
246: </listitem>
247: <listitem>
248: <para>Once the implementation of the fix is completed and communicated
249: to the PR submitter, the PR <ulink url="/Misc/pr-states.html">state</ulink>
250: should be changed to <code>feedback</code>, awaiting response from the PR
251: submitter that the fix really works.</para>
252: <para>A PR should also be put into <code>feedback</code> state when input
253: is required from the submitter to complete the analysis of the PR
254: (i.e. when you ask them a question), or when you need information
255: from some other source (essentially, <code>feedback</code> is a wait
256: state).</para>
257: </listitem>
258: <listitem>
259: <para>Once the PR submitter confirms that the fix works, the PR
260: can be <code>closed</code>.</para>
261: </listitem>
262: </orderedlist>
263: <para>At each step of the PR handling process, make sure that
264: feedback and other analysis and commentary is <ulink
265: url="../Misc/send-pr.html#appending">appended to the PR</ulink>
266: by using a proper E-mail subject line and making sure that the
267: messages are copied to
268: <email>gnats-bugs@NetBSD.org</email>. Having a complete record
269: of information about the PR is valuable both while hunting down
270: the bug and for future system maintenance.</para>
271: <para>If at all possible, it is important to get any fix committed
272: to the CVS trunk <ulink url="releng/pullups.html">pulled
273: up</ulink> by <ulink url="releng/">NetBSD Release
274: Engineering</ulink> to the "-release" branch, so that people who
275: are tracking that branch can get the fix right away rather than
276: waiting for the next major release of NetBSD. This also makes it
277: possible for the next point release of NetBSD to have the
278: fix.</para>
279: <para>The NetBSD Community is a whole lot of very smart, and very
280: experienced people. If you're having trouble analyzing a problem
281: report, ask questions in the appropriate <ulink
282: url="../MailingLists/">mailing list</ulink>; more than likely,
283: someone will be able to help.</para>
284: </sect4>
285:
286: <sect4 id="otherways">
287: <title>Other Ways Things Get Done</title>
288:
289: <para>The Ideal Process was described above. That's not the only way
290: that problem reports get handled. All of the principal people
291: involved in NetBSD are pretty busy, and can't devote full attention
292: to this project. As such, if you see a PR that you can solve that
293: hasn't been attended to yet, go claim it by setting yourself as the
294: Responsible Person with <command>edit-pr</command>, and solve
295: it.</para>
296:
297: <para>Even if you don't feel qualified to hack the code yourself, if
298: you can offer a test case or other information, send it along to
299: GNATS to be <ulink url="../Misc/send-pr.html#appending">appended to
300: the PR</ulink>. <quote>Many hands make light work.</quote></para>
301:
302: <para>Some problem reports are so trivial that the fix is obvious (or
303: perhaps the fix was provided by the submitter), that they go directly
304: from <code>open</code> to <code>closed</code> immediately after the fix is
305: committed.</para>
306:
307: <para>If for some reason you find that you're unable to finish
308: handling the PR, reset the <code>>Responsible:</code> field to
309: whoever had responsibility before you took the PR over. Don't
310: prevent others from making progress on the PR because they think
311: you're taking care of it.</para>
312:
313: <para>As long as you're marked as the Responsible Person for a PR,
314: you'll receive a monthly E-mail reminder about it. Use those
315: reminders to drive you to review PRs and put them into their correct
316: states as time passes.</para>
317:
318: <para>When a PR is in <code>feedback</code> state, the PR submitter gets an
319: E-mail monthly reminder at the same time as the Responsible Persons
320: do, to prompt or prod them into responding. Generally, if there
321: has been no response for more than three months (three reminder
322: cycles), it's pretty safe to assume that the submitter is gone or
323: no longer cares. At that point, whether to close the PR becomes
324: a judgement call for the Responsible Person - how serious is it?
325: Should it be solved without further input from the Submitter?</para>
326:
327: <para>The other way we use the GNATS PR database is to keep track of
328: problems which are waiting for larger issues to be solved. The
329: oldest PR in the database at this writing, <ulink
330: url="http://www.NetBSD.org/cgi-bin/query-pr-single.pl?number=13">lib/13</ulink>
331: (yes, of course it would be 13!) begs the entire
332: internationalization of the NetBSD system. I18N is a hard problem
333: that requires a wholesale overhaul of the system, which is why that
334: PR is still open after seven years. This doesn't mean we'll never
335: solve it; just that it isn't as critical as some other problems
336: reported in the database.</para>
337:
338: <para>In effect, this usage of the GNATS PR database is as a long-term
339: project tracking system.</para>
340: </sect4>
341: </sect3>
342:
343: <sect3 id="priorities">
344: <title> Priorities, Severities, and Releases</title>
345:
346: <para>In an ideal world, the GNATS PR database would be empty, we'd
347: release perfect software, and Microsoft would be a shadow of its
348: current self.</para>
349:
350: <para>The <code>>Priority:</code> field in the PR reflects this ideal,
351: in that <code>high</code> priority is supposed to be fixed immediately;
352: <code>medium</code> is supposed to be resolved before the next release
353: of NetBSD (major or minor?), and <code>low</code> priority gets solved
354: <quote>eventually</quote>.</para>
355:
356: <para>In practice, PR resolution is dependent on the right mix of
357: submitter interest, developer interest, problem reproduceability,
358: hardware availability, and good timing. If any of the required
359: elements is missing from the mix, the PR will sit.</para>
360:
361: <para>If we were really diligent about PRs, we would adjust the priority
362: of each PR to reflect its actual importance, and probability of
363: getting fixed according to the definitions. Unfortunately, that
364: requires an overall evaluation of Release Engineering goals and
365: targets and all PRs relative to each other, which is difficult for
366: a dispersed group to do in an organized fashion.</para>
367:
368: <para>In contrast, the <code>>Severity:</code> field is really an
369: expression of the amount of pain the user is going through with the
370: problem being reported, and it's something we really shouldn't
371: adjust without careful consideration.</para>
372:
373: <para>The proper procedure would be to review all PRs in the database
374: at each release point, to decide on a per-PR basis whether to
375: <quote>fix now</quote>, <quote>fix later</quote>,
376: <quote>suspend</quote> and adjust priorities. Perhaps one day we'll
377: have the resources and manpower for that.</para>
378:
379: </sect3>
380:
381: <sect3 id="remotegnats">
382: <title>Remote GNATS Operations</title>
383:
384: <para>For people who find logging into a remote host tedious, the
385: following csh aliases might be useful:</para>
386:
387: <programlisting>
388: alias query-pr 'ssh gnats.NetBSD.org query-pr --full \!* | tee pr-\!*'
389: alias edit-pr ssh -t gnats.NetBSD.org edit-pr
390: </programlisting>
391: </sect3>
392:
393:
394: </sect2>
395: </sect1>
396:
397: <parentsec url="./" text="&os; Developer Documentation"/>
398: </webpage>
CVSweb <webmaster@jp.NetBSD.org>