CVS log for src/usr.bin/indent/pr_comment.c

Up to [cvs.NetBSD.org] / src / usr.bin / indent

Request diff between arbitrary revisions

Keyword substitution: kv
Default branch: MAIN

indent: inline input-related macros

No binary change.

indent: use line number of the token start in diagnostics

Previously, the line number of the end of the token was used, which was
confusing in debug mode.

indent: fix scanning of no-wrap comments (since 2021.11.07.10.34.03)

The "refactoring" back then tried to be too clever.

indent: only add blank lines before actual block comments

indent: remove support for backspace in code and comments

The C code in the whole tree does not contain a single literal
backspace.

indent: clean up

Extract duplicate code for handling line continuations.

Prevent theoretic undefined behavior in strspn, as inp.s is not
null-terminated.

Remove adding extra space characters when processing comments, as these
are not necessary to force a line of output.

No functional change.

indent: miscellaneous cleanups

No binary change.

indent: rename a field of the parser state

The previous name 'comment_in_first_line' was misleading, as it could
mean that there was a comment in the first line of the file.

No functional change.

indent: allow more than 20 nested parentheses or brackets

indent: clean up handling of comments

One less moving part in the parser state.

No functional change.

indent: remove another flag from parser state

When processing a comment, the flag ps.next_col_1 was not used for the
next token, but for a line within a comment.  As its scope was limited
to a single comment, there is no need to store it any longer than that

No functional change.

indent: remove a redundant flag from the parser state

No functional change.

indent: miscellaneous cleanups

indent: in debug mode, null-terminate buffers

indent: rename and sort variables in parser state

No functional change.

indent: format its own code

indent: preserve block comments with delimiters

indent: express more clearly how delimited and no-wrap comments relate

No functional change.

indent: right-trim single-line comments

indent: clean up formatting of comments

No functional change.

indent: simplify handling of comments

No functional change.

indent: split printing of comments into smaller functions

No functional change.

indent: remove read pointer from buffers that don't need it

The only buffer that needs a read pointer is the current input line in
'inp'.

No functional change.

indent: fix out-of-bounds read when reading a comment

tests/indent: fix outdated or wrong comments

indent: extract the output state from the parser state

The parser state depends on the preprocessing lines, the output state
shouldn't.

indent: implement blank line above block comment

indent: manually wrap overly long lines

No functional change.

indent: switch to standard code style

Taken from share/misc/indent.pro.

Indent does not wrap code to fit into the line width, it only does so
for comments.  The 'INDENT OFF' sections and too long lines will be
addressed in a follow-up commit.

No functional change.

indent: directly access the input buffer

No functional change.

indent: remove support for form feed characters inside a line

Form feeds are occasionally used to split code into pages, and this use
is still supported.  Having a form feed in the middle of a line is
exotic.

indent: fix line wrapping of comments to the right of code

indent: let indent format its own code

With manual corrections, as indent does not properly indent multi-line
'?:' expressions nor multi-line controlling expressions.

indent: clean up memory and buffer management

Remove the need to explicitly initialize the buffers.  To avoid
subtracting null pointers or comparing them using '<', migrate the
buffers from the (start, end) form to the (start, len) form.  This form
also avoids inconsistencies in whether 'buf.e == buf.s' or 'buf.s ==
buf.e' is used.

Make buffer.st const, to avoid accidental modification of the buffer's
content.

Replace '*buf.e++ = ch' with buf_add_char, to avoid having to keep track
how much unwritten space is left in the buffer.  Remove all safety
margins, that is, no more unchecked access to buf.st[-1] or appending
using '*buf.e++'.

Fix line number counting in lex_word for words that contain line breaks.

No functional change.

indent: only null-terminate the buffers if necessary

The only case where a buffer is used as a C-style string is when looking
up a keyword.

No functional change.

indent: shorten variable names for indenting comments

No functional change.

indent: fix handling of multiple block comments in a line

indent: in comments, keep a leading tab

This kind of comments is used for the CVS IDs at the top of files.

indent: fix vertical spacing after declarations

A comment is not supposed to change the state of the 'blank line after
declaration', but it did.  The initialization of saved_just_saw_decl was
wrong though since it tried to capture the previous value after it had
already been overwritten.

indent: reduce code for scanning tokens

The input line is guaranteed to end with '\n', so there's no need to
carry another pointer around.

No functional change.

indent: remove foreign RCS IDs

indent: use enum instead of magic numbers for tracking declarations

No functional change.

indent: rename struct fields for buffers

No binary change except for assertion line numbers.

indent: remove statistics

The numbers from the statistics were wrong.

indent: remove broken code for handling blank lines

This fixes several bugs where blank lines were erroneously added or
removed, treating these old bugs for new bugs in different places.
These new bugs are expected to be easier to fix, as the old bugs will
not interfere anymore.

indent: clean up control flow, remove Capsicum

No functional change.

indent: group global variables related to output control

No functional change.

indent: rename dump functions to output

No functional change.

indent: add buf_add_range for adding characters to a buffer

No functional change.

indent: make fits_in_one_line independent of ps

This makes it easier to understand the function. Having the dependency
to the input line handling is already complicated enough.

No functional change.

indent: fix accidentally joined and broken comments (since 2019-04-04)

The fixed version is not perfect as it gets the indentation of the last
line of the first comment wrong, but at least indent doesn't generate
malformed output anymore.

tests/indent: demonstrate bugs in comment processing

indent: reduce casts to unsigned char for character classification

No functional change.

indent: use character input API from pr_comment.c

No functional change.

indent: replace direct access to the input buffer

This is a preparation for abstracting away all the low-level details of
handling the input.  The goal is to fix the current bugs regarding line
number counting, out of bounds memory access, and generally unreadable
code.

No functional change.

indent: rename input buffer variables

From reading the names 'save_com' and 'sc_end', it was not obvious
enough that these two variables are the limits of the same buffer, the
names were just too unrelated.

No functional change.

indent: group variables for input handling

No functional change.

indent: various cleanups

Make several comments more precise.

Rename process_end_of_file to process_eof to match the token name.

Change the order of assignments in analyze_comment to keep the com_ind
computations closer together.

In copy_comment_wrap, use pointer difference instead of pointer addition
to stay away from undefined behavior.

No functional change.

indent: remove code that accessed out-of-bounds data from buffer

Saving and restoring the exact buffer position had been necessary before
NetBSD pr_comment.c 1.114. The code accessed the buffer data out of the
bounds of [com.s, com.e), which was rather surprising. More
specifically, it accessed com.e[-1] in a case where com.e == com.s,
relying on com.e[-1] being ' ' in most cases and '*' in the case where
the comment delimiter was written to a separate output line.

Make the code easier understandable by only ever accessing the buffer
data in the bounds [buf.s, buf.e).

No functional change.

indent: only access buffer data in the range [buf.s, buf.e)

No functional change.

indent: do not expand comment buffer unnecessarily

This may have been a simple typo or a really tricky optimization that
isn't obvious even after studying the code for several months. Either of
these is bad, so use the standard form of resetting the buffer.

No functional change.

indent: remove dead code in analyze_comment

The case of an otherwise empty line is already handled further above.

indent: remove dead code from copy_comment_wrap

C99 comments are not wrapped, therefore there is no need to check for
them in copy_comment_wrap.

No functional change.

indent: move documentation from process_comment to copy_comment_wrap

indent: clean up process_comment

The assignments to the variables were redundant, they are already done
by analyze_comment.

No functional change.

indent: remove redundant assignment

At that point, ps.next_col_1 is already false.

No functional change.

indent: make end of comment detection of nowrap comments simpler

No functional change.

indent: clean up overcomplicated conditions in copy_comment_nowrap

No functional change.

indent: skip redundant conditions in copy_comment_nowrap

No functional change.

indent: clean up copy_comment_nowrap

The action for '\f' was the same as the default action.

Replacing 'switch' with 'if' makes the code shorter.

No functional change.

indent: make copy_comment_nowrap simpler

Since a nowrap comment is copied unmodified, it need not depend on any
maximum line length.

No functional change.

indent: remove dead code from process_comment_nowrap

In comments that are preserved, no additional leading ' * ' is inserted.

No functional change.

indent: remove dead code from copy_comment_wrap

No functional change.

indent: remove dead code from copy_comment_nowrap

No functional change.

indent: split copy_comment into wrapping and non-wrapping

These two cases are processed in an almost entirely different way. In
particular, copy_comment_nowrap should copy the comment verbatim, which
is not obvious from the current code, due to the many conditions and the
complex control flow.

No functional change.

indent: rename 'inbuf' functions to 'inp'

The variable 'inp' used to be named 'inbuf'. Make the function names
correspond to the variable name again.

No functional change.

indent: rename ps.curr_newline to next_col_1

For symmetry with ps.curr_col_1.

No functional change.

indent: fix parsing of C99 comments containing '*/'

indent: split process_comments into separate functions

No functional change.

indent: inline indentation_after, shorten function name to ind_add

There were only few calls to indentation_after, so inlining it spares
the need to look at yet another function definition. Another effect is
that code.s and code.e appear in the code as a pair now, instead of a
single code.s, making the scope of the function call obvious.

In ind_add, there is no need to check for '\0' anymore since none of the
buffers can ever contain a null character, these are filtered out by
inbuf_read_line.

No functional change.

indent: rename prev_newline and prev_col_1 to curr

These two flags describe the token that is currently processed.

In process_binary_op, curr_newline can never be true since newline is
not a binary operator, so remove that condition.

No functional change.

indent: fix bounds check for sc_buf

Some years ago, save_com was an array of characters, used as temporary
storage. When sc_buf was added, this code was forgotten. The bounds
check must be on the array itself, not on an iterator that points
somewhere in that array.

indent: fix assertion in fits_in_one_line

indent: merge isblank and is_hspace into ch_isblank

No functional change.

indent: fix undefined behavior in buffer handling

Adding an arbitrary integer to a pointer may result in an out of bounds
pointer, so replace the addition with a pointer subtraction.

In the buffer handling functions, handle 'buf' and 'l' before 's' and
'e', since they are pairs.

In inbuf_read_line, use 's' instead of 'buf' to make the code easier to
understand for human readers.

No functional change.

indent: use prev/curr/next to refer to the current token

The word 'last' just didn't match with 'next'.

No functional change.

indent: clean up process_comment

There is no undefined behavior since the compared characters are always
from the basic execution character set. All other cases are covered by
the condition above for now_len.

Fix debug logging for non-ASCII characters, previously a character was
output as \xffffffc3.

indent: reduce indentation in process_comment

No functional change.

indent: make reformatting of comments simpler

No functional change.

indent: split type token_type into 3 separate types

Previously, token_type was used for 3 different purposes:

1. symbol types from the lexer
2. symbol types on the parser stack
3. kind of control statement for 'if (expr)' and similar statements

Splitting the 41 constants into separate types makes it immediately
clear that the parser stack never handles comments, preprocessing lines,
newlines, form feeds, the inner structure of expressions.

Previously, the constant switch_expr was especially confusing since it
was used for 3 different purposes: when returned from lexi, it
represented the keyword 'switch', in the parser stack it represented
'switch (expr)', and it was used for a statement head as well.

The only overlap between the lexer symbols and the parser symbols are
'{' and '}', and the keywords 'do' and 'else'. To increase confusion,
the constants of the previous token_type were in apparently random
order and before 2021, they had cryptic, highly abbreviated names.

No functional change.

indent: run indent on its own source code

With manual corrections afterwards. Indent still does not get
extra_expr_indent correctly, it also indents global variables after
tagged declarations too deep.

No functional change.

indent: replace global variable use_ff with function parameter

indent: clean up comments

indent: rename blankline_requested variables

The words 'prefix' and 'postfix' sounded too much like horizontal
concepts, like in operators. The actual purpose of these variables is to
add blank lines before and after the current line, so use the same
wording as in the command line options.

No functional change.

indent: clarify that 25 is a magic number

The extra line width for comments to the very right is just that, an
arbitrarily chosen number. It neither has to be a multiple of 8, nor of
the tabsize nor of the indentation. Since 25 is neither of these, this
makes it a perfect choice, allowing these extreme comments to have 22
characters per line with -sc (leading asterisks in comment
continuations, the default) or 25 without.

No functional change.

indent: turn ps.com_ind into local variable

This makes it immediately clear that ps.com_ind is computed in the first
part of process_comment and then only used from there.

No functional change.

indent: extract fits_in_one_line from process_comment

No functional change.

indent: in process_comment, migrate int variable to bool

No functional change.

indent: in process_comment, negate box_com to may_wrap

In the new line 213, may_wrap could only be true and was therefore a
redundant condition.

No functional change.

indent: negate a few condition in process_comment

No functional change.

indent: fix formatting of single-line comments (since today)

The change in pr_comment.c 1.70 from 3 hours ago did not cover all edge
cases correctly. Now it works for comments that are aligned with tabs.

indent: replace unreachable code with assertion

indent: use high-level buffer API for processing comments

Document the trickery of copying the last word from the previous line
since that it not obvious at all.

No functional change.

indent: fix wrapping for comments in otherwise empty lines

The comment above the code was wrong. The leading 3 characters were
indeed ignored, but the first of them was '/', not ' '. Of the trailing
3 characters, 2 were not ignored. The start and end of the comment would
not cancel out, they would rather sum up.

indent: extract common code for advancing a single tab

No functional change.

indent: replace unreachable code with assertions

The input buffer is always supposed to be terminated with a newline. The
function inbuf_read_line silently skips null characters. Since the input
buffer is redirected to a temporary buffer in some cases, do not simply
remove this supposed dead code, but replace it with assertions.

In any case, if the code for calling inbuf_read_line had been reachable
and actually allocated a different buffer, continuing to use the pointer
p would have invoked undefined behavior anyway.

indent: use standard error handling for unterminated comment

Just writing it to stdout is bad, especially when indent is used in
filter mode. Silently continuing after such an error is bad as well.

echo '/*' | indent

indent: remove redundant conditions

No functional change.

indent: convert ps.box_com to local variable

This variable is only used in a single function, and that function does
not call any other function that could replace the parser state or
install a temporary parser state.

No functional change.

indent: rename fill_buffer to inbuf_read_line

No functional change.

indent: run indent on indent.h

The formatting looks mostly OK.

Some struct members had excessively long names, leaving no space for
their corresponding comments. Renamed some of them using well-known
abbreviations.

The formatting for debug_vis_range is messed up, no idea why. It is
clearly a function declaration, not a function definition, so there is
no need to place the function name in column 1.

No functional change.

indent: fix formatting of C99 comments

The first attempt at formatting C99 comments was conceptually wrong. It
accessed the next token in dump_line, even though that function should
only ever look at the buffers for the label, the code and the current
comment. (Understanding that part of the code was difficult at that time
due to the sheer number of global variables.) The complicated and
ever-growing condition for whether to output the token was a hack and in
retrospect doesn't make sense at all, that's why it only came close to
the intended effect.

Some unintended side effects were that the C99 comments had an
additional space in front of them, and that in some cases an empty line
followed the comment, and that the comments were not aligned.

Previously, the newline that terminates the C99 comment was included in
the comment. Separating the newline from the comment fixed all these
unintended side effects. The only downside is that the multi-line
statement is not indented, but that should be easy to fix.

indent: reduce scope of t_ptr in process_comment

No functional change.

indent: replace column calculations with indent, part 4/4

indent: replace column calculations with indent, part 3

No functional change.

indent: group variables for the input buffer

The input buffer follows the same concept as the intermediate buffers
for label, code, comment and token, so use the same type for it.

No functional change.

indent: use braces around multi-line statements

No functional change.

indent: let the code breathe a bit by inserting empty lines

No functional change.

indent: clean up comments

No functional change.

indent: remove redundant comments

No functional change.

indent: fix outdated comment, add 'const'

indent: fix spelling in comments

indent: use proper escape sequence for form feed

This escape sequence has been available since at least 1978.

indent: merge duplicate code into is_hspace

No functional change.

indent: clean up code for appending to buffers

Use *e++ for appending and e[-1] for testing the previously appended
character, like in other places in the code.

No functional change.

indent: merge duplicate code for reading from input buffer

No functional change.

indent: let indent format its own code -- in supervised mode

After running indent on the code, I manually selected each change that
now looks better than before. The remaining changes are left for later.
All in all, indent did a pretty good job, except for syntactic additions
from after 1990, but that was to be expected. Examples for such
additions are GCC's __attribute__ and C99 designated initializers.

Indent has only few knobs to tune the indentation. The knob for the
continuation indentation applies to function declarations as well as to
expressions. The knob for indentation of local variable declarations
applies to struct members as well, even if these are members of a
top-level struct.

Several code comments crossed the right margin in column 78. Several
other code comments were correctly broken though. The cause for this
difference was not obvious.

No functional change.

indent: merge duplicate code for token buffers

No functional change.

indent: remove dead code for printing comments after empty lines

This code has been commented out for at least 29 years.

No functional change.

indent: convert remaining ibool to bool

No functional change intended.

indent: prepare for lint's strict bool mode

Before C99, C had no boolean type. Instead, indent used int for that,
just like many other programs. Even with C99, bool and int can be used
interchangeably in many situations, such as querying '!i' or '!ptr' or
'cond == 0'.

Since January 2021, lint provides the strict bool mode, which makes bool
a non-arithmetic type that is incompatible with any other type. Having
clearly separate types helps in understanding the code.

To migrate indent to strict bool mode, the first step is to apply all
changes that keep the resulting binary the same. Since sizeof(bool) is
1 and sizeof(int) is 4, the type ibool serves as an intermediate type.
For now it is defined to int, later it will become bool.

The current code compiles cleanly in C99 and C11 mode, as well as in
lint's strict bool mode. There are a few tricky places:

In args.c in 'struct pro', there are two types of options: boolean and
integer. Boolean options point to a bool variable, integer options
point to an int variable. To keep the current structure of the code,
the pointer has been changed to 'void *'. To ensure type safety, the
definition of the options is done via preprocessor magic, which in C11
mode ensures the correct pointer types. (Add CFLAGS+=-std=gnu11 at the
very bottom of the Makefile.)

In indent.c in process_preprocessing, a boolean variable is
post-incremented. That variable is only assigned to another variable,
and that variable is only used in a boolean context. To provoke a
different behavior between the '++' and the '= true', the source code
to be indented would need 1 << 32 preprocessing directives, which is
unlikely to happen in practice.

In io.c in dump_line, the variables ps.in_stmt and ps.in_decl only ever
get the values 0 and 1. For these values, the expressions 'a & ~b' and
'a && !b' are equivalent, in all versions of C. The compiler may
generate different code for them, though.

In io.c in parse_indent_comment, the assignment to inhibit_formatting
takes place in integer context. If the compiler is smart enough to
detect the possible values of on_off, it may generate the same code
before and after the change, but that is rather unlikely.

The second step of the migration will be to replace ibool with bool,
step by step, just in case there are any hidden gotchas in the code,
such as sizeof or pointer casts.

No change to the resulting binary.

indent: remove ifdef for lint

NetBSD lint does not need them anymore, FreeBSD does not have lint.

indent: move statistical values into a separate struct

No functional change.

indent: add nonnull memory allocation functions

The only functional change is a single error message.

indent: group global variables for token buffer

No functional change.

indent: group global variables for code buffer

No functional change.

indent: group global variables for label buffer into struct

No functional change.

indent: group global variables for the comment buffer

No functional change.

indent: clean up check_size_comment

The additional parameter last_bl_ptr was only necessary because the last
blank was stored as a pointer into the buffer.  By storing the index in
the buffer instead, it doesn't need to be updated all the time.

No functional change.

indent: remove trailing whitespace

indent: clean up target column computation in process_comment

No functional change.

indent: fix off-by-one error in comment wrapping

The manual page says that the default maximum length of a comment line
is 78.  The test 'comments.0' wrongly assumed that this 78 would refer
to the maximum _column_ allowed, which is off by one.

Fix the wording in the test 'comments.0' and remove the (now satisfied)
expectation comments in the test 'token-comment.0'.

Several other tests just happened to hit that limit, fix these as well.

indent: fix lint warnings

No functional change.

indent: remove the '+ 1' from right margin calculation in comment

No functional change.

indent: distinguish between 'column' and 'indentation'

column == 1 + indentation.

In addition, indentation is a relative distance while column is an
absolute position.  Therefore, don't confuse these two concepts, to
prevent off-by-one errors.

No functional change.

indent: rename pr_comment to process_comment, clean up documentation

No functional change.

indent: remove redundant parentheses

No functional change.

indent: fix confusing variable names

The word 'col' should only be used for the 1-based column number.  This
name is completely inappropriate for a line length since that provokes
off-by-one errors.  The name 'cols' would be acceptable although
confusing since it sounds so similar to 'col'.

Therefore, rename variables that are related to the maximum line length
to 'line_length' since that makes for obvious code and nicely relates to
the description of the option in the manual page.

No functional change.

indent: document undefined behavior in processing of comments

No functional change.

indent: inline calls to count_spaces and count_spaces_until

These two functions operated on column numbers instead of indentation,
which required adjustments of '+ 1' and '- 1'.  Their names were
completely wrong since these functions did not count anything, instead
they computed the column.

No functional change.

indent: replace compute_code_column with compute_code_indent

The goal is to only ever be concerned about the _indentation_ of a
token, never the _column_ it appears in.  Having only one of these
avoids off-by-one errors.

No functional change.

indent: replace compute_label_column with compute_label_indent

Using the invariant 'column == 1 + indent'.  This removes several overly
complicated '+ 1' from the code that are not needed conceptually.

No functional change.

indent: replace pad_output with output_indent

Calculating the indentation is simpler than calculating the column,
since that saves the constant addition and subtraction of the 1.

No functional change.

indent: replace 'target' with 'indent' in function names

The word 'target' was not as specific as possible.

No functional change.

indent: use consistent indentation for 'else'

Half of the code used -ce, the other half the opposite -nce.

No functional change.

indent: reduce indentation of check_size functions

No functional change.

indent: remove redundant cast after allocation functions

No functional change.

indent: use consistent array indexing

No functional change.

indent: manually indent comments

It's strange that indent's own code is not formatted by indent itself,
which would be a good demonstration of its capabilities.

In its current state, I don't trust indent to get even the tokenization
correct, therefore the only safe way is to format the code manually.

indent: convert big macros to functions

Each of these buffers is only modified in a single file.  This makes it
unnecessary to declare the macros in the global header.

indent: fix handling of '//' end-of-line comments

indent: use all headers in all files

This is a prerequisite for converting the token types to an enum instead
of a preprocessor define, since the return type of lexi will become
token_type.  Having the enum will make debugging easier.

There was a single naming collision, which forced the variable in
scan_profile to be renamed.  All other token names are used nowhere
else.

No change to the resulting binary.

Sync with HEAD

FreeBSD indent r340138

Upgrade indent(1)

Merge all the changes from the recent FreeBSD HEAD snapshot
into our local copy.

FreeBSD actively maintains this program in their sources and their
repository contains over 100 commits with changes.

Keep the delta between the FreeBSD and NetBSD versions to absolute
minimum, mostly RCS Id and compatiblity fixes.

Major chages in this import:

 - Added an option -ldi<N> to control indentation of local variable names.
 - Added option -P for loading user-provided files as profiles
 - Added -tsn for setting tabsize
 - Rename -nsac/-sac ("space after cast") to -ncs/-cs
 - Added option -fbs Enables (disables) splitting the function declaration and opening brace across two lines.
 - Respect SIMPLE_BACKUP_SUFFIX environment variable in indent(1)
 - Group global option variables into an options structure
 - Use bsearch() for looking up type keywords.
 - Don't produce unneeded space character in function declarators
 - Don't unnecessarily add a blank before a comment ends.
 - Don't ignore newlines after comments that follow braces.

Merge the FreeBSD intend(1) tests with our ATF framework.
All tests pass.

Upgrade prepared by Manikishan Ghantasala.
Final polishing by myself.

Fix obvious contraction spelling mistakes by adding missing apostrophes.

Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22365, verified by myself.

PR/21645: Mishka: Localized comments don't work with indent.

Remove #ifndef'd __STDC__ code. ANSIfy.

WARNSify, fix .Nm usage, deprecate register, use <err.h>, KNFify (with indent!;)

merge lite-2.

RCS ID police

do block commenting, if comment begins with slash-star-newline.

Add RCS identifiers.

4.4BSD-Lite2

added, from net/2 (patch 124).