The NetBSD Project

CVS log for src/usr.bin/make/str.h

[BACK] Up to [cvs.NetBSD.org] / src / usr.bin / make

Request diff between arbitrary revisions


Default branch: MAIN


Revision 1.19 / (download) - annotate - [select for diffs], Fri Jan 5 21:56:55 2024 UTC (2 months, 3 weeks ago) by rillig
Branch: MAIN
CVS Tags: HEAD
Changes since 1.18: +9 -12 lines
Diff to previous 1.18 (colored) to selected 1.1 (colored)

make: clean up string functions

No binary change.

Revision 1.18 / (download) - annotate - [select for diffs], Fri Jan 5 21:51:27 2024 UTC (2 months, 3 weeks ago) by rillig
Branch: MAIN
Changes since 1.17: +1 -9 lines
Diff to previous 1.17 (colored) to selected 1.1 (colored)

make: inline Substring_Sub

Remove redundant assertions.

Revision 1.17 / (download) - annotate - [select for diffs], Fri Jun 23 04:56:54 2023 UTC (9 months ago) by rillig
Branch: MAIN
Changes since 1.16: +7 -2 lines
Diff to previous 1.16 (colored) to selected 1.1 (colored)

make: warn about malformed patterns in ':M', ':N' and '.if make(...)'

These patterns shouldn't occur in practice, as their results are tricky
to predict.  Generate a warning for now, and maybe an error later.

Reviewed by sjg@.

Revision 1.16 / (download) - annotate - [select for diffs], Mon Dec 5 23:41:24 2022 UTC (15 months, 3 weeks ago) by rillig
Branch: MAIN
CVS Tags: netbsd-10-base, netbsd-10-0-RC6, netbsd-10-0-RC5, netbsd-10-0-RC4, netbsd-10-0-RC3, netbsd-10-0-RC2, netbsd-10-0-RC1, netbsd-10
Changes since 1.15: +4 -10 lines
Diff to previous 1.15 (colored) to selected 1.1 (colored)

make: inline LazyBuf_AddBytesBetween

No binary change.

Revision 1.15 / (download) - annotate - [select for diffs], Wed Dec 15 10:57:01 2021 UTC (2 years, 3 months ago) by rillig
Branch: MAIN
Changes since 1.14: +2 -1 lines
Diff to previous 1.14 (colored) to selected 1.1 (colored)

make: in CLEANUP mode, free interned strings at the very end

Noticed by sjg.

Revision 1.14 / (download) - annotate - [select for diffs], Mon Dec 13 05:25:04 2021 UTC (2 years, 3 months ago) by rillig
Branch: MAIN
Changes since 1.13: +4 -1 lines
Diff to previous 1.13 (colored) to selected 1.1 (colored)

make: fix memory leak for filenames in .for loops (since 2013-06-18)

Previously, each time a .for directive pushed its buffer on the input
file stack, the current filename was duplicated.  This was a waste of
memory.

The name of a file is typically only used while it is read in.  There is
one situation when the filename is needed for longer, which is when a
target is defined.

Since .for loops are implemented as a special form of included files,
each .for loop duplicated the current filename as well.

$ cat << EOF > for.mk
.for i in 1 2 3 4 5 6 7 8 9 0
.for i in 1 2 3 4 5 6 7 8 9 0
.for i in 1 2 3 4 5 6 7 8 9 0
.for i in 1 2 3 4 5 6 7 8 9 0
.for i in 1 2 3 4 5 6 7 8 9 0
.for i in 1 2 3 4 5 6 7 8 9 0
.for i in 1 2 3 4 5 6 7 8 9 0
.endfor
.endfor
.endfor
.endfor
.endfor
.endfor
.endfor

all:
	@ps -o rsz -p ${.MAKE.PID}
EOF

$ make-2021.12.13.03.55.16 -r -f for.mk
  RSZ
10720

$ ./make -r -f for.mk
 RSZ
1716

The difference is 8 MB, which amounts to 1 million .for loops.

Revision 1.13 / (download) - annotate - [select for diffs], Sun Dec 12 23:39:34 2021 UTC (2 years, 3 months ago) by rillig
Branch: MAIN
Changes since 1.12: +1 -41 lines
Diff to previous 1.12 (colored) to selected 1.1 (colored)

make: remove unused type MFStr

The only binary change is the line number of the assertion in
Substring_Sub.

Revision 1.12 / (download) - annotate - [select for diffs], Sun Dec 12 13:43:47 2021 UTC (2 years, 3 months ago) by rillig
Branch: MAIN
Changes since 1.11: +7 -4 lines
Diff to previous 1.11 (colored) to selected 1.1 (colored)

make: fix memory leak in LazyBuf

This affects many operations on variable expressions.  Those using
LazyBuf_Done are affected, those using LazyBuf_DoneGet aren't.

$ cat <<'EOF' > lazybuf-memleak.mk
.for i in ${:U:range=5}
.  for j in ${:U:range=1000}
.    for k in ${:U:range=1000}
.      if 0 && ${VAR:Dpattern\: that needs unescaping}
.      endif
.    endfor
.  endfor
.endfor

all:
	@ps -o vsz -p ${.MAKE.PID} | sed 1d
EOF

Before using LazyBuf for the modifier ':D':

    $ make-2021.04.14.16.12.26 -r -f lazybuf-memleak.mk
        VSZ     RSZ
    1357136 1336432

Using LazyBuf for the modifier ':D':

    $ make-2021.04.14.16.59.34 -r -f lazybuf-memleak.mk
        VSZ     RSZ
    1590864 1574164

This commit alone allocates around 150 MB more data, which matches
5_000_000 strings * 30 bytes/string.

It looks very wrong that the above simple makefile uses 1.3 GB of RAM at
all, and it had already done this in 2017, long before LazyBuf was
introduced.  Before 2017.01.30.02.46.20, the above test makefile reports
way smaller numbers, but that's because the modifier ':range' did not
exist back then.

Revision 1.11 / (download) - annotate - [select for diffs], Sun Dec 5 12:17:49 2021 UTC (2 years, 3 months ago) by rillig
Branch: MAIN
Changes since 1.10: +9 -1 lines
Diff to previous 1.10 (colored) to selected 1.1 (colored)

make: save a memory allocation in each modifier ':O' and ':u'

No functional change.

Revision 1.10 / (download) - annotate - [select for diffs], Sun Dec 5 11:40:03 2021 UTC (2 years, 3 months ago) by rillig
Branch: MAIN
Changes since 1.9: +9 -1 lines
Diff to previous 1.9 (colored) to selected 1.1 (colored)

make: inline Str_Words into .for loop handling

This saves one memory allocation and a bit of copying, per .for loop.

No functional change.

Revision 1.8.2.1 / (download) - annotate - [select for diffs], Mon May 31 22:15:25 2021 UTC (2 years, 9 months ago) by cjep
Branch: cjep_staticlib_x
Changes since 1.8: +1 -2 lines
Diff to previous 1.8 (colored) next main 1.9 (colored) to selected 1.1 (colored)

sync with head

Revision 1.9 / (download) - annotate - [select for diffs], Sun May 30 21:16:54 2021 UTC (2 years, 9 months ago) by rillig
Branch: MAIN
CVS Tags: cjep_sun2x-base1, cjep_sun2x-base, cjep_sun2x, cjep_staticlib_x-base1
Changes since 1.8: +1 -2 lines
Diff to previous 1.8 (colored) to selected 1.1 (colored)

make: inline str_concat4

This function is only ever used for forming strings of the form
"archive(member)".

No functional change.

Revision 1.8 / (download) - annotate - [select for diffs], Wed Apr 14 17:39:11 2021 UTC (2 years, 11 months ago) by rillig
Branch: MAIN
CVS Tags: cjep_staticlib_x-base
Branch point for: cjep_staticlib_x
Changes since 1.7: +18 -18 lines
Diff to previous 1.7 (colored) to selected 1.1 (colored)

make: let the compiler decide whether to inline string functions

On x86_64, this reduces the binary size by 2 kB.

Revision 1.7 / (download) - annotate - [select for diffs], Wed Apr 14 16:59:34 2021 UTC (2 years, 11 months ago) by rillig
Branch: MAIN
Changes since 1.6: +4 -1 lines
Diff to previous 1.6 (colored) to selected 1.1 (colored)

make: reduce memory allocations in the modifiers ':D' and ':U'

Revision 1.6 / (download) - annotate - [select for diffs], Mon Apr 12 18:48:00 2021 UTC (2 years, 11 months ago) by rillig
Branch: MAIN
Changes since 1.5: +27 -1 lines
Diff to previous 1.5 (colored) to selected 1.1 (colored)

make: reduce memory allocation and strlen calls in modifier ':from=to'

Previously, SysVMatch was quite verbose and felt like hand-optimized
assembler code, which made it difficult to discover the underlying idea
of the code.

All this code was replaced with two simple calls to Substring_HasPrefix
and Substring_HasSuffix.  Now that the operands of that modifier are no
longer passed as C strings, there is no need to collect all information
in a single scan through the word and the pattern.

It was not necessary to call Var_Subst unconditionally.  Calling it only
when the string contains a '$' saves another memory allocation and two
string copies (because of the Buf_DoneDataCompact).

No functional change.

Revision 1.5 / (download) - annotate - [select for diffs], Sun Apr 11 22:53:45 2021 UTC (2 years, 11 months ago) by rillig
Branch: MAIN
Changes since 1.4: +6 -8 lines
Diff to previous 1.4 (colored) to selected 1.1 (colored)

make: improve performance for LazyBuf

The previous O(n^2) time complexity for parsing a long string with many
variable expressions was not meant to last for long.  I had hoped to fix
it within a few minutes, but that will take more time.

For now, make LazyBuf simpler by using a traditional C string for the
expected part instead of a Substring.  This avoids a strlen call per
Var_Parse.

No functional change, only performance.

Revision 1.4 / (download) - annotate - [select for diffs], Sun Apr 11 20:38:43 2021 UTC (2 years, 11 months ago) by rillig
Branch: MAIN
Changes since 1.3: +17 -2 lines
Diff to previous 1.3 (colored) to selected 1.1 (colored)

make: migrate ParseModifierPart to use Substring

This will reduce memory allocation for modifier parts without the escape
characters '$' or '\'.

No functional change.

Revision 1.3 / (download) - annotate - [select for diffs], Sun Apr 11 19:05:06 2021 UTC (2 years, 11 months ago) by rillig
Branch: MAIN
Changes since 1.2: +18 -1 lines
Diff to previous 1.2 (colored) to selected 1.1 (colored)

make: avoid unnecessary calls to strlen when evaluating modifiers

No functional change.

Revision 1.2 / (download) - annotate - [select for diffs], Sun Apr 11 18:44:57 2021 UTC (2 years, 11 months ago) by rillig
Branch: MAIN
Changes since 1.1: +40 -1 lines
Diff to previous 1.1 (colored)

make: migrate ModifyWord functions to use Substring

This benefits the modifiers ':T' and ':H' since these scan the word from
the end.  The SysV modifier '.c=.o' does not benefit yet, this will be
done in a follow-up commit.

Currently ModifyWords calls strlen for each single word, which degrades
performance.  This will be cleaned up in a follow-up commit as well.

No functional change.

Revision 1.1 / (download) - annotate - [selected], Sun Apr 11 12:06:53 2021 UTC (2 years, 11 months ago) by rillig
Branch: MAIN

make: add types Substring and LazyBuf

These will be used for making the string handling more efficient,
avoiding allocations, especially when evaluating variable expressions.

Since the string handling has grown quite a bit in the last months,
extract it into its own header file.

No functional change.

This form allows you to request diff's between any two revisions of a file. You may select a symbolic revision name using the selection box or you may type in a numeric name using the type-in text box.




CVSweb <webmaster@jp.NetBSD.org>