Your IP : 216.73.216.48
2016-04-21 Jim Meyering <meyering@fb.com>
version 2.25
* NEWS: Record release date.
2016-04-19 Paul Eggert <eggert@cs.ucla.edu>
dfa: remove dependency on btowc
MirOS BSD btowc is a macro that (when GCC is being used) hardcodes
btowc (0x80) == WEOF regardless of locale, which contradicts
future POSIX in the C locale. Instead of bothering to develop a
Gnulib workaround for the btowc incompatibility, use mbrtowc,
which we are using elsewhere and fixing anyway, and are caching so
it is fast here. Problem reported by Nelson H. F. Beebe via Jim
Meyering in: http://bugs.gnu.org/23269#14
* bootstrap.conf (gnulib_modules): Remove btowc.
* src/dfa.c (struct dfa): Remove mbrtowc_cache member, replacing with ...
(mbrtowc_cache): ... this new static var. All uses changed.
(dfambcache): Remove; now done by setsyntax. Call removed.
(is_valid_unibyte_character): Remove.
(IS_WORD_CONSTITUENT): Remove this macro, replacing it with ...
(unibyte_word_constituent): ... this new function. It uses
mbrtowc_cache rather than btowc.
(dfasyntax): Initialize mbrtowc_cache before using it.
2016-04-10 Paul Eggert <eggert@cs.ucla.edu>
grep: minor doc tweaks inspired by Debian
Problem reported by Santiago Ruano Rincón in: http://bugs.gnu.org/22911
* doc/grep.in.1:
* doc/grep.texi (Matching Control, grep Programs)
(Regular Expressions):
Document -e, -f, and PCRE more carefully.
2016-04-10 Jim Meyering <meyering@fb.com>
maint: remove unused mbtoupper function
* src/searchutils.c (mbtoupper): Remove now-unused function.
Also remove inclusion of <assert.h>, since this change removed
the final use of assert.
* src/search.h (mbtoupper): Remove declaration.
2016-04-10 Paul Eggert <eggert@cs.ucla.edu>
grep: in C locale, all bytes are valid characters
This works around glibc bug 19932:
https://sourceware.org/bugzilla/show_bug.cgi?id=19932
The actual bug fix was the update to the current version of Gnulib.
grep problem reported by Björn Jacke in: http://bugs.gnu.org/23234
* NEWS: Mention this.
* doc/grep.texi (File and Directory Selection): Crossref to LC_*
section. Suggest why -a or LC_ALL=C might be useful.
(Environment Variables): Mention 'locale -a'.
Say that LC_CTYPE also specifies encoding, and that every
byte is a valid character in the C or POSIX locale.
* tests/c-locale: New test.
* tests/Makefile.am (TESTS): Add it.
build: update gnulib submodule to latest
2016-04-05 Paul Eggert <eggert@cs.ucla.edu>
Give another example of binary file processing
Problem reported by Shlomi Fish
* doc/grep.texi (File and Directory Selection):
Document that 'q$' might match 'q' followed by a NUL
if --binary-files=binary is in effect.
2016-04-03 Paul Eggert <eggert@cs.ucla.edu>
tests: test egrep/fgrep help only if our grep
Problem reported by Christian Weisgerber in: http://bugs.gnu.org/23146
* tests/Makefile.am (TESTS_ENVIRONMENT):
Test egrep and fgrep only if they use our grep.
2016-03-29 Jim Meyering <meyering@fb.com>
tests: remove spurious test of egrep
* tests/reversed-range-endpoints: Do not test egrep here.
There is already a test of grep -E.
Prompted by http://bugs.gnu.org/23146
2016-03-23 Paul Eggert <eggert@cs.ucla.edu>
grep: -Pz no longer misdiagnoses [^a]
Problem reported by Michael Jess.
* NEWS: Document this.
* src/pcresearch.c (Pcompile): Do not diagnose [^ when [ is unescaped.
* tests/pcre: Test for the bug.
2016-03-22 Jim Meyering <meyering@fb.com>
maint: move new 'Improvements' blurb into proper section
* NEWS (Improvements): Move this new section from within the block
for the already-released 2.24 into the proper "next-release" block.
Also, retain the 2-blank-line separator between blocks.
2016-03-18 Jim Meyering <meyering@fb.com>
maint: avoid spurious "binary file ... matches" in generated THANKS
* Makefile.am (THANKS): Don't apply grep to a stream containing
NUL bytes. Sync this rule from the one in coreutils: it was missing
some improvements.
Reported by Bailes Magio in http://bugs.gnu.org/22899
2016-03-18 Paul Eggert <eggert@cs.ucla.edu>
grep: -oz now outputs null bytes, not newlines
* NEWS: Document this.
* doc/grep.texi (Other Options): Clarify that -z affects output
as well as input data.
* src/grep.c (print_line_middle): Output eolbyte, not newline, if -o.
* tests/null-byte: Test -o too.
* tests/pcre-context: Adjust test to match new behavior.
2016-03-17 Paul Eggert <eggert@cs.ucla.edu>
grep: use errno consistently in write diagnostics
Feature request and initial version reported by Assaf Gordon in:
http://bugs.gnu.org/23031
* NEWS: Document this.
* src/grep.c: Include <stdarg.h>.
(stdout_errno): New static var.
(write_error_seen): Remove; superseded by stdout_errno.
All uses changed.
(putchar_errno, fputs_errno, printf_errno, fwrite_errno)
(fflush_errno): New static functions.
(print_filename, print_sep, print_offset, print_line_head)
(print_line_middle, print_line_tail, prline, prtext, grep)
(grepdesc): Use them.
* tests/write-error-msg: New file.
* tests/Makefile.am (TESTS): Add it.
2016-03-10 Jim Meyering <meyering@fb.com>
maint: post-release administrivia
* NEWS: Add header line for next release.
* .prev-version: Record previous version.
* cfg.mk (old_NEWS_hash): Auto-update.
version 2.24
* NEWS: Record release date.
2016-02-28 Jim Meyering <meyering@fb.com>
maint: add dist-check.mk
This file augments "make distcheck" rules.
* dist-check.mk: New file, from coreutils via gzip.
* Makefile.am (EXTRA_DIST): Add it.
* cfg.mk: Include it.
2016-02-21 Paul Eggert <eggert@cs.ucla.edu>
grep: -Pz is incompatible with ^ and $
Problem reported by Sergei Trofimovich in: http://bugs.gnu.org/22655
* NEWS: Document this.
* src/pcresearch.c (Pcompile): Warn with -Pz and anchors.
* tests/pcre: Test new behavior.
2016-02-21 Jim Meyering <meyering@fb.com>
tests: test cleanup
* tests/z-anchor-newline: Remove test artifact that would write
to /t/x.
2016-02-20 Jim Meyering <meyering@fb.com>
grep -z: avoid erroneous match with regexp anchor and \n in text
* src/dfasearch.c (EGexecute): Clear the newline_anchor bit when
eolbyte is not '\n'.
* tests/z-anchor-newline: New file.
* tests/Makefile.am (TESTS): Add it.
* NEWS (Bug fixes): Describe it.
Originally reported by Ulrich Mueller in
https://bugs.gentoo.org/show_bug.cgi?id=574662
Reported to us by Sergei Trofimovich as http://debbugs.gnu.org/22655
tests: convert "cmd && fail=1" to "returns_ 1 cmd || fail=1"
The latter is robust, while the former can silently ignore
failure due to signals.
* cfg.mk (sc_prohibit_and_fail_1): New rule, copied from coreutils.
* tests/long-pattern-perf: Perform the above substitution.
* tests/mb-non-UTF8-performance: Likewise.
* tests/help-version: Merge from coreutils.
2016-02-09 Jim Meyering <meyering@fb.com>
maint: add a check-very-expensive target
* Makefile.am (check-very-expensive): New convenience rule,
currently merely equivalent to check-expensive.
2016-02-04 Jim Meyering <meyering@fb.com>
maint: post-release administrivia
* NEWS: Add header line for next release.
* .prev-version: Record previous version.
* cfg.mk (old_NEWS_hash): Auto-update.
version 2.23
* NEWS: Record release date.
2016-02-02 Jim Meyering <meyering@fb.com>
gnulib: update to latest
Update for this "make distcheck"-fixing change:
> verify-tests: also remove stray test-verify.Tpo
2016-02-01 Jim Meyering <meyering@fb.com>
tests/null-byte: test another code path
* tests/null-byte: Also exercise the case in which there is
a match in the block along with the NUL byte.
2016-01-31 Paul Eggert <eggert@cs.ucla.edu>
Omit excess "Binary file ... matches"
Problem reported in: http://bugs.gnu.org/22461
* src/grep.c (grep): Don't report "Binary file ... matches"
merely because the file contained both matches and binary data.
Insist that the binary data contained a match.
* tests/null-byte: Add a test for this.
2016-01-28 Jim Meyering <meyering@fb.com>
gnulib: update to latest
2016-01-23 Jim Meyering <meyering@fb.com>
gnulib: update to latest
maint: fix typo in NEWS: s/a/an/
2016-01-15 Paul Eggert <eggert@cs.ucla.edu>
grep: -x now supersedes -w more consistently
* NEWS, doc/grep.texi (Matching Control): Mention this.
* src/dfasearch.c (EGexecute):
* src/pcresearch.c (Pcompile):
Don't get confused by -w if -x is also present.
* src/pcresearch.c (Pcompile): Remove misleading comment about
non-UTF-8 multibyte locales, as PCRE doesn't support them.
Calculate buffer sizes more carefully; the old method
allocated a buffer slightly too big, seemingly due to luck.
* tests/backref-word, tests/pcre: Add tests for this bug.
tests: omit update-copyright-tests
This test does not check how 'grep' itself operates, so it is
out of place for grep's 'make check'. Problem reported by Sam Razavi in:
http://bugs.gnu.org/22376
* bootstrap.conf (avoided_gnulib_modules): Add update-copyright-tests.
2016-01-11 Jim Meyering <meyering@fb.com>
tests: do use "yes" but via an AWK replacement
Also, use sed Nq in place of head -N
* tests/init.cfg (yes): Define.
Thanks to Paul Eggert for this definition.
* tests/max-count-overread: Revert to using "yes".
* tests/mb-non-UTF8-performance: Likewise, and use
"sed Nq" in place of head -N.
2016-01-11 Paul Eggert <eggert@cs.ucla.edu>
* tests/pcre-count: Don't assume the page size is 32kB.
2016-01-08 Paul Eggert <eggert@cs.ucla.edu>
tests: port to other POSIXish platforms
I tested this on Solaris 10 and AIX 7.1.
* tests/max-count-overread:
* tests/mb-non-UTF8-performance:
Don't assume 'yes' exists, as 'yes' is not in POSIX.
* tests/mb-non-UTF8-performance:
Don't rely on 'head -1000', as that option syntax is not POSIX.
* tests/pcre-count: Don't rely on "printf '\x0'".
* tests/unibyte-binary: Don't assume \200 is an encoding error
in every unibyte locale.
2016-01-08 Jim Meyering <meyering@fb.com>
tests: fix encoding-error test failure to use of printf '\xHH'
* tests/encoding-error: Don't rely on printf having support for \xHH
hexadecimal. That is not portable. Use \OOO octal, instead.
maint: fix typo in NEWS: s/a/an/
2016-01-07 Jim Meyering <meyering@fb.com>
mb-non-UTF8-performance: avoid FP test failure on fast hardware
* tests/mb-non-UTF8-performance: Don't use a fixed size.
Otherwise, on a fast system, the fixed-size unibyte test
would complete in a nominal 0 ms, which might well be
smaller than 1/30 of the multibyte duration, provoking
a false positive test failure. Instead, increase the
size of the input until we obtain a unibyte duration of
at least 10ms.
2016-01-07 Paul Eggert <eggert@cs.ucla.edu>
doc: mention unibyte encoding fix
* NEWS: Document recent fix for encoding errors in unibyte locales.
grep: improve unibyte -P performance
This is a followon to the recent changes prompted by Bug#20526.
In <http://bugs.gnu.org/bug=20526#86> Norihiro Tanaka pointed out
that grep mistakenly assumed that unibyte locales cannot have
encoding errors. Here, the mistake hurt performance significantly.
On Fedora 23 x86-64 in the C locale, this patch improved grep's
performance by a factor of 7 when run as "grep -P 'z.*a'" on the
output of "yes $(printf '\200\n') | head -n 1000000000".
* src/pcresearch.c (multibyte_locale) [HAVE_LIBPCRE]: New static var.
(Pcompile): Set it.
(Pexecute): Use it to avoid the need to call
buf_has_encoding_errors in unibyte locales.
2016-01-06 Paul Eggert <eggert@cs.ucla.edu>
Improve on fix for Bug#22181
* src/pcresearch.c (Pexecute): Update subject when skipping past
easily-determined encoding errors, as this is faster than letting
pcre_exec skip them. On my platform this improves performance
4.7x on a benchmark created via "yes $(printf '\200\200\200\200
\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200x\n')
| head -n 1000000 >j; grep -oP y j" in a UTF-8 locale. Rework
code that deals with PCRE_ERROR_BADUTF8 return, to avoid an
incorrect (albeit currently harmless) 'bol = false' assignment.
grep: restore -P optimization (followup fix)
* src/search.h (EGexecute, Fexecute, Pexecute):
Change decls to match new implementations.
I forgot to add this file to the previous commit.
grep: restore -P PCRE_NO_UTF8_CHECK optimization
On my platform in the en_US.utf8 locale, this makes 'grep -P "z.*a" k'
220x faster, where k is created by the shell command:
yes 'abcdefg hijklmn opqrstu vwxyz' | head -n 10000000 >k
* src/dfasearch.c (EGexecute):
* src/grep.c (execute_fp_t):
* src/kwsearch.c (Fexecute):
* src/pcresearch.c (Pexecute):
First arg is now char *, not char const *, since Pexecute now
temporarily modifies this argument.
* src/grep.c, src/grep.h (buf_has_encoding_errors): Now extern.
* src/pcresearch.c (Pexecute): Use it. If the input is free of
encoding errors, use a multiline search and the PCRE_NO_UTF8_CHECK
option, as this is typically way faster. This restores an
optimization that was removed with the recent changes for binary
file detection.
2016-01-05 Paul Eggert <eggert@cs.ucla.edu>
Fix calculation of unibyte_mask
* src/grep.c (initialize_unibyte_mask): The old method worked for
UTF-8 and other typical encodings, but did not work for weird
encodings, e.g., one where all bytes other than 0x7f and 0x80 are
unibyte characters.
2016-01-01 Paul Eggert <eggert@cs.ucla.edu>
grep: fix bug with with invalid unibyte sequence
This was introduced by the recent binary-data-detection changes.
Problem reported by Norihiro Tanaka in: http://bugs.gnu.org/20526#86
* src/grep.c (HIBYTE, easy_encoding, init_easy_encoding): Remove,
replacing with ...
(uword_max, unibyte_mask, initialize_unibyte_mask): ... this new
constant, static var, and function. All uses changed. The
unibyte_mask var generalizes the old local var hibyte_mask, which
worked only for encodings where every byte with 0x80 turned off is
a single-byte character.
(buf_has_encoding_errors): Return false immediately if
unibyte_mask is zero, not whether the current encoding is unibyte.
The old test was incorrect in unibyte locales in which some bytes
were encoding errors.
* tests/pcre-z: Require UTF-8 locale, since the grep -z . test now
needs this. Use printf \0 rather than tr. Port the 'grep -z .'
test to platforms where the C locale says '\200' is an encoding
error. Use cmp rather than compare, as the file is binary and
so non-GNU diff might not work.
* tests/unibyte-binary: New file.
* tests/Makefile.am (TESTS): Add it.
2016-01-01 Jim Meyering <meyering@fb.com>
maint: update copyright year, bootstrap, init.sh
Run "make update-copyright" and then...
* gnulib: Update to latest.
* tests/init.sh: Update from gnulib.
* bootstrap: Likewise.
2015-12-31 Paul Eggert <eggert@cs.ucla.edu>
doc: clarify text vs binary match output
* NEWS:
* doc/grep.texi (File and Directory Selection):
Make it clearer that grep can now output matching text before
reporting a binary match. Problem reported by Norihiro Tanaka in:
http://bugs.gnu.org/20526#83
doc: minor clarifications
* doc/grep.in.1, doc/grep.texi: Minor clarifications suggested by
Debian documentation patches. Problem reported by Santiago Ruano
Rincón in: http://bugs.gnu.org/18651
grep: fix -l --line-buffer bug
Problem reported by Louis Sautier in: http://bugs.gnu.org/18750
* NEWS: Document this.
* src/grep.c (grep, grepdesc): If --line-buffered, flush
stdout after outputting newline (or null byte, if applicable).
2015-12-30 Paul Eggert <eggert@cs.ucla.edu>
grep: remove duplicate init
* src/grep.c (print_line_middle): Remove duplicate initialization.
grep: report line-buffered write error right away
* src/grep.c (prline): When line buffered, if there is a write
error, report it immediately rather than waiting until the next
line of output.
grep: -c should keep counting after binary data
Problem and fix reported by Jaroslav Škarvada, and test case
reported by Norihiro Tanaka, in: http://bugs.gnu.org/22028
* NEWS: Document this.
* src/grep.c (grep): Don't stop counting merely because nulls seen.
* tests/pcre-count: New file.
* tests/Makefile.am (TESTS): Add it.
dfa: port to tinycc
* src/dfa.c (add_utf8_anychar): Put 'const' after type.
Problem reported by Aharon Robbins in:
http://bugs.gnu.org/22260
grep: be less picky about encoding errors
This fixes a longstanding problem introduced in grep 2.21,
which is overly picky about binary files.
* NEWS:
* doc/grep.texi (File and Directory Selection): Document this.
* src/grep.c (input_textbin, textbin_is_binary, buffer_textbin)
(file_textbin):
Remove. All uses removed.
(encoding_error_output): New static var.
(buf_has_encoding_errors, buf_has_nulls, file_must_have_nulls):
New functions, which reuse bits
and pieces of the removed functions.
(lastout, print_line_head, print_line_middle, print_line_tail, prline)
(prpending, prtext, grepbuf):
Avoid use of const, now that we have
functions that require modifying a sentinel.
(print_line_head): New arg LEN. All uses changed.
(print_line_head, print_line_tail):
Return indicator whether the output line was printed.
All uses changed.
(print_line_middle): Exit early on encoding error.
(grep): Use new method for determining whether file is binary.
* src/grep.h (enum textbin, TEXTBIN_BINARY, TEXTBIN_UNKNOWN)
(TEXTBIN_TEXT, input_textbin): Remove decls. All uses removed.
* src/pcresearch.c (Pexecute): Remove multiline optimization,
since the main program no longer checks for encoding errors on input.
* tests/encoding-error: New file.
* tests/Makefile.am (TESTS): Add it.
2015-12-29 Jim Meyering <meyering@fb.com>
maint: correct (make sorted) order of test file names
* tests/Makefile.am (TESTS): Insert new test name in sorted order.
2015-12-28 Paul Eggert <eggert@cs.ucla.edu>
grep: --exclude matches trailing parts of args
Problem reported by Vincent Lefevre in:
http://bugs.gnu.org/22144
* NEWS:
* doc/grep.texi (File and Directory Selection): Document this.
* src/grep.c (excluded_patterns, excluded_directory_patterns):
Now 2-element arrays, with one element for subfiles and another
for command-line args. All uses changed. This implements the change.
(exclude_options): New function.
* tests/include-exclude: Test the change.
2015-12-18 Jim Meyering <meyering@fb.com>
grep -oP: don't infloop when processing invalid UTF8 preceding a match
* src/pcresearch.c (Pexecute): When advancing SUBJECT past an
encoding error, don't blindly set P to that new value, since we
will soon compute SEARCH_OFFSET = P - SUBJECT, and mistakenly
making that difference too small would allow us to match some
previously-processed text, resulting in an infinite loop.
* NEWS (Bug fixes): Mention it.
* THANKS.in: Add Christian's name and email address.
* tests/pcre-invalid-utf8-infloop: New file.
* tests/Makefile.am (TESTS): Add it.
Reported by Christian Boltz in http://debbugs.gnu.org/22181
Introduced by commit, v2.21-37-g14f8e48.
2015-11-04 Jim Meyering <meyering@fb.com>
tests: mark performance-related tests as expensive
These performance-related tests are slightly failure prone due to
varying system load during the two runs.
Marking these tests as "expensive" makes it so they are no longer run
via "make check". You can still run them via make "check-expensive".
This makes them less likely to be run by regular users.
* tests/long-pattern-perf: Use expensive_.
* tests/mb-non-UTF8-performance: Likewise.
Reported by Jaroslav Skarvada in http://debbugs.gnu.org/21826
and by Andreas Schwab in http://debbugs.gnu.org/21812.
2015-11-01 Jim Meyering <meyering@fb.com>
maint: post-release administrivia
* NEWS: Add header line for next release.
* .prev-version: Record previous version.
* cfg.mk (old_NEWS_hash): Auto-update.
version 2.22
* NEWS: Record release date.
tests: pcre-jitstack: upon failure, retry with no stack size limit
* tests/pcre-jitstack: Don't let an example that provokes inordinate
stack space use cause a test failure. Thanks to reports from and
analysis by Bruce Dubbs; see http://debbugs.gnu.org/21755
2015-10-27 Jim Meyering <meyering@fb.com>
maint: update THANKS.in
* THANKS.in: Add name+email of those who found and reported
the bug that made grep -E '^x|x$' match any "x".
2015-10-25 Zev Weiss <zev@bewilderbeest.net>
dfa: plug a memory leak in dfamust
* src/dfa.c (dfamust): Ensure MP is freed, by refraining
from returning early when, at "done:" *RESULT is NULL.
2015-10-25 Jim Meyering <meyering@fb.com>
gnulib: update to latest
* gnulib: Pull in one more portability fix:
stdalign: port to Sun C 5.9
2015-10-24 Jim Meyering <meyering@fb.com>
gnulib: update to latest, for portability fixes
* gnulib: Pull in changes like these:
fts: port to C11 alignof
stdalign: work around pre-4.9 GCC x86 bug
maint: NEWS: correct/amend
* NEWS: Move the long-regexp-performance-improvement from
"Bug fixes" to "Improvements." Say more and include an example.
The -Fw degradation was introduced in commit v2.18-125-g94555dd
tests: avoid spurious failure on OpenBSD 5.8
* tests/fedora: Don't rely on "diff - FILE" reading from stdin.
Reported privately by Nelson Beebe.
2015-10-17 Jim Meyering <meyering@fb.com>
gnulib: update to latest; also bootstrap and tests/init.sh
* bootstrap: Update from gnulib.
* tests/init.sh: Likewise.
* gnulib: Update submodule to latest.
build: avoid spurious bootstrap failure involving pkg.m4
Running ./bootstrap could fail mistakenly at the very end in
its attempt to obtain a copy of pkg.m4. It would search only
$(aclocal --print-ac-dir) and some other directories, but not
those listed in $(aclocal --print-ac-dir)/dirlist.
* bootstrap.conf (bootstrap_post_import_hook): Also search the
directories named in $(aclocal --print-ac-dir)/dirlist when that
file exists with nonzero size.
2015-10-16 Paul Eggert <eggert@cs.ucla.edu>
maint: add news item
* NEWS: Document grep -Fw speedup.
grep: simplify previous change
* src/grep.c (main): Simplify recently-changed grep -Fw test.
2015-10-16 Norihiro Tanaka <noritnk@kcn.ne.jp>
grep: use grep matcher for grep -Fw when unibyte
In single byte locales with grep -Fw, prefer the grep matcher to the
kwset matcher, as the former uses KWset and a DFA, whereas the latter
calls kwsexec many times until it matches a word.
* src/grep.c (main): Change pattern for fgrep into grep for grep -Fw in
single byte locales.
2015-10-16 Paul Eggert <eggert@cs.ucla.edu>
grep: use memchr/memrchar
* src/kwsearch.c (Fexecute): Prefer memchr and memrchr to doing it
by hand.
2015-10-16 Norihiro Tanaka <noritnk@kcn.ne.jp>
grep: improve performance of grep -Fw
* src/kwsearch.c (Fexecute): grep -Fw examined whether the previous
character is a word character after matching from the head of the
buffer. It is extremely slow. Now, if grep found a potential match,
it looks for the previous newline, and examines from there.
2015-10-13 Jim Meyering <meyering@fb.com>
maint: use single quote rather than UTF-8 multi-byte version
* tests/backref-alt: Translate unnecessary non-ASCII in comment.
2015-10-13 Paul Eggert <eggert@cs.ucla.edu>
dfa: make the executable a bit smaller
* src/dfa.c (dfamust): Hoist MB_CUR_MAX calculation out of loops.
2015-10-13 Norihiro Tanaka <noritnk@kcn.ne.jp>
dfa: fix bug in alternate of sub-patterns that differ only in constraints
Fix a bug where a line incorrectly matches alternates of sub-patterns
that differ only in the constraints, e.g., the ERE '^a|a$'.
Reported by Greg Boyd in: http://debbugs.gnu.org/21670
* src/dfa.c (dfamust): For a pattern with constraints, check that it is
matched including the constraints, to judge whether it is exact.
dfa: fix off-by-one error
* src/dfa.c (dfamust): Fix off-by-one error in computing 'must' length,
which caused the 'must' to be too short. See:
http://bugs.gnu.org/21670#28
2015-10-12 Jim Meyering <meyering@fb.com>
doc: NEWS: mention a bug fix
* NEWS (Bug fixes): Describe it.
This bug was introduced by commit v2.18-85-g2c94326
and fixed by commit v2.21-51-g256a4b4.
2015-10-11 Paul Eggert <eggert@cs.ucla.edu>
tests: add test case for Bug#21670
* tests/options: Add test #4 to catch Bug#21670.
Also, do not overescape # in shell strings.
2015-09-19 Paul Eggert <eggert@cs.ucla.edu>
Add test for pop_fail_stack bug
Problem reported by Hanno Böck in: http://bugs.gnu.org/21513
If you use --with-included-regex the bug fix is in gnulib, here:
http://git.savannah.gnu.org/cgit/gnulib.git/commit/?id=5513b40999149090987a0341c018d05d3eea1272
If you use glibc, the bug fix has not been installed yet.
* tests/Makefile.am (XFAIL_TESTS): Add backref-alt if system matcher.
(TESTS): Add backref-alt.
* tests/backref-alt: New file.
* tests/triple-backref: Remove unused var.
Don't skip if tested with glibc, as Makefile.am now handles this.
build: update gnulib submodule to latest
2015-08-19 Norihiro Tanaka <noritnk@kcn.ne.jp>
grep: avoid use of uninitialized variable
EGexecute would use "backref" uninitialized.
While that could have no bearing on correctness, it could
impact performance, via an unnecessary use of regexp.
* src/dfasearch.c (EGexecute): Initialize backref.
Reported as http://debbugs.gnu.org/21273
Introduced by commit v2.21-55-gea0ebaa.
2015-08-12 Norihiro Tanaka <noritnk@kcn.ne.jp>
grep: remove fgrep code for case insensitive match
The fgrep matcher is no longer called in case insensitive matching,
so remove the code to support it.
* src/kwsearch.c (mb_case_map_apply): Remove function.
(Fexecute): Remove now-unused code.
2015-08-12 Paul Eggert <eggert@cs.ucla.edu>
dfa: optimize [x-x]
* src/dfa.c (parse_bracket_exp): Treat [x-x] as if it were [x].
This also pacifies GCC, which otherwise complains about wc2
being set but not used.
2015-08-12 Norihiro Tanaka <noritnk@kcn.ne.jp>
dfa: remove unused multibyte support
Now regex should be used for range, collating element, equivalent class
in non POSIX locales. So remove code to support these features.
* dfa.c (struct mb_char_classes): Remove members ch_classes,
nch_classes, ranges, nranges, equivs, nequivs, coll_elems, ncoll_elems.
All uses removed.
(match_mb_charset): Remove function.
2015-08-01 Jim Meyering <meyering@fb.com>
tests: mb-non-UTF8-performance: use new function
* tests/mb-non-UTF8-performance: Rewrite to use
the user-time measuring function in init.cfg.
tests: long-pattern-perf: measure user time, not elapsed
Measuring user time makes this test less prone to false
positive failure, and also lets us use a tighter bound.
* tests/long-pattern-perf: Measure elapsed user time rather than
wall-clock time, to permit a tighter bound on the ratio of
N-to-10N timings. Suggested by Giuseppe Ottaviano.
Also, use regexps built from mostly 5-digit numbers, so that the 10:1
ratio applies to lines of "seq" output as well as to total bytes.
tests: new function to measure elapsed user time
* tests/init.cfg (user_time_): New function.
2015-07-25 Norihiro Tanaka <noritnk@kcn.ne.jp>
dfa: remove word delimiter support for multibyte locales
DFA supports word delimiter expressions, but it does not behave
correctly for multibyte locales. Even if it were to be fixed,
the DFA matcher's performance would be no better than that of regex.
Thus, this change removes DFA support for word delimiter expressions
in multibyte locales.
* src/dfa.c (dfa_supported): Return false also when a pattern uses any
word delimiter expression in a multibyte locale.
2015-07-25 Norihiro Tanaka <noritnk@kcn.ne.jp>
dfa: avoid execution for a pattern including an unsupported expression
If a pattern includes a construct unsupported by the DFA matcher,
the DFA search would fail in most cases. Make dfaexec immediately
return for any such pattern.
* src/dfa.c (struct dfa_state) [has_backref, has_mbcset]: Remove members
and all uses.
(dfaexec_main): Remove 'backref' parameter. Update callers.
(dfaexec_noop): New function.
(dfa_supported): New function.
(dfassbuild): Remove now-unused code.
(dfacomp): When a pattern uses a DFA-unsupported construct, do not
waste time performing any further analysis.
2015-07-19 Norihiro Tanaka <noritnk@kcn.ne.jp>
dfa: DEBUG: print detail of DFA states
When compiled with -DDEBUG, grep outputs tokens etc.
With this change, also print DFA states and transitions.
This change is very useful when debugging those.
* src/dfa.c (prtok) [DEBUG]: Change `%c' to `%02x' in printf format.
(state_index) [DEBUG]: Print detail of new state.
(dfastate) [DEBUG]: Print detail of DFA states.
Reported as http://debbugs.gnu.org/18707
2015-07-18 Norihiro Tanaka <noritnk@kcn.ne.jp>
tests: sjis-mb: accept two more locales
* tests/sjis-mb: Accept the ja_JP.SJIS and ja_JP.PCK locales
as well as ja_JP.SHIFT_JIS, so this test is less likely to
be skipped unnecessarily. Reported as http://bugs.gnu.org/18983
2015-07-18 Jim Meyering <meyering@fb.com>
tests: add a test for the performance fix
* tests/long-pattern-perf: New file.
* tests/Makefile.am (TESTS): Add it.
2015-07-18 Norihiro Tanaka <noritnk@kcn.ne.jp>
dfa: speed up handling of long pattern
DFA tries to find a long sequence of characters that must appear
in any matching line. However, when a pattern is long (length N),
it is very slow, because it makes O(N^2) strstr calls.
This change reduces that to O(N) by processing each sequence of
adjacent "regular" characters as a group.
Compare the run times of this command before and after this change:
(on a i7-4770S CPU @ 3.10GHz using rawhide (~fedora 22) and compiled
with gcc 6.0.0 20150627)
: | env time -f %e grep -f <(seq -s '' 9999)
Before: 0.85
After: 0.02
* src/dfa.c (dfamust): Process each string of concatenated normal
characters as a unit.
* NEWS (Improvement): Mention it.
Prompted by a bug report and patch by Ivan Yanikov
in http://bugs.gnu.org/15191#5
2015-07-17 Jim Meyering <meyering@fb.com>
tests: fix mis-applied patch.
* tests/include-exclude: I applied "|sort" to the wrong creation
of "out", and didn't push the same patch that I'd tested.
tests: avoid FS-dependent false-positive failure
* tests/include-exclude: Sort file name list, so that this test
is not sensitive to the order in which those names are returned
via readdir. I noticed the failure on a Fedora 21 system using ext4.
Also fix a typo: s/framework_failure+/framework_failure_/
2015-07-13 Paul Eggert <eggert@cs.ucla.edu>
grep: fix bug with --exclude-dir and command line
Reported by Aron Griffis in: http://bugs.gnu.org/21027
* NEWS: Document this.
* src/grep.c (grepdirent): Don't check whether the file is skipped
when on the command line, as that's the caller's responsibility.
(main): Anchor the exclude patterns.
* tests/include-exclude: Adjust test case to match fixed behavior.
Add some more test cases.
tests: fix $? typo in null-byte
* tests/null-byte: Don't assume $? survives an invocation of 'test'.
2015-07-05 Jim Meyering <meyering@fb.com>
maint: dfa: used unsigned types where appropriate
* src/dfa.c (case_folded_counterparts): Return unsigned int, not int.
Change type of two locals to unsigned int, to reflect that their
values are never negative.
(parse_bracket_exp): Adjust type of result at each use, as well
as that of related index variables.
2015-07-04 Norihiro Tanaka <noritnk@kcn.ne.jp>
dfa: build struct dfamust on demand
If we won't use KWset, do not build a "struct dfamust".
Now it is built only when needed.
* src/dfa.c (struct dfa) [musts]: Remove member.
(dfacomp): Don't build dfamust here.
(dfamustfree): New function to free a struct dfamust.
(dfamust): Make it a global function, and make it return a pointer
to a malloc'd struct dfamust.
(dfamusts): Remove it.
* src/dfa.h (struct dfamust) [next]: Remove member.
In the implementation preceding this patch, there was
never more than one of these in a given "struct dfa".
(dfamustfree, dfamust): Add prototypes.
(dfamusts): Remove prototype.
(dfaalloc): Declare with _GL_ATTRIBUTE_MALLOC.
To make that symbol usable there, move the inclusion
of "xalloc.h" from dfa.c to this file, dfa.h.
* src/dfasearch.c (kwsmusts): Adapt to use the new interface.
Update the comments to reflect reality.
This addresses http://bugs.gnu.org/17715
2015-07-04 Paul Eggert <eggert@cs.ucla.edu>
grep: use recent gnulib syntax bits
* src/grep.c (Gcompile, Ecompile): Use plain RE_SYNTAX_GREP
and RE_SYNTAX_EGREP, now that we assume a recent-enough gnulib.
maint: ignore gendocs_template_min
* doc/.gitignore: Add '/gendocs_template_min'.
build: update gnulib submodule to latest
dfa: '.' and '[^x]' now consistently match newline
* src/dfa.c (parse_bracket_exp, lex, add_utf8_anychar)
(match_anychar): RE_DOT_NEWLINE and RE_HAT_LISTS_NOT_NEWLINE
are about LF, not about eolbyte. This patch does not affect
'grep', but may affect other users of dfa.c.
grep: -z '[^x]' now consistently matches newline
Problem reported by Norihiro Tanaka in: http://bugs.gnu.org/20974#19
* NEWS: Document this.
* src/grep.c (Gcompile, Ecompile): Clear RE_HAT_LISTS_NOT_NEWLINE.
* tests/utf8-bracket: Test this.
2015-07-03 Paul Eggert <eggert@cs.ucla.edu>
grep: -z '.' now consistently matches newline
Problem reported by Balazs Kezes in: http://bugs.gnu.org/20974
* NEWS: Document this.
* tests/utf8-bracket: New file, to test for this bug.
* src/grep.c (Gcompile, Ecompile): Also specify RE_DOT_NEWLINE.
* tests/Makefile.am (TESTS): Add it.
grep: simplify print_line_middle slightly
* src/grep.c (print_line_middle): Simplify.
grep: don't mishandle left context in -P
http://bugs.gnu.org/20957
* src/pcresearch.c (jit_exec): New arg SEARCH_OFFSET.
Caller changed.
(Pexecute): Pass the left context to pcre_exec, so that PCRE
regular-expression matching can see it.
* tests/pcre-context: New file, to test for this bug.
* tests/Makefile.am (TESTS): Add it.
2015-06-28 Jim Meyering <meyering@fb.com>
tests/case-fold-backref: factor test
2015-06-26 Paul Eggert <eggert@cs.ucla.edu>
grep: don't hang on command-line fifo if -D skip
* NEWS: Document this.
* src/grep.c (skip_devices):
New function, with code taken from grepdirent.
(grepdirent): Use it. Avoid an unnecessary initialization.
(grepfile): If skipping devices, open files with O_NONBLOCK.
Throw in O_NOCTTY while we're at it.
(grepdesc): Skip devices here, too. Not only does this fix the
bug, it fixes an unlikely race condition if some other process
renames a device between fstatat and openat.
* tests/skip-device: Add a test for this bug.
grep: minor tweaks
* src/grep.c (main): Change recently-added static vars to be
constants, which makes them sharable. Prefer 'return' to 'exit'
when returning/exiting from 'main'. Move decl closer to first use
and rename local from 'ok' (which was confusing) to 'status'.
Prefer named constant STDOUT_FILENO to unnamed constant 1.
2015-06-26 Jim Meyering <meyering@fb.com>
maint: unify three argv-processing calls
* src/grep.c (main): Unify three calls to grep_commandline_arg.
maint: alphabetize anonymous enum member names
2015-05-30 Paul Eggert <eggert@cs.ucla.edu>
test: tighten tests for bracket exprs
* tests/posix-bracket: Test '[a-a[.-.]--]'.
Also, test that failures are with status 1
(nonmatching data), not status 2 (invalid expressions).
2015-04-26 Jim Meyering <meyering@fb.com>
maint: update bootstrap from gnulib
* bootstrap: Update from gnulib.
maint: reword a diagnostic not to trigger leading capital check
* src/pcresearch.c: Reword diagnostic to avoid "make syntax-check"
failure.
maint: sort test names in tests/Makefile.am and add syntax-check rule
* cfg.mk (sc_sorted_tests): New rule.
* tests/Makefile.am (TESTS): Alphabetize.
2015-04-25 Norihiro Tanaka <noritnk@kcn.ne.jp>
dfa: make find_pred return NULL for an invalid predicate
This could never happen when invoked via grep, but could have triggered
a bug if dfa.c's find_pred function were invoked by some other program.
* src/dfa.c (find_pred): Return NULL for an invalid predicate.
* tests/invalid-char-class: New file to test for this.
* tests/Makefile.am (TESTS): Add that new file name to the list.
This addresses http://debbugs.gnu.org/18631
2015-04-06 Paul Eggert <eggert@cs.ucla.edu>
build: improve pkg-config doc and error handling
Error-handling improvement suggested by Mike Frysinger in:
http://bugs.gnu.org/16757#29
* NEWS: Document pkg-config changes.
* README-prereq: pkg-config is now a prereq when building from
repository.
* m4/pcre.m4 (gl_FUNC_PCRE): Report an error if pcre is explicitly
requested but not available. Defer to user-supplied PCRE_CFLAGS
and PCRE_LIBS.
build: remove typo and don't bother with /usr/include/pcre
Problem reported by Holger Bruenjes.
* m4/pcre.m4: Remove test for /usr/include/libpng (a typo).
Come to think of it, don't bother worrying about
/usr/include/pcre, as hosts with that problem can use pkg-config
or configure with CFLAGS by hand.
build: use pkg-config (if available) to configure libpcre
Problem reported by Mike Frysinger in: http://bugs.gnu.org/16757
* bootstrap.conf (bootstrap_post_import_hook):
Copy pkg-config's pkg.m4.
* configure.ac: Invoke PKG_PROG_PKG_CONFIG.
* m4/pcre.m4 (gl_FUNC_PCRE): Rewrite to use pkg-config if
available, and to test that pcre_compile can be linked to.
* src/Makefile.am (AM_CFLAGS): Add PCRE_CFLAGS.
(grep_LDADD): Add PCRE_LIBS.
* src/pcresearch.c: Simply include <pcre.h> if HAVE_LIBPCRE,
since 'configure' arranges for the appropriate -I option now.
2015-03-11 Paul Eggert <eggert@cs.ucla.edu>
grep: output "." file name in diagnostic
This is bug C as reported by David Grayson in:
http://bugs.gnu.org/16444#18
This bug occurs only in obscure circumstances, and I didn't see
how to write a reasonable test case for it.
* src/grep.c (filename_prefix_len): Remove, replacing with ...
(omit_dot_slash): New static var. All uses of the former replaced
with uses of the latter.
(grepdirent): Don't add 2 if the filename is just ".".
egrep, fgrep: just use what's in PATH
* src/egrep.sh: Don't monkey with PATH; just use whatever 'grep'
is in the path. This is simpler, and lets the user specify
default options with a script for only grep, with no need for
egrep and fgrep scripts.
Fixes: bug#19998
doc: give a script wrapper example
* doc/grep.texi (Environment Variables): Give an example of a