Up to [cvs.NetBSD.org] / pkgsrc / devel / ruby-parser
Request diff between arbitrary revisions
Keyword substitution: kv
Default branch: MAIN
devel/ruby-parser: update to 3.22.0 3.22.0 (2025-12-20) 1 major enhancement: * Announced the EOL of ruby_parser! YAY! 6 minor enhancements: * 3.3: working down the grammar structural differences. * Added tentative 3.4 support. * Renamed assignable to old_assignable and added newer cleaner assignable * Restructured comparison rake tasks and normalization to deal with lrama changes. * Updated compare versions and added 3.4. * Updated grammars to fit most closely to ruby 3.2. Working it down the line. 11 bug fixes: * Always raise RubyParser::SyntaxError (wrap and raise). * Bumped ruby version to 3.2+. * Cleaned up user_variable productions to always return sexps. * Corrections and enhancements for gauntlet.md instructions. * Fixed rake compare's clean subtask to nuke the unpacked ruby dirs entirely. * Fixed superclass of V34. * Fixed up a couple errors created by using STRICT_SEXP=2. 3 will be neigh impossible. * I am nothing if not stubborn: fixed errors caused by STRICT_SEXP=3. * Minor fixes for 3.2+ grammars on “defined?� expressions. * Removed a lot of ancient compatibility checks and patches. (eg encodings) * This in turn cleaned up var_ref and a bunch of other productions.
devel/ruby-parser: update to 3.21.0 3.21.0 (2024-01-15) 18 minor enhancements: * (internal) Added simple RubyLexer#comment. * (internal) Removed RubyLexer#comments and #ignore_body_comments. * (internal) Removed RubyParser#comments. * (internal) Renamed files from lib/ruby##_parser.* to lib/ruby_parser##.* * (internal) Renamed lib/ruby3_parser.yy to lib/ruby_parser3.yy * (internal) Renamed lib/ruby_parser.yy to lib/ruby_parser2.yy * Added tentative 3.2 support. (torrocus) * Made `rake compare` a multitask (28s -> 10s!!). * Made `rake compare` depend on `generate` (from clean: 28s -> 18s). * No longer package versioned .y files. * Overhaul of parser/lexer comment processing. * Parser productions for k_class/k_module/k_def carry their comment with them. * Restructured build tasks to utilize Hoe's .yy support! 8.5s -> 1.2s!!! * RubyParser new_(class|module|defn|defs) handle comment processing via sexp. * RubyParser new_(class|module|defn|defs) handle normalizing production sexps. * RubyParser#next_token clears out lexer's comment when applicable. * Simplified parser productions for defn_head, defs_head. No more normalizing. * Updated `rake compare` steps for lrama in ruby 3.3+. 6 bug fixes: * Cleaned out cruft from `rake compare` code. * Fixed compare/normalize.rb for lrama differences. * Made `parser` task depend on `isolate` task to pick up racc generator. * Made racc a runtime dependency (was dev only) now that ruby 3.3 doesn't ship with the racc runtime. (wildmaples) * Updated README.rdoc updating instructions to match current filenames. * Updated compare versions and added 3.3.
devel/ruby-parser: update to 3.20.0 3.20.0 (2023-03-04) 1 major enhancement: * Added tentative 3.2 support. 1 minor enhancement: * Change minimum ruby version to 2.6. (want higher) 1 bug fix: * Fix up compare tasks for ruby 3.2 differences.
devel/ruby-parser: update to 3.19.1 3.19.1 (2022-04-05) 2 bug fixes: * Added comments to endless defn and defs. (mvz) * Fixed endless method bug handling attrset names.
devel/ruby-parser: update to 3.18.0
3.18.0 (2021-10-27)
Holy crap¡Ä 58 commits! 2.7 and 3.0 are feature complete. Strings &
heredocs have been rewritten.
o 9 major enhancements:
* !!! Rewrote lexer (and friends) for strings, heredocs, and %*[]
constructs.
* Massive overhaul on line numbers.
* Freeze input! Finally!!! No more modifying the input string for heredocs.
* Overhauled RPStringScanner. Removed OLD compatibility methods!
* Removed Sexp methods: value, to_sym, add, add_all, node_type, values.
- value moved to sexp_processor.
* Removed String#grep monkey-patch.
* Removed String#lineno monkey-patch.
* Removed string_to_pos, charpos, etc hacks for ancient ruby versions.
* Removed unread_many¡Ä NO! NO EDITING THE INPUT STRING!
o 31 minor enhancements:
* 2.7/3.0: many more pattern edge cases
* 2.7: Added `mlhs = rhs rescue expr`
* 2.7: refactored destructured args (`|(k,v)|`) and unfactored(?!)
case_body/args.
* 3.0: excessed_comma
* 3.0: finished most everything: endless methods, patterns, etc.
* 3.0: refactored / added new pattern changes
* Added RubyLexer#in_heredoc? (ie, is there old_ss ?)
* Added RubyLexer#old_ss and old_lineno and removed much of SSStack(ish).
* Added Symbol#end_with? when necessary
* Added TALLY and DEBUG options for ss.getch and ss.scan
* Added ignore_body_comments to make parser productions more clear.
* Added support for no_kwarg (eg `def f(**nil)`).
* Added support for no_kwarg in blocks (eg `f { |**nil| }`).
* Augmented generated parser files to have frozen_string_literal comments
and fixed tests.
* Broke out 3.0 parser into its own to ease development.
* Bumped dependencies on sexp_processor and oedipus_lex.
* Clean generated 3.x files.
* Extracted all string scanner methods to their own module.
* Fixed some precedence decls.
* Implemented most of pattern matching for 2.7+.
* Improve lex_state= to report location in verbose debug mode.
* Made it easier to debug with a particular version of ruby via rake.
* Make sure ripper uses the same version of ruby we specified.
* Moved all string/heredoc/etc code to ruby_lexer_strings.rb
* Remove warning from newer bisons.
* Sprinkled in some frozen_string_literal, but mostly helped by oedipus
bump.
* Switch to comparing against ruby binary since ripper is buggy.
* bugs task should try both bug*.rb and bad*.rb.
* endless methods
* f_any_kwrest refactoring.
* refactored defn/defs
o 15 bug fixes:
* Cleaned a bunch of old hacks. Initializing RubyLexer w/ Parser is
cleaner now.
* Corrected some lex_state errors in process_token_keyword.
* Fixed ancient ruby2 change (use #lines) in ruby_parse_extract_error.
* Fixed bug where else without rescue only raises on 2.6+
* Fixed caller for getch and scan when DEBUG=1
* Fixed comments in the middle of message cascades.
* Fixed differences w/ symbol productions against ruby 2.7.
* Fixed dsym to use string_contents production.
* Fixed error in bdot2/3 in some edge cases. Fixed p_alt line.
* Fixed heredoc dedenting in the presence of empty lines. (mvz)
* Fixed some leading whitespace / comment processing
* Fixed up how class/module/defn/defs comments were collected.
* Overhauled ripper.rb to deal with buggy ripper w/ yydebug.
* Removed dsym from literal.
* Removed tUBANG lexeme but kept it distinct as a method name
(eg: `def!@`).
devel/ruby-parser: update to 3.16.0
3.16.0 / 2021-05-15
* 1 major enhancement:
- Added tentative 3.0 support.
* 3 minor enhancements:
- Added lexing for "beginless range" (bdots).
- Added parsing for bdots.
- Updated rake compare task to download xz files, bumped versions, etc
* 4 bug fixes:
- Bump rake dependency to >= 10, < 15. (presidentbeef)
- Bump sexp_processor dependency to 4.15.1+. (pravi)
- Fixed minor state mismatch at the end of parsing to make diffing a
little cleaner.
- Fixed normalizer to deal with new bison token syntax
devel/ruby-parser: update to 3.15.0
Update ruby-parser package to 3.15.0.
=== 3.15.0 / 2020-08-31
* 1 major enhancement:
* Added tentative 2.7 support.
* 1 minor enhancement:
* Improved ruby_parse_extract_error's handling of moving slow files out.
* 22 bug fixes:
* Bumped ruby version to include 3.0 (trunk).
* Fix an error related to empty ensure bodies. (presidentbeef)
* Fix handling of bad magic encoding comment.
* Fixed SystemStackError when parsing a huoooge hash, caused by a splat arg.
* Fixed a number of errors parsing do blocks in strange edge cases.
* Fixed a string backslash lexing bug when the string is an invalid encoding. (nijikon, gmcgibbon)
* Fixed bug assigning line number to some arg nodes.
* Fixed bug concatinating string literals with differing encodings.
* Fixed bug lexing heredoc w/ nasty mix of \r\n and \n.
* Fixed bug lexing multiple codepoints in \u{0000 1111 2222} forms.
* Fixed bug setting line numbers in empty xstrings in some contexts.
* Fixed edge case on call w/ begin + do block as an arg.
* Fixed handling of UTF BOM.
* Fixed handling of lexer state across string interpolation braces.
* Fixed infinite loop when lexing backslash+cr+newline (aka dos-files)
* Fixed lambda + do block edge case.
* Fixed lexing of some ?\M... and ?\C... edge cases.
* Fixed more do/brace block edge case failures.
* Fixed parsing bug where splat was used in the middle of a list.
* Fixed parsing of interpolation in heredoc-like strings. (presidentbeef)
* Fixed parsing some esoteric edge cases in op_asgn.
* Fixed unicode processing in ident chars so now they better mix.
devel/ruby-parser: update to 3.13.1 === 3.13.1 / 2019-03-25 * 1 major enhancement: * Overhauled RubyLexer#lex_state to allow for multi-state value. * 5 minor enhancements: * Added RubyParser#current so scripts can poke around inside. * Cleaned up and modernized bin/ruby_parse_extract_error. * Removed RubyLexer#in_lex_state? in favor of State#=~ and #==. * Renamed RubyLexer#in_arg_state? to is_after_operator?. * Replaced all use of :expr_<*> symbols w/ bitfields. * 15 bug fixes: * Added tests to show #233 already fixed. * Fixed a lexer state bug dealing with complex edge case in hash key w/ interpolated symbol and an array value. Not a great fix. (presidentbeef) * Fixed a number of lex_state check problems. * Fixed alias and undef keyword data. * Fixed bug normalizing to utf-8 on tail of dstr (post-interpolation). (mvz) * Fixed handling of hex chars for bare unicode escape. (mvz) * Fixed handling of invalid string octals more gracefully. Needs generalization. * Fixed lexing of regexps w/ backslashed unicode. I hope. * Fixed parse_percent for EXPR_FITEM edge cases. Fixed bug #125. * Fixed parsing of op_asgn on constants (eg ::X ||= 1). * Fixed precedence of rescue modifier on assignments. (mvz/whitequark) * Fixed some EXPR_FNAME edge cases. * Fixed using unicode in gvars. Fixed bug 202. (331299) * Handle invalid string literals more gracefully. * Minor fix lexing %s[] === 3.13.0 / 2019-03-12 * 3 major enhancements: * Removed 1.8 and 1.9 support. Moved to ruby_parser-legacy gem. * Added tentative 2.6 support. * Updated ruby_parser.yy to ruby 2.4 architecture. * 22 minor enhancements: * Added debug3 and cmp3 rake tasks that use the above tools. * Added tLONELY to compare/normalize.rb * Added tools/munge.rb, tools/ripper.rb, and debugging.md * 2.6: Added trailing .. and ... support. * Extended StackState to log more state changes, making debugging easier. * Extended StackState#store to take an initial value. * Improved logging / debugging in StackState. * Improved normalization and parser compare tasks. * Improved tools/munge.rb output. * In f_arglist, track in_kwarg if no parens. * In process_newline_or_comment, handle NL if in_kwarg there vs normal. * Refactored normalized values to WORDLIST. * Refactored parser: push up to relop and rel_expr. * Removed Keyword.keyword18. * Removed RubyLexer version attr_accessors. * Removed long dead deprecations internal to RubyParserStuff. * Removed version arg to RubyLexer#initialize (default nil + no-op). * Renamed Keyword.keyword19 to Keyword.keyword. * Renamed process_bracing to process_brace_close * Renamed process_curly_brace to process_brace_open * Report first parse error message if all parser versions fail, not last. * Updated parser to track against 2.6. * 11 bug fixes: * Fix some shift/reduce errors. * Fixed BEGIN blocks having different arg value that END blocks. (mvz) * Fixed all reported unused non-terminals/rules. * Fixed bug 272. * Fixed bug in interpolated symbol lists. (strviola) * Fixed bug where block shadow arguments were not registered as lvars. (akimd) * Fixed bug where kwsplat args weren't treated as lvars. (mvz) * Fixed lex_state and other internals in many cases. * Fixed shebang in compare/normalize.rb to use env. Avoids rubygems freaking. * Fixed some more internal state bugs. * Fixed tRCURLY -> tSTRING_DEND for interpolation brace closes.
devel/ruby-parser: update to 3.11.0 === 3.11.0 / 2018-02-14 * 1 major enhancement: * Expand to support ruby 2.5 parsing. * 3 bug fixes: * Fixed %r lexing when delimiter is also used in (?X) group. * Restore newline after label to detect required kwarg. (presidentbeef) * Return `s(:self)` target for `self[]` calls. (presidentbeef)
Update ruby-parser to 3.9.0. === 3.9.0 / 2017-04-13 * 1 major enhancement: * Added tentative ruby 2.4 support. Probably missing things. * 13 minor enhancements: * Added RubyParser.latest. * Added RubyParser::Parser.version to make version range comparisons easier * Changed superclasses of all parsers to RubyParser::Parser * Cleaned up a lot of the version specific code to be more generic/flexible. * Documented how to add new versions in README. * Moved RubyParser from ruby_parser_extras.rb into ruby_parser.rb to fix bootstrap issues. * Renamed RubyParserStuff#get_match_node to new_match. Aliased and deprecated. * Renamed RubyParserStuff#logop to logical_op. Aliased and deprecated. * Renamed RubyParserStuff#node_assign to new_assign. Aliased and deprecated. * Renamed all parsers to RubyParser::V##. * Revamped grammar preprocessing to make adding new versions easier. * RubyParser.for_current_ruby falls back to latest if current not available. * Subclasses of RubyParser::Parser register themselves into RubyParser::VERSIONS. * 4 bug fixes: * Fixed `&.` after newline. (presidentbeef) * Fixed bug setting line number for hash literals to line of opening brace. * Fixed grammar preprocessing bug. * Properly handle kDO with no-parens stabby lambda. (presidentbeef)
Update ruby-parser to 3.8.4. === 3.8.4 / 2017-01-13 * 3 bug fixes: * Fix parsing if in LHS label position. (presidentbeef) * Fixed parsing of anonymous keyword splat arg (eg f(**)). * Fixed ruby 2.4+ deprecation warnings about using Fixnum.
Update ruby-parser to 3.8.1.
=== 3.8.1 / 2016-02-19
* 1 bug fix:
* Fixed problems with tLONELY in mlhs_node.
=== 3.8.0 / 2016-02-18
* 1 major enhancement:
* Added support for Ruby 2.3.0 and the &. operator. (presidentbeef)
* 2 minor enhancements:
* Add support for safe attr assign. (presidentbeef)
* Added support for safe call. (presidentbeef)
* 1 bug fix:
* Fixed parsing of { 'a': :b }. (presidentbeef)
Update ruby-parser to 3.7.2. === 3.7.2 / 2015-10-26 * 1 bug fix: * I hate regexen. Fixed a lexing bug confusing strings vs labels. (phiggins)
Update ruby-parser to 3.7.0. === 3.7.0 / 2015-05-28 * 2 major enhancements: * Added preliminary support for ruby 2.2 syntax. * Now writing all 2.x parsers from one source, generating racc files. * 1 minor enhancement: * Added RubyLexer#process_label and hooked it up to "x": and 'x': productions. * 4 bug fixes: * Fixed handling of block_args to be consistent across all parse versions. * Fixed lexing of label assoc args w/ newlines in between k/v. (jeremyf) * Fixed lexing of x?'':y for ruby22 (not a label). (presidentbeef) * clear and restore cmdarg stack around def args and body. === 3.6.6 / 2015-04-13 * 1 minor enhancement: * Added expr_labelarg state (eg: a:1) * 5 bug fixes: * Fix scan logging backtrace when DEBUG=1 * Fixed kwsplat on 2.0 & 2.1. * Fixed mandatory keyword block args. (cvortmann) * Handle #$<x> in expanded strings/regexps where x is NOT a valid gvar name. * Invalid gvar names raise a compile error.
Update ruby-parser to 3.6.5. pkgsrc change: Add pkg_alternatives support. === 3.6.5 / 2015-03-12 * 1 minor enhancement: * Add line numbers from block comments =begin/=end. (presidentbeef)
Update ruby-parser to 3.6.4. === 3.6.4 / 2015-01-16 * 1 bug fix: * Removed shebangs in tests because of bugs (aka 'features') in RPM packaging tools. === 3.6.3 / 2014-09-26 * 1 bug fix: * Fixed bug caused by latest version of oedipus_lex not exporting lineno accessor. === 3.6.2 / 2014-07-18 * 1 bug fix: * Handle return/next/break/yield w/ block call (WHY). === 3.6.1 / 2014-05-12 * 1 bug fix: * Strings that don't match the encoding of the file are now returned as ASCII-8BIT. === 3.6.0 / 2014-04-23 * 1 minor enhancement: * Added new_string and switched all parsers to it. * 1 bug fix: * Fixed line numbers of nodes following multi-line strings. (presidentbeef) === 3.5.0 / 2014-03-24 * 1 major enhancement: * Added initial support for ruby 2.1 syntax. Doesn't have numeric extensions yet. * 2 minor enhancements: * Converted ruby_lexer.rex to use new grouping mechanism. ~15% improvement in speed. * Various lexer cleanup. * 2 bug fixes: * 2.0/2.1: Fixed block kw args. * Fixed env for kwargs and friends.
Update ruby-parser to 3.4.1.
=== 3.4.1 / 2014-02-14
* 1 minor enhancement:
* Added StackState#store and #restore.
* 1 bug fix:
* 1.8/1.9/2.0: Fixed tSTRING_DBEG w/ f'd up amounts of nesting + braces + do/end. yeah. I'm as confused as you are.
=== 3.4.0 / 2014-02-04
* 1 major enhancement:
* Replaced hand-written/optimized f'd-up lexer with an oedipus_lex
generated lexer. This makes it roughly 40-50% faster.
* 30 minor enhancements:
* 2.0: Added support for a.b c() do d end.e do |f| g end
* 2.0: Added support for a.b c() do d end.e f do |g| h end
* Added -s flag to ruby_parse_extract_error to output timings.
* Added RubyLexer #command_state and #last_state to deal with oedipus_lex differences.
* Added String#lineno and #lineno= because I'm a bad bad person.
* Added a bunch of RubyLexer scanning methods: beginning_of_line?, check, scan, etc.
* Added a bunch of process_* methods extracted from old yylex. process_amper, etc.
* Added lib/.document to save my laptop's battery from pain and suffering
* Adjust lineno when we lex a bunch of blank lines.
* Attach lineno to tIDENTIFIER values (strings, ugh)
* Cleaned up and re-ordered node_assign to be faster (ordered by actual occurrance).
* Extend RubyParserStuff#gettable to set the lineno if it comes in with the id.
* Extended RubyParserStuff#new_case to take line number.
* Finally dropped RPStringScanner's BS #current_line.
* Finally dropped RPStringScanner's BS line number calculation (lineno).
* Implemented Sexp#add_all since we now have a test case for it.
* Removed :call case of node_assign. I don't think it is possible.
* Removed RubyLexer #extra_lines_added. No longer used. Complex heredoc lineno's possible screwed up.
* Removed RubyLexer#parse_number. Handled by oedipus_lex.
* Removed RubyLexer#yacc_value now that next_token returns pairs.
* Removed RubyLexer's @src. Now taken care of by oedipus_lex.
* Removed RubyParser#advance. RubyParser#next_token takes care of everything now.
* Removed RubyParserExtras#arg_add. (presidentbeef! YAY!)
* Removed lib/gauntlet_rubyparser.rb. I just don't use it anymore. Too slow.
* RubyLexer#is_label_possible? doesn't need an arg
* RubyLexer#process_token is now a normal oedipal lexer method.
* RubyParser#next_token now expects RubyLexer#next_token to return a pair (type, val).
* TRYING a new scheme to figure out encodings... but I'm about to throw in the towel. I hate this stuff so much.
* Turned off oedipus_lex's automatic line counting. (pushing to oedipus_lex soon).
* Updated to oedipus_lex 2.1+.
* 7 bug fixes:
* 1.8: Properly parse `a (:b, :c, :d => :e)`. (presidentbeef)
* Fixed lexing symbol!= vs symbol!. Please use your spacebar. Think of the children.
* Fixed line for dstr spanning multiple lines via backslash. (presidentbeef)
* Fixed line numbers for odd cases with trailing whitespace. (presidentbeef)
* Fixed line numbers on ambiguous calls w/ gvar/ivar args. (presidentbeef)
* Max out unicode hex values to 2-4 or 2-6 chars or pack will overflow and puke.
* Removed ESC_RE from RubyLexer. Must have slipped through.
=== 3.3.0 / 2014-01-14
* Notes:
39 files failed to parse out of ~834k files makes this 99.9953% or 4.07σ.
* 15 minor enhancements:
* 2.0: Parse kwarg as lvars. (chastell)
* Added RubyLexer#beginning_of_line?, check(re), end_of_stream?
* Added RubyLexer#process_token_keyword.
* Added RubyLexer#scan, #matched, #beginning_of_line? and others to decouple from internals.
* Added lexing of \u### and \u{###}."
* Added optimizations for simple quoted symbols.
* Aliased Lexer#src to ss (since that is what it is).
* Allow for 20 in parser class name.
* Modified parsers line number calculations for defn nodes.
* Removed Env#dynamic, #dynamic?, #use, #used?
* Removed RubyLexer#tern. Introduced and disused during 3.0 alpha. (whitequark)
* Removed unused RubyLexer#warnings.
* Renamed *_RE consts to just * (IDENT_CHAR, ESC, etc).
* new_defn now sets arg node line number directly.
* zero byte is allowed in symbols for 1.9 / 2.0.
* 11 bug fixes:
* 2.0: Fixed paren-less kwargs in defn.
* Don't bother with regexp encoding options on 1.9+ to avoid warnings.
* Fix constant re-build on ruby 2.0 + rake 10.
* Fix lexing of %i with extra whitespace. (flori)
* Fixed RubyParserStuff#new_body to deal with nonsensical code better (begin-empty+else). (snatchev)
* Fixed bug lexing h[k]=begin ... end. Use your space bars people!
* Fixed env scoping in new lambdas.
* Fixed handling of single array arg in attrasgn.
* Fixed test to call RubyLexer#reset between assertions.
* No longer assigning ivar/cvars to env. Only locals should be in env.
* Refactored initialize and reset to more properly re-initialize as needed.
Update ruby-parser to 3.2.2.
=== 3.2.2 / 2013-07-11
* 5 bug fixes:
* 1.9/2.0: fixed assocs in return args. (presidentbeef)
* Fixed handling of parse error when class is nested in multiple defs. (whitequark)
* Fixed lexing of %w[] w/ funny whitespace separators. (whitequark)
* Fixed more call nodes that have trailing comma syntax. (presidentbeef)
* Fixed more call_args slippage.
=== 3.2.1 / 2013-07-03
* 1 bug fix:
* 1.9/2.0: Trailing assocs were being munged into arrays. (presidentbeef)
=== 3.2.0 / 2013-07-02
* 1 major enhancement:
* Added (rough draft) 2.0 support. Still missing some small / rare things.
* 12 minor enhancements:
* Added %i(symbol-names...) support. (%I too)
* Added 140 more tests, jumping test count from 1376 to 2143. Yay for test reuse!
* Added RubyLexer#brace_nest.
* Added compare20 rake task to diff the grammar architecture against MRI.
* Added lpar_beg and paren_nest to lexer to track state of parens in stabbies
* Added shadow nodes for scoped block args.
* Compound RubyParser now defaults to 2.0.
* Fixed rake to < 10, because 10's file dependency handling is so very broken.
* Made it possible to specify version in bin/ruby_parse_extract_error w/ -v 18|19|20
* Refactored to RubyParserStuff::ENCODING_ORDER to allow custom tweaking of encoding guessing. (samlown)
* Switched `rake debug` to default to 2.0.
* Translated some fixes across 1.8 and 1.9 from 2.0.
* 42 bug fixes:
* 2.0: Fixed a number of block args scenarios w/ kwargs
* 2.0: Fixed args_tail mismatching against lexer.
* 2.0: Fixed assocs to return a hash node.
* 2.0: Fixed f_block_kw production.
* 2.0: Fixed f_block_kwarg production.
* 2.0: Fixed handling of stabby proc args in parens.
* 2.0: Fixed lexing of kwsplat nodes.
* 2.0: Implemented kwsplat nodes.
* Added tUBANG to lexer.
* Apparently ruby doesn't warn for escaped octal that goes above 7. wtf.
* Cleaned up a LOT of arg handling (block and defn, not calls) by using #args.
* ESC_RE is set to unicode. This seems problematic. *shrug*
* Either found a bug in MRI and/or fixed paren_nest count for []=? methods.
* Extended IDENT_CHAR_RE on 1.9+ to top out at \u{10ffff}... because we NEED a million unicode chars.
* Fixed % strings with interpolation.
* Fixed BEGIN {} to return a sexp.
* Fixed a[] += b. (stormbrew)
* Fixed arg_blk_pass to allow for sub-args nodes.
* Fixed assignable to allow for sexps to be passed to it.
* Fixed assoc args in 1.9.
* Fixed block_command and block_call (eg a.b c d) to #to_sym their args properly.
* Fixed bug in compound RubyParser so it rescues RubyParser::SyntaxError.
* Fixed env registration of cdecls.
* Fixed lex value for { when expr_endfn.
* Fixed lex_state for close paren/brace/bracket.
* Fixed lex_state transition for 1.9 if we lexed a defn name. Only 1.8 is odd.
* Fixed lexer problem with state mgmt of identifiers that also have registered var name.
* Fixed lexing of "1 *\n" to have the correct lex_state.
* Fixed lexing of heredocs vs chevron for some lex_states.
* Fixed op_asgn nodes to #to_sym their args properly.
* Fixed optional value block args.
* Fixed parsing of __ENCODING__ on ruby 1.8 (vcall).
* Fixed some oddity where 1.9 lexing was blowing up on "0o". Seems invalid now.
* Fixed strings with escaped octals > 128. Also... wtf.
* Fixed support for empty symbol (wtf?).
* Lexer is now declared UTF-8 internally. Hopefully this will fix the encoding mess.
* Made UTF_8 the default guess on encodings when it isn't explicit.
* Parsing of __ENCODING__ on ruby 1.9+ (in ruby 1.9+) is now colon2 sexp. (whitequark)
* Renamed RubyLexer#nest to string_nest
* RubyLexer#unescape ignores bad octal/hex and returns unicode strings.
* Switched a number of lexical constructs to use IDENT_CHAR_RE instead of \w. I wish there were something cleaner for regexps + unicode.
* Switched ruby_parse_extract_error to use binread.
Update ruby-parser to 3.0.4. Changes are too many to write here, please refer History.txt.
Update ruby-parser package to 2.0.6. === 2.0.6 / 2011-02-18 * 1 minor enhancement: * Switched to hoe's racc plugin to clean up rakefile and builds * 1 bug fix: * Fixed empty =begin/end.
Update devel/ruby-parser to 2.0.5. * Use lang/ruby/gem.mk instead of misc/rubygems/rubygem.mk. * Remove default value of GEM_BUILD. === 2.0.5 / 2010-09-01 * 1 minor enhancement: * Started merging like lexical cases to try to squeeze some optimization out
Initial import of ruby-parser. ruby_parser (RP) is a ruby parser written in pure ruby (utilizing racc - which does by default use a C extension).
Initial revision