Stage 2 Draft / May 28, 2022

Regular Expression Buffer Boundaries for ECMAScript

Introduction

Forthcoming

See the proposal repository for background material and discussion.

1 Text Processing

1.1 RegExp (Regular Expression) Objects

A RegExp object contains a regular expression and the associated flags.

Note

The form and functionality of regular expressions is modelled after the regular expression facility in the Perl 5 programming language.

1.1.1 Patterns

The RegExp constructor applies the following grammar to the input pattern String. An error occurs if the grammar cannot interpret the String as an expansion of Pattern.

Syntax

Pattern[UnicodeMode, N]::Disjunction[?UnicodeMode, ?N] Disjunction[UnicodeMode, N]::Alternative[?UnicodeMode, ?N] Alternative[?UnicodeMode, ?N]|Disjunction[?UnicodeMode, ?N] Alternative[UnicodeMode, N]::[empty] Alternative[?UnicodeMode, ?N]Term[?UnicodeMode, ?N] Term[UnicodeMode, N]::Assertion[?UnicodeMode, ?N] Atom[?UnicodeMode, ?N] Atom[?UnicodeMode, ?N]Quantifier Assertion[UnicodeMode, N]::^ $ \A \z \b \B (?=Disjunction[?UnicodeMode, ?N]) (?!Disjunction[?UnicodeMode, ?N]) (?<=Disjunction[?UnicodeMode, ?N]) (?<!Disjunction[?UnicodeMode, ?N]) Quantifier::QuantifierPrefix QuantifierPrefix? QuantifierPrefix::* + ? {DecimalDigits[~Sep]} {DecimalDigits[~Sep],} {DecimalDigits[~Sep],DecimalDigits[~Sep]} Atom[UnicodeMode, N]::PatternCharacter . \AtomEscape[?UnicodeMode, ?N] CharacterClass[?UnicodeMode] (GroupSpecifier[?UnicodeMode]Disjunction[?UnicodeMode, ?N]) (?:Disjunction[?UnicodeMode, ?N]) SyntaxCharacter::one of^$\.*+?()[]{}| PatternCharacter::SourceCharacterbut not SyntaxCharacter AtomEscape[UnicodeMode, N]::DecimalEscape CharacterClassEscape[?UnicodeMode] CharacterEscape[?UnicodeMode] [+N]kGroupName[?UnicodeMode] CharacterEscape[UnicodeMode]::ControlEscape cControlLetter 0[lookahead ∉ DecimalDigit] HexEscapeSequence RegExpUnicodeEscapeSequence[?UnicodeMode] IdentityEscape[?UnicodeMode] ControlEscape::one offnrtv ControlLetter::one ofabcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ GroupSpecifier[UnicodeMode]::[empty] ?GroupName[?UnicodeMode] GroupName[UnicodeMode]::<RegExpIdentifierName[?UnicodeMode]> RegExpIdentifierName[UnicodeMode]::RegExpIdentifierStart[?UnicodeMode] RegExpIdentifierName[?UnicodeMode]RegExpIdentifierPart[?UnicodeMode] RegExpIdentifierStart[UnicodeMode]::IdentifierStartChar \RegExpUnicodeEscapeSequence[+UnicodeMode] [~UnicodeMode]UnicodeLeadSurrogateUnicodeTrailSurrogate RegExpIdentifierPart[UnicodeMode]::IdentifierPartChar \RegExpUnicodeEscapeSequence[+UnicodeMode] [~UnicodeMode]UnicodeLeadSurrogateUnicodeTrailSurrogate RegExpUnicodeEscapeSequence[UnicodeMode]::[+UnicodeMode]uHexLeadSurrogate\uHexTrailSurrogate [+UnicodeMode]uHexLeadSurrogate [+UnicodeMode]uHexTrailSurrogate [+UnicodeMode]uHexNonSurrogate [~UnicodeMode]uHex4Digits [+UnicodeMode]u{CodePoint} UnicodeLeadSurrogate::any Unicode code point in the inclusive range 0xD800 to 0xDBFF UnicodeTrailSurrogate::any Unicode code point in the inclusive range 0xDC00 to 0xDFFF

Each \u HexTrailSurrogate for which the choice of associated u HexLeadSurrogate is ambiguous shall be associated with the nearest possible u HexLeadSurrogate that would otherwise have no corresponding \u HexTrailSurrogate.

HexLeadSurrogate::Hex4Digitsbut only if the MV of Hex4Digits is in the inclusive range 0xD800 to 0xDBFF HexTrailSurrogate::Hex4Digitsbut only if the MV of Hex4Digits is in the inclusive range 0xDC00 to 0xDFFF HexNonSurrogate::Hex4Digitsbut only if the MV of Hex4Digits is not in the inclusive range 0xD800 to 0xDFFF IdentityEscape[UnicodeMode]::[+UnicodeMode]SyntaxCharacter [+UnicodeMode]/ [~UnicodeMode]SourceCharacterbut not UnicodeIDContinue DecimalEscape::NonZeroDigitDecimalDigits[~Sep]opt[lookahead ∉ DecimalDigit] CharacterClassEscape[UnicodeMode]::d D s S w W [+UnicodeMode]p{UnicodePropertyValueExpression} [+UnicodeMode]P{UnicodePropertyValueExpression} UnicodePropertyValueExpression::UnicodePropertyName=UnicodePropertyValue LoneUnicodePropertyNameOrValue UnicodePropertyName::UnicodePropertyNameCharacters UnicodePropertyNameCharacters::UnicodePropertyNameCharacterUnicodePropertyNameCharactersopt UnicodePropertyValue::UnicodePropertyValueCharacters LoneUnicodePropertyNameOrValue::UnicodePropertyValueCharacters UnicodePropertyValueCharacters::UnicodePropertyValueCharacterUnicodePropertyValueCharactersopt UnicodePropertyValueCharacter::UnicodePropertyNameCharacter DecimalDigit UnicodePropertyNameCharacter::ControlLetter _ CharacterClass[UnicodeMode]::[[lookahead ≠ ^]ClassRanges[?UnicodeMode]] [^ClassRanges[?UnicodeMode]] ClassRanges[UnicodeMode]::[empty] NonemptyClassRanges[?UnicodeMode] NonemptyClassRanges[UnicodeMode]::ClassAtom[?UnicodeMode] ClassAtom[?UnicodeMode]NonemptyClassRangesNoDash[?UnicodeMode] ClassAtom[?UnicodeMode]-ClassAtom[?UnicodeMode]ClassRanges[?UnicodeMode] NonemptyClassRangesNoDash[UnicodeMode]::ClassAtom[?UnicodeMode] ClassAtomNoDash[?UnicodeMode]NonemptyClassRangesNoDash[?UnicodeMode] ClassAtomNoDash[?UnicodeMode]-ClassAtom[?UnicodeMode]ClassRanges[?UnicodeMode] ClassAtom[UnicodeMode]::- ClassAtomNoDash[?UnicodeMode] ClassAtomNoDash[UnicodeMode]::SourceCharacterbut not one of \ or ] or - \ClassEscape[?UnicodeMode] ClassEscape[UnicodeMode]::b [+UnicodeMode]- CharacterClassEscape[?UnicodeMode] CharacterEscape[?UnicodeMode] Note 1

A number of productions in this section are given alternative definitions in section A.1.1.

 Note 2
While the sequence \Z is already an error when the u (unicode) flag is present, it is reserved for possible future use as an extension of the \A and \z assertions.

1.1.2 Pattern Semantics

1.1.2.1 Runtime Semantics: CompileAssertion

description
It returns a Matcher.
Assertion::\A
  1. Returns a new Matcher with parameters (x, c) that captures nothing and performs the following steps when called:
    1. Assert: x is a State.
    2. Assert: c is a Continuation.
    3. Let e be x's endIndex.
    4. If e = 0, return c(x).
    5. Return failure.
 Assertion::\z
  1. Returns a new Matcher with parameters (x, c) that captures nothing and performs the following steps when called:
    1. Assert: x is a State.
    2. Assert: c is a Continuation.
    3. Let e be x's endIndex.
    4. If e = InputLength, return c(x).
    5. Return failure.

A Additional ECMAScript Features for Web Browsers

A.1 Additional Syntax

A.1.1 Regular Expressions Patterns

The syntax of 1.1.1 is modified and extended as follows. These changes introduce ambiguities that are broken by the ordering of grammar productions and by contextual information. When parsing using the following grammar, each alternative is considered only if previous production alternatives do not match.

This alternative pattern grammar and semantics only changes the syntax and semantics of BMP patterns. The following grammar extensions include productions parameterized with the [UnicodeMode] parameter. However, none of these extensions change the syntax of Unicode patterns recognized when parsing with the [UnicodeMode] parameter present on the goal symbol.

Syntax

Term[UnicodeMode, N]::[+UnicodeMode]Assertion[+UnicodeMode, ?N] [+UnicodeMode]Atom[+UnicodeMode, ?N]Quantifier [+UnicodeMode]Atom[+UnicodeMode, ?N] [~UnicodeMode]QuantifiableAssertion[?N]Quantifier [~UnicodeMode]Assertion[~UnicodeMode, ?N] [~UnicodeMode]ExtendedAtom[?N]Quantifier [~UnicodeMode]ExtendedAtom[?N] Assertion[UnicodeMode, N]::^ $ \A \z \b \B [+UnicodeMode](?=Disjunction[+UnicodeMode, ?N]) [+UnicodeMode](?!Disjunction[+UnicodeMode, ?N]) [~UnicodeMode]QuantifiableAssertion[?N] (?<=Disjunction[?UnicodeMode, ?N]) (?<!Disjunction[?UnicodeMode, ?N]) QuantifiableAssertion[N]::(?=Disjunction[~UnicodeMode, ?N]) (?!Disjunction[~UnicodeMode, ?N]) ExtendedAtom[N]::. \AtomEscape[~UnicodeMode, ?N] \[lookahead = c] CharacterClass[~UnicodeMode] (Disjunction[~UnicodeMode, ?N]) (?:Disjunction[~UnicodeMode, ?N]) InvalidBracedQuantifier ExtendedPatternCharacter InvalidBracedQuantifier::{DecimalDigits[~Sep]} {DecimalDigits[~Sep],} {DecimalDigits[~Sep],DecimalDigits[~Sep]} ExtendedPatternCharacter::SourceCharacterbut not one of ^$\.*+?()[| AtomEscape[UnicodeMode, N]::[+UnicodeMode]DecimalEscape [~UnicodeMode]DecimalEscapebut only if the CapturingGroupNumber of DecimalEscape is ≤ _NcapturingParens_ CharacterClassEscape[?UnicodeMode] CharacterEscape[?UnicodeMode, ?N] [+N]kGroupName[?UnicodeMode] CharacterEscape[UnicodeMode, N]::ControlEscape cControlLetter 0[lookahead ∉ DecimalDigit] HexEscapeSequence RegExpUnicodeEscapeSequence[?UnicodeMode] [~UnicodeMode]LegacyOctalEscapeSequence IdentityEscape[?UnicodeMode, ?N] IdentityEscape[UnicodeMode, N]::[+UnicodeMode]SyntaxCharacter [+UnicodeMode]/ [~UnicodeMode]SourceCharacterIdentityEscape[?N] SourceCharacterIdentityEscape[N]::[~N]SourceCharacterbut not c [+N]SourceCharacterbut not one of c or k ClassAtomNoDash[UnicodeMode, N]::SourceCharacterbut not one of \ or ] or - \ClassEscape[?UnicodeMode, ?N] \[lookahead = c] ClassEscape[UnicodeMode, N]::b [+UnicodeMode]- [~UnicodeMode]cClassControlLetter CharacterClassEscape[?UnicodeMode] CharacterEscape[?UnicodeMode, ?N] ClassControlLetter::DecimalDigit _ Note

When the same left-hand sides occurs with both [+UnicodeMode] and [~UnicodeMode] guards it is to control the disambiguation priority.

B Copyright & Software License

Copyright Notice

© 2022 Ron Buckton, Ecma International

Software License

All Software contained in this document ("Software") is protected by copyright and is being made available under the "BSD License", included below. This Software may be subject to third party rights (rights from parties other than Ecma International), including patent rights, and no licenses under such third party rights are granted under this license even if the third party concerned is a member of Ecma International. SEE THE ECMA CODE OF CONDUCT IN PATENT MATTERS AVAILABLE AT https://ecma-international.org/memento/codeofconduct.htm FOR INFORMATION REGARDING THE LICENSING OF PATENT CLAIMS THAT ARE REQUIRED TO IMPLEMENT ECMA INTERNATIONAL STANDARDS.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

  1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
  2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
  3. Neither the name of the authors nor Ecma International may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE ECMA INTERNATIONAL "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL ECMA INTERNATIONAL BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.