Several tests (mostly from sputnik) have a FunctionDeclaration in a Statement context. This isn't permitted by the ES5 grammar, but the tests apparently don't expect it to cause a parse error. Section 12 of ES5 recommends against supporting it.
Here's a list:
12.14-5.js
S12.10_A3.3_T4.js
S12.10_A3.3_T5.js
S12.7_A3.js
S12.7_A4_T1.js
S12.7_A4_T2.js
S12.7_A4_T3.js
S12.8_A3.js
S12.8_A4_T1.js
S12.8_A4_T2.js
S12.8_A4_T3.js
S13.2.2_A17_T1.js
S13.2.2_D20_T2.js
S13.2.2_D20_T3.js
S13.2.2_D20_T4.js
S13.2.2_D20_T5.js
S13.2.2_D20_T6.js
S13.2.2_D20_T7.js
S13.2.2_D20_T8.js
S13_D1_T1.js
S14_D1_T1.js
S14_D4_T1.js
S14_D4_T2.js
S14_D4_T3.js
S14_D6_T1.js
S14_D6_T2.js
S14_D7.js
This one's a bit tricky. The spec states verbatim:
NOTE Several widely used implementations of ECMAScript are known to support the use of FunctionDeclaration as a Statement. However there are significant and irreconcilable variations among the implementations in the semantics applied to such FunctionDeclarations. Because of these irreconcilable difference, the use of a FunctionDeclaration as a Statement results in code that is not reliably portable among implementations. It is recommended that ECMAScript implementations either disallow this usage of FunctionDeclaration or issue a warning when such a usage is encountered. Future editions of ECMAScript may define alternative portable means for declaring functions in a Statement context.
However, ES5 also states the following about (Syntax) errors:
An implementation may extend program syntax and regular expression pattern or flag syntax. To permit this, all operations (such as calling eval, using a regular expression literal, or using the Function or RegExp constructor) that are allowed to throw SyntaxError are permitted to exhibit implementation-defined behaviour instead of throwing SyntaxError when they encounter an implementation-defined extension to the program syntax or regular expression pattern or flag syntax.
Need to give this one some more thought...
Okay, so a conforming implementation isn't required to raise a SyntaxError on these cases. But the point of these tests isn't to determine whether the implementation supports FunctionDecls-as-Statements or not; the tests just *assume* that the implementation does. I think it's unwise for a test to rely on support for any feature that isn't required by the spec (let alone one that's actively discouraged by the spec).
It would be nice if they could be rewritten to not rely on support for that feature. Or at least marked/grouped somehow so that the assumption was made explicit.
For the short term, I'm disabling the set you've identified below.
We also have a number of tests which explicitly check to ensure it's *not* OK to have a FunctionDeclaration in a Statement. This is also disallowed by ES5.1 via chapter 16:
An implementation may extend program syntax and regular expression pattern or flag syntax. To permit this, all operations (such as calling eval, using a regular expression literal, or using the Function or RegExp constructor) that are allowed to throw SyntaxError are permitted to exhibit implementation-defined behaviour instead of throwing SyntaxError when they encounter an implementation-defined extension to the program syntax or regular expression pattern or flag syntax.
After consulting with the TC-39 committee, we've decided 'negative parser tests' such as this should be disabled unless ES5 explicitly calls out that they should throw. E.g., literal octals in strings under Strict Mode would be a valid negative test case we would not disable. In the long term, the test harness and website will be adapted to allow opting-in WRT running these tests. Thus far I've found the following which need to be disabled:
S12.1_A1
Trying to declare function at the Block statement
S12.5_A9_T1
Declaring function within an "if" statement
S12.5_A9_T2
Declaring function within and "if" that is declared within the function call
S12.6.1_A13_T1
Declaring function within a "do-while" loop
S12.6.1_A13_T2
Declaring a function within a "do-while" loop that is within a function call
S12.6.2_A13_T1
Checking if declaring a function within a "while" Statement leads to an exception
S12.6.2_A13_T2
Checking if declaring a function within a "while" Statement that is in a function call leads to an exception
S12.6.4_A13_T1
Declaring function within a "for-in" Statement
S12.6.4_A13_T2
Declaring function within a "for-in" Statement that is within a function call
Just removed 12.14-5.js outright after analyzing it. Corrections for the Sputnik tests will need to be done by Mark before we can close this one down.
Michael, just out of curiosity how did you generate this list of invalid tests? Sounds like whatever tool/JS impl. you used would be useful for periodic 'validity' checks on the tests:)
> how did you generate this list of invalid tests?
I built a parser from the grammar in the ES5.1 spec, fed it the tests in the test suite, and looked at the ones that unexpectedly raised a syntax error.
> Sounds like whatever tool/JS impl. you used would be useful for periodic
> 'validity' checks on the tests:)
Aren't there generally-available implementations that only accept (or have a mode where they only accept) exactly what the spec says?
Hi Michael, Of the files you list, I can't find the ones with an "_D" in them:
(In reply to comment #0)
> S13.2.2_D20_T2.js
> S13.2.2_D20_T3.js
> S13.2.2_D20_T4.js
> S13.2.2_D20_T5.js
> S13.2.2_D20_T6.js
> S13.2.2_D20_T7.js
> S13.2.2_D20_T8.js
> S13_D1_T1.js
> S14_D1_T1.js
> S14_D4_T1.js
> S14_D4_T2.js
> S14_D4_T3.js
> S14_D6_T1.js
> S14_D6_T2.js
> S14_D7.js
I also looked in external/contributions/Google/sputniktests/tests/Conformance and they're not there.
Fixed at http://hg.ecmascript.org/tests/test262/rev/a012a7e238f6
(In reply to comment #7)
> Hi Michael, Of the files you list, I can't find the ones with an "_D" in them:
In the copy I have, they're in
sputnik-v1/tests/Implementation_Diagnostics/
and
test262-ca885b11b815/test/suite/sputnik/Implementation_Diagnostics/