archives

« Bugzilla Issues Index

#48 — Add harness support for test cases covering non-ES5, yet web de-facto standards?


Should we have (at least) two categories of test cases:
1. Those that are directly from ES5 or those that can be inferred from ES5
2. Those that contradict ES5 yet are de-facto web standards

Arguably the first category could be split in two as well.


My two cents on this is it'd be extremely useful for TC-39 to know about the second category of tests so the de-facto functionality could be incorporated into ECMAScript v.Next. That said, I do not think such a category should ever be exposed to the public on http://test262.ecmascript.org. The test suite needs to remain true to the spec. That said, I think it would be absolutely fine to be able to run this from the command-line version of Test262 (to be created) if one passes the appropriate flags to the harness. Also, it's fine to keep such tests in the Test262 Mercurial repository provided they're separated from the category one tests (e.g., test\suite\nonstandard\).

A related opinion on this is that test cases covering ES5 errata should exist in the first category of tests.


Based on a command-line extension only, I'm guessing this could be done in well under a day. Populating test\suite\nonstandard\ is an entirely different story though.


We now have a test\suite\bestPractices directory. Just need to support this in some capacity from packager.py and possibly sth.js.


packager.py now generates a testcases_bestPractice.html directory alongside default.html. E.g., http://test262.ecmascript.org/testcases_bestPractice.html

All that needs to be done now is change the text on testcases_bestPractice.html so people don't confuse it with the 'required' ES test suite.


Text on testcases_bestPractice.html has been updated as well to ensure no confusion between this and default.html.