These tests, in particular:
S22.214.171.124_A1_T3 Checking by using eval fail
S126.96.36.199_A1_T3 Checking by using eval fail
S188.8.131.52_A1_T3 Checking by using eval fail
S184.108.40.206_A1_T3 Checking by using eval fail
S220.127.116.11_A1_T3 Checking by using eval fail
S18.104.22.168_A1_T3 Checking by using eval fail
S22.214.171.124_A1_T3 Use numbers and strings as pos fail
S126.96.36.199_A1_T3 Use numbers and strings as pos fail
S188.8.131.52_A1_T3 Use numbers and function object as arguments fail
They do things like this:
var charAt = String.prototype.charAt;
assuming that the global object stringifies as "[object ...]" somehow. But these methods throw a TypeError if |this| is null or undefined, via CheckObjectCoercible, so the tests are buggy.
Mark, can I have your blessing on fixing these Sputnik tests directly in test262? Alternatively, you now have the power to make these changes as well:)
This quarter I'd like to work with you to resolve all remaining issues impeding the final integration of Sputnik into test262. However, until then, I'll will resolve such things by fixing first in Sputnik and then updating the Sputnik contribution to test262. I'll take care of these specific ones. Thanks.
I'm also changing the importance from "enhancement" to "normal" because fixing a real bug is not an "enhancement".
These were disabled on the live website yesterday.
Fixed at http://hg.ecmascript.org/tests/test262/rev/db1b1731316f
Although all the files that Jeff lists are indeed incorrect (and have now been removed), most of these are incorrect for a different reason: The depend on the global "toString" binding, if present, being the same as Object.prototype.toString. This is not required and indeed some platforms validly do otherwise.