Improvements need to be made to the SputnikConverter such as preserving test case metadata attributes, and not eval'ing snippets with syntax errors. Once these changes have been made, Sputnik's conformance tests will be converted one last time and the tool can be thrown away.
Mark, please document all of your requested changes to the Sputnik=>Test262 conversion tool here. Thanks!
Blindly guesstimating around a week will be needed to implement the changes and verify they don't break anything in the process.
A few more of the suggestions I recall:
- harness should emit the failure message instead of just stating it's failed
- should we use a distinguished error (e.g., SputnikError) instead or in addition to the "true/false" function return value paradigm
- for Sputnik negative scenarios we should introduce 'expectedError', 'testSrc', etc. test case metadata