Problem 2390. Fair Scoring for Cody
The length-based scoring system for Cody has a few well-known loopholes that allow clever programmers to hide complex code inside certain functions so that it doesn't register in the score. This problem represents an attempt to generate a fairer Cody scoring function. It doesn't try to replace the length-based criterion with something more elegant; merely to eliminate places where longer code can be hidden and remain undetected by the scoring function.
I have included a few examples that I consider abusive to start with. These are just my own opinions. Please feel free to add your suggestions for others in the comments section for this problem, and I will add good ideas to the test suite when I have the time. Thanks for playing!
- regexp: count number of nodes inside (?@cmd) and add to total
- regexprep: count number of nodes inside {$cmd} and add to total
Using this scoring function, it might even be possible to allow use of eval.
- eval: count number of nodes inside argument to be evaluated and add to total
Other possibilities -- should str2num count the numbers inside its argument? Suggestions welcome! And if you support the idea of changing the scoring function along these lines, please 'like' this problem. Thanks!
Solution Stats
Problem Comments
-
9 Comments
I think this is a very interesting proposal.
So this kind of tricks still allowed in Cody, but they will be counted in a fair way.
I used this a couple of times, and i think it is useful to learn this kind of coding methods, but it is not the real meaning of Cody.
The most trick I used is the "str2num" one. To count all the numbers inside is a little bit hard, but I will accept this method also. It's fair! ;-)
I believe you need to redefine the 'newline' variable in all of your testsuite cases?
this is a very hard problem to solve in general. For example consider the following cases (the score in the first three cases should be penalized with additional 4 points, while in the latter three cases it should not): 1) regexp('',strcat('(?','@a=1)')); 2) str='(?@a=1)';regexp('',str); 3) regexp(' ',char(')@Ab>2*'-1)); 4) regexp('',strcat('(?','a=1)')); 5) str='(?@a=1)';regexp(str,''); 6) regexp(' ',char(')@Ab>2*'+1));
Alfonso, thank you for your excellent thinking on this subject. You're quite right that an approach based only on parsing the program text would be hard to perfect. One could at least make people work harder to get around the scoring system! But what do you think of a different approach: modifying the definitions of regexp and its ilk on the Cody machines so that they dynamically alter the score when called with eval-like arguments?
Hi Nicholas, that is a very good idea, the main problem would be to solve the correspondence between the different times regexp is called and the different times it might appear in the code, but perhaps using dbstack we could resolve this? (of course we could just penalize for each time it is called even if that might produce very high penalties for loops, etc.) thoughts?
The same regexp expression could potentially be called more than once with different dynamic code each time. It might be best to count the score of each unique code string from each separate invocation point -- that seems most in keeping with the spirit of the current rules. You're right that it would be quite complex to keep track.
Thanks for pointing out the problem with newline, by the way. I think it is fixed now.
The easiest way would be to add a fixed size length penalty whenever using the regexp function, for instance, a 1000 points. When a problem requires the regexp function, every player would have the same penalty so it would be meaningless. And when the problem doesn't require it, the Cody player would be disencouraged to use it.
PS: Instead of banning functions, as Cody currently does, a fixed point penalty for certain functions would disencourage their misuse while allowing players to fully explore MATLAB features.
And as a bonus, hacks would all have a high score, making them all the worst possible solutions. Never the leading ones.
I like your ideas in this regard, Rafael.
Solution Comments
Show commentsProblem Recent Solvers7
Suggested Problems
-
281 Solvers
-
226 Solvers
-
1020 Solvers
-
Find the "ordinary" or Euclidean distance between A and Z
162 Solvers
-
Create One Large Eye of size N x N Quickly?
93 Solvers
More from this Author11
Problem Tags
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!