Reflections on the croco-puzzle ladder?

Information and discussion on Puzzle Competitions
dickoon
Posts: 615
Joined: Mon 21 Jun, 2010 12:52 pm
Location: Stockton-on-Tees, UK
Contact:

Reflections on the croco-puzzle ladder?

Post by dickoon » Wed 27 Apr, 2011 10:22 pm

Seeing as we've had most of a week off from the croco-puzzle ladder, I would welcome some reflections on its design from players, non-players and potential players alike.

The rules came fairly naturally, but that tends to mean that their relative success or failure is based more on intuition than design, and there may very well be room for improvement. I'm very happy that so many of the UK croco-puzzle players are taking part in the ladder, but I'd still like more... more UK croco-puzzle players first and foremost, and more croco-puzzle ladder players secondarily. I'd be curious to know if there are any things putting the non-players off from participation, or even aspects that existing ladder players may not relish about their participation. If you've particularly enjoyed the last few days on croco-puzzle without having the ladder to worry about, and are not looking forward to its return, that's valid feedback too.

There's an obvious criticism that being on the ladder puts pressure on ladder players to take part in both croco-puzzle puzzles every day, and - especially if you're a slower, more recreational solver like me - that may be a degree of commitment that you may not relish. My feelings on this are mixed; it is possible to win matches without playing every day, showing high levels of dedication deserves reward (though, neatly, doesn't guarantee reward!) within ladder standings, and the players at the top of the ladder who might most feel this pressure are the ones most likely to be able to solve the puzzles quickly. On the other hand, the ladder would have worked just as neatly if I had said "Ladder matches will be based on the results of Monday, Wednesday and Friday's puzzles only" from the start. (Does the "bye day" system work?) Other solutions are possible; there is an argument to say that puzzles that are "too quick" or "too slow" shouldn't count towards the ladder, but this is impossible to tell in advance and retrospectively declaring puzzles ineligible feels unsatisfactory.

I'm also more than welcome to hear criticism of other features of the competition. It's true that I've had the ladder structure in mind for years and been looking for an interesting activity with which to use it, with croco-puzzle participation fitting the bill, and it's also true that a ladder structure has its own quirks. There is a fair degree of scope for repetitive matches, but (a) I really don't mind matches repeating when they're competitive, and the structure should mean that they will tend to be reasonably evenly matched, as well as leading to friendly rivalries and results that will differ depending upon which puzzle types crop up in any particular week, and (b) this wouldn't be as much of a problem if we had more players and thus a greater degree of variety in who plays whom. Other tournament formats are possible based on, say, round-robins, elimination contests or even Swiss system tournaments with some sort of match metric. Other tournaments still with formats so convoluted they would shame sports entertainment are also possible. (And fun!)

A few weeks in, I think people should have a reasonable degree of familiarity with how the system works in practice as well as in theory. I would be happy to run other croco-puzzle ladders on similar lines, with rule tweaks based on people's experiences from the first one. I ran this one for UK players because (a) having some sort of specifically UK competition feels appropriate and fun and (b) it's clearly on-topic for this forum; I would be happy to run one for, say, US players in parallel and in addition to the UK one. (It might well not be appropriate to host a US ladder on a UK puzzle forum, though!) In general, I'll run one for any group of players who wants one, though I am only comfortable doing so in the English language; it feels right to me that if (say) German players wanted one, it might best be run in the German language on the forum.logic-masters.de web site.

If someone else wants to run a ladder, I have attached the spreadsheet I use to process the results; if anyone wants instructions or documentation, let me know and I'll happily supply them. ((ETA: Troubleshooting note to self: the spreadsheet relies on the Monday to Friday spreadsheets being generated when you're logged out of croco-puzzle, or the HTML ends up with an extra line in letting you log out and that throws all the line references off by one. Told you it was ugly Excel.)) It has been written in Excel and most conveniently deals with files produced using Internet Explorer; I wrote it to suit myself and my own needs rather than for general release, though it should surely be easily adaptable to a considerably more open format. It's not too tricky, though, and processing the results only takes about ten minutes a week.

All comments, feedback and questions are welcome!
Attachments
croco-ladder.xls
Croco-puzzle ladder administration spreadsheet
(389.42 KiB) Downloaded 580 times
Last edited by dickoon on Fri 13 May, 2011 8:38 pm, edited 1 time in total.

PuzzleScot
Site Admin
Posts: 2755
Joined: Fri 18 Jun, 2010 10:45 pm
Location: Edinburgh, Scotland

Re: Reflections on the croco-puzzle ladder?

Post by PuzzleScot » Wed 27 Apr, 2011 11:41 pm

I'll be playing every day I can - I'm a bit of a creature of habit (until I get bored).
I don't take it too seriously, so don't mind what rules are in place.
Thanks for running it though - adds an extra element of interest to an otherwise routine daily ritual..

motris
Posts: 77
Joined: Tue 21 Dec, 2010 5:35 pm
Contact:

Re: Reflections on the croco-puzzle ladder?

Post by motris » Wed 27 Apr, 2011 11:44 pm

As an outsider to this process, with no real stake in the ladder, I've still found seeing the weekly results rather interesting, and the format of the ladder feels right. Croco-puzzle is stressful for me everyday with potential point losses even with an "average" performance, so I wouldn't care if I were included in a ladder ranking or not, but I can understand how other individuals (A) having to solve every weekday and (B) sometimes having to solve really long/hard puzzles because of being in the ladder is a potential negative. Still, I think you've done the best you can so far Chris to accommodate the players and I hope the UK ladder does grow.

I'm not sure there are enough players for a proper US ladder, but I'd be interested to see some kind of an international one somehow. Perhaps the top solvers from 8 countries (or maybe 5 countries plus the top 3 Germans since they have such a large percentage of players) joining in. Being on top of the UK ladder would earn the extra honor/pressure of playing in this separate league....

dickoon
Posts: 615
Joined: Mon 21 Jun, 2010 12:52 pm
Location: Stockton-on-Tees, UK
Contact:

Re: Reflections on the croco-puzzle ladder?

Post by dickoon » Thu 28 Apr, 2011 8:09 pm

motris wrote:I'm not sure there are enough players for a proper US ladder, but I'd be interested to see some kind of an international one somehow. Perhaps the top solvers from 8 countries (or maybe 5 countries plus the top 3 Germans since they have such a large percentage of players) joining in. Being on top of the UK ladder would earn the extra honor/pressure of playing in this separate league....
I definitely like that. You have just naturally evolved a Croco-Puzzle Ladder Champions' League! (Champions' Ladder, perhaps, I suppose.)

There does seem to me to be a fair bit of US talent on croco-puzzle already. I don't claim to know everyone's nationality, or even come close, but off-hand there's yourself, Rog, willwc, melon, Tablesaw... and, almost doubtlessly, many others who I either have forgotten were American or never knew. You very probably know others. Admittedly Rog doesn't play so often any more, but I'd like to think that the existence of a ladder might interest other strong talents in getting involved. One of the nice features about the ladder is that while you have to be on croco-puzzle for, what, six months in order for your rating to approximate your accurate level of performance, you can find your level on the ladder in a matter of weeks.

ColinMacLeod
Posts: 7
Joined: Sat 06 Nov, 2010 8:55 pm

Re: Reflections on the croco-puzzle ladder?

Post by ColinMacLeod » Thu 05 May, 2011 8:32 pm

I'm a US player competing on Croco-Puzzle since motris mentioned it on his blog in December. While my mum's British, I'm not, so I've watched the UK ladder from the sidelines. Since you've been gracious enough to gather the times for me, I've been comparing myself to all the competitors to see how I would rank if I competed (of note, I have three 5-5 draws with rodders in the 5 weeks so far). I play Croco-puzzle every day for pride, so participation in the ladder would not be an influence on my performance. I would be interested in a US ladder if one was started.

As for the ladder structure, the matchups are highly repetitive. Think of matchups as two coins in a chessboard square. Each column represents one week's games. When the coins/players advance to the next week's games (the next column), they would do so diagonally, staying on the same color. The only way to switch colors is to be in the top (or bottom) and stay in the same "level" of matchup. An upset (in the middle) would generate two new matches in the second week, but would probably revert to form the following week. Consider players ranked 4th and 5th: the only way for them to match up is for one of them to get to the top (or bottom) match and win (to change colors), followed by losing subsequent matches to eventually encounter the other. If the top and bottom players are strictly better/worse than everyone else, then no one will ever change colors and you'll be stuck with the same matchups every other week.

One work-around, given enough players, would be to have matches as a set of 3 players, getting one point per opponent they beat in a week. Top player advances; bottom player drops; middle player stays in the same match. The Championship match could still be a 1v1. Ties are resolved via head-to-head and then Saturday if necessary. If the middle tier is closer and has more variance in performance, this format may lead to more interchange and new matchups. In a theoretical matchup of ABD, A and B may have played two weeks prior to a draw, but D might be good enough to steal some points from A and B some days to alter the result or be a new legitimate challenger to the scene. The third player influencing the result of two others may not be as desirable as repeated matches or features of other ladder structures. One plus side of this structure is that it can scale to more players for certain tiers as long as there's enough promotion/relegation each week to keep it from getting stagnate.

ColinMacLeod

ronald
Posts: 58
Joined: Tue 08 Mar, 2011 1:22 pm

Re: Reflections on the croco-puzzle ladder?

Post by ronald » Thu 05 May, 2011 9:56 pm

Great observation!!

dickoon
Posts: 615
Joined: Mon 21 Jun, 2010 12:52 pm
Location: Stockton-on-Tees, UK
Contact:

Re: Reflections on the croco-puzzle ladder?

Post by dickoon » Thu 05 May, 2011 10:46 pm

Thanks, Colin! I welcome your comments and have pencilled you in for a US league if there is the interest. Your comments are certainly true, and I've thought about this parity (in the sense of "odd or even", not in the sense of "equal to") aspect myself.

As you say, results in the top and bottom matches can effectively change people's parity from playing people drawn from one half of the league to playing people drawn from the other half. In a division of eight or fewer players, this should happen sufficiently frequently, noting your considerable caveat that it requires the top and bottom matches to be competitive, that there ought to be reasonably frequent parity changes that ought to shake things up and also level out perceived or actual differences in strength between the two halves of the league. With more players in the league, I also tend to believe that there ought to be a greater degree of variety in the opposition to be faced to the extent that that would be less of a factor. I've also deliberately been playing up the aspect that repeated matches between the same players can build up friendly rivalries and that repeated matches between the same players are likely to remain competitive and have a variety of different outcomes based on the variety of puzzles that are offered. It's a fair criticism, but I don't know how annoyed (or, perhaps, bored) people are in practice... or will be six months down the line, if they're facing a specific opponent for perhaps the tenth time.

Three-way matches would certainly have the advantages you specify. I think my degree of reticence towards them is one of gut feeling, probably (in truth) more out of hope that the initial format that I stumbled across would work well. An objection against matches with many players, noting that this is not quite what you proposed, is that individual rounds of the competition would feel more like participating in a ladder than like participating in a head-to-head contest. This may appeal more than the current system to some people, and may appeal less than the current system to others. My gut feeling is that it would have less of an immediate, easily communicable feel to it; people relate to two-way contests in a way that they might not for others. It could certainly work, and three-way matches might prove to be a sweet spot that retains some element of immediate, tight competition with fewer problems of repetitive match-ups. (Certainly it would not eliminate them altogether if there is a definite hierarchy with some people consistently staying at certain tiers, and other people bouncing between two tiers - and, as discussed, it's not necessarily a problem in the first place.)

I'm reluctant to change league designs on an ad-hoc basis, and I tend to believe that people would prefer to play in a league which sets its procedures out and sticks to them rather than one which plays one way for a few weeks, then does something different for a while, then changes completely every few weeks. (Noting that the first two weeks of the league were different to the weeks that have followed, by design!) That said, I'm not completely wed to my initial design and welcome the suggestions you have given. It would certainly be possible to start a second league with three-way matches from the start - or even to have a single competition in which competitors' performances are compared to each other using both two-way matches and three-way matches, so that people can decide which one they prefer the feel of in practice. Given that the league design attracted some concern over potentially being complicated (or, at least, possibly just badly-explained!) to begin with, would another league actually attract players in practice?

ronald
Posts: 58
Joined: Tue 08 Mar, 2011 1:22 pm

Re: Reflections on the croco-puzzle ladder?

Post by ronald » Thu 05 May, 2011 11:42 pm

I'm happy for the league to continue as it is for now.

I would observe that guaranteed promotion or relegation is rather a harsh wager. (This perhaps relates to Colin's point)

On a 5-3 result (or even a 4-4 tie with tiebreak), one person goes up and one person goes down - leaving them a full 2 matches apart - even though they've proven to be of similar ability.

In some senses this is not a strong disadvantage. I like the head-to-head competition aspect, as you say dickoon, and the high range of motion means we can expect to see a variety of opponents (assuming that, as Colin says, the top and bottom places are not too secure).
I think it's a good idea to continue with what we're doing.

drsteve
Posts: 727
Joined: Sun 27 Jun, 2010 7:23 am

Re: Reflections on the croco-puzzle ladder?

Post by drsteve » Fri 06 May, 2011 6:15 am

One mild adjustment to try and avoid the same repetitions.

Week 1: 1v2, 3v4, 5v6, 7v8 - if you win, you either keep your place or swap with the person you've beaten.

Week 2: 1 rests, 2v3, 4v5, 6v7, 8 rests or plays 9. Same rules as above.

Repeat to fade...

This also means the ladder can be as long as you like, but I guess it will make climbing it slower...

PuzzleScot
Site Admin
Posts: 2755
Joined: Fri 18 Jun, 2010 10:45 pm
Location: Edinburgh, Scotland

Re: Reflections on the croco-puzzle ladder?

Post by PuzzleScot » Fri 06 May, 2011 8:28 am

drsteve wrote:One mild adjustment to try and avoid the same repetitions. <clip>
How bizarre, I was just about to write exactly the same thing! That would make it a true ladder as I recall other ladder systems working, where you effectively just swap positions with one person each week.

The problem with the 'true ladder' system is that a regular top/bottom player only gets a game every other week. Other ladders operate on a one-time match, usually as a result of a 'challenge' being issued, so the ladder is very dynamic, and positions change quickly. The only way I could see that working in our environment is that the table changes daily. Unless the scoring of such a process was automated, it would be way too time-consuming for anyone to manage.

KISS principle applies, so I'm content to keep it as is.

dickoon
Posts: 615
Joined: Mon 21 Jun, 2010 12:52 pm
Location: Stockton-on-Tees, UK
Contact:

Re: Reflections on the croco-puzzle ladder?

Post by dickoon » Fri 06 May, 2011 8:13 pm

I'm definitely warming to drsteve's notion - and thank you very much for making it, Steve! - as it would make the relatively rare championship and #1 contender (i.e. 2v3) matches very dramatic and interesting, and I suspect that it may work better than the current system for divisions of fewer than eight players. On the downside, it could be rather less interesting for people in the middle, by virtue that (as you say) both climbing and falling would be slower. Being in the #2 contender match is something reasonably tangible, exciting and gives a feel of being close to the top (and thus engaged in the league!) in a way that being in a 5v6 match - effectively, a #4 contender match - might not be. I consider that a reasonably substantial downside, though it's definitely arguable both ways.

I'm also not immediately sure that it wouldn't lead to reasonably similar numbers of repeat matches to the current system, though it should make them even tighter and even more competitive. On balance, this is marginally a point in its favour as far as I'm concerned.

One nice thing is that it would reasonably easily be possible as a thought-experiment to go and retrospectively simulate the league results from, say, week three onwards under this alternative fixture creation system and see whether the matches and results that this proposal kicks up feel more interesting than the true ones. At some point I may well give that a try, though probably not until there's rather more data to try it out with.

dickoon
Posts: 615
Joined: Mon 21 Jun, 2010 12:52 pm
Location: Stockton-on-Tees, UK
Contact:

Re: Reflections on the croco-puzzle ladder?

Post by dickoon » Fri 01 Jul, 2011 10:05 pm

Please find attached a document comparing the current croco-puzzle ladder match generation system with two alternatives. It is a text document, designed to be viewed in a monospaced font like Courier New so that all three systems have their own column and things line up neatly. For each system, I have recalculated the matches generated for each of weeks four to thirteen, the results of the matches and the next sets of matches that the systems would generate. Errors and omissions are excepted, and probably expected, with apologies.

It compares the current system with one implementation of drsteve's "alternate weeks have alternate match generation algorithms" system, and with one implementation of Colin's "three-way matches" system. There are not necessarily the only way to implement the systems - for instance, the "alternate weeks" system might start with (effectively) an even-numbered week rather than an odd-numbered week, and the "three-way matches" system uses a 4-2-0 scoring system (to permit ties, which might be scored 3-3-0, or 4-1-1, etc.) rather than a 2-1-0 scoring system. It's certainly possible to generate different tie-breaking methods in either method, too.

Your comments would be welcome, especially with reasons for your preferences between one system and the others, whether you are playing or not. The croco-puzzle ladder cannot be a democracy, but if I want to keep people playing, I cannot afford to (and will not) ignore people's comments on the matter.
Attachments
croco scoring systems.txt
Three different match generation algorithms for the croco-puzzle ladder
(15.55 KiB) Downloaded 509 times

drsteve
Posts: 727
Joined: Sun 27 Jun, 2010 7:23 am

Re: Reflections on the croco-puzzle ladder?

Post by drsteve » Sat 02 Jul, 2011 9:08 am

Oh, I like my version - it keeps me in a minimum of second place!!!

On a more serious note, it does seem that the issue raised early on, that of repeated match-ups, does seem to be happening still - the only way to get into the other "half" of the competitors is to defend a championship for an odd number of weeks or get relegated and rejoin an odd number of weeks later. It looks to me that some competitors, such as oenomel aka Nick, are very unlikely to end up in a match against me, whereas I'm playing my fourth or fifth match against Ronald next week (no offense, Ronald).

Not sure that my system is any better, due to the champion winning a week's holiday after a successful defence, but it does mix up the matches a bit more and variety is the spice of life.

The other concern I think is with how we deal with a missed puzzle. With the best will in the world, I'm not going to do a puzzle that starts with a 10 minute + score on the Highscoretabellen as I don't have time to do that. The penalty though, with regards a tie-break, is very high for this (9999 seconds). Could we maybe tone down that penalty into something more realistic - e.g. 1.5 x the slowest time for anyone else on the ladder who did it? Just a thought.

PuzzleScot
Site Admin
Posts: 2755
Joined: Fri 18 Jun, 2010 10:45 pm
Location: Edinburgh, Scotland

Re: Reflections on the croco-puzzle ladder?

Post by PuzzleScot » Sat 02 Jul, 2011 9:44 am

I don't really mind what system is used, so long as it's reasonably fair. If it mixes up the matches more, then all the better.

However, it does seem a little unfair to use the 'sum of solve times' as a decider.
If, as has happened, 2 players are tied, and one puzzle in the week is hideously hard (completed or not), it's the winner of that one that wins overall.
It's an extreme example, but quite indicative of who wins tie-breaks - the one who did best on the hardest one (or 2) wins overall.

There is a 'performance rating' calculation (here), that would be fairer than using raw times...
The rating 'score' for a day's puzzle is: 3000/2^((user - best)/(median-best)) [All values are solve times in seconds, though FVS are handled in a 'fairer' way]. I'm not sure if this number is displayed anywhere, but would be really useful if it did.

berni
Posts: 163
Joined: Mon 01 Nov, 2010 11:38 am
Contact:

Re: Reflections on the croco-puzzle ladder?

Post by berni » Sat 02 Jul, 2011 10:19 am

PuzzleScot wrote:I'm not sure if this number is displayed anywhere, but would be really useful if it did.
See http://www.croco-puzzle.com/Ue-Raetsel/techdata.php (Strafzeit pro Fehlversuch)

nickdeller
Posts: 604
Joined: Tue 29 Jun, 2010 11:41 am

Re: Reflections on the croco-puzzle ladder?

Post by nickdeller » Sat 02 Jul, 2011 12:13 pm

You ask for preferences, Chris, so I'll provide some!

I'm rather enthusiastic about the three-person matchups, for several reasons - first and foremost that it seems very effective at mixing up the player list week-on-week. Admittedly, it would still hit a dynamic equilibrium in a theoretical situation where every player was markedly better than the next one down, but we don't seem to be in that position.

I'd like to fly the kite that the championship match should be a three-person matchup in that system. We saw detuned win the championship match by default a couple of weeks running recently - that would be far less likely to happen when you have two opponents to consider. I think we're also in a position with our current lineup where detuned and kiwijam are measurably the best two players on the ladder (no offence to drsteve and ronaldx who are very capable of upsetting the applecart, and a few others - me, for one - who could beat the formbook in an unusually lucky week), and I think it would be beneficial all round if the championship match was usually detuned vs. kiwijam vs. a.n.other.

Other considerations - A vs B vs C matches could provide a mechanism to deal with occasional full bye weeks, with the missing player deemed to have finished second (I think such weeks would normally need to be booked at least one to two weeks in advance to prevent gaming the system). The issue of bye days seems imperfect in the three-player system, and could theoretically lead to a completely unplayed match as it stands - perhaps it would work for a single byed player to register the mean of the other two's solve times on that day, with two or more byed players causing a day off? Thinking aloud, so that probably has problems of its own!

At the bottom of the table, I'd prefer to see 3-player matches all the way down, with the waiting list never having more than two players on it. If at some future date there are enough people to have seven full matches on the ladder, why not have seven full matches!

drsteve
Posts: 727
Joined: Sun 27 Jun, 2010 7:23 am

Re: Reflections on the croco-puzzle ladder?

Post by drsteve » Sat 02 Jul, 2011 8:49 pm

nickdeller wrote: I think we're also in a position with our current lineup where detuned and kiwijam are measurably the best two players on the ladder (no offence to drsteve and ronaldx who are very capable of upsetting the applecart, and a few others - me, for one - who could beat the formbook in an unusually lucky week),
Hmm... can I let this go? Sorry, Nick, but I'd say that Ronald and myself are on a par with the two named players. I strongly disagree that this is a two-horse race and in a three-way championship, I have doubts it would be Tom, James and then me and Ronald in rotation.

Admittedly, if there was a Heyawake, Tom's going to get the top score, along with a couple of other puzzles, but it's by no means certain on others...

nickdeller
Posts: 604
Joined: Tue 29 Jun, 2010 11:41 am

Re: Reflections on the croco-puzzle ladder?

Post by nickdeller » Sat 02 Jul, 2011 10:15 pm

drsteve wrote:
nickdeller wrote: I think we're also in a position with our current lineup where detuned and kiwijam are measurably the best two players on the ladder (no offence to drsteve and ronaldx who are very capable of upsetting the applecart, and a few others - me, for one - who could beat the formbook in an unusually lucky week),
Hmm... can I let this go? Sorry, Nick, but I'd say that Ronald and myself are on a par with the two named players.
...a quick look at the last two weeks of hypothetical matchups between you and kiwijam show a 5-5 and a 6-4 match, and so I happily concede that my gut feeling about the relative strength was wrong and apologise accordingly. Without having looked at all the other hypothetical matchups involving the four of you, I'm going to assume that I was also wrong about them and offer similar apologies!

drsteve
Posts: 727
Joined: Sun 27 Jun, 2010 7:23 am

Re: Reflections on the croco-puzzle ladder?

Post by drsteve » Sat 02 Jul, 2011 10:18 pm

nickdeller wrote:
drsteve wrote:
nickdeller wrote: I think we're also in a position with our current lineup where detuned and kiwijam are measurably the best two players on the ladder (no offence to drsteve and ronaldx who are very capable of upsetting the applecart, and a few others - me, for one - who could beat the formbook in an unusually lucky week),
Hmm... can I let this go? Sorry, Nick, but I'd say that Ronald and myself are on a par with the two named players.
...a quick look at the last two weeks of hypothetical matchups between you and kiwijam show a 5-5 and a 6-4 match, and so I happily concede that my gut feeling about the relative strength was wrong and apologise accordingly. Without having looked at all the other hypothetical matchups involving the four of you, I'm going to assume that I was also wrong about them and offer similar apologies!
S'okay. I'm a bit sensitive after some sub-par performances recently... must start doing my crocs when I'm actually awake.

nickdeller
Posts: 604
Joined: Tue 29 Jun, 2010 11:41 am

Re: Reflections on the croco-puzzle ladder?

Post by nickdeller » Sat 02 Jul, 2011 10:25 pm

drsteve wrote:S'okay. I'm a bit sensitive after some sub-par performances recently... must start doing my crocs when I'm actually awake.
Me too, actually. Horrible FVS just now - I don't think it's giving away much of the plot to remark that I *should*, by now, be able to fulfil a single-digit clue (edited to completely obfuscate!) without messing it up... :oops: At least it's on a Sunday. :-)

ronald
Posts: 58
Joined: Tue 08 Mar, 2011 1:22 pm

Re: Reflections on the croco-puzzle ladder?

Post by ronald » Sat 02 Jul, 2011 10:48 pm

Although I haven't been paying such close attention to the ladder - it's definitely the case that drsteve and I have suffered for having to play each other for the challenger spot every other week.

I've been surprised that detuned has successfully defended his top spot - although some puzzle-types clearly favour him, his tagesrating is generally lower than both mine and drsteve's. (and we do seem to beat him in off-weeks)

But, on balance I think the ladder has led to some nice and friendly match-ups that might be more difficult to engineer if there was a more straight ranking system. Eventually the current pattern will be broken and replaced with something else - the introduction of kiwijam should shake things up a little.

I vote for 'keep'.

Ronald

PuzzleScot
Site Admin
Posts: 2755
Joined: Fri 18 Jun, 2010 10:45 pm
Location: Edinburgh, Scotland

Re: Reflections on the croco-puzzle ladder?

Post by PuzzleScot » Sat 02 Jul, 2011 11:56 pm

...detuned and kiwijam are measurably the best two players on the ladder (no offence to drsteve and ronaldx who are very capable of upsetting the applecart...
Hey, this isn't a 2,4(or 5)-way race - I was only 3 seconds away from being 4-0 up on Steve after 4 puzzles this week. And Liane is par with me, with Chris (et al) about the same, so don't get all presumptive about static ladders... ;)

detuned
Posts: 1953
Joined: Mon 21 Jun, 2010 2:25 pm
Location: London, UK
Contact:

Re: Reflections on the croco-puzzle ladder?

Post by detuned » Sun 03 Jul, 2011 11:12 am

Haha...there are 6 nikoli types croco rates me as better at than heyawake Steve (indeed I'm 3rd overall at masyu) - as well as killer & inequality sudoku, and Gebietssummen which is essentially the same thing. On the other hand there are a few puzzle types that I seem to really suck at.

In my defence I'd say that I get the most top 10 scores out of any British puzzler on the ladder, and have 4 overall top scores to my name. Obvious the croco ladder factors in a more overall measure of performance, which sees me have lots of measurable plateaus, but I don't see myself too far behind the British names who are currently in front of me on the ladder. Basically I am fairly consistent and then every now and again have a shocker which wipes out a couple of days progress. Wash, rinse and repeat, unfortunately.

With regards to Chris's set-up, having the odd shocker here or there doesn't matter, because I am usually getting 3-4 top UK scores a week out of 10, which then obviously makes it very hard for any other given individual to beat me. I'm not entirely sure how accurate my analysis of kiwijam is, but it strikes me that he gets a higher proportion of top 10 results than either Ronald or Steve (or anyone else, including Alan 8-) ), and I think has a slightly better shot at knocking me off my perch. Which is not to say that Ronald/Steve/Anyone else had no chance at all, far from it.

I haven't got time to do it myself, but in the other thread I've talked about getting "best of British" times - it might be interesting to retrospectively see how these are spread out between competitors. It'd be my guess that this explains my long run at the top, despite lagging behind other players on the actual croco ladder.

dickoon
Posts: 615
Joined: Mon 21 Jun, 2010 12:52 pm
Location: Stockton-on-Tees, UK
Contact:

Re: Reflections on the croco-puzzle ladder?

Post by dickoon » Wed 06 Jul, 2011 9:44 pm

Thanks for all the feedback; more feedback is welcome, including from those not currently involved on the ladder.

One of the nice things about running the UK croco-puzzle ladder is that there is no right or wrong set of goals to have when designing the ladder and deciding what principles should be used in settling issues of design. I incline towards, in descending order of priority:
1. Run a competition that is not so onerous that it cannot be run.
2. Run a competition that is sufficiently fun that people wish to play in it.
3. Run a competition that rewards strong performance in the core activity of solving croco-puzzle puzzles.
However, different people find different things fun, so item 2 is a major challenge with no definite target.

The major problem so far, as far as I can tell, has been the degree of repetition in the matches generated. I have analysed the opponents generated by ten weeks (weeks 4 to 13) using each of the three match generation schemes.

CURRENT SCHEME:

drsteve: detuned x4, rodders x1, ronaldx x4, dickoon x1
detuned: drsteve x4, rodders x2, ronaldx x2, oenomel x2
rodders: drsteve x1, detuned x2, afcwarren x2, puzzlemad x2, oenomel x2, kiwijam x1
afcwarren: rodders x2, puzzlemad x3, oenomel x1
PuzzleScot: ronaldx x3, puzzlemad x1, dickoon x5
ronaldx: drsteve x4, detuned x2, PuzzleScot x3, dickoon x1
puzzlemad: rodders x2, afcwarren x3, PuzzleScot x1, dickoon x1, oenomel x1, kiwijam x1
dickoon: drsteve x1, PuzzleScot x5, ronaldx x1, puzzlemad x1
oenomel: detuned x2, rodders x2, afcwarren x1, puzzlemad x1
kiwijam: rodders x1, puzzlemad x1

PROPOSED SCHEME:

drsteve: detuned x5, rodders x2
detuned: drsteve x5, rodders x2, ronaldx x1
rodders: drsteve x2, detuned x2, PuzzleScot x2, ronaldx x4
afcwarren: PuzzleScot x1, ronaldx x1, puzzlemad x2, dickoon x1
PuzzleScot: rodders x2, afcwarren x1, ronaldx x3, puzzlemad x1, dickoon x1, oenomel x2
ronaldx: detuned x1, rodders x4, afcwarren x1, PuzzleScot x3, puzzlemad x1
puzzlemad: afcwarren x2, PuzzleScot x1, ronaldx x1, dickoon x2, oenomel x1, kiwijam x1
dickoon: afcwarren x1, PuzzleScot x1, puzzlemad x2, oenomel x1, kiwijam x1
oenomel: PuzzleScot x2, puzzlemad x1, dickoon x1
kiwijam: puzzlemad x1

THREE-WAY MATCH SCHEME:

drsteve: detuned x5, rodders x2, afcwarren x1, PuzzleScot x1, ronaldx x3, oenomel x1
detuned: drsteve x5, rodders x2, PuzzleScot x2, ronaldx x2, oenomel x1
rodders: drsteve x2, detuned x2, afcwarren x2, PuzzleScot x3, ronaldx x4, puzzlemad x3, dickoon x2, oenomel x1, kiwijam x1
afcwarren: drsteve x1, rodders x2, PuzzleScot x3, ronaldx x1, puzzlemad x2, dickoon x1, oenomel x2
PuzzleScot: drsteve x1, detuned x2, rodders x3, afcwarren x3, ronaldx x2, puzzlemad x3, dickoon x2, oenomel x3
ronaldx: drsteve x3, detuned x2, rodders x4, afcwarren x1, PuzzleScot x2, puzzlemad x2, dickoon x1, kiwijam x1
puzzlemad: rodders x3, afcwarren x2, PuzzleScot x3, ronaldx x2, dickoon x6, oenomel x2
dickoon: rodders x2, afcwarren x1, afcwarren x2, ronaldx x1, puzzlemad x6, oenomel x1, kiwijam x1
oenomel: drsteve x1, detuned x1, rodders x1, afcwarren x2, PuzzleScot x3, puzzlemad x2, dickoon x1, oenomel x1
kiwijam: rodders x1, ronaldx x1, dickoon x1, oenomel x1

What conclusions can we draw from this? Well, you tell me. I'll start with that I'm a huge effing nerd with too much time on my hands and should spend more time doing puzzles; granted, guilty as charged.

I do not know if there is a way to determine whether we can use these opponent statistics to determine which method is more fun or not; seeing as different people find their fun in different ways, I imagine this is necessarily subjective. That said, I tend to believe that most people will believe that close matches are fun and that repeat matches are less fun. This is a personal preference and one where I would be imposing my views if I said that I tended to believe it was commonly held, but I also tend to believe that moving up and down the rankings relatively quickly is fun. It's interesting to see circumstances where people can be in the championship match under one system and relegated altogether under another, though it goes to emphasise how tight the margins are already. I have a personal preference for people finding themselves in the particularly exciting championship and relegation matches frequently, and people moving from one extreme to the other quickly is not necessarily a bad thing.

With this in mind, while I'm open to persuasion, I am not instantly convinced of the merits of the proposed (i.e., odd weeks / even weeks) match generation system. I note that it generates matches between 19 pairs of opponents, as opposed to 20 pairs of opponents for the current scheme; I do not think that this necessarily makes the current scheme better in this regard, but I do think it makes the proposed scheme not obviously better. I'm not convinced that the alternate weeks where the champion does not defend their title makes the ladder more fun to participate in, and I'm inclined to find that the alternate weeks where the player at the bottom of the ladder has no chance to advance do not make things more fun, either. Additionally, there was at least one week where all the matches "went to form" with the consequence that the ladder positions did not change at all; I have a strong sense that that does not appeal to my sense of fun, but recognise that this is a personal preference. There may well be alternate implementations of similar sets of principles which work better and I would be happy to investigate the effects of alternate investigations with an open mind.

As for the three-way matches... well, they're interesting, rather more interesting than I had thought at first. I had not seriously considered them to begin with on what I suppose might be considered, well, aesthetic grounds; it's easy to get a visceral sense of competing against one opponent, it's rather harder to get a similar sense of competing against two, particularly when the results are effectively the summations of two-player pairings within the three-way match. Nevertheless, it means that people get to meet lots of opponents, it means that there's quite a bit of motion, it may even make the stakes slightly lower. What do people think, please? Are three-way matches more, less or similarly fun to two-way matches? (There clearly are open issues, not least regarding bye days and tie-breaks.) Would bigger matches still prove better?

Responding to other issues raised: I am inclined to reward slow, successful completion of a puzzle above non-participation, noting that big puzzles can be impractical and distinctly Not Fun to try to fit into real life. While the number of occasions on which this makes a difference is thankfully relatively rare, I wonder whether there would be sense in restricting the maximum time difference between two solvers' performances on the same puzzle to - say - half an hour? So if someone solved a puzzle in 40 minutes, a non-solution might be timed at 70 minutes rather than 9999 minutes. The open issue is whether to improve a losing performance of 80 minutes to one of 70 minutes in a similar way. Alan's suggestion about using performance ratings makes a great deal of sense, but there is not currently a convenient way to calculate performance ratings on a day-by-day basis; as berni has said, the raw data is published to make it possible at all, but it's not immediately convenient.

People's thoughts, particularly with regard to what they regard as being more or less fun, are invited.

PuzzleScot
Site Admin
Posts: 2755
Joined: Fri 18 Jun, 2010 10:45 pm
Location: Edinburgh, Scotland

Re: Reflections on the croco-puzzle ladder?

Post by PuzzleScot » Wed 06 Jul, 2011 10:44 pm

Goodness me. You have more time than I thought!

I like the 3-way system as it mixes up the matches much more. Matches are getting rather repetitive, I must say.

As for settling ties, would a loser's time always <= 2 x winner's time work? (It would in a 2-way match system)
That way, whether days are missed, FVSs are made, or simply that a solve is interrupted by something important, you aren't written off when it comes to the day of reckoning.

Post Reply