If I understand correctly, this sounds like an idea that I've been thinking about for a little while.
Essentially, you do binary search-insertion into a list, where the comparison function is a prompt to the user asking "Is A better than B?" (If it's too difficult to judge "betterness" between two items, you could just as easily swap in a different comparison. "Is A funnier than B?")
One thing that people always ask when I mention this is: "What if A and B are equal?" Well, then you answer no, because A is not better than B. If your answers are consistent, then A and B will end up next to each other in the list.
yup -- I believe this is a better user experience in terms of capturing ratings (by capitalising on the availability heuristic)
What you're describing technically would work when each person compares every item (and would fall into the domain of condorcet methods).
However in practice the election becomes a graph (rather than list or x/y table) with cyclical dependencies and conflicting comparisons -- it becomes quite hard to resolve -- but it can be.
I envisioned starting with an empty list, and populating it with the user's comparisons as they come in. That way, you don't have to deal with unrated items.
Cyclical/conflicting comparisons are a function of faulty users, the algorithm can't take the blame for that! ;)
Essentially, you do binary search-insertion into a list, where the comparison function is a prompt to the user asking "Is A better than B?" (If it's too difficult to judge "betterness" between two items, you could just as easily swap in a different comparison. "Is A funnier than B?")
One thing that people always ask when I mention this is: "What if A and B are equal?" Well, then you answer no, because A is not better than B. If your answers are consistent, then A and B will end up next to each other in the list.