Skip to content

Additional voting methods #99

@jpintar

Description

@jpintar

Description of feature

It would be great if some additional voting methods were implemented to aggregate across classifiers apart from simple first-past-the-post voting. The basic idea would be to treat the classifier certainties for each cell as a ranked ballot, and compute the election results using some selection of methods, like Instant Run-off, Copeland's method, or Borda count (naturally, there's no expectation that one settles the question of which voting method is optimal – a small group of representative ones will do). There are existing Python packages, like pref_voting, that implement multiple methods and could potentially be used for this.

Additionally, even without aggregating the votes in multiple ways, treating the classifier certainties as rankings opens the possibility of computing finer-grained summary statistics for classifier agreement, like Kendall's W. If we only count up how many classifiers agree on their first choice, we leave a lot of information on the table.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions