Market icon

Will Google have the best LLM before 2024?

$3,935 Vol.

50% chance

Rules

If Google deploys an LLM that reaches the top of the Elo leaderboard on https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard by December 31, 2023, 11:59 PM ET, this market will resolve to "Yes". Otherwise, this market will resolve to "No".

Only models accessible to the public (e.g. developers or public sign-ups) will count for this market.

If Google deploys a new model in 2023 that could reach the top of the leaderboard but is not yet ranked at year's end due to time required for evaluation, this market's resolution may be delayed until February 1, 2024, 11:59 PM ET. If at this point the resolution source still does not list the new model's ELO, the underlying script(s) will be used to determine whether google's model tops the leaderboard. Note that the model must be released by the resolution date to be considered (if they release a new version of the model after the resolution date, it will not be considered for this market).

This market will resolve based on the leaderboard on Elo (https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard). If the resolution source becomes permanently unavailable by the resolution date, the underlying script to calculate google's model ELO on Chatbot Arena will be used directly.

Volume
$3,935
End Date
Dec 31, 2023
Created At
Dec 6, 2023, 11:17 PM UTC

Outcome proposed: No

No dispute

Final outcome: No

shield

Beware of external links.

$3,935 Vol.

Market icon

Will Google have the best LLM before 2024?

50% chance

About

If Google deploys an LLM that reaches the top of the Elo leaderboard on https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard by December 31, 2023, 11:59 PM ET, this market will resolve to "Yes". Otherwise, this market will resolve to "No".

Only models accessible to the public (e.g. developers or public sign-ups) will count for this market.

If Google deploys a new model in 2023 that could reach the top of the leaderboard but is not yet ranked at year's end due to time required for evaluation, this market's resolution may be delayed until February 1, 2024, 11:59 PM ET. If at this point the resolution source still does not list the new model's ELO, the underlying script(s) will be used to determine whether google's model tops the leaderboard. Note that the model must be released by the resolution date to be considered (if they release a new version of the model after the resolution date, it will not be considered for this market).

This market will resolve based on the leaderboard on Elo (https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard). If the resolution source becomes permanently unavailable by the resolution date, the underlying script to calculate google's model ELO on Chatbot Arena will be used directly.

Volume
$3,935
End Date
Dec 31, 2023
Created At
Dec 6, 2023, 11:17 PM UTC

Outcome proposed: No

No dispute

Final outcome: No

shield

Beware of external links.