Introducing Wordle Benchmark
A Python framework for evaluating AI/ML Wordle agents
Do you like Wordle? I like Wordle. I wanted to see if I could teach my computer to play a decent game, but I couldn’t find a good way to grade my solutions.
I put together a Python package to play Wordle games and keep track of game statistics for Wordle “agents”: Python software written to play Wordle.
Getting started
Let’s walk through the steps to install wordle-benchmark.
From pip
The easiest option is a pip installation.
From git
If you want to install a version that isn’t yet reflected in PyPi, you can use pip to install directly from the repo.
From a local clone
If you want to contribute to the source, you’ll want to install locally.
To call the scripts, there are some additional dependencies reflected in the requirements file.
Sample use
Want to write your own Wordle agent and see how well it can play? Let’s explore the basic logic to interact with the wordle-benchmark package. I’ll walk through the steps of this script.
An “Agent” plays a “Game” and submits “GuessWord”s to try to guess the “TargetWord”. A “Dictionary” stores all valid “GuessWord”s or “TargetWord”s. You can run multiple “Game”s with a list of “TargetWord”s and get summary statistics using “Benchmark”
An agent just needs to implement a single abstract signature “play.”
It takes a Game as an argument, the Game includes all the information you need about how many guesses have been made, what black/yellow/green letters you’ve matched, what words in the dictionary are still possible, etc.
To demonstrate the implementation, let’s take a peak at a basic agent that always predicts “crane” on the first turn, and guesses the most common word that is still possible on every subsequent turn.
And now we can evaluate it against a list of words to calculate statistics on how well it performs.
Now let’s evaluate!
This agent is just a simple example! I can’t wait to see the agents you make!
Conclusions
Wordle-benchmark provides an easy interface to experiment with AI/ML solutions to Wordle. It is a relatively simple game, making it an excellent learning exercise. Give it a shot and let me know what you come up with!
Let me know if you find any bugs! Feel free to contribute. Message or comment if you come up with interesting Wordle agents!
