How to build in Minecraft with your voice, using an AI

How do I build in Minecraft with my voice?

1. Open a world and hold V, or click the microphone in the panel.
2. Say what you want in your own words, the way you would type it.
3. Let go of the key. Your words appear in the conversation as text, and the build starts like any typed request.

V is the Voice key, and you can change it (see the questions at the end). With the microphone button, listening stops by itself after a moment of silence, and gives up after 8 seconds if you say nothing.

One listening lasts 30 seconds at most, and less than half a second of speech is taken for a tap, not a sentence.

Where does my voice go?

The speech is recognised on your own computer, by an engine called whisper.cpp that travels inside the mod. There is no account, no key and no speech service involved.

The sound is not sent anywhere and is not written to a file. What goes on is the text of your request, to your own Claude Code on your computer, exactly like a request you typed.

The speech model is a data file. The mod downloads it once from this website into config/craftai-voice/, and refuses a file whose SHA-256 fingerprint is not the expected one. After that, the recognition itself needs no connection.

Which speech model does it use, and how big is it?

There are two models, both multilingual. You choose with Voice quality, in the Voice section of the Settings tab.

Each model is downloaded once, the first time it is needed, and checked by its fingerprint.

What do I need for voice mode to work?

• A microphone. If the mod finds none, it says so and sends nothing.
• On a Mac: the launcher needs the microphone permission, in the privacy settings of the system. Without it the mod hears only silence, and says so.
• On Windows: desktop apps must be given access to the microphone, in the privacy settings of the system. The processor must support AVX2: the mod says so instead of crashing.
• On Linux: the libgomp1 package (Debian, Ubuntu) or libgomp (Fedora, Arch) must be installed, and the microphone must not be muted in your sound settings.

The speech engine is in the mod for macOS (Apple Silicon and Intel), for Windows (64-bit Intel or AMD) and for Linux (64-bit Intel or AMD, and ARM). The troubleshooting list of the guide gives the exact path on each system.

Which language can I speak?

Any language the speech model knows: the engine finds the language by itself. Your request is then answered in the language you used.

The mod also gives the engine a few sentences of the kind a builder says, in the language of the interface, so that it spells Minecraft and building words right more often. We have not measured the recognition in other languages than English, French, Spanish and German.

What you say is what you would type: what a good request contains applies word for word.

What has been tried, and what has not?

This is where voice mode stands in version 0.13.3.

• Tried: in the real game on a Mac (Apple Silicon), with the real speech engine and a recorded sentence.
• The libraries for Windows, Linux and Intel Macs start in the automated tests.
• A Windows or Linux microphone has not been tried by a person yet.
• With Forge and NeoForge, dictation has not been tried with a real microphone yet.
• The real microphone under Lunar Client could not be tried by the automated tests.

The standard model is quick, and it does make mistakes: a long or noisy sentence can be misheard. You see in the conversation what the engine understood.

Frequently asked questions