Calling Your Game With Your Phone

Whelp, I've finally put together a series of 3 posts/tutorials/guides/blogs/devlogs/whatever to describe what we did here, check it out:

https://github.com/deepgram-devs/game-twilio-deepgram-distributor

That repository contains a README which contains links to 3 branches, each with their own (version of) code  and accompanying README. Part 1 provides/describes a server which allows you to use Deepgram and Twilio to call into a video game session, streaming live transcription of your voice on the phone, with an example Godot game linked. Part 2 extends the server to enable your game to talk back to the user on the phone via a text-to-speech provider. And Part 3 finally discusses in more detail the game we built here using this tech: Robot Dreams.

Since those posts go into a lot of detail, I'll keep this post somewhat brief. Below is a diagram describing the system we've built:

When you play Robot Dreams, you may eventually reach the payphone which tells you to call a number - at this point in the game, the game connects to the `game-twilio-deepgram-distributor` server (which we'll just call "server") and the server sends the game a phone number to call, and a unique game code/password (I chose the game codes to be the names of some fruits or herbs). If you call the phone number and say the game code/password, you will automatically win the game with the "good ending"!

What's going on is described in detail in the `game-twilio-deepgram-distributor` posts, but in short, I've set up a Twilio phone number which, when you call it, will stream audio to the server, which then forwards the audio to Deepgram, receives transcripts, checks if a game code/password associated with an independently connected game client was spoken, and if it was, tell that to the game client which will then trigger the game to win. Pretty meta stuff (maybe?). The game will also send text to the server, which forwards audio of that text (via Amazon Polly text-to-speech) to the phone call to let the user know they won.

Anyways, this was a really interesting one to work on, and I intend to use this server for more games in the future - possibly adding chatbots to it to enable you to call NPCs in the game and have conversations! (Order a pizza? Dating sim? Who knows!)