Arjun Patel
8870741c3e
* capture audio with alsa * add brainstorm thoughts * use basecamp project for managing this project * add libcurl basic example * write pcm data to file * transcribe audio file with deepgram * transcribe raw audio * hit anthropic API for ai question * create full flow with stt, anthropic, tts, and playback This organizes some components like deepgram into own module, and also allows gets the full flow to work every time we run the program. * organize audio and intelligence modules * organize header files into include directory * docs: add readme * docs: explain future work * docs: add disclaimer about hardcoded audio params
Conversation
This proof of concept is for implementing conversational loop with AI at a low level with raw C.
How it works
This simple program:
- Captures audio samples for x seconds into a local file using alsa
- Sends the raw pcm data to Deepgram STT API for transcription
- Uses the transcript to call Anthropic to get an assistant's response
- Converts Anthropic's response to audio using Deepgram's TTS
- Plays back the generated audio from Deepgram using alsa
- Quits. There is no conversational loop at the time of this writing.
Requirements
Platform: the project is meant for Linux x64. It was successfully run on arch linux, with no guarantees otherwise.
- libcurl
- build-essential
- libasound2-dev
See more deps in Makefile install-deps command.
Getting started
- Add
DEEPGRAM_API_KEYandANTHROPIC_API_KEYto.bashrcas environment variables. - Build & run
make
./main
Audio configuration
Please see the audio.h and audio.c as the params for alsa were hardcoded based on my local arch linux x64 machine. You may have to adjust it based on your setup (e.g. you may have a mic with 2 channels/stereo, which would change the size of each frame).
Future work
Add an event loop, maybe add multi-threading, so that we can interrupt the AI, ask follow-up questions, and make it sound like a back and forth conversation.