talksikflowy.stream
This is a proof of concept of a showing widgets within a conversation based on intent.
Todo
- Integrate llama.cpp with local inference. This will set us up for building many parts of experience.
- Disect what llama is doing and what the Phi model is doing.
- Create conversational loop with chat and running context.
- Generate & render different types of blocks: list, email, doc, etc.
- Try different models with hugging face
- Render TUI elements based on commands
Dependencies
llama.cpp
#model weights
wget https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/resolve/main/Phi-3-mini-4k-instruct-q4.gguf
cd ./thirdparty/llama.cpp
rm -rf build
mkdir -p build
cmake -S. -Bbuild -DLLAMA_CURL=OFF
cmake --build build
The libraries should be in ./thirdparty/llama.cpp/build/bin.
NOTE: you may have to disable curl as a flag when configuring llama.cpp via cmake.
Getting started
make
LD_LIBRARY_PATH=./thirdparty/llama.cpp/build/bin/ ./stream_app