ToolS for Developers

These are various resources and tools for developers. Normies get out reee.

The basics

These are the basic repos related to llama-cpp.

GGML main repo
This is the main repo for GGML. This is the model format for llama-cpp. This might be relevant for you if you do model training and conversion.

Llama-cpp main repo
The main llama-cpp repo. Check the pull request here for upcoming features.

Llama-cpp python bindings
The llama-cpp python bindings. Can be used to run llama-cpp in chat and server mode. Has a decent api.

Integration

Sometetimes you just don't want to reinvent the wheel.

Langchain
Python framework with integrations for many tools. Has support for llama-cpp-python and various other AI tools. Has support for embeddings and vector stores.

Chainlit
For constructing a UI for llm's. This is an easy drop in tool to create a basic ui. Uses python and typescript. Good for testing models and is compatible with langchain.

VectorDb

Vector databases. You might need these for document embeddings.

ChromaDb
A local vector store. Lightweight and easy to use. Has support in langchain.

Grammar Builders

Llama-cpp has grammar option that can force the output to be in a specific format. You can use grammar builders to create the grammar files for the llm. Grammar builder for llama-cpp
Llama-cpp can be set to return output in specific format. This is a tool to create the format files more easily.

Another grammar builder for llama-cpp
Llama-cpp can be set to return output in specific format. This is a tool to create the format files more easily.

Text Parsing and NER

Parsing entities like names or locations from text automatically. Nice for embeddings metadata.

Spacy
Spacy offers pretrained models and nice features for text processing.

Sentence Transformers for Spacy
Sentence Transformers
Sentence transformers are a lightweight alternative to Spacy.

Doctran
Tool for text processing.

Textacy
Tool for text processing.

Search Algorithms

Bm25
Search engine algorithms in python.

Model finetuning

For tuning the model. Get it?

Axolotl
The swiss armyknife of finetuning.

Mergekit Ah yes, the source of frankenmerges that keeps on giving. Putting several good things together makes for something even better. I think thats science or something.