Awesome LLMOps
Section: Experiment Tracking · Sacred is a tool to help you configure, organize, log and reproduce experiments.
Entry
Appears in 6 awesome lists
Python tool to help you configure, organize, log and reproduce experiments. Like a notebook lab in the context of Chemistry/Biology. The community has built multiple add-ons leveraging the proposed standard.
Section: Experiment Tracking · Sacred is a tool to help you configure, organize, log and reproduce experiments.
Section: Tools · Python tool to help you configure, organize, log and reproduce experiments. Like a notebook lab in the context of Chemistry/Biology. The community has built multiple add-ons leveraging the proposed standard.
Section: Model Lifecycle · A tool to help you configure, organize, log and reproduce experiments.
Section: Model, Data and Experiment Management · Tool to help you configure, organize, log and reproduce machine learning experiments.
Section: Machine Learning Frameworks · Sacred is a tool to help you configure, organize, log and reproduce experiments developed at IDSIA.
Section: Experimentation · A tool to help you configure, organize, log, and reproduce experiments.
Transpile trained ML models into other languages. sklearn-porter - Transpile trained scikit-learn estimators to C, Java, JavaScript and others. mlflow - Manage the machine learning lifecycle, including experimentation, reproducibility and deployment. skll - Command-line utilities to make it easier…
Test your prompts, models, RAGs. Evaluate and compare LLM outputs, catch regressions, and improve prompt quality. LLM evals for OpenAI/Azure GPT, Anthropic Claude, VertexAI Gemini, Ollama, Local & private models like Mistral/Mixtral/Llama with CI/CD
All-in-one web-based IDE for machine learning and data science. The workspace is deployed as a docker container and is preloaded with a variety of popular data science libraries (e.g., Tensorflow, PyTorch) and dev tools (e.g., Jupyter, VS Code).
| Python | - A scalable general purpose micro-framework for defining dataflows. You can use it to build dataframes, numpy matrices, python objects, ML models, etc. Embed Hamilton anywhere python runs, e.g. spark, airflow, jupyter, fastapi, python scripts, etc.
AI-native database built for LLM applications with incredibly fast hybrid search of dense vector, sparse vector, tensor (multi-vector), and full-text. Powers RAGFlow's document engine. Apache 2.0 licensed.
Praison AI is a low-code, centralized framework leveraging AutoGen and CrewAI to simplify creating and orchestrating multi-agent systems for LLM applications, emphasizing customization and ease of human-agent interaction github | demo | website