Overview
llmware is a unified, open source, extensible framework for LLM-based application patterns including Retrieval Augmented Generation (RAG). For developers and enterprises to use Generative AI to deliver real business value, they need to connect their knowledge to AI. With llmware, you can go from enterprise data to generative AI in minutes.
This python package of llmware provides a comprehensive set of tools that anyone can use to rapidly build industrial-grade enterprise LLM-based applications. Key components include ingesting data, vector embeddings, creating datasets, connecting to proprietary and open-source embedding and inferencing models, auditing, and analytics.
Highlights
- Tools for sophisticated generative scenarios such as RAG (Retrieval-Augmented Generation). Rich model support for popular LLMs, Hugging Face generative models, and Ai Bloks READ-GPT. Prompt Handling with pre-built and custom prompts along with context prep from various sources (files, websites, audio, etc.). Post-Processing with fact-checking, response classification, and AI Auditability.
- Parsing and Text "Chunking" capabilities. Parsers for PDF, PowerPoint, Word, Excel, HTML, Text, WAV, and AWS Transcribe transcripts. Handles extraction of information and metadata into a consistent "block" format.
- Retrieval and assembly of fact-sets from a corpus of information. A comprehensive set of querying methods: Semantic, Text, and Hybrid retrieval with integrated metadata. Ranking and filtering strategies to enable Semantic query and rapid retrieval of information. Web scrapers, Wikipedia integration, Yahoo Finance API integration to help assemble fact-sets for generation.
Details
Typical total price
$0.371/hour
Pricing
Instance type | Product cost/hour | EC2 cost/hour | Total/hour |
|---|---|---|---|
t2.xlarge | $0.00 | $0.186 | $0.186 |
t2.2xlarge Recommended | $0.00 | $0.371 | $0.371 |
m5.xlarge | $0.00 | $0.192 | $0.192 |
m5.2xlarge | $0.00 | $0.384 | $0.384 |
m5.4xlarge | $0.00 | $0.768 | $0.768 |
p3.2xlarge | $0.00 | $3.06 | $3.06 |
p3.8xlarge | $0.00 | $12.24 | $12.24 |
p3.16xlarge | $0.00 | $24.48 | $24.48 |
p4d.24xlarge | $0.00 | $32.773 | $32.773 |
g5.xlarge | $0.00 | $1.006 | $1.006 |
Additional AWS infrastructure costs
Type | Cost |
|---|---|
EBS General Purpose SSD (gp2) volumes | $0.10/per GB/month of provisioned storage |
Vendor refund policy
N/A - This is a free product.
Legal
Vendor terms and conditions
Content disclaimer
Delivery details
64-bit (x86) Amazon Machine Image (AMI)
Amazon Machine Image (AMI)
An AMI is a virtual image that provides the information required to launch an instance. Amazon EC2 (Elastic Compute Cloud) instances are virtual servers on which you can run your applications and workloads, offering varying combinations of CPU, memory, storage, and networking resources. You can launch as many instances from as many different AMIs as you need.
Version release notes
This AMI contains llmware 0.1.3, Python 3, Docker, Docker Compose, MongoDB and Milvus. See /home/ubuntu/readme.txt for information about getting started.
Additional details
Usage instructions
Deployment Guidance:
- Instance Type: For best results across a wide variety of use-cases we recommend running on an instance with at least 8 CPU and 32 GB of Memory.
- EBS Storage Size: The default storage size is 100GB. If you plan to run heavy workloads involving significant numbers of documents or large models loaded locally, it is recommended to increase the default size.
Getting Started with llmware can be found in 2 places:
- llmware AWS Marketplace README on GitHub: https://github.com/llmware-ai/llmware/blob/main/README_AWS.md
- Inside the instance, see ~/readme.txt.
References:
- llmware usage examples: locally in ~/llmware/examples or on GitHub at https://github.com/llmware-ai/llmware/tree/main/examples
- llmware videos: https://www.youtube.com/@llmware
Resources
Support
Vendor support
Support is available through the llmware discussion forum and technical information on Github. Post your questions to the llmware discussions at: https://github.com/llmware-ai/llmware/discussions .
AWS infrastructure support
AWS Support is a one-on-one, fast-response support channel that is staffed 24x7x365 with experienced and technical support engineers. The service helps customers of all sizes and technical abilities to successfully utilize the products and features provided by Amazon Web Services.
Similar products
