How to Build and Deploy GenAI Inference Anywhere—in Just Hours
2
81
0
Authors:
Steve Williams, CTO, Iternal Technologies
Vaishali Ghiya, Executive Officer - Partnerships, Denvr Dataworks
Rory Finnegan, Solutions Engineer, Denvr Dataworks
May 1, 2025
Estimated read time: 4.5 minutes.
Introduction
As GenAI inference moves closer to the point of use—whether it’s a laptop in the field, a remote office workstation, or a secure environment with strict data controls—organizations face a new challenge: how to build, package, and deploy large language models (LLMs) outside the traditional cloud. The goal is to deliver intelligence where it's needed, even in environments without internet access or with stringent data privacy requirements.
In this blog, we walk through how Iternal Technologies used AirgapAI™, its fully offline inference platform, in combination with Denvr AI Services to streamline the end-to-end development of a GenAI assistant. With preloaded frameworks, managed infrastructure, and GPU-backed performance, Denvr’s AI-native platform accelerated the workflow from model prep to field-ready packaging, completing the full process in under six hours.
AirgapAI: Local Inference for Real-World Use
Developed by Iternal Technologies, AirgapAI enables organization to run powerful open-source LLMs—like LLaMA 3.2 or Mistral 7B—entirely on standard hardware, without internet access or cloud infrastructure. Designed for secure, self-contained deployments, it delivers a ChatGPT-like experience in settings where data privacy, reliability, and localized execution are paramount.
AirgapAI serves multiple audiences within an organization. Technical teams—such as AI architects, DevOps professionals, or solution owners—use Denvr AI Services to ingest documents, structure datasets with Iternal’s Blockify® technology, and package models into portable runtimes. Business users in the field—such as sales teams, consultants, or analysts—rely on AirgapAI to run AI tools locally and securely on their devices, without any cloud or internet connection.
Why Local Inference Requires a Different Approach
Local AI inference offers critical benefits:
Enhanced Security and Privacy – Data remains on-device, reducing exposure to external threats.
Reduced Latency – Immediate responses without reliance on cloud connectivity.
Operational Reliability – Continued functionality in environments with limited or no internet access.
But implementing inference locally isn't simple. Teams often face challenges in model optimization, ensuring compatibility across hardware configurations, and maintaining response quality without the continuous support of cloud services. These obstacles can delay deployment or require teams to stitch together multiple tools and environments—leading to inefficiencies and added complexity.
Built on Denvr AI Services
Denvr AI Services provided the foundation for Iternal’s development workflow—delivering a ready-to-use, AI-native environment with pre-installed frameworks, NVIDIA A100 acceleration, and persistent virtual machines. Unlike general-purpose cloud platforms, Denvr eliminated the friction of setup and environment tuning, enabling Iternal’s team to move from VM startup to active model refinement in minutes—not hours.
This AI-focused infrastructure played a central role in supporting the full workflow—from model selection and optimization to packaging and deployment—within a single environment. Pre-configured instances removed the guesswork from dependency management, while persistent storage preserved project state between sessions, reducing iteration time and avoiding rework.
By building entirely on Denvr’s platform, Iternal ensured that each stage of the development workflow—from data ingestion to model tuning and runtime packaging—was executed in a consistent, reliable environment.
No setup headaches. No guesswork. Just build and go.
Engineering GenAI for Anywhere Use
The following workflow (see Figure 1) outlines how Iternal Technologies used Blockify, AirgapAI, and Denvr AI Services with NVIDIA A100 GPUs to build a production-ready GenAI inference solution—developed rapidly using a single, streamlined environment.
Blockify, Iternal’s patented data ingestion technology, enhances retrieval accuracy by optimizing content structure and semantic vector precision. Paired with AirgapAI’s offline runtime, it enables fast, high-confidence inference in disconnected or high-security environments.
1. Model Selection & Optimization:
Open-source LLMs (e.g., LLaMA 3.2) were fine-tuned using Low-Rank Adaptation (LoRA) in Denvr AI Service’s pre-configured NVIDIA A100 GPU environments —ready-to-use without the typical setup overhead.
2. Data Ingestion & Structuring:
Documents were processed using Blockify to generate modular content and dense semantic vectors. The result: up to 78× accuracy gains, 97.5% content reduction, and ingestion speeds near 900 words per second.
3. LLM Integration & RAG:
The optimized model and dataset were integrated into a retrieval-augmented generation (RAG) pipeline to support context-aware, task-specific inference.
One of the technologies that made offline deployment possible was MLC LLM—an open-source, Apache 2.0-licensed ML compiler that generates optimized, platform-agnostic binaries. By enabling ahead-of-time compilation, it allowed Iternal to package inference-ready LLMs without runtime dependencies—an ideal fit for AirgapAI.
4. Packaging with AirgapAI:
The team used AirgapAI to create a portable runtime that bundled the LLM, embeddings, vector database, retrieval logic, and all required inference components into a single deployable unit. Thanks to MLC LLM, the package was optimized for execution on standard hardware without needing internet access, downloads, or platform-specific tuning. The entire packaging process was completed within the Denvr AI Services environment.
5. Deployment:
The self-contained application was deployed to air-gapped Windows-based field systems, requiring no internet access or runtime configuration—ideal for regulated or network-isolated environments.
The result? A fully packaged GenAI inference assistant, ready to run locally—built and deployed in a fraction of the time of traditional workflows.
What the Streamlined Workflow Delivers
This end-to-end process—integrated on Denvr AI Services—delivered measurable benefits:
Streamlined and Persistent: Denvr’s AI-native platform—combining pre-configured environments, protected VM storage, and GPU acceleration—minimized setup time and eliminated rework. This infrastructure enabled rapid progression from model preparation to field-ready deployment, compressing what would typically take days into a significantly shorter timeframe.
Accuracy and Reliability: Blockify’s modular datasets improved retrieval precision and dramatically reduced hallucinations. Combined with RAG integration, the result was up to 78× higher response accuracy compared to traditional methods.
Operational Control: Local deployment enabled strict data governance and eliminated reliance on external connectivity. Denvr’s pre-configured infrastructure minimized environment setup, letting teams focus entirely on model development and testing.
Business Applications of Inference Anywhere
The combined solution of AirgapAI, Blockify, and Denvr AI Services enables field-ready local inference for a range of critical business functions:
Professional Services: Field teams could securely access client materials and project intelligence offline, improving responsiveness and insight quality.
Healthcare: Clinicians can retrieve protocols securely without needing internet access—supporting time-sensitive decisions without risking patient data.
Field Operations: Engineering and inspectors are able to access large volumes of operational documentation on-site—reducing delays caused by connectivity issues or data gaps.
Conclusion: What It Really Takes to Deliver GenAI Inference Anywhere
Iternal Technologies’ AirgapAI demonstrates that high-performance GenAI inference can be readily built and deployed for secure, remote, or disconnected environments on standard hardware—without requiring constant connectivity or complex infrastructure. When developed on Denvr AI Services and its integrated AI-native development stack—including preloaded frameworks, GPU-accelerated infrastructure, and persistent virtual machines—Iternal created a ChatGPT-like assistant optimized for high-performance inference in places where cloud connectivity isn’t an option.
The result: AirgapAI was packaged and deployed in under six hours and achieved up to 78× improvements in retrieval accuracy using Iternal’s Blockify technology for modular content structuring and precision vector search. This project proves that deploying high-quality GenAI inference doesn’t require massive infrastructure or weeks of integration work. With the right stack and process, offline AI can be both practical and production-ready.
Ready to build GenAI inference for anywhere use? Start with Iternal Technologies’ AirgapAI and Denvr AI Services.
