There’s no denying Nvidia's historical dominance when it comes to AI training and inference. Nearly all production AI workloads run on their graphics cards. However, there’s been some optimism recently around AMD, seeing as the MI300X, their intended competitor to Nvidia's H100, is strictly

What is vLLM? vLLM is an open-source LLM inference and serving engine that utilizes a novel memory allocation algorithm called PagedAttention. It can run your models with up to 24x higher throughput than HuggingFace Transformers (HF) and up to 3.5x higher throughput than HuggingFace Text Generation Inference (TGI). How

We are thrilled to introduce the latest addition to the RunPod platform: Serverless CPU. This feature allows you to create high-performance VM containers with up to 3.75 GHz deviated cores, DDR5 memory, and NVME SSD storage. With Serverless CPU, you have the flexibility to choose between Compute-Optimized or General

When connecting to a RunPod Pod, a common issue is that SSH doesn't work out of the box. In this tutorial, we will examine a method of using a username and password to access a RunPod Pod through SSH. By the end of this guide, you'll

Coframe uses RunPod Serverless to scale inference from 0 GPUs to hundreds in minutes. With RunPod, Coframe launched their generative UI tool on Product Hunt to thousands of users in a single day without having to worry about their infrastructure failing. In under a week, Coframe was able to deploy

When Giacomo, Founder and CPO of KRNL, reached out to RunPod in May 2023, we weren’t actually sure if we could support his use case. They needed a provider that could cost-effectively scale up to handle hundreds of thousands of users, and scale back down to zero in minutes.

RunPod introduces Configurable Templates, a powerful feature that allows users to easily deploy and run any large language model. With this feature, users can provide the Hugging Face model name and customize various template parameters to create tailored endpoints for their specific needs. Why Use Configurable Templates? Configurable Templates offer

Today, we're announcing the integration between Runpod and dstack, an open-source orchestration engine, that aims to simplify the development, training, and deployment of AI models while leveraging the open-source ecosystem. What is dstack? While dstack shares a number of similarities with Kubernetes, it is more lightweight and focuses

Virtual Staging AI, an innovative startup from the Harvard Innovation Lab, is transforming the real estate industry by leveraging cutting-edge AI technology and RunPod's powerful GPU infrastructure. Their state-of-the-art solution enables realtors to virtually stage properties in just 30 seconds at a fraction of the cost of traditional

💡RunPod is hosting an AI art contest, find out more on our Discord in the #art-contest channel. In this tutorial, you will learn how to generate images using Stable Diffusion, a powerful text-to-image model, on the RunPod platform. By following the step-by-step instructions, you'll set up the prerequisites,

RunPod is excited to announce its latest integration with SkyPilot, an open-source framework for running LLMs, AI, and batch jobs on any cloud. This collaboration is designed to significantly enhance the efficiency and cost-effectiveness of your development process, particularly for training, fine-tuning, and deploying models. What is SkyPilot? SkyPilot is

In the rapidly evolving world of artificial intelligence and machine learning, the need for powerful, cost-effective hardware has never been more critical. The launch of the A40 GPUs marks a significant milestone in this journey, offering unparalleled performance and affordability. These GPUs are designed to cater to the needs of