Project Overview
Objective
Engineered a unified RunPod Serverless GPU worker for high-throughput image and video synthesis using PyTorch, Diffusers, InsightFace face conditioning, and S3 artifact storage.
Stack
PythonPyTorchDiffusersTransformersRunPod ServerlessCUDAHugging FaceInsightFaceONNX Runtime GPUAWS S3DockerFFmpegFastAPIPydantic
Delivery highlights
- Developed a unified RunPod Serverless GPU worker for the Generative Studio platform, designed to execute asynchronous image and video generation workloads on demand. Built around the official PyTorch 2.6 CUDA 12.6 runtime (`pytorch/pytorch:2.6.0-cuda12.6-cudnn9-runtime`), the service bridges incoming RunPod job dispatches to dedicated image and video diffusion pipelines, serializes output media to AWS S3, and reports status callbacks with secret-based webhook verification.
- Supported dual media synthesis pathways: an image generation pipeline leveraging Hugging Face Diffusers, Transformers, PEFT, and LoRA with optional inpainting and FaceID identity conditioning; and a video generation pipeline orchestrating frame-by-frame diffusion synthesis, optional face conditioning, and temporary MP4 packaging via FFmpeg before cloud persistence.
- Integrated face recognition and identity conditioning using InsightFace and ONNX Runtime GPU, enabling personalized character synthesis and face swapping across generated imagery and video frames. Automated bounded input handling by validating user-supplied image and face URLs, applying size constraints, and streaming binary assets safely.
- Engineered GPU memory management with an in-memory LRU pipeline caching mechanism, keeping frequently requested Diffusers models warm in VRAM while evicting idle weights to prevent out-of-memory crashes on containerized GPU instances. Added throttled asynchronous progress reporting that streams percent-complete updates back to the API webhook without flooding the network.
- The reviewed snapshot documents this worker as a deployment component of the `generative-studio` repository. Before production deployment, exposed local environment credentials must be rotated, face-image fetching should be unified with strict HTTPS domain allowlists, and safety checking or content moderation policies must be restored before public traffic.