Core Inference Through Triton

Core Inference Through Triton A workflow diagram generated by Archify. 01 / POST /detect 02 / POST /faces/recognize 03 / POST /embed/image 04 / POST /ocr/predict Decode + letterbox · 640x640 · POST /detect Decode + letterbox 640x640 YOLO end2end · yolov11 / yolo26 · POST /detect · GPU NMS YOLO end2end yolov11 / yolo26 GPU NMS Decode + resize · 640x640 · POST /faces/recognize Decode + resize 640x640 SCRFD-10G · boxes + landmarks · POST /faces/recognize SCRFD-10G boxes + landmarks Align (CPU) · Umeyama 112x112 · POST /faces/recognize Align (CPU) Umeyama 112x112 ArcFace · 512-dim · POST /faces/recognize ArcFace 512-dim Preprocess · image or tokens · POST /embed/image Preprocess image or tokens MobileCLIP2-S2 · image / text, 512-dim · POST /embed/image MobileCLIP2-S2 image / text, 512-dim Preprocess · OCR input · POST /ocr/predict Preprocess OCR input ocr_pipeline · Python BLS · POST /ocr/predict ocr_pipeline Python BLS PP-OCRv5 det · text regions · POST /ocr/predict PP-OCRv5 det text regions PP-OCRv5 rec · CTC decode · POST /ocr/predict PP-OCRv5 rec CTC decode gRPC gRPC decode + NMS gRPC gRPC BLS gRPC crops (BLS) Legend API CPU Triton GPU

One gRPC client

  • • Every route shares one Triton gRPC client, so Triton's dynamic batcher sees concurrent requests together
  • • Boxes come back normalized to 0-1
  • • Green steps run on the API CPU; orange steps are Triton models on the GPU

Composed routes

  • • /analyze runs detection, faces, CLIP and OCR on one image
  • • /ingest also writes global, face, object and OCR documents to OpenSearch for /search/*

Runtime model control

  • • Triton runs in explicit model-control mode
  • • ?model_name= selects any loaded detector; POST /models/{name}/load|unload
  • • Models a curation project promotes load the same way, as {slug}__name