Yeah, GPT 5.6 Sol is very good. Generally, Gemini models remain SOTA for VLM tasks with 3.7-flash at the top.
---
That said, considering variables like cost (say, over 100k PDF pages) and accuracy requirements (e.g., construction documents with dense images & tables), Gemini and other SOTA VLMs are expensive and inaccurate, and therefore unsuitable. This is where niche, open-weight, and task-specific OCR/VL models come in.
With a one-line change, you can switch between DeepSeek OCR 2, GLM-OCR, dots.mocr, Paddle OCR VL, PP-OCRv6, etc., and process 100K+ pages for under $60 on VLM Run Gateway
This is why we built VLM Run Gateway, one OpenAI-compatible endpoint for open-weight OCR and VLM models.
With a one-line change, you can switch between open-weight OCR VLMs (DeepSeek OCR 2, GLM-OCR, dots.mocr, Paddle OCR VL, PP-OCRv6, etc.) and process 100K+ pages for under $60 on VLM Run Gateway
> one OpenAI-compatible endpoint for open-weight OCR and VLM models
Sirus (Qax) is 90% built with LLM coding agents (mix).
I should say that I'm deeply involved in the process - architecting and reviewing (and sometimes revising) to ensure the code is clean and aligned with my preferences.
one approach to ensuring purity of work is staging good changes before asking the agent to make new ones. that way, stashing/deleting poor quality work isn't too expensive and doesn't cost you the current progress.
The comments in this post strongly validate the need for reliable video processing and understanding with VLMs.
While you can use Gemini or other local VLMs, the real challenge is token efficiency, accuracy, and coverage. For example, how do you make a VLM “watch” a 2-hour or 4GB video without losing context or meaning?
Video transcript alone can be sufficient for basic workflows needing no visual context. But when deep contextual understanding is required, e.g., self-driving, security analysis, warehouse tracking, etc., you’ll need more advanced methods like keyframe sampling, clipping, chunking, and shots+transcript.
Two approaches for passing binary files, say a 2-hour lecture video, to a VLM:
1. Send it whole: accurate, but slow to encode and process.
2. Keyframe extraction: fast, but could miss what matters.
There's no right or wrong approach here. It all depends on your use case and acceptable trade-offs.
Find useful strategies for encoding binary files in the post below. Enjoy!
That said, considering variables like cost (say, over 100k PDF pages) and accuracy requirements (e.g., construction documents with dense images & tables), Gemini and other SOTA VLMs are expensive and inaccurate, and therefore unsuitable. This is where niche, open-weight, and task-specific OCR/VL models come in.
With a one-line change, you can switch between DeepSeek OCR 2, GLM-OCR, dots.mocr, Paddle OCR VL, PP-OCRv6, etc., and process 100K+ pages for under $60 on VLM Run Gateway
This is why we built VLM Run Gateway, one OpenAI-compatible endpoint for open-weight OCR and VLM models.
Try it out quickly via OpenAI SDK:
``` client = OpenAI( base_url="https://gateway.vlm.run/v1/openai", api_key="<VLMRUN_API_KEY>", )
response = client.chat.completions.create( model="rednote-hilab/dots.mocr", messages=[{ "role": "user", "content": [{ "type": "document_url", "document_url": {"url": "https://.../invoice.pdf"}, }], }], extra_body={"document_dpi": 72}, ) ```
or via our CLI:
``` pip install vlmrun vlmrun gw models vlmrun config set --api-key 'vlmrun' # anon-user, rate-limited vlmrun gw chat <doc>.pdf -m zai-org/glm-ocr vlmrun gw chat <doc>.pdf -m zai-org/glm-ocr --json-mode vlmrun gw chat <doc>.pdf -m deepseek-ai/deepseek-ocr-2 vlmrun gw chat <doc>.pdf -m rednote-hilab/dots.mocr vlmrun gw chat <doc>.pdf -m paddleocr/pp-ocrv6 ```
Docs: https://docs.vlm.run/gateway
Catalog: https://docs.vlm.run/gateway/models
MCP: https://docs.vlm.run/gateway/mcp-server
Colab Quickstart: https://colab.research.google.com/drive/1RkuVIyuc5Po-UlcSlFy...
Read the full post here: https://huggingface.co/blog/vlm-run/intro-to-vlmrun-gateway
reply