Today we’re excited to announce that our Managed Batch Processing feature is live.
You give us a cloud storage bucket, we process every document in it through our inference cluster and write the results back. We’ll auto-provision infrastructure to manage your workload and your deadlines.
No GPUs to provision, no orchestration to build and maintain, no rate limits to work with.
The math gets real fast
Suppose your team needs to process 500,000 documents — maybe it’s being used in an indexing pipeline to create a shared memory layer for your whole organization, or part of a larger run to get training data for a new model you’re training.
500K documents doesn’t sound terrible, until you remember that some of them are 2 pages, others around 200 and some exceed 1000 pages. All of a sudden you’re looking at 10 million+ pages. This becomes extra fun when your stakeholders tell you it needs to be done 4 days from now to keep the rest of their timelines on track.
Document conversion inferencing happens by page. A single H100 running Chandra processes about 4 pages per second — roughly 345K pages per day. We can use this to roughly estimate how many GPUs we might need to process these documents within a specified time window:
| Total pages to inference | 7 days | 3 days | 1 day | 2 hours |
|---|---|---|---|---|
| 1M | 1 GPU | 1 GPU | 3 GPUs | 35 GPUs |
| 10M | 5 GPUs | 10 GPUs | 29 GPUs | ~350 GPUs |
| 50M | 21 GPUs | 49 GPUs | 145 GPUs | ~1,750 GPUs |
| 200M | 83 GPUs | ~200 GPUs | ~580 GPUs | ~7,000 GPUs |
The time dimension changes everything. A 10 million page job is comfortable on 5 GPUs if you have a week. Need it by tomorrow? That’s 29 H100s running around the clock. Need it in 2 hours? You’re looking at 350.
If it’s not your first rodeo, you know that estimating throughput isn’t always cut and dry — some pages are simple and might finish extremely fast while others with incredibly dense tables slow things down. A tiny decrease in pages inferenced per second per GPU can start to mess with your run estimates. Do you know the underlying distribution of page complexity in your documents before you process them?
Now add the operational overhead:
- provisioning GPU quota (including some buffer for when things go wrong, an unknown document distribution)
- writing a job scheduler to distribute work across machines
- building detection and retry logic for the pages that fail inferencing
- managing storage I/O to keep every GPU fed
- monitoring the whole fleet to make sure it’s chugging along so you actually hit your deadline
Accuracy at scale
Throughput and infrastructure are only part of the problem. When you’re working with document volumes at this scale, accuracy is its own beast:
Rendering pages to OCR becomes a bottleneck.
What DPI do you use for the best accuracy/throughput trade-off? Render too low and you lose fidelity on small text and dense tables; render too high and you blow up memory, slow down inference, and burn money.
Edge languages, uncommon fonts, and embedded glyphs can break naive rendering pipelines in ways you won’t catch until you’re deep into a run.
Failure modes in your parsed output increase significantly
A 0.1% subtle-error rate on 10 million pages is 10,000 broken documents. How do you know when your pages have subtle accuracy issues — a missing column in a table, a misread footnote, a chart silently dropped?
How will you build systems to detect them, score confidence, route low-confidence pages to a different model or to humans, and reprocess them without rerunning the entire job?
These are the questions that turn a “process these documents” task into a multi-month engineering project. Managed Batch Processing handles all of it for you.
How it works
You point us at your cloud storage bucket (S3, GCS, or R2) containing the documents you want processed. We provision the GPUs, deploy our models, and run your files through our pipeline. Results are written back to your bucket, organized per-document.
Our team manages the run, technically and operationally.
You get your output and a report describing all the run statistics in detail.
For teams already using our API, this is the alternative path when volume spikes beyond what’s practical to handle through real-time requests. Instead of building your own orchestration layer, you hand the work off to us.
Recent Successes
We’ve been running this system privately for a few of our customers in the last few weeks to great success.
One customer needed us to process upwards of 200 million pages in a week with a guaranteed turnaround. We provisioned infrastructure and ran those documents through within 2 days, with time for our team to QA the results.
Another customer came to us with around 16 million pages to process in a day. Midway through that run, they reached out and asked about processing another 40 million. We scaled our infrastructure to absorb the additional workload and turned it around on time.
These are the kinds of jobs that Batch Processing is built for, and we love working through these challenges.
Whether you’re training frontier models, indexing a large corpus of back content, preparing datasets, or running a one-time migration, we handle the infrastructure so you can focus on what you’re building.
What’s next
We’re actively building on top of Batch Processing, including support for bringing in user defined pipelines from Forge, progress tracking and notifications, priority queues, and more.
Get started
Batch Processing is available as a managed service with volume-based pricing — the more you process, the better the rate. We’ll configure your first job, set up bucket access, and handle the infrastructure from there.
If you’re doing heavy-duty document processing, running large batch jobs, or managing a steady stream of high-volume work, reach out to us at [email protected] and we’ll get you started.