The 2024 debate about open versus proprietary models was loud. In real enterprise projects, the decision came down to data sensitivity, latency, quality, and who would run the infra.
In 2024, you could scroll social media for hours and still be no clearer on whether open or proprietary models were "better". Inside real enterprises, the question looked different: which mix of models unlocked value soonest without blowing up risk or cost?
A Portfolio, Not a Binary Choice
The most mature teams in 2024 treated models like any other infrastructure component: they standardised interfaces and let evaluation decide which model to route to for which task. A small, well-chosen set of models - some proprietary, some open - beat ideological purity every time.
How Teams Actually Made the Choice
In practice, most enterprises ran a small evaluation matrix across a handful of candidate models: a couple of proprietary APIs, one or two managed open-source offerings, and sometimes a self-hosted open model. They measured quality on their own tasks, latency from their own regions, and cost at realistic volumes. The winning stack was rarely the one that looked best on public leaderboards.
- Quality: task-specific evaluation sets, not generic benchmarks, decided which models were 'good enough'
- Latency: proximity of hosting region and network path mattered as much as raw model speed
- Data sensitivity: highly regulated workloads favoured models that could run in a private VPC or on-prem
- Operational fit: teams preferred providers that integrated with their existing observability and security stack
Evolving Portfolios, Not One-Time Decisions
The most important mindset shift was treating model choice as a portfolio decision that would evolve over time, not a one-off platform bet. Teams that locked themselves into a single provider without a clear exit strategy found it harder to take advantage of new models or pricing changes. Teams that standardised on a few interfaces, with the ability to swap models behind them, adapted faster.
A simple governance pattern
Define a 'model review' cadence (for example, twice per year) where a small platform working group reviews new model options, pricing changes, and internal evaluation results. Use that forum to add, remove, or reassign models to specific use cases, rather than making ad-hoc changes in individual teams.
Building something in this space?
We'd be happy to talk through your use case. No pitch - just an honest conversation about what's feasible.
Book a 30-minute callKey takeaways
- Most enterprises ended up with a portfolio, not a single model
- Proprietary APIs often won on time-to-value and raw quality
- Open models shone where data residency and customisation mattered most
- Total cost of ownership depended heavily on in-house infra maturity
- Evaluation on your own tasks mattered more than leaderboard positions
