Treating model selection as a one-time architecture decision is the fastest way to ship a system you no longer understand.
OpenAI's recent announcement formalizing the Sol and Luna tiers inside GPT-5 makes explicit what was always lurking in the background: the model your product runs on is a tiered, moving target with different capability levels, access windows, and deprecation timelines depending on which customer segment you occupy. Engineers who have not yet built a response to that reality are accumulating invisible platform risk at a rate they cannot measure.
The Tier Split Is a Product Event, Not a Model Event
Most engineers read the Sol/Luna announcement as a capability story: Sol is more capable, Luna is cheaper. That framing misses the structural point. What OpenAI has done is formally acknowledge that different customers will receive different versions of what is nominally the same model, and that those versions will evolve on different schedules.
This is not new behavior. OpenAI silently updated gpt-3.5-turbo multiple times without changing the default alias, and teams discovered it through regression in production, not through a changelog. What is new is that the tier split is now explicit, which means the drift is no longer accidental. It is designed.
For engineers building products on top of these APIs, the implication is direct: your product's behavior can change without a single line of code being deployed. That is a category of risk that most engineering orgs do not have a control for.
Why Pinning Is the Minimum Viable Response
Model version pinning is the practice of specifying an exact model version in your API calls rather than relying on a default alias. It is not glamorous. It is table stakes.
Here is what an unpinned call looks like, and why it is dangerous:
```python
Dangerous: alias resolves to whatever OpenAI decides today
response = client.chat.completions.create( model="gpt-4o", messages=messages )
Safer: pinned to a specific dated snapshot
response = client.chat.completions.create( model="gpt-4o-2024-08-06", messages=messages ) ```
Pinning is necessary but not sufficient. OpenAI deprecates dated snapshots too. The real discipline is treating model version as a first-class config value: stored in your service configuration, tracked in version control, and treated as a change that requires a deployment and a review.
Store it like this:
```yaml
config/model_config.yaml
completionmodel: provider: openai version: gpt-4o-2024-08-06 tierassumption: luna # document which tier you expect to be on lastevaldate: 2025-06-01 nextreviewdate: 2025-09-01 ```
The tier_assumption field is not just documentation. It is a forcing function. When you upgrade, whoever reviews the change has to make a conscious decision about whether the tier assumptions still hold. That is the kind of institutional knowledge that separates teams who own their AI systems from teams who are passengers.
Capability Assumptions Are Now a First-Class Engineering Artifact
When you build on a versioned library, you document which interface version you depend on. When you build on a database, you document the schema. When you build on an LLM, you need to document the capability assumptions your product makes, because those assumptions will break when the model changes.
Capability assumptions are not the same as prompt instructions. They are the behavioral contracts you are implicitly relying on: the model follows multi-step instructions in a specific order, it refuses certain content categories consistently, it produces structured JSON without a parsing fallback, it handles 16k token contexts without degrading on the final instructions. Every one of those is a regression surface.
The practical form this takes is a capability manifest: a short, versioned document that lists what your product assumes about the model it is running on. It does not need to be elaborate. It needs to exist. When a new model version is available, the upgrade process starts with reviewing that manifest and running targeted evals against each assumption before you ship.
This is not theoretical overhead. It is the difference between a model upgrade that takes a week and one that silently degrades your product for a month before someone notices.
Evals Are Not a Nice-to-Have Anymore
The engineering community has been slow to treat evals with the same rigor as unit tests. The Sol/Luna tier split should accelerate that.
An eval suite for a production LLM product needs three layers:
- Behavioral regression tests: Specific input/output pairs where the expected output is well-defined. These catch the obvious breaks when a model version changes.
- Capability boundary tests: Inputs that probe the edges of your documented capability assumptions. If your product depends on consistent JSON output, you run 200 variations of your JSON-producing prompt and measure parse success rate.
- Tier-sensitive tests: If you are on a lower access tier, you need evals that specifically probe the capabilities you care about under that constraint. Do not assume Sol behavior applies to your Luna deployment.
The tooling here is real. promptfoo is the most practical open-source option for running structured LLM evals against multiple model versions in CI. Braintrust is a reasonable commercial alternative if you want persistence and team visibility. Pick one and integrate it into your deployment pipeline before your next model upgrade, not after.
A model upgrade without a passing eval suite is a config change you are shipping blind. Engineers who have internalized this at a practical level are genuinely differentiated. At Skills Tech Network, documented AI system ownership, including eval pipelines and version governance, is one of the clearest signals that separates senior engineers from mid-level engineers who happen to be working in AI.
The Deprecation Timeline Problem
OpenAI's deprecation notices for specific model versions are real, but the lead times are shorter than most teams budget for. Historically, dated snapshots have had six months or less of guaranteed availability after a new default is set. With tiered models, the calculus gets more complex: your tier may lose access to a specific version sooner than a higher-tier customer does.
Build a deprecation response into your model governance process now, before you need it:
- Set a calendar alert at 90 days before any pinned model version's announced end-of-life.
- Run your eval suite against the successor model at 60 days. Do not wait for the deadline.
- Document the delta: what changed, what assumptions broke, what prompts required adjustment.
- Deploy the upgrade with a staged rollout, not a full cutover. Shadow traffic the new version for 48 hours if your product allows it.
The 90/60/deploy cadence is a minimum, not a rigid rule. Teams with complex prompting logic or high-stakes outputs should start the eval cycle at 120 days.
Multi-Provider Strategy: When It Earns Its Complexity
The natural engineering instinct when you see platform risk is to abstract it away: build a provider-agnostic interface and route between OpenAI, Anthropic, and others as needed. I am skeptical of this instinct when applied early.
Abstraction costs. A provider-agnostic routing layer adds latency, adds a failure mode, and adds ongoing maintenance burden. More importantly, it papers over the capability differences between providers rather than surfacing them. Claude 3.5 Sonnet and GPT-4o are not interchangeable for all tasks, and pretending they are in your abstraction layer means you will discover the differences in production.
The right trigger for a multi-provider strategy is demonstrated, specific risk: you have a task where two models are close enough in performance that you can switch without degrading quality, and you have a specific reason to want provider redundancy, such as a contractual SLA requirement or documented cost arbitrage. That is a real engineering decision with a real tradeoff. General anxiety about platform lock-in is not.
Solve the pinning and eval problems first. Multi-provider routing is a valid next step for some teams, but it is not the first move.
The Career Dimension: Owning AI Systems vs. Using Them
There is a widening gap between engineers who use LLMs as features and engineers who own LLM-powered systems as infrastructure. The former group plugs in an API and ships. The latter group builds version governance, eval pipelines, capability manifests, and deprecation processes.
The gap matters for career trajectory because the problems that surface in the second year of an LLM-powered product, model drift, tier access changes, capability regressions on upgrade, are engineering problems that require the second group's skills to solve. The first group tends to discover this the hard way, usually when a model update breaks something they did not know they were depending on.
If you are aiming for staff or principal roles in AI-adjacent product companies, the portfolio evidence that matters is not "I integrated the OpenAI API." It is: "I built a model governance process that caught a capability regression before it hit production, and here is how it worked."
That is the kind of demonstrated, specific, verifiable capability that reads as senior-level ownership. If you are building this evidence and want it visible to the right engineering leaders, Skills Tech Network is built specifically for that: ranking technical talent by what you have shipped and can demonstrate, not by keywords on a resume.
Model tiering is not going to get simpler. The Sol/Luna split is the current form of a structural shift in how foundation model companies manage their customer segments, and it will have analogs at every major provider over the next two years. Engineers who build the discipline now, pinning, evals, capability manifests, deprecation cadence, are building skills that compound. Engineers who treat model selection as infrastructure someone else manages are accumulating a debt that will come due exactly when their product depends on it most.
Build a proof-backed profile
Skills Tech Network ranks technical talent by verified, demonstrated capability, not just resumes. If you are doing this work, model governance, eval pipelines, real AI system ownership, make sure it is visible to the people hiring for it. Try it here.
*The engineers who will own AI infrastructure in three years are the ones building version discipline today, not the ones waiting for the platform to stabilize.*