Those services are usually the ones that depend on identity, managed databases, key management, private circuits to partners, and partner allowlists tied to fixed addresses. They are also the ones that set the order of the whole move, so they have to be planned first.

Build the dependency map from traffic

The architecture diagram is rarely current. The map that a migration needs is built from what the estate is observed to do: network flow logs, load balancer logs, DNS query logs, the provider's API audit log and distributed traces.

The observation has to cover a month end and the batch cycles, because some dependencies appear only then. Where the logs you already keep cover those periods, the history can be read. Where they do not, the plan should state which cycles were not observed.

Classify before drawing the waves

Each workload is classified as re-host, re-platform, re-architect, replace, retire or retain, according to what it depends on. The workloads that will cost more than the plan assumed are classified first, because they decide the order of the waves.

One rule saves a great deal of trouble: a service and the datastore it writes to move in the same wave. Where that is not possible, count the round trips between the two providers for each request, and the data egress they generate, before the wave is drawn. The wave plan should state the response time that the split adds.

Stateless services: a traffic change that can be undone

For a stateless service the move is a change to where traffic goes. DNS answers are divided by weight between a source record and a target record, so traffic follows the weights approximately. The service is returned by setting the target weight to zero.

The return is not instant. Cached answers and open connections drain over the record lifetime and the connection lifetime. A rollback test should measure that drain time, so that the figure in the runbook is one that has been observed.

Stateful services: a data operation with a point of no return

A stateful service is different, because once it has taken writes on the target, returning it to the source is a data operation and no longer a traffic change.

While the service replicates to the target, the source stays the only writer. At cutover the sequence is:

1. Stop writes.
2. Wait until the target has applied the source's final position.
3. Recheck the chunks that changed since the last full parity run.
4. Start the reverse stream from that position.
5. Move write authority to the target in one step.

The order of the last two steps matters. If the target takes a write before the reverse stream is in place, that write is lost to any rollback.

Until sign-off, the service can be returned to the source by the same procedure in the other direction. At sign-off the reverse stream is removed, and from then the service can only be corrected forwards. Where the target database engine differs from the source engine and no tested reverse mapping exists, the point of no return is the cutover itself. The wave plan should say so in plain words, with the name of the person who signs it.

Count the cost of running two estates

A migration has its own cost, separate from the cost of either cloud. It includes the cost of running both estates in parallel for each wave, the data egress, and any committed spend that remains on the source.

A cost model for the move should give the month, if there is one, in which total spend falls below what you pay today. A plan without that month risks stopping halfway, with two estates to pay for.

What we would check first

If a migration has stalled part way, three questions usually show why:

• Which services write to a managed database, and which of those have a tested way back?
• Which dependencies were found only at a month end or in a batch cycle?
• What does each remaining wave cost in parallel running, and who has agreed to that cost?

Our cloud to cloud migration practice starts with an assessment of a named slice of the estate and produces the dependency map, the classification and the wave plan for that slice. The checks we use to show that a move is safe to continue are set out under how we verify a move.

Production then asks six further questions. A team that answers them before the build starts knows what the build is. A team that meets them one at a time, after a launch date has been promised, usually finds the costly ones last, when they cost the most to fix.

1. Is it right on your real data?

A proof of concept is judged on a sample, and the sample is rarely the real distribution of cases. Accuracy has to be measured on cases drawn from your own work, with a correct answer agreed for each case before the test starts. The system also has to read that data from the systems of record that hold it, which is often harder than the model.

Check first: ask two people in the business to mark the same few dozen real cases. If they cannot agree on the right answers, accuracy cannot be measured yet, and that is the first piece of work.

2. Does it hold at peak load, and what does it cost there?

Response time has to be measured at the concurrency you expect at peak, with the mix of requests you expect, and then beyond that point until it degrades, so that the margin is known. A test with one user at a time shows neither figure.

Cost follows from the same test. The cost of each type of request is measured at that load, and the monthly cost is calculated from it and from the volume you expect. Where the model runs on capacity you rent or own, the cost of a request depends on how fully that capacity is used. Where the model is priced for each token, the unit price is known from the first request, and what has to be established is the volume, the supplier's rate limits and the licence terms at that volume.

Check first: write down the peak concurrency and the response time the use case needs. Without those two numbers no sizing is possible.

3. Who and what can reach it, and what can it reach?

A test is often run with wide permissions. Production needs an answer for each component: who and what can reach it, what it is permitted to reach and do, what data it handles and stores, where its secrets are held, and what leaves your environment.

For a model there is one more question, which is what untrusted input reaches it. Text inside a document or a request can be treated by the model as an instruction, so a model that can call tools needs its permissions drawn as narrowly as any other service account.

Check first: list every outside service the proof of concept sends data to, and confirm that each one has been approved to receive it.

4. What can you show an auditor?

An auditor asks what the system did, on which data, and who approved each change. The first two come from logs that the system has to be built to keep, with a retention period and protection from alteration that you specify. The third comes from your own change approval, which every change should pass through.

Both have to be designed in from the start, because neither can be reconstructed afterwards. A log added after an incident does not describe the incident.

Check first: ask your auditors or your risk team what they would want to see for this system, before the design is drawn.

5. How does it fail?

Any component can fail. Production needs to know, for each failure that can be foreseen, its cause, its effect, how it is detected and how the system recovers. A recovery time is a target until the recovery has been rehearsed, and it is a measurement afterwards.

For an AI system the failure that matters most leaves every health check green. A wrong answer is a failure, and so is an answer that changes after the supplier updates the model. The evaluation cases from the first question are how that failure is detected, so they have to be run again whenever the model or its supplier changes.

Check first: pick the one failure that would cost the most, and ask how anyone would find out that it had happened.

6. Who owns it and who runs it?

The team that built the proof of concept is usually not the team that will run it. Your own engineers have to be able to release it, roll it back and operate it, and someone has to be on call. A named owner in the business has to accept it. In a regulated business your risk and compliance functions also have to approve it, and that approval is often the longest item in the plan.

Check first: name the owner and the operating team now. A system with neither tends to stay a proof of concept.

What to do this week

The readiness check asks these six questions about your own proof of concept and lists what is open. It takes about two minutes, and nothing is stored.

If several of the six are open, that is normal, and it is the work our AI inference systems practice exists to do. The Why Udhyam page sets out how we take a proof of concept through each question and put the answers in writing.

This article sets out how we would check a quotation, in the order we would do it.

Start from the workload

Sizing starts from what will run on the hardware. Before looking at the part list, write down:

• the model, its parameter count and its numeric precision
• the distribution of context lengths, which is how long the requests are
• the ratio of input tokens to output tokens
• the arrival rate at peak
• the response time target

The last item is the one most often missing. Without a response time target there is no way to choose how many requests the server processes together, and without that there is no way to say which limit of the hardware is reached first.

Find the limit that binds

A GPU server has several limits. It can calculate only so fast, it can move data through its memory only so fast, and it holds only so much memory. The limit that your workload reaches first is called the binding constraint, and it is the one to buy against.

Which limit binds depends on the workload:

• At low batch width, the decode stage of a language model is limited by memory bandwidth.
• As batch width rises, the limit moves towards compute.
• For long contexts, the memory taken by the KV cache usually caps the batch width first. The KV cache is the stored attention state for every token already processed.
• Multi-node training is frequently limited by the network between the nodes.

This matters for the comparison between two parts. When memory bandwidth is the limit, a part with a higher compute figure and the same bandwidth adds almost no throughput at that operating point. A quotation built on the headline compute figure can therefore be the wrong purchase for a serving workload.

The way to find the binding limit is to measure. We run a representative workload on rented capacity of the same accelerator, in an account you control. That measurement characterises the accelerator. The quoted host and network are then checked by calculation, and the findings say which figures were measured and which were calculated.

Check what surrounds the accelerator

The host, the network, and the power and cooling envelope are decided together with the accelerator. A quotation can be correct on the accelerator and still not be something your facility can run. These are the questions to put to the supplier in writing:

1. What power does one fully loaded server draw, and what does a full rack need?
2. What cooling must the data centre supply for that rack?
3. Is the network sized for serving or for multi-node training? The two need different designs.
4. What does the support contract cover, and for how long?

Put buying, colocating and renting on one basis

The three routes are priced in different units, so they have to be converted to one. The unit that works is the cost per million tokens, because it is the unit that suppliers of rented capacity price in.

For hardware you own, the figure is built like this:

The capital cost depreciated over a stated period, plus financing, power, space, support contracts and the staff to run the hardware, divided by the tokens served at the utilisation you expect.

Dividing by the tokens served at the utilisation you expect is how idle capacity enters the figure. Servers that stand idle for half the day cost close to twice as much for each token as the same servers kept busy, because the fixed costs are spread over half the tokens.

Suppliers price input tokens and output tokens separately, so the figure should be given for each, or as one figure at your measured mix.

Before committing capital it can be cheaper to rent the same capacity for a period, observe how heavily it is used, and buy only if use stays high. The useful output of the exercise is the level of sustained use at which buying becomes cheaper than renting for your workload.

Before the validity date

A quotation check has four possible answers: buy as quoted, buy less or more, buy a different part, or rent first. Any of the four is a good result if the arithmetic behind it is shown.

If you hold a quotation now, the two things that decide whether a check is possible are the validity date and a description of what you intend to run. Our AI hardware advisory practice does this work, and no supplier pays us, whichever answer we give. The method is set out in more detail under how we size compute.