Limited-time dealSupermicro HGX B200, 8 GPUs8-GPU NVIDIA HGX B200 server in stock
View the specification Call +91 92058 93902innovations@udhyamlabs.tech
We explain the mechanism behind each recommendation and show the measurement that supports it.
Where a recommendation rests on judgement because no measurement is possible, we say so. This page is written for engineers. If you are deciding whether to speak to us, the Services and Working with us pages cover what you need.
How we engineer
We use one method on every engagement: profile the system, locate the layer responsible, change one thing, and measure again.
We trace a problem to the layer that causes it
We follow a problem to the layer that causes it, whether that is the model's attention implementation, the write path of a storage engine, or the way memory is divided between the processor sockets of the host.
Tooling written for the job
Infrastructure code, migration tooling, test harnesses and load generators are written for the engagement and committed to your repository.
We read the framework's source when the fault is there
When the fault is in a framework, we read its source to find the cause. We then either work around it in your code or, with your agreement, write a patch and submit it upstream, and we document what your team has to carry until it is merged.
How we verify a move
A migration or a rebuild is safe to continue only when three things have been shown: what depends on what, that the data is the same on both sides, and that the new path behaves as the old one did.
The dependency map
The map is built from network flow logs, load balancer logs, DNS query logs, the provider's API audit log and distributed traces. Where your retention already covers a month end and the batch cycles we read the history, and where it does not the findings state which cycles were not observed.
Data parity
For each datastore we compare row counts and checksums in key-range chunks, on a canonical encoding of each row where the two engines store types differently, and we run a set of business queries on both sides. Each chunk is checksummed on the source at a recorded position and on the target once it has applied that position. Only a difference that persists after lag has drained is treated as real.
Behaviour parity
The new path answers the same live requests as the old one. Responses are compared field by field after the fields that legitimately differ, such as timestamps and generated identifiers, have been excluded by an agreed list. For a request with side effects, the comparison is made on the writes and messages the new path intended to produce.
What we work in
These are the tools our engineers work in. If your stack includes something that is not listed, ask, and we will tell you whether we know it well enough to be responsible for it.
Model and serving
The systems we build are for production inference. For training and fine tuning workloads, our work is sizing the hardware. We work at the level of how the server batches requests and how it allocates memory for the KV cache, because those two mechanisms set throughput and tail latency.
- PyTorch
- TensorFlow
- vLLM
- TensorRT
- Open weight models
- Proprietary APIs
Platform and pipeline: infrastructure
Infrastructure is defined as code, and accelerator scheduling and network configuration are treated as part of the platform.
- Terraform
- Pulumi
- Kubernetes
- Device plugins
- Topology aware scheduling
Platform and pipeline: delivery and observability
This group covers the pipeline, gradual release, and the telemetry that decides whether a release continues.
- GitHub Actions
- GitLab CI
- Argo Rollouts
- Policy as code
- OpenTelemetry
- Prometheus
Cloud or data centre
We work at the depth needed to move production between providers, which means identity, networking and managed data services as well as compute.
- AWS
- Azure
- Google Cloud
- Sovereign and private cloud
- OpenStack
- OpenShift
- Bare metal
Hardware and network
The host, the network, and the power and cooling envelope are decided together with the accelerator.
- NVIDIA
- CUDA
- NCCL
- AMD Instinct
- ROCm
- InfiniBand and Ethernet fabrics
Databases, message brokers and identity providers differ from one organisation to the next. Once the NDA is in place, name yours, and we tell you, for each, whether one of our engineers has run it in production.
How we size compute
A sizing error costs money in both directions. Capacity that is too large shows up as accelerators standing idle after they have been paid for. Capacity that is too small shows up later, as a response time or a launch date that cannot be met.
- Start from the workloadWe record the model, its parameter count and precision, the distribution of context lengths, the ratio of input to output tokens, the arrival rate at peak and the response time target.
- Find the binding constraintThe binding constraint is the resource that is exhausted first. At low batch width decode is limited by memory bandwidth. As batch width rises it moves towards compute, and for long contexts the memory taken by the KV cache usually caps the batch width first.
- Convert to cost per million tokensA token is a word or a fragment of a word. Suppliers price input and output tokens separately, so the figure is given for each, or as one figure at your measured mix. It is taken at the utilisation you expect over a day, so it includes idle capacity and the power drawn. It puts buying, colocating and renting on one basis, to be read alongside capital and contract terms.
The chart shows what renting, colocating and buying cost at different levels of use. The three bars show which limit of the hardware the work reaches first.
Describe the problem, and an engineer will reply within one working day
Tell us in general terms what you are trying to do and what has stopped it. If it is not ours to take on, we will say so.