On Prem Inference is Good For Dell

I attended the Dell Technologies Forum today. It had lots of what you get at an industry conference; vendors organize these things to sell you stuff, after all. One thing that was interesting to observe is how Dell was pitching on-prem inference. They are really leaning on on-prem inference and playing on unpredictable costs of cloud inference, especially in agentic workloads. A key word that got repeated again and again was “tokenomics.”

From a security standpoint, I am intrigued. If we could keep inference in-house, that would mean we don’t have to trust a third-party vendor with the most sensitive data we have. Their sales pitch is also attractive, as you buy the hardware once rather than renting tokens indefinitely.

Of course, the company I work for has already sold its soul to the cloud, so this idea of keeping data in-house is more attractive on paper than in practice. Something like Foundry plays better on that score, reducing the number of vendors rather than running both the cloud vendor and the inference vendor. Of course, Dell assures us that they play nice with cloud inference, and we can mix and match and use the cloud when it makes sense. It’s worth considering.

There is also the issue of hardware depreciation. When you use a cloud service, you pay a variable rate, but you have flexibility. When you buy your own hardware, your cost is fixed, but you are locked in. The math comes down to the inference patterns. If you fully utilize the on-prem hardware, you may easily recoup your investment. I believe that is why Dell is pushing agentic workloads that run independent of working hours to make this pitch.

Now the question is: why is Dell talking about this? I am guessing cloud doesn’t really play into Dell’s strong suit. Dell is a hardware company, and they would like to sell you the hardware rather than rent it to you.