| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Original HTTPS Page] |
Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.
You must be logged in to block users.
Contact GitHub support about this user’s behavior. Learn more about reporting abuse.
Report abuseWe are wasting too many tokens. As consumers, we need to get better at using tokens
Since December 2022, there has been a great flood of items on most people's to-do lists. We are doing more than ever by a large multiple, but most people aren't making any money out of it.
Going into SF, the first thing I heard a lot of people trying to start vertical AI app companies say was "Yeah, we are going to be safe from labs disrupting us is by going full-stack". For example, instead of building a legal AI product, they would build an AI-native law firm. Personally, I have a big vendetta against inexperienced software people just assuming that AI-native X legacy industry, will work simply by default. Even more against investors who think they can rely on this as a scalable strategy for disruption 🤡 with their stupid roll-ups.
There are three reasons why I feel this is going to fall flat on their face, and these are non-structural reasons. I'll just get to structural reasons later. Right off the bat, here are 3 reasons:
Every business (as on today) needs a CEO focused on taking the business to the next level due to the pace of change. This is true irrespective of the size of your business with even $10bn+ companies needing CEOs to step into an IC role (Benioff, Brian Armstrong) and set a grounded and actionable strategy.
To deliver a full-stack AI outcomes company, you are not "moving up the stack of defensibility"; you are trying to run business type 3 and business type 4 at the same time. This is an insanely hard ask for the following 2 reasons because there will be an eternal struggle to:
Additionally there is an issue from a storytelling point of view that the previous generation of fintech companies quickly found out when the market started valuing them at bank multiples just adjusted for growth rates. Once you reach this point, you have a decision to make. Do you keep spending on R&D to keep your type 3 business going, or double down on becoming a 4 business thats driving revenue?
The problem that most legacy companies are facing right now is that they need to mandatorily do three and four to even be relevant as an organization in the future. This is where the interpersonal relationships are becoming a problem. It's no longer enough to have one good CEO-level executive for the public company.
I am just very clear: how do you understand what are the older items which have become commodities in terms of their offering, but are effectively decommoditized at scale, and items that become commodities but are not decommoditized at scale? For example, compute is something that very rapidly begins to decommoditize at scale, at least when it comes to developer experience and user experience deliverable over that same set of compute, but then, say, for example, client-facing JavaScript frameworks don't do that, right?
Basically figuring out a way to scaling up commodity offerings using non-equity fundraising instruments without diminishing the enterprise value at hand.
Figuring out a way to dismantle the other assets because the parts may genuinely be more valuable. For example, a company like a Monday.com might be more valuable when broken down into saleable RL environments for labs trying to build multiplayer hardness experiences for SMB customers than as a product on its own.
Now that I'm done talking about all the examples, you may think, "Oh yes, I am the exception." Here is what you need to understand: it is a structural issue in going full stack from a vertical AI company. There are fundamentally 5 types of companies in the AI age:
The problem is that all five of these have very different tasks at hand.
A foundational lab, as we have seen, is no matter how successful the product is. They can very easily fall behind if they are not able to come up with the next generation of emergent capability in the models at the frontier. It doesn't matter what the capability costs. When it's at the frontier, it does not matter how popular the product is. Yes, GPT would never have lost to Claude. None of that is as relevant as being able to come up with the emerging property that becomes frontier and staying there for an extended amount of time.
The work of the CEO of a frontier lab is to have a unique taste level inside of exactly what it is that is currently a frontier capability and what is the next most important and valuable frontier capability. For example, Open is completely based on the idea that in chart in thread memory is everything, whereas Claude is completely based on the idea that code is everything. Eventually Grock will settle on the fact that citations are everything, and Google will settle on multi-modal and real time is everything, for their own specific reasons. If and when they don't do this, they will have a lot of issues, in one attracting people and two keeping a coherent direction of what everyone is supposed to be working towards.
An inference lab has a very specific role in the ecosystem.
The inference lab has a very, very different thing. The inference lab does not exist to discover frontier capability, but rather to discover different infrastructure constraints that exist at the time of inference, correct? This may be stemming from a latency requirement or a cost requirement or a data localization requirement as the primary three. They can solve this problem either by making smart innovations at the model architecture layer or at the chips layer or at the firmware layer, or by commoditizing certain expertise like writing CUDA kernels. This is the only thing: can they understand, as of date, what is the infra constraint that my customers have, and can I deliver either a public or a privately created, price-performance optimal checkpoint for the foundational model?
I think the third thing is basically around vertical AI companies. Of course, everyone was an OG, right? At this, everyone had their own stuff, but I think it has become very, very clear now there are only 3 to 4 things that vertical AI companies are supposed to be doing. Innovating on the core product is not one of them, right? I feel very, very simply that core AI companies need to get three things right:
So for an AI-enabled business what we are looking for is to understand two things:
AI agents consume One or two magnitudes more of the conventional infrastructure than most of the providers had ever experienced in terms of traffic. Imagine you're running sales force correct as a human, at a time, let's say. In a minute, it takes literally what you need, and then follow that path, right? A conventional dashboard would have maybe four to six static endpoints, which would be really optimized schema wrappers over, let's say, 10 to 15 DB calls. You are effectively sizing your into for 15 DB calls per user.
Now let's say you have an average agent, running a limited number of sub-agents and no swarms. You are doing at least 20 steps in a minute, not at once, and doing three to four tool calls per you, so therefore you are having to underwrite effectively 80 phone calls a minute per agent. Out of which, maybe, considering humans are who trained agents and that's what they're mimicking, you should have at least 15 net new queries or make new query patterns where you haven't had the chance to ever do optimization. So basically as a DB company you need to be prepared for 4x scale. As a code repository or storage company you need to be prepared for an equivalent amount of scale, especially when agents are writing traces to disk, because effectively traces are cumulative session logs that you need to store. At 100 sampling, like Sentry, which is used for human activity logging traces, probably had like 2-3-4% sampling. So this is what I'm trying to say basically. Then this goes down to CPU providers and RAM providers and SSD providers and all the way down to infra, like from observability software to literally electricity plants.
If you are in the vertical AI space, keep your head down and focus on the basics:
You need to have a very coordinated kind of super board of one group CEO and two deputy CEOs working on three and four independently. You need to have the ability to set that up quickly enough to leverage from your distribution, because you are not getting innovation till now, and customers are making sure that AI-native companies get distribution.
Or you have a single executive who has demonstrably shown capabilities to grow two public companies at the same time over time, which, surprise, surprise, is a very small set of one person, Elon Musk.
By 2028 you will be able to tell them apart from the outside, without the deck.
📍 Bengaluru, India
✉️ priyesh@onfinance.in · priyesh.sriv2017@gmail.com
𝕏 @fpdotmoney
All the assignments, CCs and homeworks will come up at this link
Frugal alternative to GPT4 autograding powered by cross-encoder models
Python 3
| Back | FazBrowse Home | New Git URL |