Jin Lei, from Shenzhen
Qbit | Public Account QbitAI
Fear is a very bad way of making decisions.
When companies are worried about missing the AI transformation window, Amazon CTO Werner Vogels reminds that the more anxious they are, the more they need to figure out clearly: what problem the company really needs to solve, and how AI can help.
Nowadays, new models emerge every week. People become more and more anxious, afraid of making a single step or step slower. The desire to improve efficiency and foster innovation is growing more urgent by the hour, but what technology can actually do remains unclear.
During a discussion in Shenzhen, he said bluntly that companies wouldn’t close down immediately just because they haven’t used AI yet. Therefore, before making a decision, companies need to ensure that engineers and managers have enough knowledge about the technology.
But saying this from Werner’s mouth seems a bit contradictory.
After all, this technical veteran who has served as Amazon’s CTO for over twenty years has never been willing to accept “impossible” ideas. When talking about the behind-the-scenes stories of cloud computing, he even cited a quote from computer pioneer Grace Hopper to remind developers to be wary of the phrase “we’ve always done it this way.”
A person who constantly encourages engineers to break the rules, why does it take them a while before they can use AI?
The database failure he described might explain this seemingly contradictory attitude.
Do AI still need to step into the pitfalls that databases have encountered?
December 12, 2004, was a critical moment for Amazon during the Christmas shopping season. At that time, users could enjoy free delivery when their purchases reached $25, but shipping took time. Ordering later would likely result in the package missing Christmas.
Just that day, the shared disk behind the customer database failed, preventing users from logging in or placing orders. According to Werner’s memory, the failure lastedmore than twelve hours, causing Amazonto lose millions of dollars.
The database supplier later approached them. Werner originally expected the other party to apologize seriously, but what he heard was, “You should do proper testing.”
When there’s a problem with your own product, why blame the customer?
Werner was certainly angry, but he later realized that Amazon’s scale had indeed gone far beyond the scope of this business software. Continuing to argue with suppliers wouldn’t solve the next malfunction either.
So, the team began to investigate a more fundamental question. How exactly does Amazon use the database?
The survey results show that approximately 70% of relational database usage scenarios only require key-value access.
For the shopping cart, the system usually only needs to retrieve the corresponding shopping cart based on the user identifier. Amazon does not need to fetch all users' shopping carts for a complex query.
Since the requirements are such, can the team try a different implementation approach? Considering the actual access methods, they built a key-value storage system Dynamo, which was the precursor to DynamoDB.
This experience feels very familiar today. Back then, people used relational databases as a big hammer to handle various data problems, and now large models can easily become that handy hammer.
When it comes to companies adopting AI, the questions Werner repeatedly asked were similar.What exactly do companies want to solve? Is AI suitable? Is current generation AI suitable?
He noted that in fields such as image recognition, speech processing, and prediction, a large number of effective methods have already been accumulated. Before choosing a technology, companies need to consider the task itself. Even with large models, it is not necessarily necessary to strive for first place on the rankings.
Werner is cautious about benchmark tests, as there is still a gap between artificially created tests and real business operations. If a model is already sufficient to complete the task, and the company continues to pay for higher rankings, it needs to reconsider its benefits.
And Werner has already calculated this account over more than twenty years.
When the topic of discussion shifted to the fundamental changes brought about by cloud computing, he paid special attention to the change in economic models.
In the past, when buying enterprise software, companies often had to sign contracts for five or ten years. Managers found it difficult to predict their needs for the following year, yet they still had to pay money upfront. Werner thought this was like a customer putting $1,000 into a restaurant and not getting a refund even if they don’t finish eating.
Cloud computing links resource usage with payment, and gives “resizing” real meaning. When there are fewer users at night, companies can turn off unnecessary resources. Costs can finally be discussed in the architecture context, and business departments also need to answer how much money a certain capability is worth.
In the AI era, this account has new projects as well.
For example, when asked by QuantumBit, "Why does even forgetting become an important engineering issue for agents?", Werner said bluntly that if agents have to scan everything they know before each task, the system overhead would be very high; developers need to find a way to let them retain only the content that is truly important for the task.
He also mentioned that when some programming tools are restarted, the project content is loaded again. Users already paid for these preparations yesterday, and today they have to pay again. Kiro’s cross-session memory aims to reduce such repetitive work.
So, in addition to how smart the model is, developers also need to ask themselves how much wasted work it does repeatedly to complete a task.
This also explains why Werner is willing to work with smaller models, as well as open-weight models like DeepSeek, Tongyiqianwen, and Kimi. According to his observations, some customers have found that after continuous experimentation, smaller models can meet their needs, and the cost of use is much lower.
What to use, how to use, and how much money is spent – these old problems still cannot be avoided by enterprises when it comes to AI.
Who will cover for the mistakes made by the Agent?
Even if the enterprise has calculated the costs, there is another issue waiting for the development team.
The chatbot provides a piece of text for the person to read before deciding what to do. The agent then takes action on behalf of the user, changing the itinerary, calling tools, and executing programs. Once the model gives the wrong answer, the error may turn into real consequences during the execution process.
During the conversation, Werner said that he was happy for the Agent to help modify the train ticket, but he also had specific requirements. The system needs to know how to operate it, remember his preferences, and at the same time be constrained.
And this obsession with boundaries is already mentioned in the Nitro story he tells.
Early cloud computing relied on virtualization, dividing a physical server among multiple users; however, the software that manages virtual machines, handles network and storage access, also consumes resources. As more virtual machines emerge, management becomes a potential bottleneck.
AWS later developed Nitro, offloading a large number of network and storage functions to dedicated hardware, and removed the original complete Linux management environment dom0. The team reduced performance bottlenecks, and at the same time narrowed down a significant attack surface.
Nitro also has an interesting design called “passive communication”. According to Werner’s explanation, the virtual machine monitor waits for commands from the Nitro controller. If it actively contacts the controller instead, it behaves in a way that is not expected in the normal process. The platform can then take action to shut down the relevant system and initiate an investigation.
Following this approach, it becomes easier to understand Werner’s requirements for the Agent. What the user allows the system to do must be clearly stated in advance. He gave a simple example: the user can specify that the Agent cannot spend more than $100 per day. If this limit is exceeded, human intervention is required for decision-making.
According to his introduction, Amazon has designed a rule execution mechanism for Agents. It checks operations such as Agent access to tools and running scripts, in order to limit their actions within the scope authorized by the customer.
The user’s authorization for autonomous action sets a specific scope; even if the model determines that something is worth doing, the execution system still needs to check whether it has the authority to do it.
But could these constraints hinder the efficiency of the Agent? In response to this question, Werner first spoke about...Responsibility。
You build it, you are responsible for it.
In his view, the developers of the Agent must be able to control it. No matter what actions the system takes or whether it goes beyond its limits, someone still needs to be responsible for it.
Werner also put his trust in AI within the context of distributed systems. Engineers never assume that every component is always reliable, which is why replication, isolation, and various reliability mechanisms are necessary. The team must be vigilant about individual components in order to build trust in the entire system.
However, engineers have to create these protection mechanisms into services that customers can afford and that don’t significantly slow down tasks. There are also many trade-offs involved. Lambda has faced similar challenges.
In its early days, WeTransfer had to keep running a batch of computing instances to process files uploaded to S3, waiting for new files to arrive. The customers had to maintain a complete infrastructure just to wait for these events to occur.
AWS hopes to take over these tasks, allowing customers to submit code and be charged based on execution time and memory usage. The team has figured out how to charge for it and how to implement it efficiently at the underlying level, but it remains a difficult challenge.
To isolate different customers, the virtual machines used by Lambda in the early days were still too large for function tasks and costly. Later, the team turned the isolation units into lighter microvirtual machines using Firecracker, which improved resource utilization.
After the isolation issue is resolved, the team still has to handle the cold start. Before the function is actually executed, the system needs to prepare the running environment, code, and dependencies. SnapStart reduces this preparation overhead by saving a snapshot of the initialized environment, improving the startup performance.
What the customer sees may be just a single function call. However, the engineering team needs to continuously refine isolation, resource allocation, startup, and networking mechanisms behind the scenes.
Werner refers to these tasks as “Invisible Engineering”.
When the Agent enters actual business operations, the development team also needs these details. After the demonstration runs smoothly, the team still needs to clarify how permissions are restricted, how exceptions are handled, and what range of tasks users are willing to assign to it.
Experience has no compression algorithm
When the Agent starts writing code, these issues fall back on the development team itself.
Regarding the employment of junior engineers, Werner replied that Amazon is hiring a similar number of junior engineers as in the past. Then, he shifted the topic to the learning process for new employees once they join the team.
A newly graduated person joins the S3 team and may be faced with code from twenty years ago. It is difficult for the school to teach him in advance how to understand such a system, as well as how to collaborate with those who maintain it.
Therefore, Werner values curiosity, learning ability, and experience with open-source projects. If a newcomer has participated in open-source collaboration, it means his code has been reviewed by others, and he needs to work together with others to solve problems.
AI can help newcomers understand team rules, allowing experienced engineers to spend less time reexplaining them. When Werner introduced Kiro, he also emphasized specification-driven development. Developers first clarify their requirements, architecture, and development rules before the system starts working.
However, the speed at which AI generates code has increased, and the team’s ability to understand and review code may not keep up.
Werner mentioned a concept that development teams should pay attention to: debt verification.
In the past, to meet deadlines, developers took shortcuts and noted down things that would need to be fixed later. What accumulated was technical debt. Now, in AI-generated code, some parts have not been verified yet, and the team has created another form of debt.
A few millimeters deviation in an interface element, compared to code that might leak customer information, has a very different consequence. The development team needs to conduct inspections based on risks and focus their efforts on the areas that require the most verification.
From Werner’s perspective, it is easy for the team to increase code output, but whether they can understand this code and bear the consequences it brings requires another evaluation. Otherwise, the time developers save today may be spent again on troubleshooting and rework later on.
This is also why Werner insists that junior engineers read the code and review and discuss it with senior engineers. Tools can explain knowledge, but the judgment needed when facing specific problems requires gradual accumulation.
No compression algorithm exists for experience. (There is no compression algorithm for experience.)
Werner's words are quite appropriate in addressing developers' concerns.
However, when it came to his trip to China, he told another example that made him more optimistic about AI applications. He spent a lot of time describing his visits to “rice and millet”.
The institution provides intervention services for children with developmental support needs such as autism and attention deficit hyperactivity disorder. According to Werner, the institution uses AI to analyze teaching records and observe a child’s progress, and provides plan suggestions to rehabilitation specialists. The rehabilitation specialists then make professional decisions about which suggestions to implement.
What impressed him was that AI helped professionals reduce administrative work, allowing them to devote more energy to their children and family.
When talking about this, Werner’s optimism has a specific object. What technologies can reduce the burden on some people, and who can gain support that was previously difficult to obtain? These specific applications made him see the value of AI.
Looking back at the phrase “first press the pause button”, what the enterprise needs to stop and think about becomes more specific. Whether the task is suitable for being assigned to AI, whether the cost can be borne, how the scope of action should be defined, and who will verify the results – all of these need to be answered item by item by the development team.
Werner said that the best work of developers often lies in places that others cannot see.
The AI wrote a piece of code instead of people. Engineers also need to know what will happen after this code is launched.