Having been away from the UK for over a decade, I’ve missed witnessing the rise of certain tech-driven innovations in everyday life. One such example, Amazon’s Just Walk Out technology, promises to revolutionize shopping by eliminating queues. However, the reality is far from seamless.
The Past
There’s a particular advertisement from the 90s that has stuck with me all my life. It starts with a man walking around a supermarket, looking a little suspicious. As he goes around the market he takes products off the shelves and hides them in his coat. He wanders the store and stashes a few more items before he goes to leave.
As he leaves he has to go through a scanner, which looks like a metal detector you would find in an airport. As soon as he walks through a light flashes and a machine beeps. One of the shop assistants calls out to him, “Excuse me, sir!”.
The man stops and turns around to see what the assistant wants. The shop assistant grabs a piece of paper from the machine and hands it to the man, saying “You forgot your receipt.” The idea was inspiring to me and is part of what made me want to work in technology. I dreamed of one day seeing this idea, and many like it, become a reality.
Decades later, I would see start to see that vision materialize and what it looked like in the real world.
The Future Is Now
I’ve been in London for a month and I started noticing something new in the city. I’ve been out of the country for over ten years, so I hadn’t picked up on all the new “express” stores that now cover the city. One in particular stood out to me: Amazon Fresh. I was genuinely surprised to see they had a supermarket now.
We were in London and decided that we wanted to have a quick snack. There was an Amazon Fresh nearby, so we thought it would be a good opportunity to check it out. We went in and it seemed like your standard express supermarket. Some snacks, a small selection of groceries, drinks. Nothing unusual.
But when we went to leave we were confused. We couldn’t find a checkout anywhere. We asked the staff and they said we just scan the card and walk out. The cameras see what we have bought and we get charged automatically.
I asked about a receipt and they pointed to a sign with instructions on how to install the app to be able to get one. Reluctantly I scanned my card and we both walked out. My card was charged £1 as a hold, and signs said I would be charged the correct amount later. I was left confused and annoyed since I had no idea what I was being charged, when I would get charged, and I needed to download an app and register an account just to be able to see a receipt.
Just Walk Out
What we had experienced was Amazon’s Just Walk Out technology. Purportedly, it uses AI connected to several cameras around the stores and automatically figures out what you have bought and charges you automatically.
While a nice idea, even with a working AI solution, it creates an experience that requires extra work from the customer, and can be much more frustrating when something goes wrong. It’s a concept that is fundamentally flawed from the design stage because it focuses on the technology without thinking about the customer experience.
Now I have a background in tech and a decent understanding of how things work, I couldn’t help but wonder how the technology actually works.
Challenges
Amazon isn’t using RFID chips or similar on everything. Doing so would be expensive, and would be hard to do for things like donuts or fruits unless you want to create a choking hazard. The system supposedly uses cameras and a very advanced AI, but there are a lot of challenges with this, and I don’t think Amazon has actually overcome these challenges.
Cameras
Cameras can only see what they are pointed at. Cameras might miss smaller items or fail to differentiate between similar-looking products, leading to inaccuracies. They also can’t see the weight of a product, so a supermarket like this will not be able to sell any products by weight anymore.
Artificial Intelligence
There have recently been great strides in LLM and image recognition/generation technology. AI has many actual real-world uses. We’ve all been using autocorrect, search engines, recommendation algorithms, and so on for years. All of these technologies are based on AI. But AI technology has its limits. It isn’t designed to be correct all the time, and there are some things AI just can’t figure out.
When we left the store I was paying for two people. I scanned my card and we both walked out with items in our hands. How does the AI know that I intended to pay for both of us? Can it read human interactions well enough to know that I was agreeing to pay for the stuff my partner was carrying as well?
It came as no surprise to learn that the technology was actually a group of humans from India watching people in the store and entering in the orders manually. As far as I can tell they do seem to have been putting in efforts to train an AI, but they’ve found that the AI isn’t learning fast enough, and too many people are still required years after Just Walk Out was introduced.
How Much Am I Paying?
I’m lucky enough to be in a position where money isn’t too much of a concern. But I have in the past literally had to count pennies to see if I had enough money for a loaf of bread. I know how important keeping track of purchases can be to people living paycheck to paycheck.
I didn’t know how much I was going to pay before I scanned my card. If I had wanted to know that I’d need to bring a notepad and calculator with me and figure out everything before I “just walked out”. Normally if the tab is too high you can ask to have an item taken off and the staff will put it back for you. In this scenario you would have to do extra legwork yourself.
Without a receipt, you won’t know how much you’ve been charged until sometime after you’ve left the store. For me it was within an hour or two, but I’ve found stories online of people waiting one or two days to be charged. In the real world this can be frustrating to people who need to keep track of their spending.
What if the system missed you putting the item back on the shelf and you get charged for it? If you happen to have been overcharged, you then have to find and download your receipt to figure out why, and then go back to the store to find out how to get a refund (or maybe there’s a way you can do it online).
But by that point, how do you prove that you didn’t take the item you were charged for? Do they have data centers storing all the camera data just in case? If they don’t, Amazon has a choice between accepting every refund or assuming everyone is trying to cheat the system. In the latter case customers will lose trust in the system, in the former they create a loophole that customers can and will eventually take advantage of.
Fixing Mistakes
Even if the AI becomes more advanced, it would take generations of advancement for the technology to work the way Amazon wants it to. It’s a system that is prone to mistakes, and a system that could be open to abuse by people who figure out how to game the system.
Maybe Amazon is okay with that – every supermarket accepts that there will be always be some loss of stock through crime, damage, or human error. AI Error is probably just another cost to them.
Real Life vs. Science Fiction
Science fiction often inspires futuristic technologies. The advertisement that sparked my imagination as a child depicted a utopian vision of seamless shopping. But in real life, translating such concepts into functional systems reveals flaws that writers don’t have to account for.
This isn’t meant to be a criticism of the scifi genre. Like fantasy, science fiction can provide escapism by asking what if? Often the question is “what if this technology existed?” Sometimes the answer is how the technology is bad, actually.
The lesson here is that it is easier to create a new technology for a story as you can ignore or write around any flaws it might have. But if you want to develop a technology for the real world you really have to think through the use cases and try to predict how humans will actually react to it.
Amazon is walking back the Just Walk Out technology now. While the dream of queue-free shopping may remain out of reach for now, it’s a valuable lesson in balancing innovation with usability. Perhaps with further refinement, a variation of this concept might someday strike the right balance. But, at least for now, it seems like this is a technology that will only work when we suspend our disbelief.
