SEATTLE—Given how big Amazon’s digital footprint has gotten, the company clearly wants to expand into the real-world space as much as possible. And to that end, Amazon is running some of its most ambitious experiments in its capital city before rolling them out nationwide.
As the only Seattle resident of our staff, I draw the short straw of these tests by default.
In 2015, I shopped at Amazon’s first stab at a brick-and-mortar bookstore (you know, those old things that Amazon accused of going out of business in the first place) before that chain finally launched nationwide. -language. In 2016, I delivered Amazon packages as a gig-economy driver, before this type of contract worker became a common part of the Amazon Prime Now network across the country. And in 2018, I took it through the first “cashierless”, Amazon Go convenience store filled with cameras before the same idea had arrived in other big companies.
This week, when I received word that the latter concept was expanding into something called Amazon Go Grocery, complete with a much larger selection of items to buy, I knew what I had to do. I have to wake up from your new product line, one that is very difficult to track closely with a combination of RGB and infrared sensors: productions. Can I have some plums? Wrangle some pumpkin? A bag of bananas?
Skynet above the stroopwafels
Because Amazon Go Grocery revolves around the same creepy, clock-you-shop system found in smaller Amazon Go stores, I invite anyone unfamiliar with the concept to return to my first look at Amazon Go from early 2018. Functionally, the newest store works identically . . You can’t enter the store without entering your Amazon account credentials—complete with a valid payment method—into the Amazon Go app on either iOS or Android. Which, of course, means you can’t enter a store without a smart device connected to the Internet.
Once the app has your Amazon information, it will generate a unique QR code. Tap this on a gated kiosk sensor, and after a pause, the door will open. During this short wait, the store’s cameras take a picture of you and begin tracking your every move and action.
Amazon needs this level of scrutiny to make your ad a hit-and-run. Once inside, pick up any product you want, all marked with clear price tags. Then pop it into the bag of your choice and away you go without any action required. You can put your phone and wallet in your pocket or purse; There is no further payment process involved. If you change your mind about something you took, put it back on the shelf before you leave, and don’t worry about charging for it. Whatever you store, you’ll get paid for automatically, because Amazon Go and Amazon Go Grocery (AGG) log your every move.
Where AGG differs is its selection, which is vast and varied. Instead of limiting healthier options to pre-made foods, AGG goes further to include a cold wall of raw meat and seafood, a large market of fruits, and a wall of vegetables. The latter gets the same automatic water-spritzing process you’d expect from a standard supplier. (See? Amazon knows how lettuce works.)
I haven’t been to the little Amazon Go store since it first launched in 2018, so I can’t speak to either AGG’s involved Solar sensor (image in the gallery above) has been implemented in the chain’s smaller stores. But technology has clearly moved on since the launch of the original store, because the ceiling sensor boxes now have two visible sensors, instead of the one sensor that was only visible in the chain’s first-gen boxes. This is in addition to whatever additional RGB or infrared sensors Amazon may choose to hide in the ceiling’s flimsy grid of pipes and aluminum racks. I certainly don’t expect the ceiling of a grocery store to be the most romantic sight, but I’m surprised that Amazon isn’t trying to hide or reveal its terrible Skynet program now.
But the separation of my photos and the view at the ceiling did not change any obvious distinction in the cameras. The array of staring boxes of cookies, a la carte donuts, bagels of bread, frozen pizzas, chicken nuggets, lightly spritzed kale, and lunch bags seems iconic wherever you stop and look up. Amazon has always stayed mum exactly how your system works, but our local sources have indicated that the entire scientific system is placed in the ceiling of the store, as opposed to any pressure sensitive pads that detect changes in weight or inventory when you pick something up.
Russet vs yam, round one
The last time I tried to turn Amazon Go into coughing up Free, I didn’t have any luck. I know I confused them heck out of its default tracking system, because my second visit’s purchase took a full hour to process, as opposed to my first visit of incredibly simple “grab two things, put them in the bag, and send” process. My second visit in 2018 involved removing a jacket and a lot of silly jewelry.
On Tuesday, I doubled down on the stupidity of the purchase, although mostly in the manufacturing sector—where AGG must deal with items whose color and design can vary significantly within the same product line. Avocados come in “medium” and “large” sizes, and they are priced differently. Loose fruits and root vegetables like oranges, potatoes, and pumpkins are more differentiated within their own families, not to mention the eternal battle between the russet potato and the yam. And the traditional bananas sit next to the minerals; the latter is bundled with yellow tape like most groceries, so we should cause a banana switcheroo in a Piggly Wiggly! But it is also a possible point of camera sensor confusion.
Sometimes, I just grabbed and juggled a few products of different types before putting them back. At times, my behavior became strange. At one point, I picked up various fruits and vegetables, held them in my arms, and proceeded to the jetty to hold some of my bundle. Then I took different things with both hands, reached behind my back and under my backpack to change which was in which hand before putting them back in their places. (I do not return items to the wrong places for the sake of my testing, for the sake of this store do get stockers, and I don’t mess with retail staff. That’s my golden rule, experiments are stupid.)
Finally, I arranged a special banana in a bundle, almost like a synthetic genetic machine. After walking the stage around the store, I went back to grab a bunch of traditional bananas with one hand and an organic pack with the other. Then I passed both sets behind my back…but got a loose banana stuck between my back and my backpack. I put the rest of the bananas back in their respective containers, then walked to something I’d never seen in an Amazon Go store before: a bathroom.
The store’s large bathroom is lined with sensors and cameras, but the bathrooms themselves don’t appear to have any kind of camera or sensor in them. (I didn’t take photos of the bathroom, because I’m not DrDisrespect. You’ll have to trust me on this one.) The hallways also include a small tray outside each bathroom door where customers are encouraged to leave merchandise. before use. spare parts. I left the rest of the fruit in my hand at that time, a single pear, on that tray.
As soon as I entered the bathroom, I immediately opened my backpack and put the banana in there. Bananas are suspicious! I love my “ole”, then breathe. I probably don’t really foil cameras with a single wacky juggle of banana peels, I thought to myself. What am I doing here?
This moment includes a surprising turn to the bathroom mirror, which is when the light bulb went off in my head. I took off my jacket and put it in my backpack before entering the store. Can I confuse cameras with changing clothes?
It sure looks like it. As of press time, it has been two hours since my shopping trip ended, and AGG is it yet processing my trip. I believe this is, in part, because I was able to walk into the restroom wearing a yellow, long-sleeved shirt and a backpack on my back, then walk out with a gray jacket and a blue T-shirt. while holding my backpack to my chest. In addition, I removed the two glasses that were on top of my head. If I had been an invested member of the AGG team, I could have gone back and done some A / B tests of what happened when I emerged from the bathroom with a different size of clothes change; was the infection as simple as the backpack’s position? The jacket? Aviator shades? I would probably not know.
Solving a problem that doesn’t exist
Sam Machkovech
The time spent wondering how my shopping was doing was fun at first. Later, I wondered how the legwork was going to go in my store experience—and how likely someone would reverse the recorded image and trouble me for it. watch as I wander from store section to warehouse section, snapping photos and juggling fruit all the way.
Is this the future of retail management? It’s one thing to admit that major retail chains use security cameras in the war against theft. But AGG’s future appears to be trading general retail trust with the cost of having someone hide behind the screen to study and correct the robots’ attempts to do that basic, commercial work. It all seems like Amazon is trying to solve a problem that doesn’t exist—and creating an entirely new problem in its place. That’s in addition to the inconvenience of pulling out a phone and waiting for an Amazon app to load before walking into a store, as opposed to the organic wait-and-take-purse advantage we get in a traditional retail experience (or even stream rising water. of automated restaurant checkout kiosks, which have their own issues).
Maybe there is a future in which walking in and out of AGG is easy. Something seamless like Apple Pay or Google Pay, which works almost instantly with a fingerprint scan or face unlock and a tap of a phone’s NFC system, could work. Or maybe Amazon will go full Orwell and require an Amazon-certified face scanner at some point. I don’t know.
What I do know is that Amazon has had two years to consider its plans for Amazon Go as a live product. This week’s new store is proof that the concept isn’t… ahem… Going anywhere. Amazon wants its fingers in every possible sector of the data world, and on the retail side, Amazon Go is a big upgrade for them. The company doesn’t test every click, purchase, and conversion between digital storefronts and apps. With the store’s effective sensor system, you can see exactly how people react to sights and sounds in a real-world setting. The store’s ability to confirm my production hijinks could be a Trojan that will take Amazon Go technology into other retail binds and further breaks down the existing retail social contract.
I’m not sure that a sneaky banana heist — which, again, I haven’t proven successful as of press time — is worth it.
Update: Hours after this article went through the Ars Technica edit queue, and four hours after my shopping trip ended, I received a notification from Amazon Go: My order has been processed! Interesting results:
The product above is what I took out when I entered the bathroom, as mentioned earlier. My ridiculous juggling of beefsteak tomatoes, avocados, potatoes, and bananas stymied AGG’s solar sensor. However, after leaving the restroom, I picked up two non-productive items, totaling $6, plus mortar try to be sneaky about it. I even talked to an Amazon representative about what I would be carrying. Those two things are not listed here. What’s more, in the previous photo gallery, you may have noticed I left an avocado on the tray “outside the restroom”. I didn’t bring that one back, but I was rescued from it. (That’s in line with the app’s warning that whatever you take without return, you buy.)
Surprisingly, then, my “change of clothes” turned on Amazon Go Grocery. Everything I’ve carried before ducking into the loo has been charged correctly. After that, the app clearly wiped me out, which could fit the claim of a 2-hour, 23-minute shopping trip, well over the 20 minutes I was there. And Amazon needed another hour and a half to complete that I had picked up those first items, gone into a bathroom, and then been incapacitated by a jacket-wearing idiot with a beard and similar hair. I hope they catch that guy. You can come armed with bananas!
I already got a house on the other side of Seattle after learning I haven’t been charged for a couple of things, so I’ll try to return those things this week. I do has an option within the Amazon app to request a refund on avocados, but I think I’ll let that slide.