Under Review

In early 2019, when I was twenty-three and trying to break into journalism, a friend told me that The Strategist, New York magazine’s product recommendation vertical, was looking for an intern. Everyone I knew seemed to own at least one item that The Strategist had written about: a blue Matisse nude print, a pleated lampshade, a fisherman’s net for carrying groceries. I went in for an interview and made the case that I would be well suited for the work: upon moving to New York, I’d developed a hustle buying designer furniture at estate sales in New Jersey and Connecticut, ferrying my finds back to the city, and flipping them on Craigslist. I had an eye for merchandise, I said. I was hired.

One of the first things I learned at The Strategist was that it made lots of money. The Strategist had affiliate marketing deals with Amazon, Target, and Nordstrom; whenever a reader bought something from these retailers using a Strategist product link, The Strategist received a small commission. The amounts added up. “A good article can crack thirteen thousand dollars,” Alexis Swerdloff—then the editor of The Strategist, now New York’s deputy editor—once told me. During a meeting with the business team, Camilla Cho—then general manager of the magazine’s e-commerce, now senior vice president of e-commerce for its parent, Vox Media—announced that in 2018 The Strategist had netted around fourteen million dollars. The business team hoped to double that in 2019.

The Strategist was not the first product recommendation vertical, nor is it the largest, though it might be the most successful at leveraging its publication’s brand for profit. Contributors told me that, lately, The Strategist takes in at least two million dollars per month; without sharing specifics, Cho told me the margins “are very, very healthy.” And it’s a category with wide appeal: In recent years, numerous legacy outlets seeking viable revenue streams have formed or acquired e-commerce sites (the New York Times with Wirecutter, the Wall Street Journal with Buy Side, CNN with Underscored). Style magazines that once dabbled in product recommendations have transformed into veritable department stores (GQ, Harper’s Bazaar, Marie Claire). Even The New Yorker now publishes gift guides and book reviews embedded with affiliate links. (“Pellet Ice Is the Good Ice,” according to a recent headline; the piece directs you to an Amazon listing for a six-hundred-dollar Wi-Fi-powered ice maker.)

These endeavors are not all alike, however. “The truth is, most players in this space don’t test products at all,” Ben Frumin, Wirecutter’s editor in chief, told me. “Most players in the online product review and affiliate revenue space are kind of skeezy. They have a certain cynicism for their readers, and they’re like, ‘Here’s some stuff we found on Amazon without ever physically interrogating these products.’” I encountered some of that cynicism as a Strategist intern. Most of the site’s money—more than 80 percent, at the time—came from Amazon, which offered a sprawling catalogue of products and the most generous revenue-sharing agreements of any retailer. (The Strategist disputed this figure but declined to comment further.) Amazon didn’t just pay a percentage of whatever product was purchased through an article recommendation, but a percentage of a shopper’s entire Amazon cart. To capitalize on the arrangement, New York started “The People’s Choice,” a column showcasing Amazon’s top-rated merchandise. The Strategist claimed that every product was “vetted by editors,” which didn’t involve testing so much as making recommendations based on Amazon reviews—many of them specious, some fictitious. (A 2019 study by Fakespot, a customer-review monitoring service, estimated that a third of all Amazon reviews were “fake or unreliable.” The same year, the Federal Trade Commission successfully brought its first case against a company using fraudulent Amazon reviews to advertise.)

Part of my job was updating old People’s Choice articles. As I went along, I noticed a strange pattern. Some of the product pages that The Strategist linked to had customer reviews of entirely different items. This, I learned, was called “review hijacking,” a process by which Amazon vendors introduced new merchandise by updating disused listings—replacing photos and descriptions but retaining old feedback in order to give the product a boost in Amazon’s queue. Vendors wagered that most shoppers looked at aggregate ratings without reading the reviews themselves. Media companies in the product recommendation game needed to keep rereading reviews and updating their articles in order to maintain any semblance of control over their authenticity.

Since then, the market of product review verticals has evolved and grown. Digital advertising rates have fallen. E-commerce—“a high-margin business,” as Cho told me—has proved to be “a fairly easy way to diversify, because it is taking service journalism and optimizing that toward monetization.” It’s also increasingly important to outlets’ bottom lines. “Nobody who grows up and wants to be a journalist thinks, ‘I want to cover Amazon Prime Day every year and write a story on an Instant Pot sale,’” Lauren Joseph, a former e-commerce writer, told me. “But journalists need salaries; publications need revenue streams.” Be that as it may, when considered in terms of editorial value, product reviewing may rate poorly.

Product recommendations as we know them today took shape in the early twentieth century. The Great Depression fanned public distrust in private enterprise; a militant new consumer advocacy took root. Consumer cooperatives—retail outlets, grocery stores, laundry services—flourished. Americans opened consumer research labs, where they tested products for safety and called for enforced commodity standards. One of these labs, Consumers Union, founded in 1936, published its findings in a magazine: Consumer Reports.

Consumer Reports was uniquely combative toward corporations. Unlike its rival, Consumers’ Research, Consumer Reports criticized products by name. When the magazine evaluated an item of merchandise—milk, soap, Alka-Seltzer—reviewers took into account the corporate-labor relations of the manufacturer. The magazine refused ads, maintaining that Consumer Reports exclusively catered to and acted on behalf of subscribers, or “members.”

Following World War II, the GI Bill and a booming economy enabled millions of Americans to buy homes and fill them with furniture, appliances, and cars. Now, instead of worrying over corporate greed, shoppers wondered which was better: a Zenith or Magnavox television set. Between 1944 and 1954, the readership of Consumer Reports ballooned, from fifty-five thousand to seven hundred thousand. During that time, as Lizabeth Cohen, a Harvard historian, writes in her book A Consumers’ Republic, the magazine transformed to become a “buying guide to consumer durables for a more and more affluent, educated, professional middle-class audience.” In the following decades, other emerging periodicals took a similar approach—including New York, founded in 1968, which aimed, among other things, to protect city residents-on-the-rise from buyer’s remorse. As an early house ad explained, “Our ‘Underground Gourmet,’ ‘Best Bets,’ ‘Passionate Shopper,’ and ‘Urban Strategist’ columns tell you where to go for great meals under $5 (and where to avoid those $42 per couple fiascos).”

The internet transformed what was possible, allowing retailers to track when a recommendation precipitated a sale. In 1994, William Tobin started the first affiliate e-commerce network, for PC Flowers & Gifts. Two years later, Amazon launched its own affiliate network, offering media outlets and bloggers as much as 15 percent commissions on purchases. That was a steep cut, but it solved a practical problem: when Amazon established its marketplace for third-party sellers, in 2000, the company began onboarding retailers faster than it could vet them. The result was an exploding, indecipherable catalogue listing hundreds of seemingly identical products of almost no description; some were scams. Product reviews sifted through the muck while giving readers a sense of security. “This was when you could literally get the goods, put them in your garage in Brooklyn, open up a door, and sell them out of the back of your truck,” Carol Lappin, the director of business development for Consumer Reports, told me. “There was a lot of bait and switch in terms of pricing and upselling and all of that.” Consumer Reports was an early adopter of affiliate networks, including Amazon’s. “We felt we had to help our consumers get a safe shopping experience,” Lappin said. 

Empowering people to make sense of a complicated world grants service journalism a noble purpose. But given the revenue potential of partnerships with retailers, media companies are incentivized to emphasize just how fraught online shopping can be—to suggest, for instance, that buying a toaster without consulting a guide would be disastrous. “So much of what we do is take the stress and negative emotions out of shopping,” as Wirecutter’s Frumin put it. “We really try to invest in journalism that helps people solve problems that they don’t know about.”

Wirecutter’s office occupies a gray-brick former envelope-printing facility in Long Island City, on a street with warehouses, auto repair shops, and a beer hall. One afternoon, Frumin met me on the main floor, which was brightly lit and filled with dozens of empty desks. “We employ a hundred and twenty journalists,” he said. “Maybe thirty of them are in the New York City area. A majority of our staff are scattered around the country.” There was action to see downstairs, in the testing facility. It was garage-like, with concrete floors and products stacked everywhere: induction cooktops, vacuum cleaners, several cans of tomatoes. “Two months ago we had seven treadmills,” Frumin said.

Andrea Barnes, a Wirecutter reporter, was evaluating three washing machines. She started with an industry-standard stain strip test, involving a rectangular piece of cotton fabric dyed with squares of sebum (body oil), carbon black (a mixture of soot and mineral oil), cocoa (chocolate and milk), pig’s blood, and red wine. “The joke in the industry is that this is the most useful and useless test for laundry,” she said. “Washing machines are designed to pass this test.” Next, she planned to introduce a curveball: a garment she brought from home that her dog had peed on. “Consumer Reports is all lab, lab, lab,” she told me. “They’re doing the standards tests that the washing machine companies are already doing themselves. It’s not the same thing as what we do.” (Later, Jen Shecter, the senior director of content for Consumer Reports, told me that’s not true: Consumer Reports has an extensive team for real-world testing and its own rating system. “Unlike other traditional media outlets,” Shecter said, “if we find an issue with a product, we won’t just say there’s a safety issue. We’ll actually ask for a recall to have that product removed from the marketplace. And sometimes we petition for that—we’ll ask the company, we’ll ask the government.”)

For about a decade now—Wirecutter was founded in 2011—product recommendation sites have elevated the inspection of household goods to a science worthy of a peer-reviewed journal. A recent Wirecutter guide to air purifiers, for instance, took months to complete and ran at twenty-two thousand words, including ozone analysis and reporting on what withstood assault by a toddler. After pieces are published, they become living documents, updated as products change and evaluation methods evolve. At Wirecutter, editorial independence is paramount: “We don’t want any of our writers and editors making any decisions, even subconsciously, that may be influenced by knowing whether or not we have any monetary relationship with a business partner,” Linda Li, the senior vice president and general manager, told me.

Still, for every satisfied shopper, there’s another for whom a review-endorsed item was all wrong. “I bought pillows from The Strategist because I saw they had an article on best pillows for side sleepers, and the top recommendation was the one I bought,” Stephanie DeVaan, of Seattle, told me. “It wasn’t any better than my five-dollar pillow from Target.” Kit Zauhar, of New York, has a Wirecutter-endorsed dish rack. “It doesn’t hold many dishes, and it gets moldy. It’s honestly the worst dish rack I could have ever imagined,” she said. Astra Taylor, of Asheville, North Carolina, told me, “I bought Wirecutter’s best coffee maker, and it sucked. It leaked all over the floor.”

For all the focus on optimization, no amount of testing can overcome the fact that there are really no “best” products, even for a plurality of people. Months ago, I was afflicted with back pain and bought Wirecutter’s pick for the best heating pad of 2023, the PureReliefKing XL. It kept slipping; it was never as warm as I wanted it to be. Eventually, I stopped at a Walgreens and grabbed a generic microwavable neck pillow at random. It worked much better. E-commerce writers can scientifically break down items into their most basic and quantifiable parts, but a method-based write-up can never fully account for shoppers’ disparate and strange preferences.

Since my Strategist internship ended, in May of 2019, a few things have changed. The most obvious is the setting of New York’s offices: in 2020, shortly after the company was bought by Vox Media, the magazine moved to a glittering skyscraper in Battery Park City, twenty floors above a shopping mall. When I visited recently, Maxine Builder, who now edits The Strategist, said the mall provided a metaphor: “Sometimes you need a new pair of sneakers or a blender, and you can find a really good recommendation for that. But sometimes you get distracted by a little kiosk or the smell of a pretzel. I want to have stories on the site that offer delight and surprise and help you find the thing you never knew you needed.”

​There were other differences, too. At its founding, Swerdloff told me, The Strategist placed less emphasis on scientific analysis than on embodying New York’s sensibility: humorous, stylish, celebrity-conscious. “Sometimes you want to know something other than ‘It works really well, and we tested it for eight hundred hours,’” she said. But in 2021, Google updated its system for ranking recommendation sites in search results: “We know people appreciate product reviews that share in-depth research,” a Google blog post explained. Under the new guidelines, Google would prioritize “high quality reviews”—articles with “express expert knowledge” that feature “quantitative measurements.” Almost immediately, People’s Choice—The Strategist’s cash cow, composed of customer-feedback summaries—took a hit. Builder responded by hitting pause on that and establishing a new column, Best in Class. For Best in Class, writers would sometimes test the merchandise they covered or reach out to category experts (Jeff Halevy, a former health correspondent for the Today show, spoke on the best protein powders; Parisa Fowles-Pazdro, the founder of a luxury dog-gear store, weighed in on the best dog carrier). Each Best in Class article included a didactic introduction, explaining the metrics by which a product was judged, down to its most trivial features (“brush head size” for toothbrushes, “opening mechanism” for panini presses). In other words, The Strategist’s new flagship shopping guide looked a lot like Wirecutter’s.

For Strategist writers, the shift to Best in Class was monumental. They were sent a spreadsheet with hundreds of old articles to be converted to the Best in Class format. Retrofitting stories about fashion and beauty turned out to be easy enough; everything else, however, had to be completely reconceived, with new interviews and hours of research. In some cases, products once borrowed from companies were no longer available. Current and former Strategist contributors described to me a culture of burnout, an expectation that they become experts in a growing number of product categories and deepen their research under ever tighter deadlines. A third of the writers quit. Recently, they have been asked to publish or update four articles per week—a schedule that makes some contributors feel rushed. (“We take our editorial integrity incredibly seriously, and the same journalistic rigor that is applied to everything New York magazine publishes applies to all Strategist articles,” Lauren Starke, the senior vice president of communications at Vox Media, told me. “The entire enterprise is based on reader trust, which we work hard to earn and never take for granted.”)

I wondered whether the financial incentives of e-commerce had led publishers to make impossible demands. “I think it’s just journalism,” Builder said. “I don’t think it’s specific to e-commerce in particular.” It’s true that the average reporting job is taxing, but the average reporter isn’t usually responsible for generating hundreds of thousands of dollars for their employer per annum. At The Strategist, writers are acutely aware of the stakes. The site has always kept its editorial and business teams in contact; Cho described the boundary as “a transparent wall.” Between calls with retail partners, she said, “we’re constantly feeding Alexis and Maxine’s team information and insights that we believe just constantly improve the service journalism we’re doing.” Swerdloff told me: “Readers love anything to do with sleep.” Anything to do with tech is usually a bust. (Starke said editors have “complete independence” when it comes to using the business team’s intel.) Until 2021, writers were shown financial data, and the figures were astounding: a single article could sometimes generate the equivalent of a writer’s entire salary. The cessation of that practice relieved writers of some stress, even as it conveniently obscured the outsize value of their labor.

At other media companies, the pressure to deliver results can feel even greater. Condé Nast has a centralized search team that coordinates with editorial staff across the company’s e-commerce verticals. Writers told me that they’re fed increasing revenue goals on a regular basis. Lately, verticals at several magazines have been asked to perform “weekly refreshes”—that is, to make small changes to e-commerce articles that Google registers as an “update,” bumping those pieces to the top of search results. Refreshes might seem innocuous—usually they involve adding products to a story, or simply rewording it—but taken together, they can have a sinister effect: Condé’s search team is known to issue the same recommendations across its e-commerce sites; as one of the company’s editors told me, teams are effectively pitted against each other while shilling the same merchandise in the same way. (“It’s a co-collaboration,” Melissa McVeigh, Condé Nast’s VP of consumer product for commerce and brands, said. “Each piece of content is different. We’re taking lessons from one brand—what sells—into another. And that’s just good practice.”)

Digital media’s collective embrace of product reviews brings to mind lessons of the “pivot to video” debacle, when dozens of companies poured resources into video verticals at the pleasure of Facebook, only to lose out, hemorrhaging revenue and laying off hundreds of journalists. Amazon presents its own cautionary tale: in 2020, a Wall Street Journal investigation concluded that the company exploited privileged data it collected on independent sellers while developing copycat products for Amazon Basics, its in-house product line. And Google has made updates to its e-commerce service, Shopping, that portend trouble: the recent unveiling of an AI-generated-review feature, with merchandise descriptions accompanied by pros, cons, and photos, appears to be in direct competition with the product review sites that now show up farther down the search results page.

Nevertheless, the media executives I talked to seem confident. “Nothing beats editorial voice,” Condé’s McVeigh told me. “From The Strategist’s beginning,” Starke, of Vox Media, said, “we’ve distinguished ourselves by publishing an editorially excellent product.” But the vast majority of product reviews are similar enough to have plausibly appeared anywhere, and their length is often more baffling than useful (“39 Dreamy Mattress Deals You Can Shop over Memorial Day Weekend”; “The 21 best lip balms for treating dry, chapped lips”). Last year, McKinsey & Company estimated that media companies could reap as much as fifty billion dollars from “commerce media” by 2026. To achieve that, McKinsey’s senior partners urged publishers to “become more like a retailer.”

Visiting The Strategist, I was reminded of why I abandoned e-commerce journalism. During my internship, editors encouraged me to pitch article ideas, and finally I had one: an ode to a Danish company that made mobiles, the hanging kinetic sculptures that turn freely in space. Usually, you find them in a baby’s nursery; I had bought one that I loved for my apartment. I was about to draft a note—but then stopped myself. I didn’t like the possibility that my mobile could become another Strategist design trend. Nor did I relish the prospect that my recommendation might be copied by other e-commerce verticals or dropped into year-end lists and holiday gift guides (“The 30 Best Mobiles for a Tranquil Living Space, According to Amazon Reviewers”). I kept the idea to myself.

Before it was time to depart from New York once more, I took a moment to look out of the office’s panoramic windows. In the distance was New Jersey’s Port Elizabeth, where millions of products were being offloaded by enormous cranes and trucked to warehouses and distributors—ready to be transported and monetized, yet again, as content of the internet.

Editor’s Note: This piece has been updated to correct quote attribution and the neighborhood of the Vox Media office; a response from Vox Media was added.

Will Tavlin is a writer who lives in Brooklyn.

The Business Model Issue »