Award Banner
Award Banner

For tech giants, AI like Bing and Bard poses billion-dollar search problem

For tech giants, AI like Bing and Bard poses billion-dollar search problem
The logo for Google LLC is seen at the Google Store Chelsea in Manhattan, New York City, US on Nov 17, 2021.
PHOTO: Reuters file

MOUNTAIN VIEW — As Alphabet Inc looks past a chatbot flub that helped erase US$100 billion (S$134 billion) from its market value, another challenge is emerging from its efforts to add generative artificial intelligence to its popular Google Search: the cost.

Executives across the technology sector are talking about how to operate AI like ChatGPT while accounting for the high expense.

The wildly popular chatbot from OpenAI, which can draft prose and answer search queries, has "eye-watering" computing costs of a couple or more cents per conversation, the startup's Chief Executive Sam Altman has said on Twitter.

In an interview, Alphabet's Chairman John Hennessy told Reuters that having an exchange with AI known as a large language model likely cost 10 times more than a standard keyword search, though fine-tuning will help reduce the expense quickly.

Even with revenue from potential chat-based search ads, the technology could chip into the bottom line of Mountain View, California-based Alphabet with several billion dollars of extra costs, analysts said. Its net income was nearly US$60 billion in 2022.

Morgan Stanley estimated that Google's 3.3 trillion search queries last year cost roughly a fifth of a cent each, a number that would increase depending on how much text AI must generate.

Google, for instance, could face a US$6-billion hike in expenses by 2024 if ChatGPT-like AI were to handle half the queries it receives with 50-word answers, analysts projected.

Google is unlikely to need a chatbot to handle navigational searches for sites like Wikipedia.

Others arrived at a similar bill in different ways.

For instance, SemiAnalysis, a research and consulting firm focused on chip technology, said adding ChatGPT-style AI to search could cost Alphabet US$3 billion, an amount limited by Google's in-house chips called Tensor Processing Units, or TPUs, along with other optimisations.

What makes this form of AI pricier than conventional search is the computing power involved.

Such AI depends on billions of dollars of chips, a cost that has to be spread out over their useful life of several years, analysts said. Electricity likewise adds costs and pressure to companies with carbon-footprint goals.

The process of handling AI-powered search queries is known as "inference," in which a "neural network" loosely modelled on the human brain's biology infers the answer to a question from prior training.

In a traditional search, by contrast, Google's web crawlers have scanned the internet to compile an index of information. When a user types a query, Google serves up the most relevant answers stored in the index.

Alphabet's Hennessy told Reuters, "It's inference costs you have to drive down," calling that "a couple year problem at worst."

Alphabet is facing pressure to take on the challenge despite the expense.

Earlier this month, its rival Microsoft Corp held a high-profile event at its Redmond, Washington headquarters to show off plans to embed AI chat technology into its Bing search engine, with top executives taking aim at Google's search market share of 91 per cent, by Similarweb's estimate.

A day later, Alphabet talked about plans to improve its search engine, but a promotional video for its AI chatbot Bard showed the system answering a question inaccurately, fomenting a stock slide that shaved US$100 billion off its market value.

Microsoft later drew scrutiny of its own when its AI reportedly made threats or professed love to test users, prompting the company to limit long chat sessions it said "provoked" unintended answers.

Read Also
digicult
Microsoft's Bing plans AI ads in early pitch to advertisers

Microsoft's Chief Financial Officer Amy Hood has told analysts that the upside from gaining users and advertising revenue outweighed expenses as the new Bing rolls out to millions of consumers.

"That's incremental gross margin dollars for us, even at the cost to serve that we're discussing," she said.

And another Google competitor, CEO of search engine You.com Richard Socher, said adding an AI chat experience as well as applications for charts, videos and other generative tech raised expenses between 30 per cent and 50 per cent.

"Technology gets cheaper at scale and over time," he said.

A source close to Google cautioned it's early to pin down exactly how much chatbots might cost because efficiency and usage vary widely depending on the technology involved, and AI already powers products like search.

Still, footing the bill is one of two main reasons why search and social media giants with billions of users have not rolled out an AI chatbot overnight, said Paul Daugherty, Accenture's chief technology officer.

"One is accuracy, and the second is you have to scale this in the right way," he said.

Making the math work

For years, researchers at Alphabet and elsewhere have studied how to train and run large language models more cheaply.

Bigger models require more chips for inference and therefore cost more.

AI that dazzles consumers for its human-like authority has ballooned in size, reaching 175 billion so-called parameters, or different values that the algorithm takes into account, for the model OpenAI updated into ChatGPT.

Cost also varies by the length of a user's query, as measured in "tokens" or pieces of words.

One senior technology executive told Reuters that such AI remained cost-prohibitive to put in millions of consumers' hands.

"These models are very expensive, and so the next level of invention is going to be reducing the cost of both training these models and inference so that we can use it in every application," the executive said on condition of anonymity.

Read Also
digicult
Microsoft-backed OpenAI to let users customise ChatGPT

For now, computer scientists inside OpenAI have figured out how to optimise inference costs through complex code that makes chips run more efficiently, a person familiar with the effort said.

An OpenAI spokesperson did not immediately comment.

A longer-term issue is how to shrink the number of parameters in an AI model 10 or even 100 times, without losing accuracy.

"How you cull (parameters away) most effectively, that's still an open question," said Naveen Rao, who formerly ran Intel Corp's AI chip efforts and now works to lower AI computing costs through his startup MosaicML.

In the meantime, some have considered charging for access, like OpenAI's US$20 per month subscription for better ChatGPT service.

Technology experts also said a workaround is applying smaller AI models to simpler tasks, which Alphabet is exploring.

The company said this month a "smaller model" version of its massive LaMDA AI technology will power its chatbot Bard, requiring "significantly less computing power, enabling us to scale to more users."

Asked about chatbots like ChatGPT and Bard, Hennessy said at a conference called TechSurge last week that more focused models, rather than one system doing everything, would help "tame the cost".

ALSO READ: 'As significant as the invention of the internet': Bill Gates says ChatGPT 'will change our world'

Source: Reuters

homepage

trending

trending
    $320m crypto heist suspect: How a Choa Chu Kang student went from gaming to a life of luxury
    Man to be charged for abusing airline staff and kicking wall at Changi Airport over lost phone
    Long delays at Woodlands checkpoint: Over 2-hour wait as school holidays start
    Tay Ping Hui says acting in China was a 'super humbling experience'
    From serving passengers to taking flight: How an ex-SIA flight attendant fulfilled her dream of becoming a pilot
    Rain holds surprise performance in Singapore, feasts on crab afterwards
    'Spectacular sight': SM Lee shares stunning photo of rare sun halo over Singapore
    SM Lee unveils future improvements in Ang Mo Kio, calls for residents' continued support
    Elderly man dies after falling in bus, family appealing for witnesses
    'They deserve an award': Pre-school teachers hailed for carrying children to safety amid Punggol fire
    Keep off the tracks: SMRT issues warning after man climbed over barriers at Boon Lay MRT station to retrieve phone
    This newly MOP-ed 4-room HDB flat at Bidadari sets new record with almost $800k profit

Singapore

Singapore
    • 'Deliberately confrontational': Shanmugam calls out activists for disrupting PAP's meet-the-people sessions
    • Ex-Kinderland teacher gets jail for abusing children in her care
    • GE2025: 5 hot wards to watch
    • Suspected gang boss, who ran gambling dens in Geylang, arrested in Malaysia after 4 years on the run
    • At least $1.7m lost in Singapore to scammers using insurance policies as bait
    • 'If I open this door, what is behind it?' How Coast Guard responded to illegal boarding of tanker in Singapore Strait
    • Mediation by senior establishment figures led to CDL truce, sources say
    • Daily roundup: 5 hot wards to watch in GE2025 — and other top stories today
    • Bus driver, 62, dies after accident involving lorry at Outram Road
    • 4 Dutch crewmen plead guilty for causing Singapore's worst oil spill in 10 years

Entertainment

Entertainment
    • Ya Hui toughs it out in new job as creative director: 'I'm not a flower you need to take care of'
    • Kim Soo-hyun admits to dating late Kim Sae-ron, refutes allegations she was underaged
    • 'Here's to new beginnings': Kiki Lim leaves Mediacorp after 5 years
    • 'Where am I supposed to say I'm from?' Jackson Wang addresses controversial clip of him saying 'I'm Chinese'
    • Ellen Pompeo says she won't let her daughter watch Grey's Anatomy
    • 'It's watered down': Will.i.am says TikTok has changed the music industry
    • Jonathan Majors reveals he was 'sexually abused by men and women' since age 9
    • 'I felt this raw nerve on display': Johnny Depp recalls being 'freaked out' by fame
    • 'It's funny, teaching him how to do laundry': Home Alone star Macaulay Culkin's fiancee
    • Oasis to release documentary film in conjunction with concert tour

Lifestyle

Lifestyle
    • 2 Singapore pizzerias earn spots on 50 Top Pizza Asia-Pacific 2025 list
    • Luckin Coffee launches Maltese-themed drinks and limited-edition merchandise as part of collaboration
    • 23 local attractions offering SG60 deals, here's what Singaporeans can enjoy
    • Hunt the Mouse is back, with $100k worth of coins hidden at Sentosa waiting to be found
    • All-new Skoda Superb Combi unveiled
    • The truth about dual-key living in Singapore: What 3 homeowners want you to know
    • Toyota Camry review: A classic nameplate that vastly amps up the luxury and fun factor
    • Dear teachers, I owe you an apology: Lessons I learnt as a parent
    • Neta X review: Affordable Chinese EV with more than a few flaws
    • Touring Lucky View: Freehold landed homes from $5m in the east

Digicult

Digicult
    • A $500 wake-up call: How the Samsung Galaxy Ring made me realise my stress
    • Monster Hunter Wilds producer explains how game has remained unique and fresh over 20 years
    • China's Baidu launches 2 new AI models as industry competition heats up
    • China's top universities expand enrolment to beef up capabilities in AI, strategic areas
    • International Women's Day: Meet the Singapore women levelling up in gaming
    • US indicts slew of alleged Chinese hackers, sanctions company over spy campaign
    • Meet the women powering innovation in tech in Singapore and beyond
    • Games in March: JRPGs, co-op games and wrestling
    • Australia fines Telegram for delay in answering child abuse, terror questions
    • Embracing the future: A personal guide to how to get started with artificial intelligence

Money

Money
    • Amid AI push, DBS to cut 4,000 temp jobs over next 3 years
    • Singapore's economy grew 4.4% in 2024, beating forecasts
    • DBS to give staff $1,000 special bonus, introduce capital return dividend
    • Travel insurance for infectious diseases: What you need to know (2025)
    • Best bank offers in Singapore (March 2025): Lady Gaga concert presale, up to 50% off flights and more
    • How high will Singapore property prices get in 2025? Here's what AI predicts
    • Before you say 'I do' to a home: The checklist every newlywed couple needs in Singapore
    • Are older HDB maisonettes and jumbo flats worth the premium? A deep dive into pricing trends
    • Chocolate Finance imposes $250 spending limit on debit card to manage demand for miles rewards
    • Trump threatens further tariffs as EU, Canada retaliate for those already in place

Latest

Latest
  • Chinese training fighter jet pilot parachutes to safety after crash, PLA Navy says
  • Judge temporarily blocks Trump's use of wartime powers to target Venezuelan gang members
  • Peru fisherman lost for 95 days in Pacific Ocean returns home
  • Improving Pope reduces use of ventilation for oxygen, Vatican says
  • US to pay El Salvador to jail 300 alleged gang members, AP reports
  • Angry Canadians get their 'elbows up' in face of Trump threats
  • Pope Francis, showing plans to stay on, starts new Catholic reform process
  • Massive rallies across South Korea ahead of key ruling on impeachment of Yoon
  • Australia's New South Wales sweats through heatwave, faces 'extreme' bushfire risk

In Case You Missed It

In Case You Missed It
  • Malaysian woman stunned to find car trapped among diners after car park turns into outdoor eatery at night
  • Johor invests $1.5b in water treatment infrastructure to reduce dependency on Singapore after 2030
  • Ceiling collapses at KSL City Mall in JB, viral videos show
  • Food poisoning outbreak: 187 cases linked to Total Defence Day ready-to-eat meals, probe underway
  • Blazing car on Nicoll Highway: 1 dies, 2 taken to hospital
  • FairPrice giving away 75,000 sets of snacks and drinks to Muslim customers to break fast
  • Disabled Malaysian boy seen 'walking' with hands in school gets walkers from kind donors after TikTok video went viral
  • CDL's Kwek Leng Beng seeks to fire son Sherman as CEO over alleged 'attempted coup'
  • 10-year-old girl hurt by pebbles thrown from condo's 11th storey by toddler
This website is best viewed using the latest versions of web browsers.