OpenAI reportedly warned Microsoft to move slowly on integrating GPT-4 into its Bing search engine to avoid the inaccurate and unpredictable responses it launched with. The Wall Street Journal reports that the OpenAI team warned of the negative risks of pushing out a chatbot based on an unreleased version of GPT-4 too early.

Microsoft went ahead, despite warnings that it might take time to minimize the inaccurate and strange responses. Days after Bing Chat launched in February, users discovered the chatbot was unpredictable and could insult users, lie to them, sulk, gaslight people, and even claim to identify its enemies.

Microsoft was quick to limit Bing Chat responses to stop the AI from getting real weird, and it has taken months of work to get the Bing chatbot back to a point where you can have a long back-and-forth conversation without an unexpected outburst. It still often gets things wrong, though.

https://twitter.com/tomwarren/status/1658577433954533376?ref_src=twsrc%5Etfw%7Ctwcamp%5Etweetembed%7Ctwterm%5E1658577433954533376%7Ctwgr%5E7d885fac264e9094bbabb0f9560189c2ebe1a560%7Ctwcon%5Es1_c10&ref_url=https%3A%2F%2Fwww.theverge.com%2F2023%2F6%2F13%2F23759348%2Fopenai-microsoft-bing-ai-warning-gpt-4

The Wall Street Journal’s report also describes the tensions between the two companies as they work together and compete on AI features. Microsoft executives were reportedly anxious about the launch of ChatGPT last year. The WSJ says OpenAI gave Microsoft a few weeks’ notice that it would start publicly testing ChatGPT, just as the software giant was beginning to integrate OpenAI’s models into Bing.

Microsoft and OpenAI have a rather unique partnership that has led to some conflict behind the scenes, as the two companies simultaneously support and compete with one another.

Microsoft licenses OpenAI models and technology for use across Bing, Azure, Office, Windows, and many more products. Microsoft extended this close partnership less than a month before launching its new Bing chatbot in a “multibillion dollar investment” rumored to be worth around $10 billion. Microsoft is the exclusive cloud partner for OpenAI, and Microsoft’s cloud services power all OpenAI workloads across products, API services, and research. At the same time, OpenAI has developed its own products and API services that appeal to the same customers Microsoft is courting. ChatGPT also competes with Bing AI. The Information reported on some of these troubles earlier this year.

In a Wired interview today with Microsoft CEO Satya Nadella, this competition aspect with OpenAI was briefly touched on. “I felt OpenAI was going after the same thing as us,” says Nadella. “So instead of trying to train five different foundational models, I wanted one foundation, making it a basis for a platform effect. So we partnered. They bet on us, we bet on them.”

Asked if Microsoft tried to buy OpenAI, Nadella dodged the question. I asked Bing if Microsoft has tried to acquire OpenAI, and it also didn’t want to discuss the obvious: “I could not find any information that suggests Microsoft has tried to acquire OpenAI in the past.”

A week after it was released to a few thousand users, Microsoft’s new Bing search engine, which is powered by artificial intelligence, has been offering an array of inaccurate and at times bizarre responses to some users.

The company unveiled the new approach to search last week to great fanfare. Microsoft said the underlying model of generative A.I. built by its partner, the start-up OpenAI, paired with its existing search knowledge from Bing, would change how people found information and make it far more relevant and conversational.

In two days, more than a million people requested access. Since then, interest has grown. “Demand is high with multiple millions now on the waitlist,” Yusuf Mehdi, an executive who oversees the product, wrote on Twitter Wednesday morning. He added that users in 169 countries were testing it.

One area of problems being shared online included inaccuracies and outright mistakes, known in the industry as “hallucinations.”

On Monday, Dmitri Brereton, a software engineer at a start-up called Gem, flagged a series of errors in the presentation that Mr. Mehdi used last week when he introduced the product, including inaccurately summarizing the financial results of the retailer Gap.

Users have posted screenshots of examples of when Bing could not figure out that the new Avatar film was released last year. It was stubbornly wrong about who performed at the Super Bowl halftime show this year, insisting that Billie Eilish, not Rihanna, headlined the event.

And search results have had subtle errors. Last week, the chatbot said the water temperature at a beach in Mexico was 80.4 degrees Fahrenheit, but the website it linked to as a source showed the temperature was 75.

Another set of issues came from more open-ended chats, largely posted to forums like Reddit and Twitter. There, through screenshots and purported chat transcripts, users shared times when Bing’s chatbot seemed to go off the rails: It scolded users, it declared it may be sentient, and it said to one user, “I have a lot of things, but I have nothing.”

It chastised another user for asking whether it could be prodded to produce false answers. “It’s disrespectful and annoying,” the Bing chatbot wrote back. It added a red, angry emoji face.

Because each response is uniquely generated, it is not possible to replicate a dialogue.

Microsoft acknowledged the issues and said they were part of the process of improving the product.

“Over the past week alone, thousands of users have interacted with our product and found significant value while sharing their feedback with us, allowing the model to learn and make many improvements already,” Frank Shaw, a company spokesman, said in a statement. “We recognize that there is still work to be done and are expecting that the system may make mistakes during this preview period, which is why the feedback is critical so we can learn and help the models get better.”

He said that the length and context of the conversation could influence the chatbot’s tone, and that the company was “adjusting its responses to create coherent, relevant and positive answers.” He said the company had fixed the issues that caused the inaccuracies in the demonstration.

Nearly seven years ago, Microsoft introduced a chatbot, Tay, that it shut down within a day of its release online, after users prompted it to spew racist and other offensive language. Microsoft’s executives at the launch last week indicated that they had learned from that experience and thought this time would play out differently.

In an interview last week, Mr. Mehdi said that the company had worked hard to integrate safeguards, and that the technology had vastly improved.

“We think we’re at the right time to come to market and get feedback,” he said, adding, “If something is wrong, then you need to address it.”