A Silicon Valley startup that used ChatGPT for all its coding says it lost US$10,000 (AU$15,000) in sales after an error in the off-the-shelf code was discovered.
Reworkd, a company founded by a small team of computer science students, is a service that helps extract and analyse web data for large firms. Like many startups in the industry, Reworkd integrated OpenAI’s ChatGPT into its business.
Sounds pretty sweet, right? In July 2023 the startup raised CA$1.65 million (AU$2.49 million) from investors. Reworkd is also backed by iconic Silicon Valley incubator Y Combinator (“YC”), which is behind some of the most successful tech startups in history such as AirBnB, Uber, Reddit and DoorDash.
To be funded by YC is kind of a big deal. So, Reworkd’s founders had much to be excited about — until one day, they decided to use ChatGPT.
ChatGPT vs. coding
On June 9 Reworkd co-founder Asim Shrestha wrote a now-deleted blog post admitting why the startup’s website was down throughout last week.
Months ago, when migrating the company’s website, Shrestha and his team decided to turn to ChatGPT to generate a few lines of code. The chatbot spat out some lines so legit that the startup decided to use almost all of ChatGPT’s codes.
“We found that ChatGPT did a pretty exceptional job doing this translation and so we used it for almost the entire migration,” Shrestha explained.
At the end of May Reworkd finally decided to turn on a monetisation setting on the website, and that was when the trouble began. Shrestha said his staff faced a bombardment of complaints from clients saying they weren’t able to access Reworkd’s services.
“E v e r y t h i n g seemed to have set on fire overnight,” Shrestha said. “None of these users could subscribe. We had no idea why.”
After five days, countless angry emails from customers, and hours of staring at lines of code, the co-founder said they finally found the error within a single line of code. Hooray! We saved the company!
Right? Right … ?!?!?! No.
AI’s costly mistake
Keep in mind that nearly all the code used by Reworkd was taken from ChatGPT. This meant that the team had to scrap nearly all the work done by the chatbot.
“We ended up just copying over the same format that ChatGPT wrote for our existing models,” Shrestha said in his blog post. “What we failed to notice was that we were copying over the same issue with the way we were generating IDs in all our models.”
That’s right folks, bro bet his company’s reputation on “Copy”, “Paste” and ChatGPT.
Over the five days the company says it lost a grand total of US$10,000 (AU$15,000) in sales — “and that was only from people who cared enough to complain”, the co-founder wrote.
ChatGPT’s hallucinations
Perhaps one would be asking, why use ChatGPT for a task like coding that needs extreme attention to detail? Cost efficiency and time saving are probably why even tech whizzes are using ChatGPT for work.
Although ChatGPT has ascended to a higher level of intelligence with GPT4-o, AI chatbots still have the capacity to make factual mistakes. Tech experts call these mistakes “hallucinations”.
Was it painful? Yes. But did Shrestha regret it? No. He stated that it was all part of the journey of being a tech startup founder. “We’re just happy that we can now look back at those days and laugh,” he said.
That’s a positive outlook to have, I guess.
