{"id":1260560,"date":"2023-07-24T08:11:23","date_gmt":"2023-07-24T02:41:23","guid":{"rendered":"https:\/\/trak.in\/stories\/?p=1260560"},"modified":"2023-07-24T08:14:06","modified_gmt":"2023-07-24T02:44:06","slug":"chatgpt-is-getting-slower-dumber-less-accurate-as-per-this-stanford-uc-berkeley-research","status":"publish","type":"post","link":"https:\/\/trak.in\/stories\/chatgpt-is-getting-slower-dumber-less-accurate-as-per-this-stanford-uc-berkeley-research\/","title":{"rendered":"ChatGPT Is Getting Slower, Dumber & Less Accurate As Per This Stanford, UC Berkeley Research"},"content":{"rendered":"\n

Users are reporting a deterioration in OpenAI\u2019s ChatGPT’s performance, with complaints that it is getting extremely slow and far less capable of providing accurate answers.<\/p>\n\n\n\n

Researchers at Stanford and UC Berkeley found that the performance has worsened over time, thus making its answers less accurate.<\/p>\n\n\n\n

\"ChatGPT<\/figure>\n\n\n\n

Task-wise performance<\/strong><\/p>\n\n\n\n

They analyzed different versions of ChatGPT from March 2023 and June 2023 versions of GPT-3.5 and GPT-4 on four primary tasks.<\/p>\n\n\n\n

They were assessed on tasks involving solving math problems, answering sensitive questions, generating code and visual reasoning.<\/p>\n\n\n\n

In March, ChatGPT-4 could identify prime numbers with a 97.6 per cent accuracy rate.<\/p>\n\n\n\n

However, in June it got only 12 questions right, plunging to 2.4 per cent accuracy.<\/p>\n\n\n\n

It also performed poorly in generating code.<\/p>\n\n\n\n

Cost-cutting<\/strong><\/p>\n\n\n\n

One theory explains that to offset the high cost of operating the systems, companies like OpenAI aren\u2019t <\/a>putting out the best versions of chatbots to the public.<\/p>\n\n\n\n

\u201cRumors suggest they are using several smaller and specialized GPT-4 models that act similarly to a large model but are less expensive to run,\u201d said AI expert Santiago Valderrama.<\/p>\n\n\n\n

There has also been speculation that changes made to speed up the service and thereby reduce costs leads to quicker responses but degraded competency.<\/p>\n\n\n\n

Side effect of continuous tweaks<\/strong><\/p>\n\n\n\n

GPT 3.5 and 4 are language models that are continuously <\/a>updated but OpenAI doesn\u2019t announce many of the changes made to them.<\/p>\n\n\n\n

In the paper, they conclude that the behavioral changes are a side effect of unannounced updates to how the models function.<\/p>\n\n\n\n

This leads to a fluctuation in the quality of these models.<\/p>\n\n\n\n

\u201cA LLM like GPT-4 can be updated over time based on data and feedback from users as well as design changes.<\/p>\n\n\n\n

However, it is currently opaque when and how GPT-3.5 and GPT-4 are updated, and it is unclear how each update affects the behavior of these LLMs”, the researchers write.<\/p>\n\n\n\n

Safety comprises quality<\/strong><\/p>\n\n\n\n

Another possibility is that changes introduced to prevent ChatGPT from answering dangerous questions impairs its usefulness for other tasks.<\/p>\n\n\n\n

They found that ChatGPT\u2019s newer version refused to answer certain sensitive questions.<\/p>\n\n\n\n

Jim Fan, senior <\/a>scientist at Nvidia, wrote on Twitter, \u201cUnfortunately, more safety typically comes at the cost of less usefulness.\u201d<\/p>\n\n\n\n

Regular quality checks<\/strong><\/p>\n\n\n\n

They said that companies that depend on OpenAI should consider conducting regular quality assessments in order to monitor for unexpected changes.<\/p>\n\n\n\n

In the same vein, some have called for open-source models like Meta’s LLaMA that enable community debugging.<\/p>\n\n\n\n

The study stresses the importance of regularly monitoring performance so that problems or issues that arise can be identified and addressed promptly.<\/p>\n","protected":false},"excerpt":{"rendered":"

Users are reporting a deterioration in OpenAI\u2019s ChatGPT’s performance, with complaints that it is getting extremely slow and far less capable of providing accurate answers. Researchers at Stanford and UC Berkeley found that the performance has worsened over time, thus making its answers less accurate. Task-wise performance They analyzed different versions of ChatGPT from March […]<\/p>\n","protected":false},"author":24,"featured_media":1260567,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[21],"tags":[584,1541],"class_list":["post-1260560","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-business","tag-chatgpt","tag-openai"],"acf":[],"yoast_head":"\nChatGPT Is Getting Slower, Dumber & Less Accurate As Per This Stanford, UC Berkeley Research - Trak.in - Indian Business of Tech, Mobile & Startups<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/trak.in\/stories\/chatgpt-is-getting-slower-dumber-less-accurate-as-per-this-stanford-uc-berkeley-research\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"ChatGPT Is Getting Slower, Dumber & Less Accurate As Per This Stanford, UC Berkeley Research - Trak.in - Indian Business of Tech, Mobile & Startups\" \/>\n<meta property=\"og:description\" content=\"Users are reporting a deterioration in OpenAI\u2019s ChatGPT’s performance, with complaints that it is getting extremely slow and far less capable of providing accurate answers. Researchers at Stanford and UC Berkeley found that the performance has worsened over time, thus making its answers less accurate. Task-wise performance They analyzed different versions of ChatGPT from March […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/trak.in\/stories\/chatgpt-is-getting-slower-dumber-less-accurate-as-per-this-stanford-uc-berkeley-research\/\" \/>\n<meta property=\"og:site_name\" content=\"Trak.in - Indian Business of Tech, Mobile & Startups\" \/>\n<meta property=\"article:published_time\" content=\"2023-07-24T02:41:23+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-07-24T02:44:06+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/trak.in\/stories\/wp-content\/uploads\/2023\/07\/Untitled-design-12-3.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1000\" \/>\n\t<meta property=\"og:image:height\" content=\"465\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Shreya Bose\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Shreya Bose\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/trak.in\/stories\/chatgpt-is-getting-slower-dumber-less-accurate-as-per-this-stanford-uc-berkeley-research\/\",\"url\":\"https:\/\/trak.in\/stories\/chatgpt-is-getting-slower-dumber-less-accurate-as-per-this-stanford-uc-berkeley-research\/\",\"name\":\"ChatGPT Is Getting Slower, Dumber & Less Accurate As Per This Stanford, UC Berkeley Research - Trak.in - Indian Business of Tech, Mobile & Startups\",\"isPartOf\":{\"@id\":\"https:\/\/trak.in\/stories\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/trak.in\/stories\/chatgpt-is-getting-slower-dumber-less-accurate-as-per-this-stanford-uc-berkeley-research\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/trak.in\/stories\/chatgpt-is-getting-slower-dumber-less-accurate-as-per-this-stanford-uc-berkeley-research\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/trak.in\/stories\/wp-content\/uploads\/2023\/07\/Untitled-design-12-3.png\",\"datePublished\":\"2023-07-24T02:41:23+00:00\",\"dateModified\":\"2023-07-24T02:44:06+00:00\",\"author\":{\"@id\":\"https:\/\/trak.in\/stories\/#\/schema\/person\/9817221c96a9aadc34f34cc7b8767dc6\"},\"breadcrumb\":{\"@id\":\"https:\/\/trak.in\/stories\/chatgpt-is-getting-slower-dumber-less-accurate-as-per-this-stanford-uc-berkeley-research\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/trak.in\/stories\/chatgpt-is-getting-slower-dumber-less-accurate-as-per-this-stanford-uc-berkeley-research\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/trak.in\/stories\/chatgpt-is-getting-slower-dumber-less-accurate-as-per-this-stanford-uc-berkeley-research\/#primaryimage\",\"url\":\"https:\/\/trak.in\/stories\/wp-content\/uploads\/2023\/07\/Untitled-design-12-3.png\",\"contentUrl\":\"https:\/\/trak.in\/stories\/wp-content\/uploads\/2023\/07\/Untitled-design-12-3.png\",\"width\":1000,\"height\":465},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/trak.in\/stories\/chatgpt-is-getting-slower-dumber-less-accurate-as-per-this-stanford-uc-berkeley-research\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/trak.in\/stories\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"ChatGPT Is Getting Slower, Dumber & Less Accurate As Per This Stanford, UC Berkeley Research\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/trak.in\/stories\/#website\",\"url\":\"https:\/\/trak.in\/stories\/\",\"name\":\"Trak.in - Indian Business of Tech, Mobile & Startups\",\"description\":\"Trak.in is a popular Indian Business, Technology, Mobile & Startup blog featuring trending News, views and analytical take on Technology, Business, Finance, Telecom, Mobile, startups & Social Media Space\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/trak.in\/stories\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/trak.in\/stories\/#\/schema\/person\/9817221c96a9aadc34f34cc7b8767dc6\",\"name\":\"Shreya Bose\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/trak.in\/stories\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/1404d637610526a87c014a8eac9883a6c61491f1e6553d1946b0dd0b135713be?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/1404d637610526a87c014a8eac9883a6c61491f1e6553d1946b0dd0b135713be?s=96&d=mm&r=g\",\"caption\":\"Shreya Bose\"},\"url\":\"https:\/\/trak.in\/stories\/author\/shreya-bose\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"ChatGPT Is Getting Slower, Dumber & Less Accurate As Per This Stanford, UC Berkeley Research - Trak.in - Indian Business of Tech, Mobile & Startups","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/trak.in\/stories\/chatgpt-is-getting-slower-dumber-less-accurate-as-per-this-stanford-uc-berkeley-research\/","og_locale":"en_US","og_type":"article","og_title":"ChatGPT Is Getting Slower, Dumber & Less Accurate As Per This Stanford, UC Berkeley Research - Trak.in - Indian Business of Tech, Mobile & Startups","og_description":"Users are reporting a deterioration in OpenAI\u2019s ChatGPT’s performance, with complaints that it is getting extremely slow and far less capable of providing accurate answers. Researchers at Stanford and UC Berkeley found that the performance has worsened over time, thus making its answers less accurate. Task-wise performance They analyzed different versions of ChatGPT from March […]","og_url":"https:\/\/trak.in\/stories\/chatgpt-is-getting-slower-dumber-less-accurate-as-per-this-stanford-uc-berkeley-research\/","og_site_name":"Trak.in - Indian Business of Tech, Mobile & Startups","article_published_time":"2023-07-24T02:41:23+00:00","article_modified_time":"2023-07-24T02:44:06+00:00","og_image":[{"width":1000,"height":465,"url":"https:\/\/trak.in\/stories\/wp-content\/uploads\/2023\/07\/Untitled-design-12-3.png","type":"image\/png"}],"author":"Shreya Bose","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Shreya Bose","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/trak.in\/stories\/chatgpt-is-getting-slower-dumber-less-accurate-as-per-this-stanford-uc-berkeley-research\/","url":"https:\/\/trak.in\/stories\/chatgpt-is-getting-slower-dumber-less-accurate-as-per-this-stanford-uc-berkeley-research\/","name":"ChatGPT Is Getting Slower, Dumber & Less Accurate As Per This Stanford, UC Berkeley Research - Trak.in - Indian Business of Tech, Mobile & Startups","isPartOf":{"@id":"https:\/\/trak.in\/stories\/#website"},"primaryImageOfPage":{"@id":"https:\/\/trak.in\/stories\/chatgpt-is-getting-slower-dumber-less-accurate-as-per-this-stanford-uc-berkeley-research\/#primaryimage"},"image":{"@id":"https:\/\/trak.in\/stories\/chatgpt-is-getting-slower-dumber-less-accurate-as-per-this-stanford-uc-berkeley-research\/#primaryimage"},"thumbnailUrl":"https:\/\/trak.in\/stories\/wp-content\/uploads\/2023\/07\/Untitled-design-12-3.png","datePublished":"2023-07-24T02:41:23+00:00","dateModified":"2023-07-24T02:44:06+00:00","author":{"@id":"https:\/\/trak.in\/stories\/#\/schema\/person\/9817221c96a9aadc34f34cc7b8767dc6"},"breadcrumb":{"@id":"https:\/\/trak.in\/stories\/chatgpt-is-getting-slower-dumber-less-accurate-as-per-this-stanford-uc-berkeley-research\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/trak.in\/stories\/chatgpt-is-getting-slower-dumber-less-accurate-as-per-this-stanford-uc-berkeley-research\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/trak.in\/stories\/chatgpt-is-getting-slower-dumber-less-accurate-as-per-this-stanford-uc-berkeley-research\/#primaryimage","url":"https:\/\/trak.in\/stories\/wp-content\/uploads\/2023\/07\/Untitled-design-12-3.png","contentUrl":"https:\/\/trak.in\/stories\/wp-content\/uploads\/2023\/07\/Untitled-design-12-3.png","width":1000,"height":465},{"@type":"BreadcrumbList","@id":"https:\/\/trak.in\/stories\/chatgpt-is-getting-slower-dumber-less-accurate-as-per-this-stanford-uc-berkeley-research\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/trak.in\/stories\/"},{"@type":"ListItem","position":2,"name":"ChatGPT Is Getting Slower, Dumber & Less Accurate As Per This Stanford, UC Berkeley Research"}]},{"@type":"WebSite","@id":"https:\/\/trak.in\/stories\/#website","url":"https:\/\/trak.in\/stories\/","name":"Trak.in - Indian Business of Tech, Mobile & Startups","description":"Trak.in is a popular Indian Business, Technology, Mobile & Startup blog featuring trending News, views and analytical take on Technology, Business, Finance, Telecom, Mobile, startups & Social Media Space","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/trak.in\/stories\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/trak.in\/stories\/#\/schema\/person\/9817221c96a9aadc34f34cc7b8767dc6","name":"Shreya Bose","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/trak.in\/stories\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/1404d637610526a87c014a8eac9883a6c61491f1e6553d1946b0dd0b135713be?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/1404d637610526a87c014a8eac9883a6c61491f1e6553d1946b0dd0b135713be?s=96&d=mm&r=g","caption":"Shreya Bose"},"url":"https:\/\/trak.in\/stories\/author\/shreya-bose\/"}]}},"jetpack_featured_media_url":"https:\/\/trak.in\/stories\/wp-content\/uploads\/2023\/07\/Untitled-design-12-3.png","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/trak.in\/stories\/wp-json\/wp\/v2\/posts\/1260560","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/trak.in\/stories\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/trak.in\/stories\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/trak.in\/stories\/wp-json\/wp\/v2\/users\/24"}],"replies":[{"embeddable":true,"href":"https:\/\/trak.in\/stories\/wp-json\/wp\/v2\/comments?post=1260560"}],"version-history":[{"count":1,"href":"https:\/\/trak.in\/stories\/wp-json\/wp\/v2\/posts\/1260560\/revisions"}],"predecessor-version":[{"id":1260568,"href":"https:\/\/trak.in\/stories\/wp-json\/wp\/v2\/posts\/1260560\/revisions\/1260568"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/trak.in\/stories\/wp-json\/wp\/v2\/media\/1260567"}],"wp:attachment":[{"href":"https:\/\/trak.in\/stories\/wp-json\/wp\/v2\/media?parent=1260560"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/trak.in\/stories\/wp-json\/wp\/v2\/categories?post=1260560"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/trak.in\/stories\/wp-json\/wp\/v2\/tags?post=1260560"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}