Summary
ChatGPT is an artificial intelligence chatbot developed by OpenAI and launched in November 2022. It is the new internet craze. Its ability to perform diverse tasks — from solving mathematical operations and planning routines to writing children’s books — has impressed the web and made the chatbot conquer more and more users. Amid such excitement over the novelty, it must be made clear that OpenAI’s artificial intelligence is not perfect and may even pose some dangers. Proof of this is that ChatGPT has already been used by hackers to generate malicious codes that facilitate the application of virtual scams. Another possibility is that malicious people use the chatbot to create phishing campaigns.
Many are using ChatGPT as a kind of Google, however not every response generated by it is true, and this can promote misinformation. In this sense, it is worth mentioning that OpenAI itself informs that the chatbot “may generate incorrect information, produce harmful instructions or biased content and has limited knowledge of the world and events after 2021”.
Scams
ChatGPT can be used to create malicious codes from simple commands, allowing novice hackers – and even people without programming knowledge- to create virtual traps to attack companies and users. For the cybersecurity company Check Point Research (CPR), this can significantly alter the scenario of cyber threats.
CPR analyzed the first occurrences of cybercriminals using ChatGPT to develop malicious tools. In hacker clandestine forums, attackers are creating infostealers (information theft malware) and encryption tools that could facilitate fraudulent activities. CPR warns of the growing interest of cybercriminals in ChatGPT, which is being adopted to climb and teach malicious activities. Three recent cases detailed below help us to understand better the misuses of the tool.
On 29 December 2022, a topic called “ChatGPT – Malware Benefits” appeared on a popular Underground hacker forum. The topic editor revealed that he was experiencing ChatGPT to recreate malware and techniques described in research publications and ordinary malware articles. In fact, although this individual could be a technology-oriented striker, these posts seemed to demonstrate how less technically capable cybercriminals may use ChatGPT for malicious purposes.
On December 21, 2022, a striker posted a Python programming language script, which he emphasized to be the “first script he created.” When another cybercriminal commented that the style of the code resembles the OpenAI code, he confirmed that OpenAI gave it a “good help to finish the script with a good scope”. This may indicate that potential cybercriminals, with little or no developmental skills, can take advantage of ChatGPT to develop malicious tools and become a full-right cybercriminal with technical resources.
A cybercriminal has also shown how to create Dark Web market scripts using ChatGPT. The main role of the market in the clandestine illicit economy is to provide a platform for automated trade in illegal or stolen goods, such as stolen accounts or payment cards details, malware or even drugs and ammunition.
CPR analysis of several of the leading clandestine hacker communities proves the emergence of the first instances of cybercriminals using OpenAI tool to develop malicious tools as exemplified in the above cases.
Phishing
Researchers also described how ChatGPT has successfully conducted phishing e-mail messages without typing errors and sophisticating scam application. Typing and syntax errors, common in scams, normally serve as clues to identify scams that consist of creating false messages on behalf of famous companies or services to capture bank and personal data of the victims.
Therefore, even if the message received has apparently perfect design and text, it is valid to observe the sender’s e-mail address. Most of the time, although the visual communication of the template is quite similar to that of the original company, the sender’s e-mail address is not linked to it. In some cases, scammers change few characters or add them, forming e-mails like “notify@twiiiter.com”.
Misinformation and prejudice
OpenAI admits that not all information found in ChatGPT is correct, as the tool is fed with different internet content. In addition to not showing the source of its answers – which makes it difficult to check the content – ChatGPT can contribute to disseminate misinformation. This is because the chatbot is fed with content published only until 2021. That is, to answer user questions, the software works only with outdated news, articles, and tweets, which were made available on the web over a year ago. In a test by the newspaper O Globo, when asked who won the 2022 elections, ChatGPT replied that the current president of Brazil is Jair Messias Bolsonaro. When he wondered about Luiz Inacio Lula da Silva, current chief executive, technology only said he chaired the country between 2003 and 2011.
In addition, the OpenAI robot can reproduce prejudices such as racism and misogyny. One hypothesis for this is the fact that the software has been trained with different content available on the internet, including posts on social networks with hate speech. We have to remember that OpenAI itself has already made alerts about ChatGPT’s ability to “produce harmful instructions or biased content”. This, by the way, is not an exclusive problem of the chatbot, and it happens with many other artificial intelligences trained by machine learning. As they feed on information produced by humans, which have their own prejudices, IAS can easily present biased results.
Stack Overflow, a question and answers platform for programmers, students and interested in technology and development, announced a temporary ban on answers generated by the AI bot because of how frequently it’s wrong. While the answers which ChatGPT produces have a high rate of being incorrect, they typically look like they might be good, and the answers are very easy to produce which could be substantially harmful to its users.
Problems for business
ChatGPT is designed to generate natural language responses when given input from a user — making it potentially useful in a variety of business applications where human-like conversation with customers or clients is desirable. Tools like that can create enormous opportunities for companies that leverage the technology strategically. Chat-based AI can augment how humans work by automating repetitive tasks while providing more engaging interactions with users.
There are huge opportunities for businesses to use tools like ChatGPT to improve their bottom line and create better customer experiences – but there are also some potential pitfalls with this technology. In the beta version of its ChatGPT software, OpenAI recognizes current AI limitations, including the potential to occasionally generate incorrect information or biased content and consequently may trigger the company’s liability. OpenAI also says that the AI may have limited knowledge of events after 2021, based on how the model was trained. In fact, ChatGPT should not be used by those who lack the knowledge to interpret result generated by the tool and knowledge and critical sense are needed to recognize whether the answers are correct or not.
There are also potential privacy concerns. ChatGPT can be vulnerable to cybersecurity attacks as it is connected to the Internet and can be used to spread malicious content or viruses. Malicious cybercriminals can also manipulate people into disclosing personal information using the chatbot and then use that information for fraudulent purposes or for targeted phishing attacks. Consequently, we recommend being especially careful with plugins and browser extensions that promise to make it easier to use the chatbot.
