Generative AI Security and Privacy
Guidance from the New School Information Security and Privacy Office
Using generative AI may provide us with an opportunity to generate fresh ideas, organize our thoughts and relieve us of the burden of some tedious work, freeing us to focus on creativity and more deep thinking. However, artificial intelligence is not always that “intelligent” and in some cases, can be simply malicious.
Be Aware of Privacy and Cyber Risks When Using Generative AI
The emergence of generative AI introduces heightened cybersecurity risks as the attacks can be more complex and personalized. If you are trusting AI to write your meeting notes or write the summary of an article you read, just think how easy it will be for AI to write a phishing email that includes details about where you live, where you work and what courses you are teaching. Grammar errors and misspelled words will not be present in phishing emails anymore, so you have to think before you click.
Presume anything you submit could end up on the front page of a newspaper.
The information you enter into ChatGPT and other Generative AI systems could be viewed by anyone else who uses that same service. If the Generative AI thinks your data makes its answer better, it will be included in its reply.
AI Large Language Models may use any information fed into it. AI LLM’s require vast amounts of data to train and improve their language processing capabilities, which may include sensitive information such as personal information, financial data, and confidential business information.
The data you provide could be enhanced and build an even more detailed profile of you and your life. Generative AI systems use bots to scrape data from the internet constantly and they pick up data from social media websites, online newspapers and any other website that is publicly available. It is also possible that data that was released in a data breach where your data was compromised could be scooped up and combined with everything else.
Cyber threat actors may use generative AI in their attacks in the following ways:
- Writing AI-powered, personalized phishing emails:
- With the help of generative AI, phishing emails no longer have the tell-tale signs of a scam—such as poor spelling, bad grammar, and lack of context. Plus, with an AI like ChatGPT, threat actors can launch phishing attacks at unprecedented speed and scale.
- A hacker could use a Chat GPT-powered chatbot to trick someone into divulging sensitive information, such as login credentials or financial data
- Generating deep fake data: Since it can create convincing imitations of human activities—like writing, speech, and images—generative AI can be used in fraudulent activities such as identity theft, financial fraud, and disinformation.
- Cracking CAPTCHAs and password guessing: Used by sites and networks to combat bots seeking unauthorized access, CAPTCHA can now be bypassed by hackers. By utilizing AI, they can also carry out password guessing and brute-force attacks with much more success.
Be sure to read through the terms and conditions of whatever platform you join to see what data they collect, from your data submissions or questions and the data that they collect from your devices. They might be pulling your contacts from your phone, your email address, physical address and lots of other data in the background after you agreed to this by clicking “I agree” when you purchased the app or created an account with the company.
Only provide information to Generative AI tools that is appropriate for public disclosure. This includes any text, photos, videos, or voice recordings you share with the AI. Be aware that the AI output may include unexpected personal information from another user, so be sure to remove it before publishing.
Your use of generative AI needs to be indicated in any products that you create for work or school. Check with your supervisor, department chair/director or dean to ensure you are citing it correctly and in accordance with university policy.
If you would like to learn more about how to protect your and others’ privacy when using generative AI or other data processing apps, please contact ispo@newschool.edu.