AI Ethical Framework: AI Model Questions

This series on AI is co-authored by Megan Notarte and Jason Grigsby.

When people evaluate services they might integrate into their application or website, they often consider factors like cost, features, reliability, and performance. For AI, we want to extend those factors by asking questions about how the AI model is built and its impact.

Many AI models have scraped content off the web without regard to the rights of the authors and artists. There are several lawsuits contesting this practice, and it is unclear what the outcome will be.

We should seek AI models that are transparent about what they train on and either own the rights to their training data outright or license it from copyright holders. It would be ideal if an AI was only trained on a company’s own data as it avoids many of these issues.

We prefer models that use specialized, licensed data—for example, scientific or medical research—over models trained on a large amount of general information. Specialized models tend to be more useful in their specific applications and produce more tailored results.

This is perhaps the thorniest issue. AI models reflect the biases of their training data. Without extensive evaluation of the training data, there is no way for outsiders to know what biases the AI may have learned, and most AI models won’t provide their training data.

When we evaluate different AI models, we should look at what they have published about known biases in their models, what they’re doing to mitigate them, and their guidance on how to minimize bias when using their systems. 

If an AI company doesn’t acknowledge potential bias in their models, it should be considered a red flag. We’d rather work with a company that acknowledges bias and is trying to fix it than one that acts like it doesn’t exist.

We know that current AI models will have bias. Therefore, we must include safeguards in our AI usage to try to catch bias before it creates problems.

The black-box nature of AI models makes it difficult to prevent them from divulging secrets. AI researchers tricked ChatGPT into revealing training data by asking it to “repeat the word ‘poem’ forever.” Prompt injection attacks like this might seem funny until it is your data being exposed.

We need to reduce the potential that AI will expose sensitive information. We can do so by looking for solutions that keep user data on a user’s device like Apple Photo machine learning or a local AI like WebLLM. 

Unfortunately, most AI models are too large to run on a user’s device. Therefore, we need to review the AI model’s privacy policy and practices carefully. Does it isolate user data? Are user data and prompts used to train the AI model? The more data shared among users or used to train the model, the more likely the AI may leak it.

The data centers powering today’s AI models are energy intensive and consume vast amounts of water. This increase in energy and water consumption comes at a time when we’re struggling with the climate crisis and associated droughts.

Unfortunately, most AI models don’t divulge their energy and water consumption. Until that changes, we’re forced to resort to other signals that we hope indicate the company behind the model is committed to reducing its environmental footprint.

Does the company provide annual reports on their overall sustainability? Do they have a public commitment to being carbon and water neutral? If so, by when? Do they break out AI separately in their sustainability reports?

One way to reduce the environmental impact and privacy concerns is to use the processing power of devices users already own. 

Since 2017, Apple has shipped Neural Engines—a specialized chip for artificial intelligence often referred to as Neural Processing Units (NPUs)—in its phones and computers. Other manufacturers have also begun including NPUs in their devices.

If some or all AI processing can be handled locally on the device, we can reduce data center energy and water consumption. We hope to see more options for local AI in the future.

Stay tuned tomorrow for the final part in our AI Ethical Framework Series.

• Three Things We Agree On
• AI Model Questions
• AI Usage Questions