First, a caveat — I do use generative AI quite a bit in my work. As a software engineer, I use AI-augmented tools such as Claude Code to help me develop software at a rate that is faster and often more comprehensive than were I to program the solutions by hand. I also use generative AI as an editor to pass over my writing, primarily to identify errors in grammar, spelling, and flow. In some cases, I use it as a brainstorming space, where I can bounce off ideas before committing to any one of them.

Over the past two years, I’ve spent a substantial portion of my time learning about generative AI. I’ve used AI myself. I’ve seen how other organizations used AI. I’ve learned the fundamentals of Large Language Models, which are the primary foundation for most of what people call “AI” today. I’ve studied how it’s permeated through the market — what companies are using it and why, who is selling it and how, what are experts saying about it?

Through all of that, I’ve come up with principles that organizations can use to determine whether generative AI is right for them.

How many errors can you tolerate?

Generative AI makes mistakes. More specifically, it has no way of evaluating if what it’s saying is true or not.

Humans, of course, can validate the output. But the point of generative AI is to save your organization time and resources, and you may not be able to do that if you have to go over every AI output with a fine-tooth comb.

A good rule of thumb I use is the “9 out of 10” rule. If generative AI gets it right nine times out of ten, and makes a mistake the tenth time, how much of a problem is that?

Consider a common use case for generative AI today, which is checking grammar in writing. Human beings make grammatical mistakes in writing all the time, so an AI missing a comma isn’t that different, and is often picked up anyway by a human rereading it. On the other hand, if you summarize 100 intake forms per day, and AI reliably gets 10 of them wrong each day, you either need to read all 100 forms (which defeats the point), or accept that some summaries will contain inaccuracies.

If verifying the output costs more than producing the output yourself, AI isn’t saving you anything.

What happens when the AI is wrong?

Related to the question of how tolerant your organization is of errors — what happens if an error gets through?

If you use generative AI to summarize meeting notes, and it fails to mention one element of the meeting, that’s not dissimilar from common human error, and easily recoverable. On the other hand, if you use generative AI to give medical advice, and someone gets hurt as a result, that creates major liability for your organization.

This matters especially if Generative AI is involved in processes which directly affect customers or would be scrutinized by funders. If, for example, you’re a nonprofit evaluating the transparency of various government institutions, and those institutions come knocking asking why their institution was rated wrong, you can’t just say “AI told me” without risking your organization’s credibility.

If you want to use LLMs to summarize the medical and criminal history data of individuals (something I have personally seen attempted), I would suggest extreme caution, and likely not using an LLM at all. The legal and ethical risks in such a domain are too high to deploy in cavalier fashion.

Nor can AI be trusted to audit its own responses and provide a reliably correct description of why it produced a certain output. AI is probabilistic in nature, and its outputs are determined by the specifics of its model and the data it was trained on, in a way that even the developers of the AI system cannot fully predict. You can ask AI “Why did you suggest a user use glue to make cheese stick to pizza”, but it won’t actually be able to tell you why.

This also matters in terms of determining accountability. If AI makes an error, who is responsible? The person who approved the AI’s output? The person who implemented the system in your organization? The original AI vendor?

In the use cases I leverage AI in, errors are acceptable and easily caught. If a bug occurs in code, there are opportunities to detect and correct that many times over, so an error is manageable. On the other hand, I never use AI to draft emails or other correspondence with clients. The risk of errors in that case — of the AI saying something to the client that I do not agree with, or me not remembering what the AI said, or the client being turned off by an inconsistent or awkward communication style — are so substantial that I do not use AI at all.

A good rule of thumb is this: If a single error could trigger legal exposure, lose a funder, or harm someone you serve, the savings aren’t worth it, no matter how often the AI gets it right.

Are you comfortable with AI possibly getting much more expensive?

We are relatively early on in using AI at its current scale and breadth. AI’s position in the economy is far from settled.

Not just because the technology is still being developed, but because its price is likely heavily subsidized by investor money and competitive pressures.

Currently, multiple companies — Anthropic, OpenAI, Meta, Google, Microsoft, and so on — are in a race to develop the most advanced models and get as much market share as possible. They are encouraged in this domain by a heavy amount of investor money. Those investments help them develop more advanced “frontier” models, but also offer use of the models to consumers at a discount.

This is not necessarily a stable state. If investor funds run dry, or one of the companies achieve’s market dominance, the cost to consumers is subject to change.

Uber used to be in a similar situation — folks may remember how cheap Uber used to be, which was because it was backed by a lot of investor money that allowed them to lower their prices in order to get market dominance. Over time, they raised their prices to something more sustainable and profitable for the business after they achieved that dominance.

We don’t know what the “sustainable” price is for AI. And that’s without considering the environmental and power costs. Even if AI works well for your use case right now, it’s worth considering your time horizon, and how comfortable you are with the possibility that prices may increase, and possibly quite a bit.

When I use AI myself, this is something I carry in the back of my head. If the cost of using Claude Code increases, I need to know if I have the budget to keep using it at my current levels, or if I would need to make adjustments in how I use Claude Code. Those cost increases may never come to pass. But it’s better to have a contingency plan in case it does, than be caught by surprise and having to adapt on the fly.