How do I properly test an AI-generated product?

Start from what your product must do, not from the code. Proper testing has five parts: (1) write down the things your product must never stop doing, (2) save a test for each one, (3) make every test fail once, so you know it can, (4) run the tests on every change before your users see it, and (5) add a test each time something breaks. Every part can be done without opening the code.

Why clicking through stops being enough

Most founders test by opening the app and trying it. Early on, that works. The product is small, and you can try everything in a few minutes.

Then the product grows. Each change can affect a page you didn’t ask about. Checking everything by hand takes longer every month, so you check less. A break in a part you skipped is found by a customer first.

Asking the AI to write tests helps, but generated tests have a problem of their own. A developer who tracked vibe-coded projects for a year after launch wrote that the tests in them were also generated and might not be testing the right things (opens in a new tab).

A test protects you when it checks something you care about and when it can fail. You can confirm both yourself.

What a test is

A test is a short piece of code that does one thing in your product and checks what happens. For example, it signs up as a new user and confirms that the welcome page appears.

A person clicking through does that job once. A saved test repeats it in seconds, each time the product changes.

Some tests check one rule, such as “a 10% discount on a $50 order comes to $45.” Others go through the product the way a customer would, from signup to payment. A product usually needs both kinds.

Five steps to tests you can trust

1. Write down what your product must never stop doing

Start with sentences in plain language. Each one names who does what and what they should see.

• “A new customer signs up with an email address and reaches their dashboard.”
• “A customer with a valid card pays and receives a receipt by email.”
• “A customer whose card is declined sees a clear message and is not charged.”

Include the cases where something goes wrong, such as a declined card, an expired link, or a form left half empty. A list without them may lead to tests that cover only the case where everything goes right.

Five sentences are enough to start.

2. Ask for one saved test per sentence

Give the AI your list and ask for a test for each sentence. Ask it to save the tests in your repository. A repository is where your code is kept, along with every earlier version of it.

Then ask for a list showing which test covers which sentence. If a sentence has no test, you know what is missing. If a test matches no sentence, ask what it is for.

3. Make every test fail once

A pass is only useful if the test would fail when the feature is broken. Some tests can’t fail. They confirm that a page loads without checking that the price on it is right.

To find out, choose one sentence, such as the discount rule. Ask the AI to disable the discount as a trial, run the test, and show you the result. The test should fail. Ask for the trial on a copy of your code that isn’t live, with test accounts rather than real payment or email details, so no customer is affected.

Then ask it to undo the trial and run the test again. It should pass. If the test passed while the discount was off, it wasn’t checking the discount, so ask for a better one.

4. Run the tests on every change, before your users see it

Tests that run only when you remember them can miss the change that matters.

Ask for the tests to run automatically each time a change is proposed. If your code is on GitHub, a change can arrive as a pull request, which holds the change until you approve it. GitHub can run your tests on each pull request and show a pass or a fail beside it.

GitHub shows the fail, but it still lets the change go in unless a rule requires the tests to pass first. Ask for that rule too, so a change with a failing test waits until it is fixed. If your code isn’t in a repository yet, moving to a normal GitHub workflow explains how to get it there.

5. Add a test each time something breaks

A first list will have gaps. When a customer finds a bug, write the sentence that would have caught it.

Ask for a test for that sentence before the fix is made, and watch it fail. After the fix, it should pass. The test then guards against that bug returning. Why the AI keeps fixing one thing and breaking another explains what brings a bug back.

Three signs your tests aren’t protecting you

A test was changed so that it would pass. When a change breaks a test, the AI can edit the test and leave the product broken. Ask it to tell you whenever it changes or removes a test, and why. A test should change when you’ve changed your mind about what the product should do.

You get a summary without the results. “All tests passed” doesn’t say which tests ran. Ask for the list of tests that ran and the result of each one.

There are many tests, and your list still has gaps. Two hundred tests can still miss your payment page. Count how many of your sentences have a test.

What tests won’t tell you

Tests check what you wrote down. They don’t tell you whether a new page looks right or whether a feature is worth building. Try each new feature yourself once, then let a test repeat the check.

Tests also don’t replace a security check. For those, see whether your app is safe to keep building on.

How Forged handles this

Forged decides how a change will be checked before it writes any code. It then shows you the result of each check.

Dimension	The Forged approach
What gets checked	Acceptance criteria you approve before the build starts
The method for each one	By a test where one is possible, and by a read of the code where it isn’t
When a check fails	The work goes back to be fixed before you are asked to review it
What you see	How each criterion was checked, and whether it passed
Where you review it	A pull request in your repository, beside the checks GitHub runs
How your existing tests are judged	A Product Health check scores testing as one of six areas

“Done” is written down first. Forged breaks a feature into small action items. Each one has acceptance criteria, which say what must be true for the work to count as finished. They do the job of the sentences in step 1, and you approve them before the build starts.

Each criterion gets its own check. Forged verifies every criterion, by a test where it can and by reading the code where it can’t. A criterion that fails goes back to be fixed. The pull request then says how each one was checked.

You see every result. The feature’s page in Forged counts the criteria that passed a test, passed on a read of the code, failed, or weren’t checked. The checks GitHub runs on the pull request appear with them.

Your questions, answered

How many tests do I need?

Fewer than you might expect.

One test for each sentence on your list is the starting point. Five is a reasonable number. Add one each time a bug reaches a customer.

Can I trust tests the AI wrote?

After you’ve seen them fail.

A generated test can pass without checking anything useful. Break the feature as a trial, as in step 3. If the test fails, it is checking something real. For the rules that matter most, such as payments, try more than one kind of break.

Can I do this without a developer?

Yes.

Your part is the list of sentences and three questions: which test covers which sentence, which tests ran, and what each result was. A developer can review the tests in a few hours if you’d like a second opinion.

Do I still need to click through my app?

Yes, for anything new.

A person is still a better judge of whether a new feature looks and feels right. After that first look, a saved test repeats the check.

Watch a change get checked before you approve it

Give Forged your repository and one change you need made.

You agree what “done” means first. The pull request then shows how each acceptance criterion was checked.