Build or buy an AI tool: decide from the task, data and acceptance test
Neaptide · October 15, 2026 · 6 min read
Compare an AI product and custom development using your sources, permissions and acceptance tasks. A guide to a measurable pilot.
On this page
Start by testing whether an existing product can complete a well-defined business task with your data and acceptance criteria. Commission additional development when a material requirement remains unmet: access to authoritative information, permissions, a necessary system action or a workflow the product cannot support adequately.
Buying a tool does not remove implementation work. Building an application does not require training a foundation model. A practical solution may combine a purchased model or product with a small integration that the business owns.
A real source audit before choosing a tool
On 9 October 2026, we extracted 24 published English FAQ entries from Neaptide’s web, SEO, GEO and AI service pages. We inspected their source URLs and stored the questions and answers with the collection date. This is an inventory of existing company knowledge, not a sample of customer conversations or a test of a model’s accuracy.
The AI service page provides two useful boundaries. It explains the difference between retrieval-augmented generation and fine-tuning, and it describes an initial assessment as taking one to two weeks. Both statements have a public source. Neither confirms a start date or an approved scope for a particular prospective client.
An assistant could use the page to explain the published process. Confirming a booking would require different information and authority. That distinction should drive procurement before model preferences do.
Method and observation records.
Describe one accepted result
Write the task as an input, a required output and a rule for acceptance. Identify who decides whether the result is usable. Include the source the result must agree with and any business action it would trigger.
Use genuine work already handled by the team, with appropriate permissions and removal of unnecessary personal information. Preserve difficult and incomplete requests as well as straightforward ones. Do not let a supplier select only the cases that make its product look capable.
Keep part of the material outside initial configuration. Otherwise a demonstration can show familiarity with the setup examples without establishing performance on fresh work. The size of the evaluation set should reflect the range and consequences of the task; no universal small sample proves production readiness.
Separate answering from acting
| Requirement | Evidence to ask for |
|---|---|
| Explain published information | An answer linked to the correct current source |
| Read private information | Access restricted to the authorised user and task |
| Draft a proposed action | A reviewable draft with its supporting record |
| Execute an action | Explicit permission, a recorded result and a recovery procedure |
| Operate over time | An owner for updates, evaluation and exceptions |
A product that answers a policy question has not thereby demonstrated that it can change an account correctly. Test each required action separately. Decide which actions need human approval and what the system must do when the relevant information is missing.
Anthropic distinguishes predefined workflows from agents that choose their own steps and tools. Its guidance favours starting with a simple approach and adding complexity when it improves the task. This is useful architectural guidance, not evidence that a particular product will meet your acceptance criteria. Building effective agents.
Compare the complete operating arrangements
For a purchased product, verify the exact plan, available connectors, data export, permission model and support route. Run the acceptance work in the configuration you would actually buy. A feature listed on a marketing page may still require setup or a different subscription.
For custom work, require a clear boundary between purchased components and project-specific code. The proposal should identify who maintains integrations, responds to changed APIs and reruns evaluations after a model or source update.
For either option, record licence or usage charges, integration work, staff review, monitoring and the cost of switching. A subscription price and a development estimate cover different things until their scope is aligned.
Make the pilot answer the purchase question
Before starting, agree on the cases, acceptance decisions and required records. Count accepted results, rejected results, manual corrections and unresolved requests. Record staff time as it occurs. Keep test actions away from live irreversible operations unless explicitly authorised and controlled.
Choose the option that satisfies the material requirements at an operating cost the business can maintain. If neither passes, narrow the task or improve the source material before expanding the system. A bounded assistant that reliably supports a real process can be a better first investment than a broad mandate with no measurable completion rule.
Neaptide’s AI implementation work can start with the source inventory and acceptance task. These establish what needs to be bought, connected or built.