How to Build AI Systems That Work

If you want something to 'work' - you first have to define what 'working' means. In the context of AI systems for small and medium sized businessses ("SMB"), 'working' usually means delivering value to users by saving time or increasing output.

Start with the User

The first step in building an AI system that works is to deeply understand the user and their needs. This involves:

Asking questions about the user's pain points, goals, and workflows.
Observing how users currently perform tasks and identifying inefficiencies.
Deeply understanding the user's current data, tools, technologies, and constraints.
Defining clear objectives for the AI system based on user needs.

Design the AI System

Once you have a clear understanding of the user, the context they workin in, and their goals, you can start designing the AI system. This involves:

Designing the user interface and experience to ensure ease of use and accessibility.
Defining the system architecture: trigger/perception, reasoning, memory, and execution modules.
Defining the data requirements: what data is needed, how it will be collected, stored, and processed at each step.
Planning the integration and deployment strategy for the AI system.

When planning the system, it is helpful to have a mental model for the big picture:

Polling Conducted by Kat Abughazaleh's campaign indicates she is trailing Biss. — *Figure adapted from Fundamentals of Building Autonomous LLM Agents (Castrillo et. al. 2024).

Build for Today, Plan for Tomorrow

With the rapid advancements in AI technology, it's important to build systems that can deliver value fast and can evolve over time. This involves choosing tools that are flexible and scalable and leveraging modular architectures that allow for easy updates and improvements. Consider starting with simpler AI techniques and gradually incorporating more advanced capabilities as the system matures and user needs evolve. What is the simplest thing that could possibly work? Start there, build trust with users, then iterate.

Sync on Some AI Terminology

This space is full of buzzwords and jargon. People use the same words differently and different words for the same thing. Here's what I mean by the following:

Automation System - A software system that perform repetitive tasks without human intervention, often to improve efficiency and reduce errors. This may use AI, but it doesn't need to.
AI System - A software system that uses artificial intelligence techniques to perform tasks.
AI Agent - this is a type of AI system that can autonomously perform tasks, make decisions, and interact with users or other systems.
Perception Module - a component of an AI system that processes and interprets sensory or other input data (like images, sounds, or text) to understand the environment. The perception system is often the 'trigger' of an AI work flow.
Reasoning Module - a component of an AI system that uses logic and knowledge to make decisions, solve problems, or draw conclusions based on the input data.
Memory Module - a component of an AI system that stores and retrieves information to support learning, decision-making, and task execution.
Execution Module - a component of an AI system that carries out actions or tasks based on decisions made by the reasoning module.
RAG - Retrieval Augmented Generation, the process where Reasoning and Memory modules work together to create more relevant outputs:
- "Retrieval" - This means that before responding with it's 'base' training, the LLM is searching for and accessing the information that is most-relevent to the problem. This can include referencing a store of private documents, like an Employee Handbook for an internal benefits chatbot, for example.
- "Augmented Generation" - This just means that the LLM is using the retrieved information to 'augment' or improve it's response. Instead of just generating a response based on it's training data, it is using the specific, relevant information it just retrieved to create a more accurate and useful output.

Conclusion

Building AI systems that work requires a user-centric approach, careful design, and a focus on delivering value quickly while planning for future growth. By starting with the user, designing thoughtfully, and building flexibly, you can create AI systems that effectively meet the needs of small and medium sized businesses. If you'd like to discuss an AI project of yours, please send me an email - brian@turbotabs.com.