Generally, AI is non-deterministic software, unlike most software, which typically produces deterministic outcomes, meaning the result is predictable: if this happens, then that follows. This predictability allows for unit tests and clear functional requirements. If the software behaves unexpectedly, we file a bug and revise the code until it behaves as expected. In contrast, Generative AI (GenAI), which is based on large language models (LLMs), is predictive and does not always produce the same output each time. An LLM is simply a type of machine learning model. GenAI, through tools like DALL-E, MidJourney, and others, can generate content such as videos and audio.

AI has numerous potential applications, but its effective use is always context-dependent. Architects must distinguish between the hype surrounding AI and practical software solutions, designing systems around tangible components like LLMs instead of vague AI concepts. The decision of when, how, and where to incorporate AI should be based on traditional trade-off analysis. First, assess whether AI is the right solution for your application. Like any technology, AI can be creatively applied but also misused. Next, determine the best approach for implementation, weighing options like using AI-as-a-service APIs versus self-hosting. Architects can also enhance their decision-making and communication with AI, leading to more effective designs and improved stakeholder understanding.

Explained in more detail by Thomas Betts: https://www.infoq.com/articles/architectural-intelligence/