ML software (SW) architecture is the technical (both structural as well as dynamic) design and implementation of all components that either support or are directly part of the ML lifecycle. It addresses via a 1-to-1 mapping the identified phases, activities, and concrete tasks of the ML lifecycle and suggests SW components to implement them. Just as much as the lifecycle itself does, an ML software architecture typically addresses:
Software Architecture (SA), as a discipline of Software Engineering (SE), is generally concerned with describing the fundamental buildings blocks and components of a larger software system, as well as their interaction within the system. Architecture in this context is a metaphor, acknowledging the similarities between large buildings and the elements of which they are comprised, on the one hand, and complex software systems and the interrelationships of their components, on the other. Naturally, when reasoning about SAs we are often looking for finding the right abstractions and hierarchies to create reusable elements.
“The primary purpose of architecture is to support the lifecycle of the system. Good architecture makes the system easy to understand, easy to develop, easy to maintain, and easy to deploy. The ultimate goal is to minimize the lifetime cost of the system and to maximize programmer productivity. The goal of an architect is to minimize the human resources required to build and maintain the required system” - Robert C. Martin
This understanding of an SA maps to the structure of this guide. We will move from the definition of the lifecycle onwards to a discussion of SAs that play a particularly important role in ML. In this context, there is often an emergence of patterns: collections of best practices that may be used to solve specific problems in defined contexts. When searching for such patterns or best practices, one cannot avoid the first law of software architecture stated by Mark Richards in Fundamentals of Software Architecture: “Everything in software architecture is a trade-off. If an architect thinks they have discovered something that isn’t a trade-off, more than likely they just haven’t identified the trade-off yet.”
Due to the role an architect plays in balancing the technical needs of different stakeholders, it is paramount for the architect to think through carefully about when to apply which patterns. As the second law of software architecture by Mark Richards states that the "why is more important than the how." We expect in SA for ML just as much as in classical SA that the decisions we make as architects will be challenged by other developers, other architects, and other business stakeholders. Therefore, as architects, we search for the least bad design and evaluate the trade-offs.
SAs often try to balance particular needs in the lifecycle of the development. These needs, or "architecture characteristics" (see Fundamentals of Software Architecture), can be associated directly with technical qualities, such as performance of fault tolerance, or with the process- and lifecycle- or development-related qualities, such as agility and testability (see the image on the right side as an example of such characteristics). Characteristics of this sort are desirable in an ML context as much as they are in SW 1.0. Consider agility and testability, for example:
‣ from the Fundamentals of Software Architecture DIT344 by Dr. Sam Jobara Chalmers | University of Gothenburg
However, some of these characteristics need to be revisited in the context of ML, both on a micro- as well as a macro-level. Take testability and security, for example. In normal SW, we can find processes and activities for which, through testing, the right level of security can be nearly guaranteed, and many SAs reflect that fact. In ML, however, we do not have that luxury. As we laid out in The ML lifecycle, there are architecture characteristics from a lifecycle perspective that we called "themes" and which relate in ML to as-yet unconquered challenges (e.g., trustworthy AI, responsible use of AI, etc.). See the next chapter for a more detailed discussion of this point.
Over the past decades, sets of architectures have emerged that may be called "styles." An architecture style is a family of architectures sharing certain characteristics that can be used to solve common problems in the SE lifecycle. To some degree, styles reflect patterns, though they are not as concrete as design patterns normally are. See on the right an exemplary mapping of dominant architectures to their architectural characteristics.
‣