Architectural change

Thoughts on architecture and agile methods (in the example of SAFe)

Agile software development pactices try to embed the design of the sofware system architecture into other development activities, instead of doing it as a separate activity.

Although the software system architecture can be developed in various ways, there are some implications if this happens incrementaly and is driven by random stream of requirements.

Using ArchiMate for modeling ...

ArchiMate language does not distinguish between types and instances.

Because among the enterprise architecture models there are both type and instance models (sometime it even makes sense to put both types and instances on the same model), some technique must be used to distinguish between types and instances.

...

Recently I stumbled upon a strange „feature“ of ArchiMate language that brings some ambiguity into the models.

In the ArchiMate specification (see Open Group ArchiMate 3.1) chapter 3.6 is written „The ArchiMate language intentionally does not support a difference between types and instances.“.

This design decisions brings ambiguity into the meaning of the diagrams. For example, what is described inthe following diagram:

Does this mean that:

• „There exists a set of data objects of type A, each consisting of 'one-or-more' data objects of type B, and 'one-or-more' data objects of type C“, or
• „There exists a data object A, which consists of data object B and data object C“, or even some mix like
• „There exists a data object A, which consists of 'one-or-more' data objects of type B, and a data object C“?

There are situations, where it would be possible to deduce, whether diagram is describing the types or instances (because for example certain reflexive relationships, like „composition“, would not make sense for the instances), like:

But in some cases it would be impossible to know without some indication of what kinds ofelements are in the model or on the diagram, like:

Therefore indicate the usage of types or instances explicitly, using for example through ArchiMate specializations (similar to UML sterotypes) and the naming convention similar to UML, either for the types:

or for the instances:

Although in ArchiMate specification it is stated that „At theEnterprise Architecture abstraction level, it is more common to model types and/or exemplars rather than instances.“, there are still many cases where the model of the actual enterprise architecture consists of instances (like capability maps, process maps, or application landscapes, which all represent certain enterprise portfolios), and therefore it makes sense to clearly indicate on the diagrams, where are the types (for example when describing conceptual models or solution patterns) and where are the instances, especially if both are mixed on the diagram.

Hierarchies with the single elements and fixed levels

There is certain wish, when handling the hiertarchies, to fix the numer of levels, with the reasoning that this should make things simpler and clearer.

This leads to the hierarchy levels whichj contain only single element, which could make sense in the class/type hierarchies, but does not have any meaning for the composition hierarchies.

...

I have seen many times the wish to fix the number of levels in the decomposition hierarchies (like for example in the maps of business capabilities), with the reasoning that it would make things easier by always having known number of levels. One result of this approach is the appearance of branches with single elements on certain levels.

There are such hierarchies, where the elements with just one sub-element will make sense, and there are other hierarchies, where this doesn’t make sense.

For example in case of taxonomy, each element is representing a set of features – it is a class hierarchy.

In the class hierarchy it makes sense to have even single sub-class for a particular class, because this sub-class represents a different set of features than its super-class. It also could make sense in class hierarchy, if needed, to have a fixed number of levels – because on each level of each branch you have different set of features (which you can distribute evenly).

It makes sense to have:

either , or .

On the other hand, in case of functional decomposition,like hierarchy of business capabilities or application components, each element is a whole – it is a composition hierarchy.

In the composition hierarchy(which consists of instances, not of classes) it doesn’t make sense to have an element that has only one part, because these are then the same thing.

It makes sense to have:

, but not .

It also would not make sense in the decomposition hierarchy to have fixed set of levels – because this would either enforce the designers to come up with "dummy" levels just to follow the scheme, and results decomposition that is not natural, causing branches with single elements just to „fill“ the levels of hierarchy, introducing multiple ways to name what actually is the same thing.

This is the reason I wouldn't advise to create for example business capabilities with only one sub-capability or application components with only one sub-component.

Using models in software development ...

Compared to the beginning of software engineering, the current practice tends to diminish or completely drop the use of models (formal represenation of requirements towards the software system).

But if there's any wish to automate the software development activities beyond the simple pipelines of build tools, there is need for the strong formal models that specify what to automate.

Business Capability – a short clarification

A "business capability" describes ability to perform certain set of related activities for providing a set of business, by combining people, processes, resources and governance.

Business capabilities are connected to other business capabilities through the business services and form a value network.

API vs. ESB (and other related tools)

I would like to clarify some questions related to API management, ESB and specific service interfaces.

From one side, API (Application Programming Interface) is nothing more than a specification – a text, containing a set of more or less formal statements, which specify the available (service) operations and data that flows through the interface when these operations are performed.

Therefore managing an API is nothing morethan managing any other formal document – which can be done using any basic text editor or something more fancy, called API management tool, with lots of bells and whistles and with heavy price tag.

From another side, ESB is nothing more than a system that transports and routes service requests, and returns matching responses, usually providing several different physical mechanisms to do so, independent of how these service requests are defined or do they together at all form an API.

Therefore, if we implement API management tool and ESB, then this will not in any way result in a specific API (or service interface).

Developing a specific API is quite time-and resource-consuming task, which needs to be planned and designed as anyother large development. What makes API design even more important, is that API's guide the architecture of future developments and affect strongly their properties.

A good analogy here is with DBMS and actual database schema – although we have tools for designing database schemas(e.g. ERwin) and DBMS to run these schemas (e.g. Oracle or TeraData), we cannot assume, that database schema (e.g. for Enterprise DW), suitable for current and future needs just emerges from separate developments. It needs a special (some-time very large) effort, to develop a suitable concrete data-base schema.

API design and development requires same way as data-base schema design and development:

• conceptual models for both functionality (service operations) and data (information),
• logical models that specify the interfaces, and
• physical models that specify the interfaces for specific implementation technologies.

And the above mentioned artifacts are not produced neither by the API management tools nor by ESBs.

Thoughts on "Architect Your Business - Not Just IT"

When reading MIT CISR  Research Brief No 12 from 2014 "Architect Your Business - Not Just IT", I agree very much with the statement that “… despite the title, business architects rarely design their company’s business.”!

The main puzzle for me is, that even when everybody in the organization sees and agrees, that “…their processes, structures, and systems are not providing the agility they need …” (i.e. the business architecture of the company is not adequate), I don’t usually see any dedicated effort for designing a new business architecture, not to mention employment of a specialist with business architect skills for doing that.

Here I must agreeagain with the statement that “… the dominant design approach for large companies is ‘divide and conquer’ in which individual leaders accept responsibility for success over a specific set of closely related business activities.”. Because of the Conway's Law, this approach leads to a business and IT architectures that copy the power-structure of the organization.

The above mentioned approach could work, but only if the domains of power and integration/interaction points between those separate “kingdoms” are very clearly defined and controlled, and designing these interfaces and controls should be the main task for the actual business architect.

Complexity in/of the enterprise architecture

The negative effects of the overall complexity of business and IT in the enterprise, manifest themselves as unreliability and excessive cost of operations, and excessive cost and time to make changes.

Business complexity has additional negative effects due to the difficulties in selling more complex products and customer dissatisfaction due to unclear and time consuming business processes.

Therefore complexity of the business and IT in enterprise needs to be controlled and managed.

To be able to control and manage the complexity, we need to be able to measure it.

If looking into different treatments of the complexity of systems, we can define the complexity as

the number of different elements and their interconnectedness (number of interconnections between these elements).

Based on such definition, we propose to measure business and IT complexity by counting the elements of business (like business models, customer segments, offered products, business functions, business services, business processes, etc.) and IT (like data stores, applications, technologies, etc.), and their interconnections.

In both business andIT we can differentiate between:

• external complexity, caused by the external factors that are notunder our control or depend on large scale strategic decisions (that define in which business the enterprise is in), which cannot be reduced without the large changes in the enterprise business strategy, and

• internal complexity, caused by our tactical choices and decisions of how we organize ourselves or how we operate (that also defines the complexity for the customers), which can and should be reduced to improve the overall efficiency and agility of the enterprise.

The internal business complexity (e.g. how we organize or operate the business) defines also large part of the external IT complexity, the other part being defined by the external technological factors.

Extending EA meta-models to contain the environment ...

Current EA meta-models describe in great detail the internals of the enterprise, but leave the environment in which the enterprise operates either totally out, or describe it in considerably less detailed way.

There are definitely some meta-models, for example Nick Malik's EBMM (contains Influencer) and new ArchiMate motivation extension (contains Driver), which try to deal with the (inconveniences) of outside world, but this is not that elaborate and structured, as these parts of meta-models which deal with inside world.

If we see the role of EA function as supporting the orientation of the enterprise according to John Boyd's OODA loop, there is need to have sufficiently good models for both representing and interpreting the environment, and representing and interpreting the enterprise itself.

We should add something similar to the dynamic financial analysis (DFA) models to the EA meta-models, to be able represent the impact of environment to the enterprise, as elements representing competition, markets, regulations, etc. (see for example A. Bergbauer, V. Chavez, T. Fischer, R. Perera, A. Roehrl, S. Schmiedl, Back to the future: Dynamic Financial Analysis (DFA) for decision making, 2004 (Fig. 3), or M. Eling, T. Partnitzke Dynamic Financial Analysis: Classification, Conception, and Implementation, 2005 (Fig. 2), or M. A. Taylor, Business Environment Model, 2013).

Do/should we have "internal customers"?

If we use in the enterprise architecture framework IBM Component Business Model (CBM) and A. Osterwalder's business model canvas (BMC) for describing the business and its parts in a business domainarchitecture.

The CBM can be used to describe the overall business functionality and the functional decomposition of whole enterprise into business components, which can be viewed as small independent businesses. The business components in a CBM are connected through the business services that they produce and/or consume from the other business components, forming a value network. Some of those business services are produced and/or consumed by the external parties (including the enterprise’s customers). So from that viewpoint, for every business component, an external (to the given business component) party could be in two different role – customers/consumers of the produced services and suppliers/producers of the required services.

The A. Osterwalder’s business model canvas can be used to describethe overall business logic of how the business works for whole enterprise, and/or for each functional sub-division down to the business components, identified in the CBM. Business model canvas also identifies external parties in two different roles – partners and customers. This separation is beneficial because business usually needs to employ different relationship management techniques for the external parties playing those different roles, and usually also the channels through which the value is delivered to the business, and through which business delivers value, are different.

Above described conceptual models (together with their language) provide the modularity and encapsulation, needed to manage the inherent complexity of the business functionality that whole enterprise comprises. Employing this view in business organization/operations allows us to achieve self-optimization of the operations of whole enterprise by optimizing the operations of separate business components, and robustness by encapsulating the business components behind the well formed service agreements.

In principle we should be able to separate and replace any business component (including IT or its parts) as an independent business entity, without changing the internal workings of that particular business component and affecting the operations of overall value network.

So in the behavior of a business component, there should not be any difference, whether the external (to the given business component) party is also external to the whole enterprise or just another part of the enterprise, but the business component should definitely have different behavior (down to the clear service agreements) towards the parties to whom it delivers services and towards the parties that deliver services to it.

To avoid confusions with the usage of word “partner” in the A. Osterwalder's business model ontology it would not be good to denote such business components, which do not directly deliver services to the enterprise’s customers, with the same word “partners”, for both consumers of their services and suppliers of services they need.

There might be political reasons for which we want that in our language enterprise’scustomers should stand out from business components that are internal to the enterprise, and because the value network inside the enterprise uses different ways to account the value, the word “internal customer” might not be appropriate for consumers of internal services.

So should we then use the word “consumer” throughout the enterprise architecture models/descriptions to denote the business service consumers in the business models instead of word “customer”?

In case the same business component provides the same business service to both enterprise’s customers and other business components in the enterprise (as in many cases IT related business components do), should we treat those as two different service consumer classes, and use different words to denote these?

Reducing the complexity and increasign the modularity ...

... of enterprise systems, based on the results from (The Evolutionary Origins of Modularity), could be achieved, by imposing an additional cost (a kind of "tax") upon the direct connections between the enterprise systems.

Reserves created from such "tax", could be invested into the improvement activities of enterprise architecture.

Controlling the enterprise (IT) architecture

We can control/develop enterprise (IT) architecture by following:

• using Conway's law -- enterprise organization should be designed such that we would like the enterpise IT architecture to become -- there must always exist an interested party for developing certain architecture feature/element (e.g. developing the architecture in certain way must be in somebody's interest);
• limiting the resources in suitable manner would stimulate thinking and (as a consequence) reduction of complexity, increase of reuse, and simplification of maintenance -- that is architecture can be developed by the owner/steward of necessary resources -- in case we have a governance body for architecture like architecture committee, it must own or command resources;
• ownership of main (architectural) principles by top management (they must belong to the enterprise identity and value system) -- only then they will influence the architectural decisions;
• differentiated actions according to the governance model:
  • in the enterprises with strong central governance (power) it is possible to design the architecture according to certain goals, and the result would be foreseeable,
  • but in the enterprises with weak central governance (power) or completely decentralized enterprises it would only be possible to create favorable situation for architecture to emerge and develop (grow), and the result would not be foreseeable nor guaranteed;
• differentiated capital investments -- the part of the architecture, which function is most static (i.e. infrastructure/platform) could afford the biggest capital investments -- to stay long time without changes it must perfrom very generic functions (as an opposite to the frequently changing parts of the architecture which could perfrom very specific functions) -- the more variable/volatile is the function, the smaller should be the capital investments;
• limit the amount of "technical debt" allowed for the IT architecture -- large technical debt limits the options of change and "takes over" control of the architecture.

Measuring the Effect of Enterprise Architecture

Some things that can be measured for estimating/following the success of enterprise architecture activities:

• Overall trends in enterprise, for which EA should correspond:
  • Response time trend for business changes – EA has been successful, if the organization has become more agile.
  • Efficiency trend for business operations – EA has been successful, if the organization has become more efficient.
  • Reliability trend of business operations and development – EA has been successful, if the amount of failures (both in operation and development) has been reduced.


• Special measures for EA value:
  • Maintenance of big-picture and reuse of EA information – the ratio of questions related to strategic decisions that can be answered, based on collected and maintained EA information, to the amount of questions that require a special one-time effort (study or project).
  • Alignment of activities to business goals (the ratio of projects/solutions that are connected to the business goals, to the total number of projects/solutions).
  • The degree of standardization in business – the ratio of different "ways of working" to the overall number of business functions.
  • The degree of standardization is IT – ratio of different used IT technologies to the overall number of IT systems.
  • The degree of complexity in IT – ratio of supported business functions to the overall number of IT systems.
  • The impression of stakeholders, how good/clear is the target ("TO-BE") picture – rating according to some subjective scale.
  • The impression of stakeholders, how well complexity is handled – rating according to some subjective scale.


• Measures for strength of EA function:
  • Strength of architecture governance – ratio of projects/solutions evaluated by architecture board, to the total number of projects/solutions.
  • Conformance to the architectural principles – ratio of projects/solutions following the architectural principles to the total number of projects/solutions.
  • How well the architecture is followed – ratio of architectural exceptions to the projects/solutions evaluated by architecture board.

About the requirements analysis ...

1. Requirements Analysis and Specification
   1. Purpose – to understand what is required and to communicate that understanding to other parties.
   2. Team
      1. Lead analyst – driving the analysis and requirements process (knowledgeable in analysis and requirements specification techniques).
      2. Domain Experts – bringing the domain knowledge to the analysis and specification process.
      3. Technical Experts – bringing the technical knowledge to the specification process (assuring that the solution which satisfies the stated requirements is feasible).
   3. Process
      1. Iterative process, where analysis and specification tasks alternate until the satisfactory requirements specification is produced.
   4. Tasks
      1. Requirements Analysis
         1. Purpose – to understand what is required.
         2. Methods
            1. Study of documents – to collect formal information.
            2. Interview of involved parties – to collect informal information.
            3. Workshops – to assure, that all the relevant viewpoints are taken into account.
            4. Modeling and prototyping – to assure, that all the relevant questions are asked.
      2. Requirements Specification
         1. Purpose – to communicate what is required to other parties (developers, testers, ...).
         2. Methods
            1. Writing specification documents – to describe the requirements.
            2. Modeling and prototyping – to assure, that requirements are
               1. unambiguous – all parties understand the same thing,
               2. complete – nothing that is necessary has been left out,
               3. correct – there are no errors,
               4. consistent – there are no contradictions,
               5. verifiable – result can be tested against the requirements,
               6. feasible – result can be achieved,
               7. prioritized – to allow scoping and risk management.
            3. Workshops – to assure, that specifications are correct.
   5. Results
      1. Requirements Specification Documents.
      2. Models and prototypes.


2. Important parts of the business software system (should be defined in the results of requirements specification)
   1. Overview – what is the overall purpose and the context (environment) of the system and what are the non-functional requirements (performance, response time, etc.).
   2. Data – what information the system maintains (business entities and relationships, their life-cycle and corresponding business rules).
   3. Roles – who are the users and what rights they have to perform functions that system provides.
   4. Functions – what functions system provides to the users (business processes, business transactions and corresponding business rules).
   5. User Interface – what data is presented to and can be entered by the user and what functions are accessible to the user, how the system responds to the user actions.
   6. Printouts and reports – what data is collected from the system and how it is presented to the user.
   7. External Interfaces – what are the connection points of the system with its environment.

Because every requirement has a price attached (in terms of time and money), they should be handled with care and precision (as money is handled).

About freedom ...

IT people use Android, but business people use Apple iOS.

Both want freedom, but these are two different freedoms:

• IT wants freedom of choice (or freedom to build), but
• business wants freedom from choice (or freedom to operate).

The same case is with the agility:

• agility for IT is freedom from thinking about business
• agility for business is freedom from thinking about IT

What system aspect is above others – who rules in governance ...

I don’t want to put all the modeling disciplines into the information modeling (as sub-domains) just because most of these are somehow related to information modeling, or just because we can view any modeling activity itself as information processing activity.

For me information modeling is part of the larger domain of modeling the structural aspects of the domains or systems, and answers in this context to the following questions:

1. What information we capture (represent) about particular domain – here information model should have the connection to the domain model, which itself answers to the question what the domain is and gives the semantics (meaning) to the information model elements.
2. What information we capture (represent) for a particular purpose – here information model should have the connection to the behavioral models (use-case, process, state-transition, etc. models) of either the domain or the system under study, to show for what and how the information model elements are used.
3. What are the formal rules for assessing the quality of information (for given purpose) – here information model should have the connection to the domain model constraints for static or structural rules, and connection to the behavioral models for dynamic and context sensitive rules.

It is important to have in mind that because different modeling disciplines cover different aspects of the system under study, we get the full holistic picture only by applying several modeling disciplines, not by putting one discipline above others – real world systems have both structure and behavior at the same time.

That is why I would like to see that neither information owner nor process owner are ruling over each other in information or process governance – the governance should be set over full picture that describes all aspects of given domain or system (EA in case the system is the enterprise).

To achieve the consistency, stability and encapsulation, there is no need to try to establish global governance over all levels, just higher level composition owner should govern the interfaces (incl. both structure and behavior) between the next level components – doing so we are able to partition the models to fight the complexity, and allow freedom of development/governance inside the components.

Business components solve problems of process-orientation

Most of the problems in process-orientation come from the fact, that processes are viewed as global artifacts independent and above of the functions (business components). Hence for example the problem of cross-functional processes and lack of authority of process owners in the organization.

If we would build the whole enterprise using component-orientation -- that is to encapsulate the processes inside the business components, and allow only interactions of business components based on well-defined interfaces (SLAs), then there will be no such problems, as every process owner is in full control of its processes and resources needed to perform these.

Additionally enterprise would be way more agile (and self-organizing), as responses to the possible changes are localized into the components, any components can be replaced with the best in industry (outsourced) without any adverse effects to business as whole, and the global flows of activities will adjust themselves to the optimal in given context.

If you look from the customer's perspective, then processes are encapsulated inside the component with which you interact.

For example if customer requests for loan offer from sales component, and receives the response from sales component, then it doesn't matter to him/her what other business components are involved and how.

In this case sales component management is completely in charge of the process which is used to produce the response (or service), as it is not crossing borders of component, and component management has SLAs for all the services that it needs/uses from other components -- so there is no need for matrix-organization (with the dual and often conflicting command lines) at all.

To the sales component doesn't actually matter how (using what process) the underwriting is done by the risk management component as long as the SLA for underwriting is not breached.

Software "stack" -- business logic and reusable assets

(A business function is a function that gives business benefit to the user, it is often specified/described by a business use-case.)

How to start describing business (architecture)

1. Use IBM's CBM method and generic industry CBM to identify the business components (business functions) -- identify three cases for components:
   1. that match fully (exist in generic industry CBM),
   2. that match partially, and
   3. are missing from generic industry CBM.
2. Map the resulting business components to the existing organizational structure (verify this mapping with the responsibility descriptions of organizational units)
3. Define the external boundary (interface) of business system(s) – identify:
   1. the service types delivered/produced and
   2. the service types required/consumed to both external and internal customers.
4. Map the resulting service types to the business components.
5. Consolidate/split business components (business functions) not depending on the existing organizational structure, but depending on the coupling between components (functions).
6. Define the used/planned business model(s) by defining (according to Osterwalder's method):
   1. customer segments (types)
   2. channels
   3. value offerings (products) -- packages of the predefined services deliverable to the customer segments
7. Delivered service types on the external boundary (interface) identify the primary business processes, needed to produce services.
8. If primary business processes are identified, then describe these on very high level -- not yet even the diagram of activities, but inputs, outputs, performers, material resources used/needed, purpose, dependencies, etc. (for example using SIPOC).
9. Supporting business processes could be then identified from:
   1. the (human and material) resources needed for primary processes -- resources need to be managed, and
   2. the primary and supporting business processes themselves -- processes also need to be managed (planning and controlling).

Specifications and actuals (and UML powertypes)

What I have in mind, is the duality of concepts in the domain, where:

• some concepts refer to the actual things/objects (like product, property, …), and
• other concepts refer to different things/objects, which are specifications (or descriptions) for the previous ones (like product specification, property description, …).

Both sets of concepts being on the same modeling level, and always have at least one-to-many relationships between each other. The difference of specification from the actual is, that specification can, and in many cases do exist without corresponding actuals. Also specifications have usually own behavior, which is different from behavior of actual.

Participants of this (pattern?) could be represented in models using corresponding stereotypes, or just grouping these together on diagrams.

This pattern has helped me to identify concepts/relationships, and to organize the model.

Maybe this would be possible to model using UML powertypes, but I would avoid mixing different modeling levels (model, meta-model, ...) in the same model, and would prefer to use stereotypes instead of these (to show that certain classes are from given class denoted by stereotype).

If we want to model the business rules for meta-model elements, then we should have a separate meta-model, that shows the specialized elements and respective rules/constraints attached to these.

This distinction in modeling levels is good to be maintained also into the solution, down to the physical models -- that is if we choose to have dynamically configurable system, then from the meta-models we derive the physical models representing the configuration information, and from the models we derive the actual content of configuration information for the system. Failure to make this distinction clearly is the source of problems in maintenance of dynamically configurable systems.

What Fowler describes as specification pattern seems to me too narrowly applied only for selecting objects from object sets, but maybe I am mistaken.

On EA function and the "customer" of EA

Gartner Pub. "Develop Next-State Enterprise Architecture to Improve Usability" states: "Too many EA programs are suffering because their content is being largely ignored by some of the very people they are trying to serve -- project teams."

Is it really so that EA program is trying to or must serve project teams?

I would argue, that the main purpose of the EA programs is not to serve, but to constraint the project teams -- doing so of course in the name of serving the whole enterprise including the project teams, but still the main purpose is to impose the constraints that restrict the project teams.

It is the same, as police is establishing speed limits for motorists -- for benefit of whole society some of the members need to be restricted in their doings.

This of course doesn't mean, that restrictions/constraints developed by EA program, doesn't have to be "actionable" and "pragmatic".

The detail level and overall nature of restrictions/constraints has to match the maturity of project teams -- by analogy, for small children the restrictions must be very precise and categorical, but when dealing with grown-ups you can rely on some level of common sense (or maybe not).<7p>

...

The overall question is about the primary stakeholder/counter-party -- who EA function serves.

Does the EA function serve management in development of enterprise, working for global goals, or does it serve the project teams, working for local goals? I would focus more on serving the management and global goals, because project teams are already served by the application architects.

There was an important remainder in the article -- you can't have road map (next steps), before you have future state!