Book Review: Clean Architecture, by Robert C Martin

I thought this book did a good job of presenting a number of ideas about software architecture. The three standout discussions in the book for me were the following: (1) The proposal that each of the three major programming paradigms (i.e. structured programming, object-oriented programming and functional programming) removes capabilities from the programmer and imposes discipline rather than adding new capabilities. (2) The definition of high-level and low-level components based on how far or near the component is to the system's inputs and outputs. (3) The importance of writing high-level components (i.e. business rules) so that they can be tested independent of low-level concerns like the database and the user interface.

Yes, this book did have an emphasis on dependency inversion and yes, taken to its extreme, I can see how someone might go on a rampage and create an interface for each and every class in the system. However, I think that is more a fault of the reader than the book itself. Perhaps, in fairness, the book could have called out the dangers and drawbacks of taking dependency inversion to this kind of extreme. All in all, I thought this book could have done with a bit more editing and condensing but it was perfectly readable and there is plenty in it to think about and take away.

See below for some definitions and quotes from the book.

Definitions

Programming paradigms: Ways of programming unrelated to any particular programming language. A paradigm tells you which programming structures to use and when to use them. 

Paradigms -> Structured Programming: Imposes discipline on direct transfer of control. Unrestrained jumps like goto statements are replaced with if/then/else and do/while/until constructs. 

Paradigms -> Object-Oriented Programming: Imposes discipline on indirect transfer of control. Promotes polymorphism through the disciplined use of function pointers. 

Paradigms -> Functional Programming: Imposes discipline upon assignment. A foundational notion is immutability. 

Dependency Inversion: The source code dependency between a low-level class L and the interface I that it implements points in the opposite direction to the flow of control from a high-level class H and the call that it makes to I. The source code dependencies are inverted against the flow of control. 

The Single Responsibility Principle: A module should be responsible to one and only one actor. Think of "actor" as the singular group of users or stakeholders who may provide a reason for the module to change. Think of "module" as a source file or a cohesive set of functions and data structures. 

Cohesion: The force that binds together the code responsible to a single actor.

Architectural Boundary: A line that separates software elements from one another and restricts those on one side from knowing about those on the other. Ideally, this line divides the system into two components: one abstract and the other concrete. The abstract component contains all the high-level business rules of the application. The concrete component contains all the implementation details that those business rules manipulate.

Dependency Rule: Dependencies cross the architectural boundary lines in one direction: toward more abstract entities containing the higher level policy. 

Components: The units of deployment. They are the smallest entities that can be deployed as part of a system. 

Reuse/Release Equivalence Principle: The granule of reuse is the granule of release. Another way to look at it: Classes and modules that are grouped together into a component should be releasable together. The fact that they share the same version number and the same release tracking, and are included under the same release documentation, should make sense both to the authors and to the users. 

Common Closure Principle: Gather into components those classes that change for the same reasons and at the same times. Separate into different components those classes that change at different times and for different reasons. This is the Single Responsibility Principle restated for components: A component should not have multiple reasons to change. Another way to look at it: If two classes are so tightly bound that they always change together, then they belong in the same component. 

Common Reuse Principle: Don't force users of a component to depend on things they don't need. Another way to look at it: When we depend on a component, we want to make sure that we depend on every class in that component. This principle tells us more about which classes shouldn't be together than about which classes should be together. 

The Acyclic Dependencies Principle: Allow no cycles in the component dependency graph. A cycle in the component dependency graph means that the components in the cycle have, in effect, become one large component. The developers working on any of these components will be stepping over one another because they must all use exactly the same release of one another's components. Also, to test any one component in this cycle, the developers must build and integrate with all of the components. 

Stability: How easy or difficult it is to change a software component. A component with lots of incoming dependencies is very stable because it requires a great deal of work to reconcile any changes with all the dependent components. 

The Stable Dependencies Principle: Depend in the direction of stability. Any component that we expect to be volatile should not be depended on by a component that is difficult to change. By conforming to this principle, we ensure that modules that are intended to be easy to change are not depended on by modules that are harder to change. 

The Stable Abstractions Principle: A component should be as abstract as it is stable. Taken together with the Stable Dependencies Principles, this implies that dependencies run in the direction of abstraction. 

Level (as in: high-level vs low-level): The distance from the inputs and outputs. The farther a policy is from both the inputs and outputs of the system, the higher its level. The policies that manage input and output are the lowest-level policies in the system. 

Entity: An object within our computer system that embodies a small set of critical business rules operating on critical business data. The object either contains the critical business data or has very easy access to that data. The interface of the object consists of functions that implement the critical business rules that operate on that data. 

Use Case: A description of the way that an automated system is used. It specifies the input to be provided by the user, the output to be returned to the user, and the processing steps involved in producing that output.

Quotes

Foreword: "Not only does a good architecture meet the needs of its users, developers, and owners at a given point in time, but it also meets them over time."

Foreword: "The only way to go fast, is to go well."

Preface: "Why should such different systems all share similar rules of architecture? My conclusion is that the rules of software architecture are independent of every other variable."

Introduction: "Getting software right is *hard*. It takes knowledge and skills that most young programmers haven't yet acquired. It requires thought and insight that most programmers don't take the time to develop. It requires a level of discipline and dedication that most programmers never dreamed they'd need. Mostly, it takes a passion for the craft and the desire to be a professional."

Introduction: "... when you get software right, something magical happens: You don't need hordes of programmers to keep it working. You don't need massive requirements documents and huge issue tracking systems."

Introduction: "When software is done right, it requires a fraction of the human resources to create and maintain. Changes are simple and rapid. Defects are few and far between. Effort is minimised, and functionality and flexibility are maximised."

Introduction: "Have you worked on systems that are so interconnected and intricately coupled that every change, regardless of how trivial, takes weeks and involves huge risks?"

Introduction: "It is far more common to fight your way through terrible software designs than it is to enjoy the pleasure of working with a good one."

What is design and architecture / The Goal? "The goal of software architecture is to minimise the human resources required to build and maintain the required system."

What is design and architecture? / The Goal? "The measure of design quality is simply the measure of the effort required to meet the needs of the customer. If that effort is low, and stays low throughout the lifetime of the system, the design is good. If that effort grows with each new release, the design is bad."

What is design and architecture? / What Went Wrong? "The developers may think that the answer is to start over from scratch and redesign the whole system – but that's just the Hare talking again. The same overconfidence that led to the mess is now telling them that they can build it better if only they can start the race over."

A Tale of Two Values / Architecture: "Software was invented to be 'soft'. It was intended to be a way to easily change the behaviour of machines. If we'd wanted the behaviour of machines to be hard to change, we would have called it *hardware*."

A Tale of Two Values / The Greater Value: "... there's no such thing as a program that is impossible to change. However, there are systems that are practically impossible to change, because the cost of change exceeds the benefit of change."

A Tale of Two Values / The Greater Value: "... it is the responsibility of the software development team to assert the importance of architecture over the urgency of features."

Paradigm Overview / Food For Thought: "Each of the [programming] paradigms [i.e. structured programming, object-oriented programming and functional programming] *removes* capabilities from the programmer. None of them adds new capabilities. Each imposes some kind of extra discipline that is *negative* in its intent. The paradigms tell us what *not* to do, more than they tell us what *to* do."

Paradigm Overview / Food For Thought: "... each [programming] paradigm [i.e. structured programming, object-oriented programming and functional programming] takes something away from us."

Structured Programming / Proof: "... all programs can be constructed from just three structures: sequence, selection, and iteration."

Structured Programming / Science To The Rescue: "Science does not work by proving statements true, but rather by *proving statements false*. Those statements that we cannot prove false, after much effort, we deem to be true enough for our purposes."

Structured Programming / Science To The Rescue: "... mathematics is the discipline of proving provable statements true. Science, in contrast, is the discipline of proving provable statements false."

Structured Programming / Tests: "... a program can be proven incorrect by a test, but it cannot be proven correct. All the tests can do, after sufficient testing effort, is allow us to deem a program to be correct enough for our purposes."

Structured Programming / Tests: "Software development is not a mathematical endeavour, even though it seems to manipulate mathematical constructs. Rather, software is like a science. We show correctness by failing to prove incorrectness, despite our best efforts."

Functional Programming / Squares of Integers: "Variables in functional languages *do not vary*."

Functional Programming / Immutability and Architecture: "All race conditions, deadlock conditions, and concurrent update problems are due to mutable variables. You cannot have a race condition or a concurrent update problem if no variable is ever updated. You cannot have deadlocks without mutable locks."

The Single Responsibility Principle: "... separate the code that different actors depend on."

The Single Responsibility Principle: "... separate code that supports different actors."

Open-Closed Principle: "... the behaviour of a software artifact ought to be extensible, without having to modify that artifact."

Open-Closed Principle / A Thought Experiment: "If component A should be protected from changes in component B, then component B should depend on component A."

Open-Closed Principle / A Thought Experiment: "We want to protect the *Controllers* from changes in the *Presenters*. We want to protect the *Presenters* from changes in the *Views*."

Components: "... well-designed components always retain the ability to be independently deployable and, therefore, independently developable."

Component Cohesion / The Tension Diagram for Component Cohesion: "The three [component] cohesion principles tend to fight each other. The REP [Reuse/Release Equivalence Principle] and CCP [Common Closure Principle] are *inclusive* principles: Both tend to make components larger. The CRP [Common Reuse Principle] is an *exclusive* principle, driving components to be smaller. It is the tension between these principles that good architects seek to resolve."

Component Coupling / Top-Down Design: "... component dependency diagrams have very little to do with describing the function of the application. Instead they are a map to the *buildability* and *maintainability* of the application."

Component Coupling / The Stable Dependencies Principle: "One sure way to make a software component difficult to change, is to make lots of other software components depend on it."

What is Architecture? "The architecture of a software system is the shape given to that system by those who build it. The form of that shape is in the division of that system into components, the arrangement of those components, and the ways in which those components communicate with each other. The purpose of that shape is to facilitate the development, deployment, operation, and maintenance of the software system contained within it. The strategy behind that facilitation is to leave as many options open as possible, for as long as possible."

What is Architecture? "... the architecture of a system has very little bearing on whether that system works. There are many systems out there, with terrible architectures, that work just fine. Their troubles do not lie in their operation; rather, they occur in their deployment, maintenance, and ongoing development."

What is Architecture? "Good architecture makes the system easy to understand, easy to develop, easy to maintain, and easy to deploy. The ultimate goal is to minimise the lifetime cost of the system and to maximise programmer productivity."

What is Architecture? / Keeping Options Open: "... software has two types of value: the value of its behaviour and the value of its structure. The second of these is the greater of the two because it is this value that makes software *soft*."

What is Architecture? / Keeping Options Open: "The way you keep software soft is to leave as many options open as possible, for as long as possible. What are the options that we need to leave open? *They are the details that don't matter.*"

What is Architecture? / Keeping Options Open: "The goal of the architect is to create a shape for the system that recognises policy [i.e. the business rules and procedures] as the most essential element of the system while making the details [i.e. IO devices, databases, frameworks etc.] *irrelevant* to that policy."

What is Architecture? / Keeping Options Open: "If you have a portion of the high-level policy working, and it is agnostic about the database, you could try connecting it to several different databases to check applicability and performance... The longer you leave options open, the more experiments you can run, the more things you can try, and the more information you will have when you reach the point at which those decisions can no longer be deferred."

What is Architecture? / Keeping Options Open: "A good architect pretends that the decision has not been made, and shapes the system such that those decisions can still be deferred or changed for as long as possible."

What is Architecture? / Keeping Options Open: "A good architect maximises the number of decisions not made."

What is Architecture? / Conclusion: "Good architects carefully separate details from policy, and then decouple the policy from the details so thoroughly that the policy has no knowledge of the details and does not depend on the details in any way."

Independence / Use Cases: "The most important thing a good architecture can do to support behaviour is to clarify and expose that behaviour so that the intent of the system is visible at the architecture level... Developers will not have to hunt for behaviours, because those behaviours will be first-class elements visible at the top level of the system."

Independence / Decoupling Layers: "... separate those things that change for different reasons... collect those things that change for the same reasons..."

Boundaries: Drawing Lines: "What kinds of decisions are premature? Decisions that have nothing to do with the business requirements – the use cases – of the system. These include decisions about frameworks, databases, web servers, utility libraries, dependency injection, and the like. A good system architecture is one in which decisions like these are rendered ancillary and deferrable. A good system architecture does not depend on these decisions. A good system architecture allows those decisions to be made at the latest possible moment, without significant impact."

Boundaries: Drawing Lines / Which Lines Do You Draw, And When Do You Draw Them: "The database is a tool that the business rules can use *indirectly*. The business rules don't need to know about the schema, or the query language, or any other details about the database. All the business rules need to know is that there is a set of functions that can be used to fetch or save data."

Boundary Anatomy / The Dreaded Monolith: "High-level components remain independent of lower-level details."

Boundary Anatomy / Local Processes: "... the architectural goal is for lower-level processes to be plugins to higher-level processes."

Boundary Anatomy / Services: "Lower-level services should *plug in* to higher-level services."

Policy And Level: "... lower-level components should be plugins to the higher-level components."

Business Rules: "How the data gets in and out of the system is irrelevant to the use cases."

Screaming Architecture / The Purpose of an Architecture: "Good architectures are centred on use cases so that architects can safely describe the structures that support those use cases without committing to frameworks, tools, and environments."

Screaming Architecture / The Purpose of an Architecture: "... consider the plans for a house. The first concern of the architect is to make sure that the house is usable – not to ensure that the house is made of bricks. Indeed, the architect takes pains to ensure that the homeowner can make decisions about the exterior material (bricks, stone, or cedar) later..."

Screaming Architecture / Frameworks Are Tools, Not Ways of Life: "Look at each framework with a jaded eye. View it skeptically. Yes, it might help, but at what cost? Ask yourself how you should use it, and how you should protect yourself from it. Think about how you can preserve the use-case emphasis of your architecture. Develop a strategy that prevents the framework from taking over that architecture."

Screaming Architecture / Testable Architectures: "If your system architecture is all about the use cases, and if you have kept your frameworks at arm's lengths, then you should be able to unit-test all those use cases without any of the frameworks in place. You shouldn't need the web server running to run your tests. You shouldn't need the database connected to run your tests. Your Entity objects should be plain old objects that have no dependencies on frameworks or databases or other complications. Your use case objects should coordinate your Entity objects."

Screaming Architecture / Conclusion: "Your architecture should tell readers about the system, not about the frameworks you used in your system. If you are building a health care system, then when new programmers look at the source repository, their first impression should be, 'Oh, this is a health care system.' Those new programmers should be able to learn all the use cases of the system, yet still not know how the system is delivered."

The Test Boundary / Design for Testability: "... design the system, and the tests, so that business rules can be tested without using the GUI."

Clean Embedded Architecture: "Although software does not wear out, it can be destroyed from within by unmanaged dependencies on firmware and hardware."

Clean Embedded Architecture: "Being able to test only in the target hardware is not good for your product's long-term health."

The Four Types of Test Double: Dummy, Stub, Fake, Mock

A test double is a pretend object used in place of a real object for testing purposes. The four types of test double are as follows:

Dummy

• An object that is passed around but never used.
• Used to help fill parameter lists or fulfil mandatory field requirements where the object will never get used but gets the code to run.
• In many cases it's just an empty or null object.

Stub

• An object that always returns the same canned response.
• Used when you want to replace a real implementation with an object that will return the same response every time.

Fake

• An actual working implementation (close to but not of production quality or configuration) that can replace the real implementation.
• Can be seen as an enhanced stub that almost does the same work as the production code but takes a few shortcuts in order to fulfil the testing requirements.
• An example is an in-memory database (used instead of a real, persistent database).

Mock

• An object that represents a series of expectations and provides canned responses.
• Can be seen as a programmable stub which can be told the sequence of calls to expect and how it should respond to each one.
• Combines well with Dependency Injection in that it allows pretend objects to be injected that will behave in precisely known ways.

(Source: The Well-Grounded Java Developer, by Benjamin J. Evans and Martijn Verburg)

Book Review: The Well-Grounded Java Developer, by Benjamin J Evans and Martin Verburg

Although Java 9 is now out and this book was written way back when Java 7 was new I still found this to be a very useful and educational read. The book explains the fundamentals of the Java platform (e.g. bytecode, garbage collection) and programming principles in general (e.g. concurrency, dependency injection, functional programming) in a way that I've not seen in any other book. The introduction and contrast of the Groovy, Scala and Clojure JVM languages I also thought was done well. Some aspects of the book are now dated but it's still worthwhile skimming over for a historical context of how the platform has developed. If you're new to programming and new to Java development however this probably isn't the right book to start with.

Types of programming languages

Interpreted vs Compiled

• In an interpreted language each step of the source code is executed as is.
• A compiled language uses a compiler to convert the human-readable source code into a binary form as an initial task.

Dynamic vs Static

• In a dynamic language a variable can contain different types at different times.
• In a static language the type information is all about the variable itself, not the value in the variable.

Imperative vs Functional

• An imperative language models the running state of a program as mutable data and issues a list of instructions that transform that running state.
• A functional language operates on values but, instead of altering the inputs, functions act like mathematical functions and return new values.

(Source: The Well-Grounded Java Developer, by Benjamin J. Evans and Martijn Verburg)

Book Review: The Elements of Scrum, by Chris Sims and Hillary Louise Johnson

I got this book after reading the abridged version by the same authors. Like the abridged version this book is well written and the concepts well explained. The book opens with a good comparison of "waterfall" vs "agile" development. Following this is a very enjoyable exposition of the agile values and principles (individuals and interactions over processes and tools etc etc). Following this is the core part of the book: explanations of the roles (product owner, scrum master, team member), ceremonies (sprint planning, daily standup, sprint review, sprint retrospective, backlog grooming, story time) and artifacts (product backlog, sprint backlog, burn charts, task board, definition of done) that make up scrum. Within the core part of the book is a good explanation of what user stories are and how they help with understanding what needs to be built, for who and, importantly, why. And closing out the core part of the book is a discussion on some tried and tested strategies for estimating stories.

I've not read any other books on scrum so I can't compare but I'd be very very amazed to find a better one out there than this one. This book covered absolutely lots of ground whilst remaining compact and, most importantly, very readable.

Below is a small selection of quotes from the book:

"...only by demonstrating working software to your customer can you find out what they really want... agile processes include constant feedback from customers, projects stay relevant and on track..."

"...The main idea behind the agile approach is to deliver business value immediately, in the form of working software, and to continue delivering value in regular increments..."

"...Document as you go, and only as needed..."

"...It’s only human nature to become attached to something you worked hard to produce; if you spend the same amount of time drafting your documentation as Tolstoy spent writing War and Peace, you may find yourself treating your masterpiece as if it were the work product itself..."

"...In order to be successful, a contributor-scrum master must put the needs and success of the team above the desire to directly contribute..."

"...Being a scrum team member isn’t about getting your job done, it’s about getting the job done..."

"...Velocity should never be used as a performance metric: it’s not about demonstrating to management that you’re working fast enough; it’s about gaining predictability of schedule..."

Book Review: Scrum: A Breathtakingly Brief and Agile Introduction, by Chris Sims and Hillary Louise Johnson

Good, concise, to-the-point introduction and summary of the Scrum way of working for teams and companies. The book covers the Roles (Product Owner, Scrum Master, Team Member), the Artifacts (Product Backlog, Sprint Backlog, Burn Charts etc) and the Sprint Cycle elements (Daily Scrum, Story Time, Sprint Review etc) that make up a Scrum workflow.

Go To: Software Superheroes, by Steve Lohr

I bought this book back in 2002 and only got around to reading it in 2015! Good thing too as it probably wouldn't have made much sense back then! Good book on the evolution of software and programming from the 1950s through to the 2000s I thought. I liked the last few chapters in particular which chronicled the birth of the internet and web programming and then the introduction of Java and lastly the growth of Apache and the open source community.

Why is programming fun?

"First is the sheer joy of making things...

Second is the pleasure of making things that are useful to other people...

Third is the fascination of fashioning complex puzzle-like objects of interlocking moving parts...

Fourth is the joy of always learning, which springs from the non-repeating nature of the task...

Finally, there is the joy of working in such a tractable medium..."

(Snippet from 'The Mythical Man-Month' by Frederick P. Brooks, Jr. [Chapter 1 - The Tar Pit])

Processes and Threads

A 'process' is an 'instance' of a 'computer program' that is being 'executed'. It contains the 'program code' and its 'current activity' (state?).

A 'thread of execution' is the smallest unit of 'processing' that can be scheduled by an operating system. It generally results from a 'fork' of a computer program into two or more concurrently executing 'tasks'. Typically, a thread or multiple threads are contained inside a process and share resources such as state and memory.

An advantage/use of multi-threading: the ability for an application to remain responsive to input, e.g. in a single threaded program, if the main execution thread blocks on a long running task, the entire application can appear to freeze. By moving such long running tasks to a worker thread that runs concurrently with the main execution thread, it is possible for the application to remain responsive to user input while executing tasks in the background.

Compilers

About to give away my book 'modern compiler implementation in Java'. Was skimming through it and thought I'd Blog some key words/definitions for future reference and as a reminder of some of what I learnt during my undergraduate days:

Compiler - used to translate a programming language into executable code.

Syntax - the way in which words are put together to form phrases, clauses, or sentences.

Abstract - disassociated from any specific instance.

Semantic - of or relating to meaning in language.

Stack - an orderly pile or heap.

Translate - to turn into one's own or another language.

Canonical - reduced to the simplest or clearest schema possible.

Instruction - a code that tells a computer to perform a particular action.

Register - a device for storing small amounts of data.

Debug - to eliminate errors in or malfunctions of.

Garbage - unwanted or useless material. (Heap-allocated records that are not reachable by any chain of pointers from program variables.)

Garbage Collection - process by which memory occupied by garbage is reclaimed for use in allocating new records. (Process performed not by the compiler but by the runtime system: support programs linked with the compiled code.)

Software Development Processes

Just read a bit about Software Development Processes. In particular, the 'Waterfall Model' (where progress flows sequentially from top to bottom through various phases) and 'Iterative and Incremental Development' (where progress is through cycles (iterative) and in smaller portions at a time (incremental)).

Regarding the former, an example of the phases (in order): Requirements specification; Design; Construction (AKA implementation or coding); Integration; Testing and debugging (AKA Validation); Installation; Maintenance.

Regarding the latter, a quote from Wikipedia: "Learning comes from both the development and use of the system, where possible key steps in the process start with a simple implementation of a subset of the software requirements and iteratively enhance the evolving versions until the full system is implemented. At each iteration, design modifications are made and new functional capabilities are added."