Shift Left
Shifting left involves a variety of application engineering, software testing and operations management skills and practices that empower teams to find and prevent defects early in the software delivery process. Companies that effectively shift left recognize that software quality is a key, foundation driver of customer success and the company's ability to conduct business without disruption. It also involves developing an engineering culture based on the shared responsibility of quality.
The term "Shift Left" is actually derived from the Shift Left Testing philosophy and the corresponding "Test early and often" maxim that Larry Smith coined back in 2001 as part of his progressive perspective on how to have software testing and system testing performed earlier in the software development life cycle. His article is often considered the foundational, required reading for those interested in learning about where software development shift came from.
Since the article was published nearly two decades ago, Shift Left has become an industry-wide movement that is comprised of many individual disciplines, of which code quality and software testing practices are foundational. It emphasizes using modern, progressive software engineering skills and agile practices to relentlessly produce high quality software and provide better results for businesses.
Successful shift left teams demonstrate early detection of software issues, high defect prevention capabilities and realize significant efficiencies in delivering new feature work and defect remediation.
Why do it?
Those that are highly successful in adjusting to shift left typically enjoy a greater sense of collaboration and teamwork that is rooted in a shared responsibility of quality instead of the old school "throw it over the wall" approach to quality assurance.
The idea that a "QA" team is solely responsible for testing is antithetical to shifting left. Instead, organizations intentionally investment time, effort and attention into adopting development strategies like Three Amigos to empower organizations to work cohesively together.
What does good look like?
Successful shift left practitioners typically demonstrate within their work products that they recognize that those coding the application are subject to well documented confirmation bias challenges and that the independent, third-party verification provided by highly capable and well skilled software testing professionals is essential to success. Continuous testing by the entire team is the foundation of continuous integration, continuous deployment and ultimately continuous delivery.
The effective use of modern source code management (SCM) tools (GitHub, GitLab, BitBucket or similar alternative) integrated with code quality analysis, pre-installation open box testing, building, deploying, and post-installation closed box testing are standard operating procedure. These disciplines become part of the DNA of the codebases being built. It is also important that teams can differentiate between the mechanics that perform these operations from the quality controls, practices and disciplines that keep testing highly effective and visible while achieving a progressive DevTestOps culture.
All of this becomes achievable when application engineers and software testing professionals are empowered within a shift left supportive engineering culture.
Cost of NOT shifting left
There are many other "your mileage may vary" types of studies and articles, like the following, that illustrate the cost per defect perspective relative to the increased cost per phase approaches that have been widely accepted as industry best practices and software development capabilities that business leaders throughout the industry find highly desirable.
The ROI of Shift Left Testing
How much do bugs cost to fix during each phase of the SDLC?
-
The Systems Sciences Institute at IBM reported that it cost 6x more to fix a bug found during implementation than to fix one identified during design. Furthermore, according to IBM, the cost to fix bugs found during the testing phase could be 15x more than the cost of fixing those found during design.
*****************************
The team advocates and champions Shift Left and DevTestOps philosophies that clearly state that the testing disciplines are not siloed away from Dev and Ops. Instead, we seek to raise the visibility of testers and all testing disciplines as integral parts of the modern software quality culture. Software testing practices and the professionals that use them are first class citizens within agile projects and there is a focus on testing early and often, not after-the-fact or as an afterthought.
Foundational disciplines
The Developer Enablement team will be focusing their 2021 efforts on the supporting the following disciplines within multiple projects that comprise the sixty top-tier applications that were supported for the 2020 AIP bug reduction effort.
Code Quality
Bugs, vulnerabilities, security hotspots and code smells are well established measurements that are identified by using static code analysis to evaluate projects against language-specific rules.
Together with duplications, complexities, test code coverage and other ratings, a comprehensive picture of how maintainable, extensible, secure and reliable the source code of a project is produced.
Within the business world, code is not written for an individual.Projects need to be crafted to support current team members as well as those that will work within the codebase in the future.
Individuals that are rogue coding their way through tickets, without a focus on cohesiveness or the easily reproducible evidence that their code works as intended, are often creating the next future technical debt challenge.
Release Confidence and its Fulcrum Score have been designed specifically to help teams continuously improve within this and other related disciplines.
Source Code Management (SCM)
The "for those that code" section below has a variety of practices like commit often, reference Jira issues within your commit messages, ensure you're working from latest version, peer reviews and minimum necessary change (MNC) listed to create awareness but considerably more guidance, as well as standards, will be provided for this discipline in the future.
The Developer Enablement team will be championing the move away from racially insensitive master/slave references in source code management practices and supporting teams in making the transition.
Software Testing
Open Box Testing
-
A pillar of the shift left movement, this is a software testing discipline in which the internal structure, design and implementation of the program being tested is known and is available to those that are performing quality focused activities. These pre-installation testing activities are expected of application engineers and it is often said that this is where shift left testing begins.
-
Empower software engineers are often great at solving problems deep within their project's code bases to test, debug and support their software.
-
Software engineers that write their own tests are often highly invested in fixing them when they fail.
-
Software engineers who understand the feature requirements, and the testing implications, generally write better code.
-
The Developer Enablement team has been championing the move away from the racially insensitive "white box testing" terminology.
Closed Box Testing
-
Equally important and as challenging as the open box disciplines, this testing discipline is where the internal structure, design and/or implementation of the program being tested may not be known by the tester. These post-installation activities are primarily focused on the functional and non-functional requirements of the software and are typically expected by the independent, third-party software testing professionals.
-
Software testing professionals that successfully adopt practices that support high code quality and the shift left software testing disciplines typically foster more collaborative and inclusive partnerships with their business and development partners.
-
The Developer Enablement team has been championing the move away from the racially insensitive "black box testing" terminology.
Release Management
-
The deliberate and disciplined use of feature, release and hotfix branches, in conjunction with semantic versioning practices, are essential to providing a path for fully analyzed, tested, verified and finalized release artifacts to be installed for end users.
Agile Practices
-
Successful agile practitioners are sustainably able to deliver new features with high overall quality. While many agile teams struggle at some point in their journey with incorporating the necessary testing time, there is usually an explicit recognition that traditional testing methodologies simply do not fit into an agile context.
-
Teamwork and the long term sustainability of the work produced by teams that often have personnel changes over time benefit from standardized, industry proven best practices interwoven with innovations that come from within the organization.
Open source compliance and security checks
-
The exceptionally progressive article Shift Left Testing in Agile Development: a Seismic Shift highlights how disciplines like open source license compliance and security vulnerability can be shifted left as well. This is a particularly interesting area of innovation and technical maturity that will be explored within the Shift Left program.
Performance Engineering
-
Shifting performance testing left means enabling developers and testers to conduct performance testing in the early stages of development cycles. ... A shift-left performance testing strategy instead allows smaller, ad hoc performance tests to be performed by testers against individual components as they are developed.
For those that code
For those that are producing application, testing, or operational and infrastructure assets that are code based, the fundamentals of shifting left are something that every engineer can and should be doing
Commit often
-
Code is effectively invisible to the organization until it exists in the source code control system.
-
Write code out in the open! Put effort into not working privately in an attempt to contribute "perfect" code, as if no mistakes had ever been made. Programming is an inherently social activity, but many treat it as an arena for personal heroics. That is not what we do.
-
While checking in broken code is a bad practice, committing incomplete code periodically provides visibility, empowers collaboration, safeguards the company’s software development investments and corresponds nicely with an incremental/iterative approach to software development.
-
While the statement “if the code isn't checked into source control, it doesn't exist” might be overly harsh it does emphasize the importance of actively committing code changes.
-
Do not go dark. Modern software development and testing is an inherently social activity, so transparency and team responsible behavior is crucial to shifting left. Jim McCarthy's 1995 book Dynamics of Software Development documents the going dark phenomenon.
Reference Jira issues within your commit messages
GitLab has an extremely powerful integration with Jira that cross-references activity within the GitLab project to the individual Jira issues. It is as simple as explicitly referring to the Jira issue within the git commit message:
git commit -m "PROJECT-1 Description of what was done"
Ensure you're working from latest version
-
Source Code Management (SCM) solutions like git enables rapid updates from multiple developers. It is easy to have a local copy of the codebase fall behind the global copy. Engineering professionals need to be sure to fetch the latest code before making updates. This will also help avoid conflicts at merge time.
Peer Reviews
-
In order to improve the overall quality of the software being built, we engage in peer reviews which is the systematic examination of computer source code by colleagues in order to find mistakes or possible oversights during the creation of the code.
-
Do not expect code reviews to be perfect. Among the benefits is the ability to help us collectively learn from our mistakes. Teams that adopt meaningful code review practices often directly benefit from the “none of us is as smart as all of us” philosophy being practiced within their codebases and work product.
Minimum Necessary Change (MNC)
-
Who has the time to review thousands of lines of code? Whether you are making changes to existing functionality or creating brand new functionality, effort must be made to manage development tasks in such a way that individuals provide small units of work that can be reasonably understood, reviewed, commented upon and improved upon.
Write Testable Code
-
Understanding how to write testable code is one of the biggest frustrations many engineers encounter when they transition from their academic studies into their professional careers.
-
Learning and practicing SOLID design principles with the projects that engineers support can help projects to be more modular and to have increased testability.
-
DRY (Don't Repeat Yourself) was first introduced in the book The Pragmatic Programmer and its "Every piece of knowledge must have a single, unambiguous, authoritative representation within a system" tenet is among the most important within the principle. A "piece of knowledge" is often explained as specific functionality in the business domain of an application or a distinct algorithm within a project. A common example would be a the concept of a shipment within an e-commerce application and that the logic of that entity should only appear once within the application.
-
The Writing Testable Code article has great discussion points and excellent examples of how to use the SOLID principles:
-
S (Single-responsiblity Principle)
-
O (Open-closed Principle)
-
L (Liskov Substitution Principle)
-
I (Interface Segregation Principle)
-
D (Dependency Inversion Principle)
-
-
Those that follow the Law of Demeter (LoD) principle produce code that is not highly-coupled to implementation details and is substantially more reusable. Software that breaks this principle often requires excessive mocks or the creation of instances of explicit collaborators of the class and also all the implicit chain of collaborators required by it. Each unit should have only limited knowledge about other units: only units “closely” related to the current unit.
-
Each unit should only talk to its friends; don’t talk to strangers.
-
Only talk to your immediate friends.
-
Static Code Analysis
-
Projects must implement SonarQube (https://sonarqube.cicd.aws.nml.com) to provide continuous inspection of code quality and repeatedly use static code analysis data to detect bugs, vulnerabilities and many other potential issues as early as possible.
-
Do not misuse SonarQube exclusions. Excessive exclusions often result in projects reporting significantly fewer lines of code than they actually have. Excluding valid source code that is within projects artificially skews the code quality analysis and test coverage metrics reported for a project. While it is perfectly acceptable to exclude explicit test resources and configuration files from analysis, misuse of this SonarQube feature adversely impacts data accuracy.
-
Example: As of 3/5/21 there were 559 projects with exclusions that result in the majority of the code within those projects not being analyzed for code quality and any open box test coverage statistics are going to be significantly skewed
-
Test Relentlessly
-
Developers explicitly write code that tests their project on their local machines whenever possible and the same tests run within CI/CD build pipelines.
-
Automation executes the same tests within pipelines so that the project does not become a victim of Works on My Machine.
-
Automated tests are run prior to a build artifact being considered installed and ready for software testing professionals.
-
After installation is complete and verified, software testing professionals conduct a variety of activities that include, but are not limited to, functional and non-functional automation, exploratory testing and other progressive disciplines.
-
All members of the team constantly validate that what was working previously is still working as a result of whatever change was made.
-
When new functionality is being introduced, ensure that the business logic and conditions within the code are covered by open box tests and that the functional and non-functional requirements are validated by closed box tests.
Intentionally Release Software to Users
-
Whether you are a proponent of trunk based development, GitFlow or any of the alternative strategies that exist, the intentional use of feature, release and hotfix branches along with semantic versioning are essential to providing a path for fully analyzed, tested and finalized release artifacts to be installed for end users. Skillful diligence and well-designed automated ensures that unintended code and functionality is not inadvertently introduced into production systems.
-
To be effective within Shift Left disciplines, it is important that teams avoid spending exorbitant amounts of time coping with a"tangled thicket of branches", as Martin Fowler described in his May2020 article Patterns for Managing Source Code Branches. Instead, teams need to use the mechanics provided by the GitLabCI/CD tool in ways that work for them while explicitly performing git tagging on their project's source code at release time and use GitLab's Releases capability to intentionally release software into production systems.
-
Teams are also required to demonstrate Hotfix and rollback capabilities. Hotfixes involve explicitly fixing the version of software that is currently running in production, using the git tagged version to avoid releasing new, potentially untested features to production.Rollbacks involve restoring the previous version of the software, and any related external dependencies, to its previous operational state.
Plan for Debugging and Support
-
Be prepared with strategies to enable the right level of logging and/or remote debugging capabilities whenever possible.
-
Having an established practice to support details about how to reproduce issues from other environments can also be a tremendously valuable tool in the team's toolbox.
-
Teams need to be prepared to track and prioritize issues that are reported in the various phases of the Software Development LifeCycle (SDLC), which includes incidents that may occur in a production environment impacting customers. The diligent use of release and hotfix branches is a foundation to the application engineering and software testing work that needs to be done to remediate software incidents.
-
Estimating defects in Agile is an anti-pattern, but following the standardized recording and reporting of issues and incidents provides visibility and enables data-driven decision making.
Recognize and Avoid Complacency
-
The article Complacency — The Developer’s Kiss of Death is an exceptional perspective of how programming skills are similar to building a major muscle group. Just like building muscle at the gym, consistent yet challenging work is required.
Fail Fast
-
Humans tend to make mistakes and software tends to have bugs. The only code that has no bugs is the code that has never been written.
-
The short circuit/circuit breaker description that is often associated with this discipline comes from electrical engineering and illustrate show when error occurs, the system fails immediately and visibly without setting the entire system on fire. Yes the system may be offline as the short-circuiting or tripping occurs, but detecting a fault provides engineers an opportunity to learn from and remediate the issue.
-
The "Fail Fast" article by Jim Shore explains how to effectively use this concept in software development.
-
Assertions are the key to failing fast. An assertion is a tiny piece of code that checks a condition and then fails if the condition is not met. So when something starts to go wrong, an assertion detects the problem. Making problems visible, understandable and actionable without being penalized or shamed is crucial to the success of Shift Left disciplines.
Ultimately, professionals that are writing code are doing these things with an understanding that there is not one single solution that fits all.Instead, we build effective strategies across all phases of the software release cycle that support the building, testing, deploying and monitoring of the finished software product that is delivered to users.
As shift left capabilities improve, the business will benefit from all of these activities that are constantly focused on quality and, hopefully, reduce the number of issues that reach production.