For a long time, development and operations had been divided into isolated modules with both separate concerns and responsibilities. Developers wrote the code and made sure that it worked on their development systems, while the system administrators were responsible for the actual deployment and integration in the organization's IT infrastructure.

As there was limited communication between these two isolated modules, both teams worked mostly separated on their projects. However, they heavily depended on each other because there was no cross-platform knowledge across the different teams.

This fitted in nicely with the Waterfall Methodology that was used for most projects. The Waterfall Methodology is based on the Software Development Life Cycle (SDLC), which has clearly defined processes for creating software. The Waterfall Methodology is a breakdown of project deliverables into linear sequential phases, where each phase depends on the deliverables of the previous phase. This sequence of events may look as follows:

Figure 1.1 – Waterfall Methodology

The Waterfall Methodology is well suited for projects in the following circumstances:

• Early in the development life cycle, customers and developers agree on what will be delivered, with minimal to no changes during the development of the project.
• For integration with external systems, it is common for multiple components of the software to be designed in parallel. In these cases, it is desirable to have the design document complete at an early stage in the development life cycle.
• Various team members are involved in other projects simultaneously as well. For example, business analysts can gather the requirements and create the design while developers are working on another project.
• Where it is not possible to break down the requirements phase, customers are not fully engaged in smaller deliverables.

However, customers may not exactly know what their requirements are before they see working software. This can result in changing the requirements, thus leading to redesign, reimplementation, and reverification. This can dramatically increase the costs of the project.

Due to this, Agile and DevOps were introduced in 2009 and have slowly taken over the world of software development. They replaced the Waterfall Methodology for most projects that are out there. DevOps is a natural extension of Agile and continuous delivery approaches, and it stands for development and operations. It is a practice that merges development, IT operations, and quality assurance into one single, continuous set of processes.

The following diagram illustrates the different parts that DevOps consists of:

Figure 1.2 – DevOps methodology

It is a team-based and iterative approach to development where all stakeholders, such as developers, administrators, testers, and a representative of the customer, are part of the same team. Applications are delivered in functional components, and rather than creating schedules and tasks at the start of the project, the project is divided into smaller phases, called sprints. The duration of each sprint is defined up front and has a list of deliverables that are planned at the start of each sprint. All those deliverables are defined together with the customer and prioritized by business value by the customer. At the end of each sprint, when work is completed, it is reviewed and evaluated by the team through daily builds and end-of-sprint demos.

This results in the following advantages:

• By working directly with the project team throughout the whole project, the customer will experience a stronger sense of ownership.
• The customer has opportunities to see the work being delivered in an early stage of the project and can make appropriate decisions and changes to it.
• Development is more business and value focused. This is a result of working closer with the customer and having a better understanding of their needs.
• An Agile way of working enables us to quickly create a base version of the product, which can be built upon in the next iterations.

Now that we have covered a very brief introduction to DevOps, we are going to look at the different DevOps principles.

There are a lot of different definitions when it comes to DevOps. Most of them are good at explaining the different aspects of finding the right flow in delivering software and IT projects. In the upcoming sections, we will highlight six DevOps principles that we think are essential when adopting a DevOps way of working.

Principle 1 – Customer-centric action

Nowadays, it is important that software development projects have short cycles and feedback loops, with end users and real customers integrated into the team. To fully meet the customers' requirements, all activity around building software and products must involve these clients. DevOps teams and organizations must continuously invest in products and services that will allow clients to receive the maximum outcome, while also being as lean as possible to continuously innovate and change the chosen strategy when it is no longer working.

Principle 2 – Create with the end in mind

Organizations need to act more like product companies. They should focus more on building working products that are sold to real customers. This engineering mindset needs to be shared by all employees. This is required to realize those products. This means that they should let go of the approach where each unit focuses on a particular role with their own scoped responsibility.

Principle 3 – End-to-end responsibility

In most traditional software development projects, the software and services that are developed are handed over to operations, where they then deploy and maintain those solutions after the initial development process. By adopting a DevOps way of working, the DevOps teams become fully responsible and accountable for the project they deliver. This means that once the product has been delivered by the team and it needs to be maintained, it still remains under the responsibility of the team. The team will also provide support for the product until it reaches its end of life. This greatly increases the level of responsibility of the team and the quality of the products that are developed.

Principle 4 – Cross-functional autonomous teams

Organizations that work with vertical and fully responsible teams will need to let these teams work completely independently throughout the whole life cycle. To enable these teams to work completely independently, a broad and balanced set of skills are required. Team members need to have T-shaped profiles instead of old-school IT specialists who are only skilled in their own role. Examples of skills that every team member should have include development, requirement analysis, testing, and administration skills.

Principle 5 – Continuous improvement

Another part of end-to-end responsibility is that, for organizations, it is important to adapt changes continuously. There can be a number of changing circumstances, such as new technology that has been released, changing customer requirements, and so on. Continuous improvement is a strong focus in DevOps when it comes to optimizing for speed and costs, minimizing waste, easy of delivery, and to continuously improve the software and services that are being built and released. An important activity to embed inside these cycles is experimentation. This will allow teams to develop a way of learning from their failures, which is essential to continuous improvement.

Principle 6 – Automate everything

To fully adopt and embed a continuous improvement culture inside an organization, most organizations have a lot of waste and tech depth to eliminate. To work with high cycle rates and to process the instant feedback from customers and end users as soon as possible, it is imperative to automate everything. This means that not only the software development process should be automated using continuous delivery (which includes continuous development and integration), but also the whole infrastructure landscape needs to be automated. The infrastructure also needs to be ready for new ways of working. In this sense, automation is synonymous with the drive to renew the way in which the team delivers their services to their customers.

In this section, we have covered the six principles that are very important when adopting or migrating to a DevOps way of working. In the next few sections, we are going to look at what Azure DevOps has to offer as a tool that supports teams so that they can work in a DevOps oriented manner.

Azure DevOps provides a wide variety of services for DevOps teams so that they can plan, work, collaborate on code development, and build and deploy software and services. Most DevOps teams rely on several tools and build custom toolchains for each phase in the application life cycle.

The following diagram shows the phases that are defined in the application life cycle:

Figure 1.3 – Application life cycle phases

In the following sections, we'll explain these phases and the corresponding Microsoft tooling and products in more detail.

Plan

During the planning phase, teams can use Kanban boards and backlogs to define, track, and lay out the work that needs to be done in Azure Boards. They can also use GitHub for this. In GitHub, an issue can be created by suggesting a new idea or stating that a bug should be tracked. These issues can be organized and assigned to teams.

Develop

The development phase is supported by Visual Studio Code and Visual Studio. Visual Studio Code is a cross-platform editor, while Visual Studio is a Windows- and Mac-only IDE. You can use Azure DevOps for automated testing and use Azure Pipelines to create automatic builds for building the source code. Code can be shared across teams with Azure DevOps or GitHub.

Deliver

The deliver phase is about deploying your applications and services to target environments. You can use Azure Pipelines to deploy code automatically to any Azure service or on-premises environments. You can use Azure Resource Manager templates or Terraform to spin up environments for your applications or infrastructure components. You can also integrate Jenkins and Spinnaker inside your Azure DevOps Pipelines.

Operate

In this phase, you implement full-stack monitoring for monitoring your applications and services. You can also manage your cloud environment with different automation tools, such as Azure Automation, Chef, and more. Keeping your applications and services secure is also part of this phase. Therefore, you can use features and services such as Azure Policy and Azure Security Center.

To support the full life cycle of analyzing, designing, building, deploying, and maintaining software and infrastructure products and services, Azure DevOps provides integrated features that can be accessed through any web browser.

Azure DevOps offers a combination of solutions and tooling that can be used to create unique and custom workflows throughout each of the application life cycle phases. These solutions will be described in the upcoming sections.

Continuous integration and continuous delivery (CI/CD)

You can automate each DevOps process with CI/CD (and continuous deployment) in Azure DevOps. CI is used in the development phase of a project and refers to building and testing code in a fully automated way. Every time you commit changes to the master branch, the changes will be validated and then packaged into a build artifact automatically. With CD, the delivery phase is automated. Every time a build artifact is available, the artifact is automatically deployed to the desired environment. When continuous integration and continuous deployment are both used by development teams, the code remains ready for production at any time. The only thing that teams must do to deploy a working application into production is trigger the transition from development to deploy. This will make the automated build artifact available for deployment. This triggering can be as simple as pressing a button.

With Azure DevOps, you also implement continuous deployment. Adding this to your development life cycle means that you can automate the entire process, from code commit to production. The trigger between the development and delivery phase is completely automatic. So, when code changes are validated and pass all the tests that are performed during the development phase, the changes will be published to production automatically as well. This means that customers will receive the new version, along with the improvements for it, as soon as they are available.

Agile development support

Azure DevOps supports teams that adopt Agile development methods with planning, tracking, and reporting capabilities. This will result in shorter release cycles and full visibility in the software development process. You can use Azure Boards, which will be covered in more detail in the next section of this chapter, to manage backlogs and define, assign, and track work items. You can also use advanced analytics and reporting and create custom dashboards to track progress.

Version control

A version control system, also known as a source control system, is an essential tool for multi-developer projects. It allows developers to collaborate on the code and track changes. The history of all the code files is also maintained in the version control system. This makes it easy to go back to a different version of the code files in case of errors or bugs.

Azure DevOps supports two different types of source control: Git (distributed) and Team Foundation Version Control (TFVS). With Git, each developer has a copy of the source repository on their development machine. All branch and history information is included inside the source repository. Each developer works directly with their copy of the repository and all the changes are shared between the local and source repositories as a separate step. Changes can be committed on the local filesystem, and version control operations can be executed without a network connection. Branches can be created easily on the dev machine and later, they can be merged, published, or disposed by the developer separately. With TFVC, developers have only one version of each file on their local dev machines. All the others, as well as the historical data, are maintained only on the server. The branches are created on the server as well.

Infrastructure as Code

Teams can also manage the infrastructure in Azure DevOps. Infrastructure components that are used in a project, such as networks, virtual machines, and load balancers, can be managed using the same versioning features and capabilities that are used for the source code.

Used together with continuous delivery, an Infrastructure as Code (IaC) model generates the same environment every time it is deployed. Without IaC, teams need to configure and maintain the settings of all the individual deployment environments manually, which is a time-consuming and error-prone task. The most plausible outcome is that, over time, each environment becomes a snowflake, which is a unique configuration that cannot be reproduced automatically anymore. This inconsistency across environments will lead to issues during the deployment phase.

Configuration Management

Configuration Management refers to all the items and artifacts that are relevant to the project and the relationship between them. Those items are stored, retrieved, and uniquely identified and modified. This includes items such as source code, files, and binaries. The configuration management system is the one true source of configuration items.

Using Azure DevOps, resource configuration across the entire system can be managed by teams to roll out configuration updates, enforce desired states, and automatically resolve unexpected changes and issues. Azure offers multiple DevOps tools and capabilities for configuration management, such as Chef, Puppet, Ansible, and Azure Automation.

Monitoring

You can use Azure Monitor to practice full-stack continuous monitoring. The health of your infrastructure and applications can be integrated into existing dashboards in Grafana, Kibana, and the Azure portal with Azure Monitor. You can also monitor the availability, performance, and usage of your applications, whether they are hosted on-premises or in Azure. Most popular languages and frameworks are supported by Azure Monitor, such as NET, Java, and Node.js, and they are integrated with DevOps processes and tools in Azure DevOps.

In this section, we are going to introduce the different services that are offered by Azure DevOps. These services can be used to support teams throughout the whole life cycle of realizing business value for customers.

Azure Boards

Azure Boards can be used to plan, track, and discuss work across teams using the Agile planning tools that are available. Using Azure Boards, teams can manage their software projects. It also offers a unique set of capabilities, including native support for Scrum and Kanban. You can also create customizable dashboards, and it offers integrated reporting and integration with Microsoft Teams and Slack.

You can create and track user stories, backlog items, tasks, features, and bugs that are associated with the project using Azure Boards.

The following screenshot shows an example of an Azure Board:

Figure 1.4 – Azure Boards

Azure Repos

Azure Repos provides support for private Git repository hosting and for Team Foundation Server Control (TFSC). It offers a set of version control tools that can be used to manage the source code of every development project, large or small. When you edit the code, you ask the source control system to create a snapshot of the files. This snapshot is saved permanently so that it can be recalled later if needed.

Today, Git is the most used version control system among developers. Azure Repos offers standard Git so that developers can use the tools and clients of their choice, such as Git for Windows, Mac, third-party Git services, and tools such as Visual Studio and Visual Studio Code.

The following screenshot shows an example of the commits you can push to a repo in Azure:

Figure 1.5 – Azure Repos

Azure Pipelines

You can use Azure Pipelines to automatically build, test, and deploy code to make it available to other users and deploy it to different targets, such as a development, test, acceptance, and production (DTAP) environment. It combines CI/CD to automatically build and deploy your code.

Before you can use Azure Pipelines, you should put your code in a version control system, such as Azure Repos. Azure Pipelines can integrate with a number of version control systems, such as Azure Repos, Git, TFVS, GitHub, GitHub Enterprise, Subversion, and Bitbucket Cloud. You can also use Pipelines with most application types, such as Java, JavaScript, Node.js, Python, .NET, C++, Go, PHP, and XCode. Applications can be deployed to multiple target environments, including container registries, virtual machines, Azure services, or any on-premises or cloud target.

The following screenshot shows an example of a run for an Azure Pipeline:

Figure 1.6 – Azure Pipelines

Azure Test Plans

With Azure Test Plans, teams can improve their code quality using planned and exploratory services in Azure DevOps. Azure Test Plans offer features for planned manual testing, exploratory testing, user acceptance testing, and for gathering feedback from stakeholders. With manual testing, tests are organized into test plans and test suites by testers and test leads. Teams can begin testing from their Kanban boards or from the Work Hub directly. With user acceptance testing, the value that's delivered to meet customer requirements is verified. This is usually done by designated testers. Exploratory testing includes tests that are executed by the whole development team, including developers, product owners, and testers. The software is tested by exploring the software systems, without the use of test plans or test suites. Stakeholder feedback gathering is done outside the development team by marketing or sales teams. Developers can request feedback on their user stories and features from Azure DevOps. Stakeholders can then respond directly to the feedback item.

The following screenshot shows an example of an Azure Test Plan:

Figure 1.7 – Azure Test Plan

Azure Artifacts

With Azure Artifacts, you can create and share NuGet, npm, Python, and Maven packages from private and public sources with teams in Azure DevOps. These packages can be used in source code and can be made available to the CI/CD pipelines. With Azure Artifacts, you can create multiple feeds that you can use to organize and control access to the packages.

The following screenshot shows an example of a feed in Azure Artifacts:

Figure 1.8 – Azure Artifacts

Extension Marketplace

You can download extensions for Azure DevOps from the Visual Studio Marketplace. These extensions are simple add-ons that can be used to customize and extend your team's experience with Azure DevOps. They can help by extending the planning and tracking of work items, code testing and tracking, pipeline build and release flows, and collaboration among team members. The extensions are created by Microsoft and the community.

The following screenshot shows some of the extensions that can be downloaded from the marketplace:

Figure 1.9 – Extension Marketplace

The services that we introduced in the previous sections will be explained more thoroughly in the upcoming chapters of this book. In the next section, we will introduce the scenarios that will be used throughout this book.

Throughout this book, we will use two different scenarios for our demos. We are going to use sample projects that can be generated and installed in your Azure DevOps environment by using the DevOps generator. For this book, we are going to install Tailwind Traders and Parts Unlimited. Tailwind Traders is an example retail company that showcases the future of intelligent application experiences, while Parts Unlimited is an example e-commerce website.

Creating the starter project

To create the scenario project, we are going to use the Azure DevOps demo generator, which will generate the sample project for us. These projects are free to use. Before you generate the project, you need to install two different Azure DevOps extensions from the marketplace, both of which are used by the Tailwind Traders project. These extensions are as follows:

• ARM Outputs: This extension reads the output values of ARM deployments and sets them as Azure Pipelines variables. You can download and install the extension from https://marketplace.visualstudio.com/items?itemName=keesschollaart.arm-outputs.
• Team Project Health: This extension enables users to visualize the overall health of builds, thereby delivering a visual cue similar to Codify Build Light. You can download the extension from https://marketplace.visualstudio.com/items?itemName=ms-devlabs.TeamProjectHealth.

Once the extensions have been installed inside your Azure DevOps organization, you can generate the sample project:

1. First, navigate to the following site: https://azuredevopsdemogenerator.azurewebsites.net/.
2. Click the Sign in button. If you don't have an Azure account yet, you can sign up for a trial account by clicking the Get started for free button:
   

   Figure 1.10 – Azure DevOps demo generator

3. Name the project Tailwind Traders, select an organization, and select a template by clicking the Choose template button. Select Tailwind Traders from the list and click Select Template.
4. After filling in these details, the page should look as follows:
   

   Figure 1.11 – Creating a new project

5. Click the Create Project button.
6. With the project created, navigate to https://dev.azure.com/.
7. Log in with your credentials and select the organization where you created the project. Select the Tailwind Traders project to see if anything has been generated.
8. Repeat these steps to create the Parts Unlimited project in your DevOps environment.

   Tip

   For more information about the Tailwind Traders sample project, refer to the following site: https://github.com/Microsoft/TailwindTraders. For more information about the Parts Unlimited example, refer to https://microsoft.github.io/PartsUnlimited/.

In this chapter, we covered some of the basics of DevOps and covered the six different DevOps principles. Then, we covered the key concepts of Azure DevOps and the different solutions that Azure DevOps has to offer to support teams throughout each of the application life cycle phases. After that, we looked at the different features that Azure DevOps has to offer, and we introduced and created the two scenarios that we will use in the upcoming chapters of this book.

In the next chapter, we are going to cover how to manage projects with Azure Boards.

Check out the following links for more information about the topics that were covered in this chapter:

• Extension Marketplace: https://marketplace.visualstudio.com/azuredevops
• Azure Automation documentation: https://docs.microsoft.com/en-us/azure/automation/
• Azure DevOps demo generator: https://docs.microsoft.com/en-us/azure/devops/demo-gen/use-demo-generator-v2?view=azure-devops&viewFallbackFrom=vsts
• An overview of the Tailwind Traders reference apps for Azure: https://www.youtube.com/watch?v=EP-PME-1tq0