Blog

Preface – This post is part of the SAP on Azure series.

Introduction

Azure DevOps provides an option to create a pipeline that can be configured in such a way that it can take the latest code from a branch, and build it into artifacts (deployable file) and then deploy the same in a specific cloud platform. This full process can be automated, and it is called Continuous Integration and Continuous Deployment (CI CD Pipeline). In this article, we will discuss the later part of pipeline that is Release and Continuous Deployment.

Creating a Release Pipeline for Continuous Deployment

1. Go to Releases, as shown below:
   

   

2. Create or import a pipeline (for that you need to export the existing one and then import)
   
3. When you create a new pipeline, choose Empty Job, as shown below:
   
4. Now, add your stage name
   
5. Change the name of the pipeline
   
6. Now click on “Add an artifact” and choose the source as the builder we have created above:
   
7. Now, add a trigger for automation (Continuous Deployment), add your branch by following steps as shown below:
   
8. Now, add artifact filter, just in case you don’t want every stage to run for all the builds:
   Here, you need to choose the branch for which this stage is meant to run the pipeline
9. The pipeline can be easily copied from existing one by uploading pipeline json which has given code.
   Also, you must change given builder name with yours under Tasks (for each of the stages)
   
10. Once you are done with the above activities, then go to variables and given info there:
    
11. Now simply save and run the pipeline
    

Hence, we have successfully created an end to end CICD Pipeline.

Continuous Deployment

In CICD Pipeline, we can automate the deployment part with Continuous Deployment. We have shown how to do that in step 07, above.

Preface – This post is part of the SAP on Azure series.

Introduction

Azure DevOps provides an option to create a pipeline that can be configured in such a way that it can take the latest code from a branch, and build it into artifacts (deployable file) and then deploy the same in a specific cloud platform. This full process can be automated, and it is called Continuous Integration and Continuous Deployment (CI CD Pipeline). In this article, we will discuss the first part of pipeline that is Build and Continuous Integration.

Build Pipeline Creation based on YAML

Click New Pipeline

Click on Azure Repos Git

Select your Repository

Choose Starter Pipeline

Save and Run (Just add comment: “Initial Configuration”)

Note: In case you get error “Pushes to this branch are not permitted; you must use a pull request to update this branch.”  Then you need the permission to Push directly.
Now you can see the newly created pipeline.

4. Now, we have a pipeline. The next thing we can do is to manually setup the whole pipeline or use the existing pipeline code. Here, we will use from the existing one.
   Click on the three dots next to the newly created pipeline:

Add variables here:

Add following variables:

version.major = 1

version.minor = 4
version.patch = $[counter(variables['version.minor'], 0)]

Then add the given code in your yaml now(you can edit as per your branch name):

# Variable 'version.major' was defined in the Variables tab
# Variable 'version.minor' was defined in the Variables tab
# Variable 'version.patch' was defined in the Variables tab
# Cron Schedules have been converted using UTC Time Zone and may need to be updated for your location
trigger:
  branches:
    include:
    - refs/heads/dev
    - refs/heads/release
  batch: True
schedules:
- cron: 0 0 * * *
  branches:
    include:
    - refs/heads/dev
- cron: 0 8 * * *
  branches:
    include:
    - refs/heads/devui
- cron: 0 16 * * *
  branches:
    include:
    - refs/heads/devui
name: v$(version.major).$(version.minor).$(version.patch)__$(SourceBranchName)
jobs:
- job: Job_1
  displayName: Agent job 1
  pool:
    vmImage: ubuntu-18.04
  steps:
  - checkout: self
  - task: Bash@3
    displayName: Generate semantic version variable with value
    inputs:
      targetType: inline
      script: 
        if [ '$(Build.SourceBranchName)' == 'dev'];

        then
            echo "##vso[task.setvariable variable=semanticversion]dev_$(version.major).$(( $VERSION_MINOR +1 )).$(version.patch)"
        elif [  '$(Build.SourceBranchName)' == 'release']

        then
            echo "##vso[task.setvariable variable=semanticversion]v$(version.major).$(version.minor).$(version.patch)"
        else
            echo "##vso[task.setvariable variable=semanticversion]tmp_$(version.major).$(( $VERSION_MINOR +1 )).$(version.patch)"
        fi
  - task: NodeTool@0
    displayName: Use Node 10.x
    inputs:
      versionSpec: 10.x
  - task: CmdLine@2
    displayName: Build MBT Command
    inputs:
      script: >-
        npm install -g mbt

        npm config set @sap:registry https://registry.npmjs.org/

        mbt build
  - task: CopyFiles@2
    displayName: Copy Artifact to Staging Directory
    inputs:
      SourceFolder: $(agent.builddirectory)/s/mta_archives
      TargetFolder: $(Build.ArtifactStagingDirectory)
  - task: CmdLine@2
    displayName: Rename MTAR by the semantic versioning
    inputs:
      script: >+
        mv $(Build.ArtifactStagingDirectory)/*.mtar $(Build.ArtifactStagingDirectory)/TEST_$(semanticversion).mtar

  - task: file-creator@6
    displayName: Create file with variables
    inputs:
      filepath: $(Build.ArtifactStagingDirectory)/variables.json
      filecontent: >-
        {
            "semanticversion": "$(semanticversion)"
        }
  - task: PublishBuildArtifacts@1
    displayName: 'Publish Artifact: drop'
  - task: git-tag-on-release-task@9
    displayName: Tag Artifacts
    enabled: False
    inputs:
      staticTagName: $(semanticversion)
...

Now, save your pipeline and run it. It will fetch the latest code from the base branch (here devui) and create a fresh build.

Build Pipeline Creation based on tasks

In this method, we will create a blank pipeline and tasks manually as shown below.

1. Click on Create a new pipeline
2. Click “Use the classic editor” from the bottom
3. Now select a source and other details as shown below:
   
4. Select “Empty Job” from the next step:
   
5. Now you have reached to the main screen where all other information for the pipeline builder will be added:
   Here you can change the project name initially and add the variables in the variable tab as we did earlier. For example I changed the builder name and added a test variable.
   

   In our use case, add these variable with given value:

   1. major = 1
   2. minor = 4
   3. patch = $[counter(variables[‘version.minor’], 0)]
6. Now, One by one add given tasks:
   
7. For Bash, these are the values, we have added:
   Display name: Generate semantic version variable with value
   Type: Inline
   Script:
   

   if [ '$(Build.SourceBranchName)' == 'dev'];
   
   then
       echo "##vso[task.setvariable variable=semanticversion]dev_$(version.major).$(( $VERSION_MINOR +1 )).$(version.patch)"
   elif [  '$(Build.SourceBranchName)' == 'release'  ]
   
   then
       echo "##vso[task.setvariable variable=semanticversion]v$(version.major).$(version.minor).$(version.patch)"
   else
       echo "##vso[task.setvariable variable=semanticversion]tmp_$(version.major).$(( $VERSION_MINOR +1 )).$(version.patch)"
   fi
   

    

8. For Node.js tool installer, make given changes:
   Display name: Use Node 10.x
   Version Spec: x
9. For Command line, make given changes:
   Display name: Build MBT Command
   Script:
   

   npm install -g mbt
   npm config set @sap:registry https://registry.npmjs.org/
   npm i -g @sap/cds-dk
   mbt build

    

10. For Copy files, add given changes:
    Display name: Copy Artifact to Staging Directory
    Source Folder: $(agent.builddirectory)/s/mta_archives
    Target Folder: $(Build.ArtifactStagingDirectory)
11. For Command line, make given changes:
    Display name: Rename MTAR by the semantic versioning
    Script:
    

    mv $(Build.ArtifactStagingDirectory)/*.mtar $(Build.ArtifactStagingDirectory)/TEST_$(semanticversion).mtar

12. For File Creator, add given changes:
    Display name: Create file with variables
    File path: $(Build.ArtifactStagingDirectory)/variables.json
    File content:
    { “semanticversion”: “$(semanticversion)”}

13. For Publish Build Artifacts, add given changes:
    Display name: Publish Artifact: drop
    Path to publish: $(Build.ArtifactStagingDirectory)
    Artifact name: drop
    Artifact publish location: Azure Pipelines
14. Till now, you have added items and variables, now you can add given Triggers:
    
15. Now go to Options tab, and make given changes:
    
    Build number format: v$(version.major).$(version.minor).$(version.patch)__$(SourceBranchName)
16. Now save and queue your pipeline, you have successfully created a pipeline. Now run it with the branch you want.

Enabling Continuous Integration in Azure Pipelines

In the above steps, at step number 14, we have added triggers. This is exactly what is known as Continuous Integration. Here, you enable it with a checkbox and add branches for which it is applied. Once the branch is changed by a Pull Request in future then the CI is automatically triggered and build is created. In the next article we will discuss regarding Continuous Deployment, the next part in CICD pipeline.

Preface – This post is part of the SAP on Azure series.

Introduction

Azure DevOps is a powerful version management tool. Its CI/CD pipeline keeps the work in flow without interruption. But sometimes, even Azure DevOps ends with errors. These error can be categorized as follows:

• Build Definition based errors: It means the error is in the Pipeline build.
• Release Definition based errors: It means the error is in pipeline release.
• Build Artifacts based errors (Error in code): It means the error is in the code. Sometimes the code is converted successfully into build artifacts (or drop files), but generates error during deployment.
• Deployment environment generated errors: It means the error is generated by external deployment environment. The error may be in pipeline configuration or the build code.

Debugging and Troubleshooting Errors on Azure DevOps

All the debugging and troubleshooting Errors on Azure DevOps starts from log. Once the deployment fails, you can see error directly in the console as shown below:

You can click the error above to read more or simply download the log and explore later, as shown below:

You can further turn debugging mode on, and run pipeline again, and then download logs for debugging. To turn the debugging mode on, you need to enable system diagnostics as shown below:

In case something went wrong, then the environment too returns log for analysis. For example, in case deployments in SAP Environment fails, they give cloud foundry commands to download the logs, something like this:

-> cf deploy -i f450e444-8632-11ea-abb3-eeee0a8f6ba7 -a monitor                                  

Updating application "uiDeployer"...

Application "uiDeployer" attributes are not modified and will not be updated                     

Uploading application "uiDeployer"...

Content of application "uiDeployer" is not changed - upload will be skipped.                     

Starting application "uiDeployer"...

Application "uiDeployer" started

Application "uiDeployer" executed

Stopping application "uiDeployer"...

Deleting discontinued configuration entries for application "uiDeployer"...

Service key "onboarding-workflow-credentials" for service "workflow" already exists

Uploading content module "onboarding" in target service "workflow"...

Deploying content module "onboarding" in target service "workflow"...

Skipping deletion of services, because the command line option "--delete-services" is not specified.

Process finished.

Use "cf dmol -i f450e444-8632-11ea-abb3-eeee0a8f6ba7" to download the logs of the process.

Then, you can use terminal/command prompt to download the logs using the command as mentioned above. This will only work, once you login into their environment using the CLI.

Now, you have successfully downloaded the log and you need to identify the issue so that you can troubleshoot, you can follow below steps to do the same:

• Download the log
• Identify the type of error, as discussed in the introduction
• In case it is a pipeline error, check where exactly it failed. If it failed during build extract, you can simply try redeployment. If security files or YAML has issues, then you need to Google and identify the fix of the same. If the pipeline itself has error like “Deployment YAML file is not found”, then again you need to check if you are providing the right configuration in the YAML or Release pipeline or not
• In case it is an error related to the code, then identify if it is a part of Frontend, Backend or Database, then accordingly assign the bug to respective developer
• In case it is an error related to the environment, then check if the environment configuration is correct or not, download the log provided by the environment to identify the root cause. In case, you are still unable to fix environment based error, then raise a ticket to respective Admin team

Common Errors on Azure DevOps

Pipeline won’t trigger

• UI settings override YAML trigger setting
• Pull request triggers not supported with Azure Repos
• Branch filters misconfigured in CI and PR triggers
• Scheduled trigger time zone conversions
• UI settings override YAML scheduled triggers

Pipeline queues but never gets an agent

• Parallel job limits – no available agents or you have hit your free limits
• Can’t access Azure Key Vault behind firewall from Azure DevOps
• You don’t have enough concurrency
• Your job may be waiting for approval
• All available agents are in use
• Demands that don’t match the capabilities of an agent
• Check Azure DevOps status for a service degradation

Pipeline fails to complete

• Job time-out
• Issues downloading code
• My pipeline is failing on a command-line step such as MSBUILD
• File or folder in use errors
• Intermittent or inconsistent MSBuild failures
• Process stops responding
• Line endings for multiple platforms
• Variables having ‘ (single quote) appended
• Service Connection related issues

References

Troubleshoot common errors in Azure DevOps CLI

DevOps: Common mistakes and how to avoid them

Troubleshoot connecting to a project

Troubleshoot pipeline runs

Review logs to diagnose pipeline issues

Preface – This post is part of the SAP on Azure series.

Introduction

Azure DevOps pipeline configuration starts with Azure Repository and Branches. We firstly create an Azure DevOps Repo, and then create our branches within it. Then only we can configure Azure Pipelines. In this article we will explore both Azure Repository and Branches.

Azure DevOps Repository

Azure DevOps Repository is a space where all the branches are created. We create separate Repos for individual projects.

Steps to Add Repository

1. Create a new Repository (You need Authorization to do so)
   
2. Assign a name to your Repository e.g. “Test Repo”, and choose Type as Git:
   

You can find your Repo, like mentioned below:

Azure Git Branches

When a new Repository is created, it just has master branch. As per the requirement of your project, you can create multiple branches. A simple flow of code includes three branches i.e. Develop, Release and Production. The Develop is the one where all the development is done, Release is where all the testing is performed by the testers and the Production is the one that has stable code, available for end users.

Adding Branches

According to your use case you can create Develop, Release and Production branch with certain policies (The branch will be on top of Master)

Enter required details:

Add in Favorite:

Adding Policies in DevOps branches

To add or change branch policies, you need to click on the three dots next to the branch and then click on “Branch policies”.

Following are some ideal policies:
1. Require a minimum number of reviewers

2. Mark “Check for linked work items” as “Required”

3. Mark “Check for comment resolution” as “Required”

4. Set “Limit merge types” as below:

We can make more changes as per the project requirement. We can read more about policies here.

5. Now add reviewers

And then add these: