SonarQube

 

Preface to Sonar

Why to use Sonar?

• With SonarQube, a developer has the proprietorship of his/her code.
• It ensures fast, efficient report generation and processing across multiple fields of analysis.
• It detects tricky issues, supports multiple languages, DevOps integrations, and centralize quality.

Is it difficult to operate?

• No, it's very easy to handle, and it gives you the result without keeping in concern how many projects you have or how complex your reporting requirements are!
• The results are shown in a very fancy UI having charts, graphs, and red-blue light.

Prologue

Welcome to the course on SonarQube (formerly known as Sonar).

You are probably wondering what SonarQube is all about? So read on to find it out.

Where can you use it?

• SonarQube can be used in any project to inspect the code. It supports more than 25+ languages.

How is it different from other analyzing tools?

• Only tool in the market that supports a leak approach as a practice to code quality.

Topics to Expect

In this course, you will come across these topics.

• What is Sonar?
• Installation of SonarQube
• Features of SonarQube
• Types of Issues
• What are the rules in SonarQube?
• Importance of Code Viewer
• Administration
• How to scan the code?

So without much delay, let's start.

What is Sonar?

SonarQube (formerly known as Sonar) is an open source platform that allows you to track and improve the quality of source code.

• It's a code analysis tool for continuous inspection of the code quality.

• One of the most important aspects in analyzing code is how much of your source code is being tested? (code coverage)

• For this, Sonar unites with the most popular open source code coverage tool ( JaCoCo, Cobertura, Emma)

• By default, it uses the JaCoCo engine.

JaCoCo

Now a question arises, why JaCoCo is selected among the three?

• JaCoCo is continuously developing and improving.
• It's fast.
• It's the only tool that analyses bytecode in a flash.
• It has its own Java agent for code analysis.

Analysis of Maven Project with Jenkins

In this video, you will see an analysis of the Maven project using SonarScanner and integrating it with Jenkins.

https://youtube.videoken.com/embed/jh7utASgKj4

Installation of SonarQube

SonarQube incorporates with Eclipse, Visual Studio, and IntelliJ ID development environment through SonarLint plugins or else you can install in your local machine.

Steps:

1. Install SonarQube.
2. Unzip the file.
3. Move it to the desired directory (e.g., opt).
4. Go to the directory, open the terminal, then run

/opt/sonarqube/bin/[OS]/sonar.sh console




Architecture

The SonarQube platform constitutes four processes:

1. SonarQube Server

It is comprised of 3 main processes viz.,

1. Web Server - to look for quality snapshots and configures the SonarQube instances.
2. Search Server - to get results from UI.
3. Compute Engine Server - to save the analysis report in the SonarQube Database.

2. SonarQube DataBase

1. The configuration of the SonarQube instance (security, plugins settings, etc.)

2. The quality snapshots of projects, views, etc.

3. SonarQube Plugins

Installed on the server.

4. SonarQube Scanners

Running on your Build / Continuous Integration Servers to analyze projects.

Integration

• SonarQube integrates with other Application Lifecycle Management (ALM) tools.

• Developers use SonarLint plugin in their IDEs and to do the local analysis.

• Developers push their code into their favorite SCM: git, etc.

Integration

• The Continuous Integration Server triggers an automatic build and the execution of the SonarQube Scanner required to run the SonarQube analysis.
• The analysis report is sent to the SonarQube Server for processing.
• The results are stored in the SonarQube Database.
• Developers review, comment, and challenge their issues to manage and reduce their Technical Debt through the SonarQube UI.
• Managers receive Reports from the analysis.
• Ops use APIs to automate configuration and extract data from SonarQube.
• JMX is being used by Ops to monitor SonarQube Server.

Architecture of Sonar

https://youtube.videoken.com/embed/z2paQm_BfKU

Prerequisite

The essential requirement to run SonarQube is Java, it should be installed in your system.

Hardware Requirements:

• The SonarQube server needs 2GB of RAM to run efficiently.
• Hard drives that have excellent read and write performance as for searching the server should be up and run to a large amount of I/O will be done.
• SonarQube JAVA analyzer is compatible with any kind of Java source files despite the version. But SonarQube analysis and the SonarQube Server require the specific version of JVM.
• It is recommended to use Oracle JRE 8 and OpenJDK 8.
• And in databases, you can use PostgreSQL 8.x/9.x l, Microsoft SQL Server-2014/2016, and MYSQL 5.6/5.7.

Getting Started

Steps to start with SonarQube:

1. Download the latest version of Sonar.

2. Unzip the file: unzip /home/<yourusername>/Downloads/sonarqube-6.7.4.zip.

3. Go to the file: /home/<yourusername>/Downloads/sonarqube-6.7.4/conf/sonar.properties

(To make changes in this file, refer to the next card).

4. Now move the sonar.properties file to "/etc/sonarqube" folder.

5. Start the SonarQube server:

   /etc/sonarqube/bin/[OS]/sonar.sh console

6. Then log into http://localhost:9000 with System Administrator credentials.

For more information, you can check here.

sonar.properties File

You need to make some changes and configure Sonar properties in sonar.properties File.

sonar.jdbc.url=jdbc:sqlserver://localhost;databaseName=sonar
sonar.jdbc.username=sonarqube
sonar.jdbc.password=mypassword

To set up server on port 9000, add the below configurations:

sonar.web.host=127.0.0.1
sonar.web.context=/sonar
sonar.web.port=9000

Now you can start the service,

sudo /opt/sonar/bin/linux-x86-64/sonar.sh start

Replace the start keyword with stop to shutdown the server.

For more information, refer configuration of database.

Installing Database

SonarQube has an inbuilt database available along with installation that supports small projects. This database can be made use of by executing the following commands.

wget https://sonarsource.bintray.com/Distribution/sonarqube/sonarqube-7.1.zip

unzip sonarqube-7.1.zip
mv sonarqube-7.1 /opt/sonar

Alternately, you could also install the database of your choice depending on the project needs. Following are the set of commands given as an example to install MySQL and make it as a SonarQube DB.

After installing MYSQL, run mysql -u root -p. Username and password would be set at the time of installation.

Let's create a sonar database by executing these commands.

CREATE DATABASE sonar CHARACTER SET utf8 COLLATE utf8_general_ci;
CREATE USER 'sonar' IDENTIFIED BY 'sonar';
GRANT ALL ON sonar.* TO 'sonar'@'%' IDENTIFIED BY 'sonar';
GRANT ALL ON sonar.* TO 'sonar'@'localhost' IDENTIFIED BY 'sonar';
FLUSH PRIVILEGES;

Overview

After the installation of SonarQube, you can install an analyzer to analyze a project. So install the Sonar Scanner (analyzer of SonarQube) and configure according to your needs.

• For Gradle Project-Scanner for Gradle

• For MSBuild- Scanner for MSBuild

• For Jenkin Project-Scanner for Jenkins

• For Maven Project-Scanner for Maven

• For Ant-Scanner for Ant

• For Azure Devops-Scanner for Azure

What Analysis Produces?

SonarQube performs analysis on more than 20 languages. The result of the analysis depends on issues and measures.

• The analysis will vary from language to language.

• A static analysis of source code is performed on all languages (.class file in java, .dll files in C#).

• A dynamic analysis is done on certain languages.

Files SonarQube will Analyze

The files that are being recognized by the analyzer will be loaded by default.

For example, if we have plugins like SONARJAVA, SONARJS, then only .js and .java files will be taken into consideration.

Behind the Scene During Analysis

Sonar Scanner is taken as a default scanner to analyze code with SonarQube.

• During analysis, the server requests data.
• Then the files are being analyzed, and then the analyzed data is sent to the server.
• The analysis of the code is done sequentially and are being queued.
• The updates sometimes take a long time, and you can see the in-progress icon in your SonarQube project.

There are some background tasks after the analysis of reports.

Configuration of Parameters

There are three types of analysis parameters.

1. Global analysis parameters
2. Project analysis parameters
3. CommandLine Parameters

Global analysis parameters are set in the UI (Administration > Configuration > General Settings).

Project analysis parameters are determined in the project analysis configuration file and override global analysis parameters.

Command line parameters are set while launching an analysis.

Sample Project Report

Why Sonar Scanner?

Sonar is the code quality platform.

• It is a server.
• SonarRunner is the old name of Sonar Scanner. It analyses the code.
• The result of code analysis is issues and measures.
• For all languages, static code analysis of source code is done.
• For only a few languages, static analysis of compiled code is done.
• Dynamic analysis is done in certain languages.

Note: For a maven-project, you do not need Sonar Scanner.

For more details on Sonar Scanner installation, you can go through this link.

Creating a Database

• Following is the SQLite3 command that is used to create a new database.

sqlite3 DatabaseName.db

Note: Database name should be unique.

• You can view the databases that you have created by running .databases command in terminal.
• You can get out from the sqlite3 through .quit command.
• You can also export the whole database into a file using .dump command.
• Following is the command that you should use to export a db to a file.

sqlite3 DatabaseName.dump > FileName

Installation of Sonar-Scanner

• Download sonar-scanner.
• Extract the zip file and move it to opt directory.

cd opt/sonarscanner/conf/sonar-scanner.properties

Uncomment the following line:

#----- Default SonarQube server
#sonar.host.url=http://localhost:9000

Then add the directory to your path:

vi ~/.bashrc
export PATH=$PATH:/opt/sonarscanner/bin

Go to the project directory and run the following in command line:

source ~/.bashrc
sonar-scanner

Use of Sonar Scanner
You need to create a configuration file in the root directory of the project namely,
->sonar-project.properties.
sonar.projectKey=project_name
sonar.projectName=project_name
sonar.projectVersion=1.0
sonar.sources=.
Then open the command prompt in your project base directory and run sonar scanner.
sonar-scanner
NOTE: Project Key should be unique.
Analysis of Java Project
Installation of Sonar Server.
Install SonarJava (other plugins if you want). By default, SonarJava is given.
Analysis: For Maven Project: (use SonarQube Scanner).
Execute the command from the root directory of the project.
mvn sonar:sonar

For Gradle projects: declare the org.sonarqube plugin in build.gradle file
Execute the command from the root directory of the project.
./gradlew sonarqube -Dsonar.host.url=[SonarQube URL]
https://youtube.videoken.com/embed/KX2FFmDPXD4




Plugins
Sonar is not just restricted to Java-based projects. It also supports a wide variety of projects and tools using a list of plugins supported to extend the code analysis, defects hunting, and reporting functionalities.
SonarQube installs available plugins from Marketplace automatically if the SonarQube is on the open internet. However, in case of lack of internet, needed plugins can also be installed manually.
As an example, the best plugin for java code analysis is SONARJAVA. To test Java code, you need certain test libraries properties. You can use additional java-specific plugins such as Cobertura or Android Lint.

Plugin for Maven Project

https://youtube.videoken.com/embed/jxgxDz930ig


Find-Bugs Plugins
https://youtube.videoken.com/embed/atNG7UiTMi8

Features of SonarQube

https://youtube.videoken.com/embed/z43QGB8Pmjk

    

Sample Project Analysis

https://youtube.videoken.com/embed/UVtQZp1n3Kw

Dashboard Overview

https://youtube.videoken.com/embed/3KhxgTsvcIM

Features of Sonar

To work smoothly with this tool, you should know some important features of Sonar.

• Leak
• Quality Gate
• Quality Profile
• Issues

You will get to understand more about these features in the forthcoming cards.

Leak

In a project, the code review comes much later in the development, and by then the collaborator (developer) would be moving on to develop newer versions of their project(s).

The best and convenient way to address code quality issues is to fix the leak ie., focussing on the parts of the code that is developed after the earlier stable release.

Quality Gate

In Sonar, you can define various rules and checks that determine the code quality. In case of any violation against these rules, it is necessary to be alerted immediately. For instance, we want coverage for new code to be more than 80%, or else we will get warnings. In this, we have many metrics like duplicate lines, maintainability rating, reliability rating, etc.

Quality Gate is the best way to enforce quality policies. You can define as many Quality Gates as many applications since quality requirement will vary for every application.

The quality gate "Sonar way" is provided by SonarSource is built-in, read-only and activated by default.

Three information measures allow you to enforce a given Rating of Reliability, Security, and Maintainability. All of these three are recommended and comes as a part of default Quality Gate.

More on Quality Gate

The Quality Gate status can be seen at the top of Project Page .

• You can also subscribe to quality gate status to get the notification.
• Quality Gates can be accessed by any user and can view every aspect of a quality gate.
• To make any changes users must be granted the Administrator Quality Profiles and Gates permission.
• A project administrator can choose which quality gates his/her project is associated with. See Project Settings for more.

• To manage quality gates, go to Quality Gates (option in the menu bar). Each Quality Gate condition is a combination of :

1. measure
2. period: Value (to date) or Leak (differential value over the Leak period)
3. comparison operator
4. warning value (optional)
5. error value (optional)

A Glance to Quality Gate

https://youtube.videoken.com/embed/wBQHloFbFE8

Quality Profile

• SonarQube works on profiles.
• The set of coding rules according to different criteria is defined through the Quality Profile related to the project.
• The Quality Profile service is main to SonarQube, as it's where you define requirements by defining rules.
• To manage Quality Profiles, go to Quality Profiles (top bar), where you'll find profiles grouped by language. To know more about Quality Profile, you can go through this link.

Quality Profile Walkthrough

https://youtube.videoken.com/embed/BsLxji7_qVs

Handling of Issues

When SonarQube needs to handle issues?

• Whenever the code breaks the rule
• Whenever the piece of code disobeys the rules you have set for the project in Quality Profiles.SonarQube will raise an issue.
• Now when the issue is raised, the developer requires to correct the code. In this video, you will see how SonarQube helps in handling the issues

Introduction to Rules

There are three basic types of rules:-

1.Reliability rules-related to bugs in a project.

2.Maintainability rules are enforced to detect code smells (code smell is an indication that there are chances of bugs, failure in future).

3.Security rules - related to vulnerability and security hotspots.

Except for security rules, others are expected to have zero or no false positive rules. To avoid this security rules they are given a different standard. As an issue is raised it is passed on to a human auditor to handle it.

You can click on the top "Rules" menu item to see rules. By default, you will see all the available rules.

Language: the language to which a rule applies.

Type: Bug, Vulnerability or Code Smell rules.

Tag:- it is possible to add tags to rules

Repository:- the engine that contributes rules to SonarQube.

Default Severity: the original severity of the rule - as defined by the plugin that contributes to this rule.

Status:- rules can have 3 different statuses(Beta, Depreciated, Ready).

Template:- display rule templates

Quality Profile:- Quality Profiles are collections of rules to apply during an analysis..

The following two actions are available only if you have the right permissions.

Add/Remove Tags: It is possible to add existing tags on the rule or to create new ones.

Note that some rules have built-in tags that you cannot remove.

Extend Description: Extending rule descriptions is useful to users as you come to know how your organization is using a particular rule for instance.

Note that the extension will be available to non-admin users as a normal part of the rule details.

Creation of Rules

SonarQube already has some minimal number of rules. You can activate/deactivate rules according to your need.

To create a custom rule you need following details:-

1. Name
2. Key
3. Description
4. Type
5. Severity
6. Status
7. factoryMethod

Adding Rules

There are two ways to extend coding rules: click the below links for more.

1. Coding rules using Java via a SonarQube plugin

2. Adding XPath rules via SonarQube web interface

Measures in SonarQube

https://youtube.videoken.com/embed/a0CuchlzPoM