- Key CI/CD/Jenkins Concepts
- Jenkins usage (features and functionality)
- Building Continuous Delivery (CD) Pipelines
- CD-as-Code Best Practices
- Define continuous integration, continuous delivery, continuous deployment
- Continuous Integration
Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day.
- Continuous Delivery
Continuous Delivery is a software development discipline where you build software in such a way that the software can be released to production at any time.
- Continuous Deployment
Continuous deployment goes one step further than continuous delivery. With this practice, every change that passes all stages of your production pipeline is released to your customers. There's no human intervention.
- Continuous Integration
- Difference between CI and CD
CI- The developer's changes are validated by creating a build and running automated tests against the build.
CD- An extension of continuous integration to make sure that you can release new changes to your customers quickly in a sustainable way.
- Stages of CI and CD
- CI
- Code checkout
- Code analysis, linting
- Compile
- Unit test
- Integration test
- Package artifacts
- Publish artifacts
- CD
- Provision environments
- Deploy artifacts to environments
- Performing tests
- Automated UI test
- Performance and load test
- Smoke test
- Acceptance test
- Manual release to production (Optional, for Continuous Delivery)
- Monitoring and telemetry
- CI
- Continuous delivery versus continuous deployment
Continuous delivery- a software development discipline where you build software in such a way that the software can be released to production at any time. Every change is delivered to a staging environment using complete automation. It can be deployed to production with a push of a button when the business is ready.
Continuous deployment- the next step of continuous delivery: Every change that passes the automated tests is deployed to production automatically.
- What are jobs in Jenkins?
- Types of jobs
- Freestyle
- Maven
- Pipline
- Multi-configuration
- Multibranch pipeline
- External job
- Scope of jobs (Can't find this item)
- What are builds in Jenkins?
- What are build steps, triggers, artifacts, and repositories?
- Build steps are the basic building blocks for the Jenkins freestyle build process. They are what let you tell Jenkins exactly how you want your project built
- Triggers are how you tell Jenkins when to kick off a build
- Artifacts are files which are associated with a single build. Artifacts can be used to represent data created as a side-effect of running a Jenkins build.
- Build tools configuration
- What are source code management systems and how are they used?
- Cloud based SCMs
GitHub, Bitbuckte, AWS CodeCommit, and etc.
- Incremental updates vs. clean check out
Incremental build: if it doesn't think it needs to rebuild a project, it won't. It may also use partially-built bits of the project if they haven't changed
Clean solution will remove the build artifacts from the previous build
- Checking in code
To check in (ci or, commit) is to write or merge the changes made in the working copy back to the repository. The terms 'commit' and 'checkin' can also be used as nouns to describe the new revision that is created as a result of committing.
- Infrastructure-as-Code
Infrastructure as code (IaC) is the process of managing and provisioning computer data centers through machine-readable definition files, rather than physical hardware configuration or interactive configuration tools
- Branch and Merge Strategies
ABCs of a Branching and Merging Strategy is a good article. The most commonly used strategies are Gitflow and GitHub Flow
- Benefits of testing with Jenkins
Test Automation software is the best way to increase the effectiveness, efficiency and coverage of your software testing. ... Once automated tests are created they can easily be repeated and they can be extended to perform tasks impossible with manual testing (see also Top 10 Benefits of Automated Testing)
- Define unit test, smoke test, acceptance test, automated verification/functional tests
Unit test- a software testing method by which individual units of source code, sets of one or more computer program modules together with associated control data, usage procedures, and operating procedures, are tested to determine whether they are fit for use. (Wikipedia)
Smoke test is preliminary testing to reveal simple failures severe enough to, for example, reject a prospective software release. (Wikipedia)
Acceptance test- a formal testing with respect to user needs, requirements, and business processes conducted to determine whether a system satisfies the acceptance criteria and to enable the user, customers or other authorized entity to determine whether or not to accept the system. (Wikepedia)
Functional test is a quality assurance (QA) process and a type of black-box testing that bases its test cases on the specifications of the software component under test. (Wikipedia)
- Types of notifications in Jenkins
Email, RSS, IM (IRC, Slack, and etc.), HTTP call
- Importance of notifications
Better visibility, more transparent, faster feedback loop, continuous improvement, and etc.
- What are distributed builds?
The reason being that workloads are often best "farmed out" to distributed servers. This may be for scale, or to provide different tools, or build on different target platforms. Another common reason for remote agents is to enact deployments into secured environments (without the master having direct access).
- Functions of masters and agents
A "master" operating by itself is the basic installation of Jenkins and in this configuration the master handles all tasks for your build system.
An agent is a computer that is set up to offload build projects from the master and once setup this distribution of tasks is fairly automatic.
- What are plugins?
Jenkins defines extensibility points, which are interfaces or abstract classes that model an aspect of a build system. Those interfaces define contracts of what need to be implemented, and Jenkins allows plugins to contribute those implementations.
- What is the plugin manager?
The Web UI to manage Jenkins plugins.
- How to interact with it
BASH (curl), PowerShell (Invoke-WebRrequest or Invoke-RestMethod), web client, etc.
- Why use it?
Interact with Jenkins programmatically
- Authentication versus authorization
- Matrix security
- Definition of auditing, credentials, and other key security concepts
An IT audit is the examination and evaluation of an organization's information technology infrastructure, policies and operations. Information technology audits determine whether IT controls protect corporate assets, ensure data integrity and are aligned with the business's overall goals.
Credentials refer to the verification of identity or tools for authentication. They may be part of a certificate or other authentication process that helps confirm a user’s identity in relation to a network address or other system ID. Other key security concepts from Wikipedia
- What are fingerprints?
Jenkins supports file fingerprinting to track interdependencies of artifacts in different projects.
- How do fingerprints work?
The fingerprint of a file is simply a MD5 checksum. Jenkins maintains a database of md5sum, and for each md5sum, Jenkins records which builds of which projects used. This database is updated every time a build runs and files are fingerprinted.
- How to use artifacts in Jenkins
- Archive artifacts in upstream projects: In the Jenkins project (or job) configuration, create a Post Build Action, select Archive the artifacts, also in General section, set the Permission to Copy Artifact
- Use the archived artifacts in downstream projects: In the Jenkins project (or job) configuration, create a Build step, select Copy artifacts from another project
- Storing artifacts
It is being archived on the master server (even if the build were on a slave) in the following folder:
$JENKINS_HOME/jobs/<job>/builds/<build>/archiveBut you can configure a different location using the 'Advanced' setting of the job (where you can set a different workspace folder) or using plugins that are made for this purpose such as Copy Artifact Plugin (Stack Overflow)
- How to use 3rd party tools
Not very sure what is the scope of 3rd party tools. A lot of plugins provide integration with 3rd party tools. Also 3rd party tools can use Jenkins Rest API or Jenkins CLI to work with Jenkins.
- What is the Jenkins Installation Wizard?
It is the post-installation setup wizard. It takes you through are a few quick "one-off" steps to unlock Jenkins, customize it with plugins and create the first administrator user through which you can continue accessing Jenkins.
- How to use the Wizard?
- Which configurations are covered by the Installation Wizard?
Unlock Jenkins by providing the Administrator password. Customizing Jenkins with plugins. Creating the first administrator user
- Organizing jobs in Jenkins
Views (my view or custom view) and folders are the ways to organize projects
- Parameterized jobs
Use parameterized build, The parameters are available as environment variables. The Parameterized Trigger Plugin lets you trigger new builds when your build has completed, with various ways of specifying parameters for the new build.
- Usage of Freestyle/Pipeline/Matrix jobs
Matrix jobs are Multi-configuration projects now.
Freestyle projects are good start point and can be treated as an empty template. Pipeline in Jenkins is a suite of plugins that supports implementing and integrating continuous delivery pipelines into Jenkins. Pipeline provides an extensible set of tools for modeling simple-to-complex delivery pipelines "as code" via the Pipeline DSL.
- Setting up build steps and triggers
In freestyle projects, build steps are defined in configuration (build section). In pipeline projects, build steps are defined by Pipeline DSL.
Build triggers are defined in the project configuration (Build Triggers section). The triggers can be another upstream project, a schedule, a SCM hook, polling SCM changes, or an API call.
- Configuring build tools
You can configure build tools in Manage Jenkins, Global Tools Configuration.
There are sections for Maven, JDK, Git, Gradle, Ant, Docker, and etc.
- Running scripts as part of build steps
In freestyle projects, you can select running scripts as a build step.
In pipeline projects, you can run scripts via the
shdirective.
- Polling source code management
The schedule of polling SCM can be defined with cron job syntax (with minor difference) with additional time zone specificcation)
- Creating hooks
There is a great tutorial from Atlassian, the maker of Bitbucket, on Git hooks.
- Including version control tags and version information
How do you achieve a numeric versioning scheme with Git from Stack Overflow
- Testing for code coverage
For Java projects, Cobertura plugin is a tool to generate code coverage reports
- Test reports in Jenkins
For Maven projects, there is no extra configuration required. For Java projects, user needs to add the Publish JUnit test result report as a Post-build Actions. For projects in other languages, similar configuration is required after adding xUnit plugin.
- Displaying test results
Jenkins displays test result after getting the test reports. The result includes trend, test list, test details, and time costs. (from book Jenkins: The Definitive Guide)
- Integrating with test automation tools
- Breaking builds
- Setup and usage
There are different notification strategies, and different notification channels.
A good aproach is to notify the one who caused build failure first, then notify others when the failure repeats. The notification channels include email, editable email, RSS feed, instant messaging, SMS, desktop notification, noisy sound, and some special devices
- Email notifications, instant messaging
There are regular email notification and (advanced) editable email notification.
The instant messaging notifications include Jabber, IRC, Slack, Trello, and etc.
- Alarming on notifications
Use plugins to make noisy sound alarm?
- Setting up and running builds in parallel
- Setting up and using SSH agents, JNLP agents, cloud agents
To use SSH agents, put master's ssh public key in slave's
~/.ssh/authorized_keysdirectory. To use JNLP agents, initial the connection request from slave. There are several plugins for cloud agents, such as for AWS, GCP, or OpenStack - Monitoring nodes
You can check the status of nodes in Configure Jenkins -> Configure nodes. You can also set the retention strategy for nodes
- Setting up and using Plugin Manager
Managing Plugins page addresses Jenkins Plugin Manager
- Finding and configuring required plugins
From Plugin Manager's available tab, upload an
.hpifrom advanced tab, or using the Jenkins CLI
- Using Pipeline (formerly known as "Workflow")
- Integrating automated deployment
- Release management process
- Pipeline stage behavior
- Using REST API to trigger jobs remotely, access job status, create/delete jobs
- Setting up and using security realms
- User database, project security, Matrix security
- Setting up and using auditing
- Setting up and using credentials
- Fingerprinting jobs shared or copied between jobs
- Copying artifacts
- Using artifacts in Jenkins
- Artifact retention policy
- Making basic updates to jobs and build scripts
- Troubleshooting specific problems from build and test failure alerts
- Value stream mapping for CD pipelines
- Why create a pipeline?
- Gates within a CD pipeline
- How to protect centralized pipelines when multiple groups use same tools
- Definition of binary reuse, automated deployment, multiple environments
- Elements of your ideal CI/CD pipeline - tools
- Key concepts in building scripts (including security/password, environment information, etc.)
- Triggering jobs from other jobs
- Setting up the Parameterized Trigger plugin
- Upstream/downstream jobs
- Triggering Jenkins on code changes
- Difference between push and pull
- When to use push vs pull
- Benefits of Pipeline vs linked jobs
- Functionalities offered by Pipeline
- How to use Pipeline
- Pipeline stage view
- How to control access to items in Jenkins with folders
- Referencing jobs in folders
- Setting up test automation in Jenkins against an uploaded executable
- Passing parameters between jobs
- Identifying parameters and how to use them: file parameter, string parameter
- Jenkins CLI parameters
- Promotion of a job
- Why promote jobs?
- How to use the Promoted Builds plugin
- How to radiate information on CD pipelines to teams
- Usage of Multibranch jobs
- Scanning GitHub and BitBucket Organization
- Scanning basic SCM repositories
- How to share code across Pipelines
- Usages of the Shared Libraries
- Interaction with Folders and Repository scanning
- Security and Groovy sandbox
- Distributed builds architecture
- Fungible (replaceable) agents
- Master-agent connectors and protocol
- Tool installations on agents
- Cloud agents
- Traceability
- High availability