What is CI (Continuous Integration) / Sudo Null IT News

CI (Continuous Integration)

- literally translated as “continuous integration”. This refers to the integration of individual pieces of application code with each other. The more often we put the code together and check:

Is he going to?
Are there auto tests?

All the better! CI allows you to do such checks automatically. It is used in advanced development teams that write not only code, but also automated tests. They ask him at interviews - at least an understanding of what it is. Yes, even among testers. Therefore, I will tell you in the article what it is. How CI works and how it will be useful to your project. If you prefer the video format, you can watch my video on YouTube on the same topic.

What is CI

CI is the assembly, deployment and testing of an application without human intervention.
Now I will explain with an example. Let's say that we have two developers - Masha and Vanya. And tester Katya.

Masha writes code. Adds it to the version control system (from the English Version Control System, VCS

). This is something like a dropbox for code - a storage place where all changes are saved and at any time you can see who changed what and when.

Then Vanya finishes his piece of functionality. And it also saves the code in VCS.

But this is just source code - a set of files with the extension .java, or any other. In order for Katya to test the changes, she needs to:

Compile a build from source code
Run it on a test machine

Assembling a build is when we create one executable file from a set of source code files:
You can assemble the build manually, but this is an extra hassle: you need to remember what order to run, which files depend on each other, not to make a mistake in the command... Usually they use a special program. For java it is Ant, Maven or Gradle. With the builder, you set up the build process once and then launch it with one command. Example launch for Maven:

mvn clean install This is semi-automation - you still need a person to enter the command and assemble the build manually. Let's say the developers are doing this. When Katya asks for a build for testing, Vanya updates the version from the repository and assembles the build.

But putting together a build means getting an application for testing. It still needs to be launched! This is done by the application server. There are different servers: wildfly, apache, jetty...

If this is wildfly, then you need:

Place the build in the standalone/deployments directory
Start the server (after configuring the service once)

And this is again semi-automation. Because the developer needs to copy the resulting archive after assembly to the test bench and enable the service. Yes, this is done by a couple of teams, but still by a person.

But if we remove the person from this scheme, we get CI!

CI is an application that allows you to automate the entire process. It pulls changes from the code repository. Same! There are two settings options:

CI polls the repository “Hey peek-a-boo, do you have any changes??” once every N hours/minutes, as configured.
The repository waves at CI when committing: “Hey, hi! And I got an update here!” (this is git hook or equivalent in your VCS)

When CI has received the changes, it starts the build and autotests.

If the build fails (the tests failed, or the project failed to build), the system writes an email to all interested parties:

To the project manager (so that he knows what is being done!)
To the developer who made the changes
For anyone else - as you set it, so it will be.

If the build is successful, CI deploys the application to the test machine. And as a result, Katka can test the new assembly!

Yes, of course, you will have to set it all up once - tell the CI server where to get changes from, what autotests to run, how to assemble the project, where to build it later... But once you set it up - and then it’s on its own!

You will also have to write autotests yourself, but what can you do =)

To put it simply, the CI ( Continuous Integration
) system is a kind of program that monitors your Source Control, and when changes appear there, it automatically pulls them together, builds them, and runs autotests (of course, if they are written).
In case of failure, she lets all interested parties know about it, first of all, the last committer. (With)

What is it used for?

The Common Interface on Samsung TVs is designed to integrate a module with an inserted card, which is provided by satellite television operators. Thanks to this card, the user can view paid channels that are undergoing the decoration procedure. Thus, the CI+ module is protection against illegal viewing of pay television channels. This module is designed for digital broadcasting, which is known to be of high quality.

This opinion is incorrect, since it is only an interface that provides the ability to integrate cards.

Owners of TVs with such slots can enjoy all the advantages of the interface, abandoning traditional receivers:

It is worth noting that there is only one control panel, which is already an important advantage. In the case of a receiver, the user has to use two remote controls: for the TV and for the receiver;
the cost of the CI module is significantly lower when compared with a receiver, which, among other things, needs to be allocated free space in front of the TV.

What does it look like

Let's see what this looks like from the user's point of view.
I will show you using the TeamCity system as an example. When I log in, I see all the tasks. There are different tasks:

Assemble the build
Run autotests
Deploy the application on a test bench
Run GUI tests (or Postman tests) on this bench
Notify everyone interested by email about the results of the build and testing

Tasks can be grouped. Let's say we have a CDI project. Going inside, I see the tasks for this particular project:

CDI Archetype and CDI Core
are builds. They check that the application is even built. They work it out in a couple of minutes and run it for every code change.
CDI Core with tests
- assembly of the project with all autotests, of which, as you can see in the screenshot, there are 4000+ pieces. The tests take half an hour, but they are also run on each commit.

In addition to automated launch, I can rebuild the build at any time by clicking on the “Run” button:

This is necessary to:

Restart the tests, correcting the jamb - this may be an environment setting, not a code one. Therefore, fixing the setting will not rerun tests that only monitor code version control.
Restart the build if TeamCIty is configured to check for changes once an hour - and we need to check for fixes now
Restart the build if corrections in VCS were made not to this project, but to a related one.
Check the stability of the crash - sometimes tests crash for unknown reasons, but if you restart them, they will run successfully.

When I go inside any task, I see the build history. When did it launch? Who made the changes and how many were there? How many tests passed and how many failed?

So even if I'm not subscribed to email alerts about build status, I can easily see what the current state of the system is. You open the graphical interface of the program and look.

Actions if the module does not connect

Many users are faced with a situation where the installation process is completed, but the module does not connect. In many cases, the problem can be solved within a few minutes:

The most common errors include incorrect installation of the card or the adapter itself. To eliminate errors, you should turn off the household appliance from the network, remove the adapter, install the card correctly and place the module in the slot. After this, we check the notification about connecting a new device by connecting the TV to the electricity network.
After connecting, it is possible that the channels are not configured. If the equipment is connected according to all the rules, and the TV finds a new device, but does not connect channels, you should contact your service provider for advice.
If there is no automatic connection, TV channels are configured manually.

Televisions are one of the essential household appliances in the home. They differ in size, production technology and additional functions. Among the wide variety presented on the modern market, every buyer will be able to choose the best option for himself. TVs with an additional CI module are becoming widely popular. Such a device will become a full-fledged family member in any home.

How CI works

How and where does CI assemble a build and run autotests?
I'll use TeamCity as an example, but other systems work in much the same way. TeamCity itself does not collect anything. Autotests are assembled and run on other machines called “agents”:

"Agent" is a simple computer. A piece of hardware or a virtual machine is not the point. But how does this computer understand what it needs to do?

TeamCity has a server and a client. The server is the same application in which you will then poke buttons and see a beautiful picture of “how successful everything was.” It is installed on one machine.

And the “client” application is installed on “agent” machines. And when we click the "Run" button on the server:

The server selects a free client and sends him all the instructions: what exactly needs to be done. The client assembles the build, runs autotests, collects the result and returns to the server: “Here, hold it, draw.”

The server displays the result to the user and sends an email to all interested parties.

At the same time, we always see which specific agent the build took place on:

And you can choose where to run autotests. Because it happens that autotests fail on only one build agent. This means that there is something wrong with his configuration.

Let's say that initially we had only one build agent - Buran.
The name can be absolutely anything; the administrator comes up with it when he connects a new machine to TeamCity as a build agent. We assembled the project on it, carried out autotests - everything worked. And then we bought a second machine and called it Apollo. It seems they set it up the same way as Buran, they even installed the same operating system - CentOs 7.
But when we launch the build on Apollo, it crashes. And it crashes strangely, there is not enough memory or something else. We restart on Apollo - it crashes again. We're launching on Buran - it's a success!
We begin to figure it out and find out that Apollo forgot about some setting. For example, they did not increase the number of open file descriptors. We fixed it, ran the assembly on Apollo - yes, it works, hurray!

We can also configure for each build a list of agents on which it can run.
For what? For example, half of our agents are Linux, and half are Windows. And we only assembled it for one system. Or some kind of floating bug appears on Windows, but fixing it is time-consuming and expensive, and all the clients are on Linux - well, why then? It also happens that agents are divided between projects so that there is no fight - this project uses Buran and Apollo, and that one uses Chip and Dale. The trick is that only one task can be executed on one agent. Therefore, there is no point in buying a cool car for an agent; you still can’t run a bunch of tests there at once.

In summary, how it works: first, the administrator purchases computers for “agents” and installs the TeamCity client application on them. They shouldn’t be too cool, because they won’t do many tasks at once.

At the same time, you pay TeamCity for the number of licenses for build agents. So the fewer there are, the better.

The administrator installs the TeamCity server on a separate machine. And configures it - sets up assemblies, indicates which assemblies can be raced on which cars, etc. The server needs a place to store artifacts—the results of the build.

We have two projects - Single Client and Factor, which interact with each other. The Unified Client tester may not collect the Factor locally. He starts the build in TeamCity and downloads the finished build from artifacts!

Next, the developer chooses which assembly he wants to run and clicks “Run”.
What happens at this moment: 1. The TeamCity server checks the list on which agents this assembly can be run on. Then he checks which of these agents is currently free:

Found one available? I gave him the problem!

If all agents are busy, the task goes into the queue. The queue operates on the FIFO principle - first in, first out. The one who stood up first gets the slippers.

The queue can be adjusted manually. So, if I see that the queue is full of builds that the version control system has launched, I will move mine to the very top. If I see that people have launched assemblies, that means they are also important, I’ll have to wait.

This is normal practice if there is not enough agent capacity throughout and a queue is created. Look who started it:

Robot? This means this is just a routine check to make sure nothing unnecessary is broken. This one can wait 5-10-30 minutes, no big deal
Colleague? This assembly is important to him, since he did not wait for the scheduled launch. We get in line, there’s no point in jumping ahead.

Sometimes you can even cancel the build from the version control system if it’s really hot and all the agents are busy with hour-long tests. In this case you can:

raise your queue to the very top so that it starts on the first free agent
go to the agent, cancel the current build
restart it! Even though it will end up at the very bottom of the queue, it’s not nice to just cancel the build

2. The agent performs the task and returns the result to the server

3. The server renders the result in a graphical interface and saves the artifacts. So I can go to TeamCity and look at the full logs of past autotests in the artifacts, or download the project assembly to deploy it locally.

It is highly recommended to configure in advance the number of assemblies that CI will store. Because if the artifacts contain builds of 200+ MB and there are a lot of them, then the next run of the build will fail with the error “out of disk space”:

4. The server sends out emails - it’s up to you to configure it. He can also send positive mailings “the assembly was completed successfully”, or he can send mail only in case of failure “Oh-oh-oh-oh, something went wrong!”

Continuous Deployment (CD)

The process aims to deploy a new version of an application into a production or testing environment. Usually this stage is not isolated as a separate stage; deployment is included in the delivery process. It depends on the requirements placed on it and the tools used, but, as a rule, the implementation option for the process lies on the surface.

VCS Integration

I talked about different options for setting up CI - VCS integration:

CI polls the repository “Hey peek-a-boo, do you have any changes??” once every N hours/minutes, as configured.
The repository waves at CI when committing: “Hey, hi! And I got an update here!” (this is git hook or equivalent in your VCS)

But when is which one used?

It is best, of course, for the version control system to notify the CI server. And run the entire cycle for each change: collect, test, deploy. Then any code change will immediately go to the test environment, which will be as up-to-date as possible.

Plus, every change runs autotests. And if the tests fail, it’s immediately clear whose commit broke them. After all, it worked before and after Vasya’s edits it suddenly broke - which means it was his commit that led to the crash. With rare exceptions, when the fall is floating.

But in real life such a scheme is rarely applicable. Just think - you can have many projects, many developers. Everyone commits something at least once every half hour. And if you run 10 builds for half an hour for each commit, the queues in TeamCity will never be cleared!

One of our products has a core module, and we have 15+ Customers.
Each has its own autotests. Customer assembly is core + customer features. That is, a change in the root project could affect 15 different assemblies. This means that they all need to be launched when committing to core. When we had 4 build agents, all, all, all assemblies and tests for these customers were launched on Tuesday night. And by 10 am there was still a queue at TeamCity for a couple of hours.

Another option is to purchase many agents.
But this is the price for the car itself + for a license in TeamCity, which is already much more expensive, and you also have to pay every month. So they usually do something like:

1. Very fast and important builds can be left for any commit - if it takes 1-2 minutes, let it race.

2. The remaining assemblies check whether there have been changes in the VCS - for example, once every 15 minutes. If there were, then we launch it.

3. Long tests (for example, performance tests) - once every few days at night.

conclusions

I hope you now clearly understand the differences between CI and CD, and also understand the importance of automating these processes.

To summarize all of the above, the implementation of CI/CD itself is a continuous process and is constantly modified depending on the new requirements of an individual company. At the same time, CI/CD processes require additional tools and implementation time, which can be reduced by using ready-made services to automate them.

Therefore, everyone must draw their own conclusion about the need to automate a particular process. I believe that automation of the processes discussed should be present in every company that is involved in development.

CI in testing

If we are talking about developing our own application, then testing is part of the standard cycle. You or your developers write autotests, which are then run by CI. These can be unit, api, gui or load tests.

But what if you are testing a black box? There is an application, but no source code. This is the harsh reality of integration testers - the vendor gives you a new release of the application, which you need to test before putting it into production.

Let's say you have API tests in Postman. Or GUI tests in Selenium. Is it possible to set up a CI cycle for them?

Of course you can!

CI does not set strict limits like “I only work in projects with automated tests” or “I only work when I have access to the source code.” He may or may not look at the version control system. This is an optional condition!

Have you written autotests? Tell the CI server how often to run them - and enjoy the result =)

Tricolor smart card

A smart card is a kind of key that allows access to encrypted channels. It specifies from the satellite which channels you are allowed to watch (in accordance with the paid tariff). The decoding itself is performed by the CAM module, where the smart card is inserted.

How to insert a Tricolor card correctly

The smart card must be installed in the access module slot with the barcode and the chip facing up. The correct direction will be indicated by the arrow next to the chip on the smart card (it must coincide with the direction of the arrow on the module).

Carefully insert the card into the module so that it fits evenly. There is no need to press hard.

Turn off the TV and insert the CAM module into the CI+ connector on the rear panel. In this case, the module should be located facing away from the TV.

Registration of Tricolor TV smart card

Go to the Tricolor TV website and go to the “Client Registration” section;
Enter ID number;

How to find out ID? Your ID number can be seen on the access card (below the barcode).

Next, you will be asked to enter personal information: full name, passport details, registration, phone number, etc.
The password to enter your personal account will be sent via SMS and a link will be provided where you can download the subscription agreement in electronic form.

Tariff “Unified Ultra HD Tricolor”

more than 200 channels for every taste;
including 35 channels in HD format;
and most importantly 8 channels in UHD format,

allowing you to get the most out of your 4K TV!

How much does a single Tricolor package cost? The cost of the “Unified Ultra HD Tricolor” Tariff is 2,500 rubles per year (as of June 2021).