8 Essential DevOps Tools - Learn What Separates Good Engineers from Great Ones

Here at Netguru, we are on the bandwagon cheering the hype. We believe DevOps brings a set of practices that can help you build a scalable, stable, and secure software architecture fast. It also offers sysadmins some completely new career opportunities. This is why we’ve prepared this guide to show you 8 DevOps tools the Netguru team can't imagine working without.

We understand DevOps as a set of practices that enhance trust, speed up development, resolve issues, and increase project security. DevOps emerged from the spread of Agile methodology, which is deeply rooted in the foundations of Netguru's culture.

DevOps Explained

Agile software development has become so popular because it makes projects and organizations much more adaptive to the changing environment. With the new bespoke software development culture, SysAdmins and SysOps have evolved into DevOps. In the DevOps methodology, all members of the development team are engaged in the project and can participate in the work throughout the product's lifecycle.

The DevOps boom created a new recruitment challenge for tech companies and new opportunities for experienced specialists. Companies like Google, Amazon, or LinkedIn are looking for people who know how to use interconnected tools. This is the knowledge that separates good DevOps engineers from great ones.

Gaining experience with different configurations, and different scales and categories of applications is crucial, and Netguru is one of the best places in Europe to achieve it. We build and support various apps for top-tier clients from all over the world. Our small DevOps team led by an Ex-Googler is responsible for supporting over 100 projects!

Here's the DevOps stack we use at Netguru – let’s see the 8 tools our team cannot imagine working without.

1. Docker

Docker lets you put all the parts of an application into one package, which is an innovative solution in software engineering, called "containerization". All the relations and functions are enclosed in a Docker package and are independent of the server they are installed on. Thanks to Docker, you can move the software and connect it with external applications. Docker is an open platform for developers and sysadmins to build, ship, and run distributed applications, whether on laptops, in data center VMs, or in the cloud.

Docker itself is growing fast. The company developing the tool has achieved the status of a unicorn, being worth over $1 billion. Recently, it has received $92 million in new fundingand is targeting a $192-million funding round. This proves the tool is likely to stick around in the future and has a chance to become the standard, as it brings the value to the businesses that use it. Companies like PayPal, Visa, or Societe Generale use the Docker Enterprise container platform for their projects.

At Netguru, we use Docker to isolate applications for easy local development. When we have to deploy a Ruby on Rails application that requires a specific version of Operating System dependencies, we don’t have to worry about installing the dependencies on the server host, and versioning isn’t an issue either. We simply ship the application with all the dependencies it requires already contained within the Docker image. Additionally, we don't have to worry about the version of ImageMagick on the host machine, because it has already been shipped with the relevant version within the Docker image.

2. Terraform

Terraform lets you define server infrastructure at the code level. The concept is called "infrastructure as code". You can write a few short configuration files and create an execution plan that will send out a complete configuration to the server.

Usually, these tasks are done manually or with the use of in-house solutions. Terraform really makes it easier to set up infrastructure. The project is developed by HashiCorp and is available in both standard and enterprise versions, while the language used in Terraform is called HCL (HashiCorp Configuration Language).

Terraform enables us to use the Infrastructure as Code process. With Terraform, we can define the infrastructure in a way that can be easily reproduced, tested, verified, and audited if necessary. Thanks to managing infrastructure with code, we are able to automate some parts of the setup and deployment, which saves us lots of time.

3. Ansible

Ansible aims at automating software provisioning, that is how software is delivered and assessed by different teams. The tool allows configuration and deployment management.

NASA used Ansible to move 65 applications from a traditional hardware-based data center to the cloud. The migration needed to be done very quickly, so the apps were transferred "as-is". As a result, the time needed to update nasa.gov went down from 1 hour to 5 minutes.

In Netguru, we use Ansible to define further stages of the environment directly with code. After the initial configuration, we can perform further configuration of the server, where we define what should be installed , what kind of user roles to grant, etc.

4. AWS Amazon

Amazon is the leading cloud service provider, and Amazon Web Services sales are growing fast and have already exceeded 10% of the tech giant's revenue. There are many AWS solutions we use at Netguru as it's our default service provider, which is no surprise considering its dominant market position. Thanks to AWS, we can provide load-balanced, reliable infrastructures that scale automatically.

Here's a shortlist of the Amazon services we use the most:

• with Relational Database Service (RDS) we set up and maintain SQL Databases, such as PostgreSQL or MySQL;

• Elastic Cloud Computing (EC2) is our Linux server stack setup, which comes with load balancing, autoscaling, and many other fantastic features;

• Elastic Container Service (ECS) is a Docker-based application orchestration and hosting tool;

• Elastic Container Registry (ECR) is a private registry for Docker images;

• by using Elasticache we can set up key-value databases, such as Redis or Memcached;

• Cloudfront Content Delivery Network (CDN) is essential for caching web assets in endpoints – it allows us to load websites much faster and reduces unnecessary requests to the server;

• Simple Cloud Storage Service (S3) stores our assets, relevant files, and hosts static websites;

• Route53 helps us manage DNS records or buy new domains;

• we use Certificate Manager for requesting free SSL certificates from AWS and attaching them to Cloudfront or Loadbalancer.

There are many other AWS services we use, but we don't want Amazon to dominate this post too much.

In the DevOps methodology, all members of the development team are engaged in the project.

5. Datadog

Datadog is a New York-based software company that provides a complex SaaS, which we use for monitoring servers and resources, executing queries, and building dashboards and graphs.

Thanks to Datadog, we are able to monitor a cluster of over 150 servers – CPU, Memory, IOPS, storage usage, and more. We also create alerts based on queries that allow us to inform stakeholders of potential issues. The communication can go through various channels, but we chose Slack.

The Datadog service is agent-based. If an instance goes down, we are informed about it, because Datadog alerts us about the lack of response from an agent.

6. Graylog

Every app produces logs. With time and scale the logs grow bigger, and keeping them in order becomes a true challenge. Graylog is a log management tool. The logs are centralized and easily searchable. What's most important is that the system is ready for any scale and compliant with many legal requirements. Graylog is our main tool for log aggregation and analysis at Netguru.

7. Let's Encrypt

Let's Encrypt performs a small but very useful task – it generates SSL certificates for free. Unlike some of the services above, Let's Encrypt is not a huge business. It is developed by a small team of eight people. Let's Encrypt is a free, automated, and open certificate authority.

At Netguru, we use Let’s Encrypt to generate SSL certificates for our staging environments. The practice helps us get a structure as similar to the production environment as possible. Additionally, we use Let’s Encrypt when an SSL certificate expires or when we have to create an infrastructure with dozens of different domains, in situations which a wildcard (a public key certificate for multiple sub-domains) couldn't handle.

We also use a tool to enhance this tool: Certbot. Certbot is a very handy solution for generating certificates for Let's Encrypt.

8. PagerDuty

Finally, Pagerduty, a tool for incident resolution. Being responsible for a web service is a serious job. Even if it's a piece of art perfectly made, you need a qualified DevOps team to be available in case of emergency, and you need a process for real-time incident detection and automatic response. PagerDuty can be integrated with over 300 tools. It uses machine learning to notify on-call engineers about only the important signals, which probably are incidents. PagerDuty automatically sends alerts to your team. You can also adjust the alert channels and adapt them to your organization’s culture as well as the legal environment. Pagerduty can call you, send a text message, or an e-mail... or everything at the same time.

PagerDuty has recently closed a $90m Series D, which increased the company’s value to $1.3 billion. Yet another unicorn startup in the DevOps tools world.

We use PagerDuty to create teams with different schedules and alert levels. It's very useful in managing alerts. Thanks to PagerDuty, we can reduce the support response time and increase the uptime of our applications.

PagerDuty the last tool we find indispensable today. It may change tomorrow because our DevOps stack is constantly changing. We are exploring new opportunities, discussing possible solutions, and we always focus on using the best set of available tools.

If you are looking for a tech team for your project or a company you'd like to work for, and you'd like to get to know more about our current stack of top-notch DevOps solutions and integrate them into a perfect ecosystem managing teams and software, you should definitely talk to us about that. We will be happy to help.

If you have any tools you think we should add to this list, let us know.