Sep 21, 2020, by Chris
This article aims to provide an additional layer of security to your nodes according to swiss cheese model by preventing SSH login via public WAN. Using VPN and (V)LANs both relay- and block-producing nodes get isolated from the web.
This article will provide additional tips about protecting your cardano relay- and block-producing nodes by using VPN, (V)LANs and one additional firewall node against malicious attacks.
The purpose of this guide is to add one additional "layer of security" to your relay- and block-producing nodes according to swiss cheese model.
But please note that there is no 100% security! Also after reading this guide you will not be immortal against hacking attacks. As a stake pool operator, the best way to assure safe and reliable nodes is to monitor them regularly. Also, one efficient way is to imagine yourself in the role of a hacker intruding into your own stake pool. Often, this role play helps to identify security leaks in your own setup and topology.
According to Security Magazine, globally there is one hacking attempt every 39 seconds. About 300,000 new malware are created every day (Source: Cybersecurity Ventures) and in average, hackers steal 75 records every second (Source: Breach Level Index). And finally, Denial of Service/Distributed Denial of Service (DoS/DDoS) attacks are counted among the most often used hacking techniques.
In this posting, junada, the stake pool operator of the korean pool HAPPY describes, how his stake pool was hacked.
We hope this never happens again to any stake pool out there.
Most of the stake pool operators are no security experts. Therefore they spend lots of free time to read manuals, study documentations and guides in order to setup stake pools as good as possible.
Very often it is also required to invest a certain amount of money to have the proper hardware which is capable of running reliable relay and block-producing nodes with low latency and high uptime.
By setting up such stake pools we put our efforts, time and money into improving the decentralized topology of cardano.
Thanks to all SPOs out there for all your efforts and keep up your outstanding work and keep in mind that cardano is only as secure as the block-producing nodes we create!
DoS/DDoS is a classic technique used to bring down systems or networks by overloading them with login attempts, data requests, repetitive tasks, etc.
Attacks range from the fairly basic (configuring a system to continually bombard a site or server with requests), to the orchestrated (infecting a multitude of systems with malware to form a "botnet" that proceeds to flood a target network with unmanageable traffic), to the specific and sophisticated (buffer overflow attacks which allow hackers to gain access to personal information by filling online form fields with excess data, so they freeze up).
A successfull DoS/DDoS attack could lead to your node becoming unresponsive and thus missing blocks.
Another technique often used is trying to scan for unprotected SSH ports and to get access to the server by brute force attacks.
SSH Intrusion is one of the most frequently used methods of gaining access to a server.
Right now, they are also working on an article which should teach pool operators how to properly secure their keys and sign transactions offline.
To sum up the published article:
Also, the official cardano documentation suggests the above mentioned methods. One should also use a different port than the default TCP port 22.
We also would like to emphasize that the above mentioned items are the least you should do to protect your server against unauthorized access.
However, when supporting other stake pool operators, we often notice that many of them follow the advice of using RSA/key method for SSH authentication BUT they do not protect the web-based server control panel which offers their provider!
Please keep in mind that most server control panels provide full root access to your node. You should also try to protect your web-based server control panel as good as possible, ideally by 2FA (2-Factor-Authentication).
Also check if it is possible to restrict access to your web-based server control panel to a certain range of IP addresses.
There are additional ways of increasing the safety of your node. In an ideal scenario the login prompt of SSH should only be accessible and visible to you - and to nobody else.
The easiest way to achieve this is by defining a firewall rule which allows access to SSH only from your unique IP address.
The contra is: This method only works when you have a static IP. Most stake pool operators have internet connections without static IPs!
If you have a dynamic IP address and create a firewall rule that allows access from only your specific IP address, one day, you will lock out yourself from your node.
What we also often notice is that block producing nodes are exposed directly to the internet although it is suggested that block producing nodes are isolated from the web and the only link they should have is one to the relay node.
In most cases, direct access from internet to block-producing nodes was necessary because otherwise the SPO didn't have any chance to administrate this server node.
For your understanding:
It is a security benefit when there is a relay node which protects the block-producing node against direct access from the internet.
However, this security benefit will vanish once the block-producing node is reachable from the internet the same way as the relay node.
The problem is: In many cases, removing the direct access between internet and block-producing node will also remove the possibility to administrate this block-producing node. In this cases, the SPO does not have a chance to login via SSH anymore.
In this example we would like to show a method of isolating both your relay node(s) and block-producing nodes from direct access from the internet while still having full control via SSH.
The benefit of this method is: Regardless of which method you are using for authenticating via SSH (username/password or RSA/key): there is no direct access via public WAN possible to SSH of your relay node or block producing node.
When comparing this with swiss cheese model you add one additional "slice of security" to your setup of nodes.
In contrast to a traditional setup, this setup requires one additional machine and the possibility of creating a VLAN or physical LAN between all of your three machines (firewall, relay node, block-producing node).
For sake of simplicity we leave this example at one relay node.
In this example the role of the firewall machine will include:
It will be beyond the scope of this article to explain the firewall node setup in detail. This guide should only give you an idea about how you could increase the security of your node topology.
Nevertheless we would like to share some must-items when configuring your firewall.
The main benefits of the above mentioned solution are that no SSH ports are available at all to the public - they will be only available for requests coming from your local via VPN tunneled subnet - and that your relay node as well as your block-producing node will be isolated as suggested from the internet. Nevertheless, you will have comfortable access via SSH to administrate the machines.
We hope this guide gives you some helpful ideas of increasing the security of your stake pool.
Please feel free to contact us if you have any questions. We also appreciate any feedback.
Have a great day!
written by: Chris published at: Sep 21, 2020
Feb 2, 2021 by Chris
Cardano's [CAFE] and [CAFE2] are updated to newest version 1.25.1. For all stake pool operators (SPOs) it is important to know that with release of 1.25.1 the mainnet-config.json in the repository of CF / IOHK was consolidated and is a lot slimmer than previous! Some names of statistic variables have changed. In most cases the wording "ChainDB_" was removed. So be sure to update your grafana dashboard so that you can still enjoy your statistics after update to 1.25.1.
Nov 21, 2020 by Chris
Prometheus and grafana are helpful tools to monitor your nodes. They have a built-in alert function which notifies you about arising problems. This guide will show how to setup your infrastructure to do this monitoring with a separate machine thus not requiring additional installations on your vital cardano nodes.
Nov 8, 2020 by Chris
RTView is a new monitoring tool released by IOHK. When configuring or installing RTView, you might need the help of yaml to json converters.
Oct 3, 2020 by Chris
A great way to help new stake pool operators in setting up their own stake pool is learning from the docs of the official Cardano Stake Pool School. It was launched by the Cardano Foundation in August 2020.