Rookout: Scalable and Agile Observability
Today, we are honored to welcome Liran Haimovich to the show. Liran is co-founder and CTO of Rookout, a company that builds a robust live debugging and observability platform which collects data on-demand from live code and pipeline.
Liran Haimovich
Co-Founder&CTO at Rookout
Liran, can you tell us the difference between the offering Rookout offers to the market and other traditional observability tools?
Traditional observability tools fall into one of two options for instrumenting your code. The first approach is that you-do-it-yourself approach. You get a set of APIs, whether it’s a logging framework, or metric framework, or an APM, and it’s up to you to decide what to collect. There you can deep dive into your business logic and tailor the data you’re collecting based on your need. The downside is that every small change means changing the code, testing the change, releasing and deploying a new version that will take from a few hours to a few months, depending on your release cadence.
The second type of instrumentation is the one provided by vendors. There you get free instrumentation and don’t have to write any code. But because a third-party vendor wrote this code, you will get very generic feedback about HTTP requests, about databases. It’s not going to go down deep into your business logic, into the decisions your code is making.
Rookout is bringing the best of both worlds by bringing the automated nature of APMs, along with the surgical nature of coding, so you get to select the data you want to get, and you don’t have to worry about overhead because you only pay for what you collect. It’s much more lightweight and accurate. And you can go as deep as you want in an agile manner. We do the instrumentation, but we allow you to control it. You can instruct us to instrument whatever you want with a click of a button – if you want a new logline, just click on the line, and we’re going to add a logline on that line instantly.
Does it mean you can scale up and down the tool’s performance and only pay for the data you’ve retrieved?
That’s exactly right. When we meet small companies, they usually tell us they’re running with all the logs enabled, and they’re running in debug or trace repository. They can afford it because it’s a new product that has very little traffic, very few transactions; therefore, you’re not worried about the costs. Though, once you scale up, then all of a sudden, we can only afford to operate at info level or sometimes even at the error level.
The agile nature of Rookout powers you with the ability to select data in real-time and ultimately takes away that fear of missing out.
Because even if the log lines you’ve initially decided on are not going to be the ones you need, you will still have the ability to determine what you need right now and instantaneously turn it on. This ability allows many of our customers to be much more agile in observability posture.
What is it about the move to the cloud that makes the application so difficult to understand?
A few things change about the move to the cloud and move to the cloud-native applications. First and foremost, SaaS allows us to deliver software in a much more efficient manner. The new cloud-native paradigm is allowing us to build ever more complex and scalable applications. It will enable us to manage that complexity and break it down both from the tech side of containers and from the people side, allowing engineers to have their areas of operations and ownership.
That complexity also means engineers spend most of their time testing and developing in the cloud instead of locally. It means more tests being carried out in integration environments. From there, it becomes harder to emulate real life in a lab. Therefore, we need to provide engineers with tools to operate in those environments, move fast, build high-quality software, release it and support it with better tools that operate at the code level.
I hear you recently integrated with the Open Telemetry project and build new visuals there. Can you tell us a little bit more about that?
We’re big fans of Open Telemetry and use it internally to operate and understand our SaaS platform. Collecting the Open Telemetry information allows you to correlate. Once you have the transaction ID from Open Telemetry, you can also correlate it with your APM, your logs, and it’s easy to cross-reference the data from Rookout to other tools. And as an added benefit for that, Rookout allows you to export data, export new log lines, include transaction data, and create new metrics even on the fly. As you export all the data to other services, you can seamlessly integrate, cross-reference, and visualize both traditional logging, and Rookout generated logs on the fly using the same visualization tools.
Does contributing to the open-source ecosystem help you to accelerate the pace of building your solutions?
Rookout works closely with the runtime developers, the people behind Python, Ruby, and Java. We often find new features that might not be working as expected, and we need to report those bugs or occasionally open a suggested fix for them. To be part of that community is critically important for us as we are making instrumentation and other code manipulation technologies more manageable and more accessible for everyone.
Once Rookout is installed in production, it works with sensitive client data. How do you guarantee the protection of the data collected by Rookout services? And how do you comply with security standards like GDPR?
I spent a decade of my life doing cybersecurity, and it’s one of the topics we as Rookout take great pride in. Rookout is an excellent tool from a security standpoint. We offer a policy-based, fully audited approach for enabling secure access to the data you need.
We take a lot of compliance burden upon ourselves. On top of tons of compliance, including data reduction, role-based access controls, and audit logs, we’re ISO 27001 certified and GDPR, HIPAA compliant. We’re more than happy to process that data on your behalf and keep it very secure.
Because many enterprises still prefer to keep the data on their behalf, we have a hybrid SaaS solution where we separate the data plane from the control plane. We do all the heavy lifting around database management, indexing, backup being authorization authentication, or the business logic, all the software upgrades. You just have to install a couple of very lightweight, simple containers. Those two containers will keep all your data within your premise, whether it’s a VPC in the cloud or a data center. And once you do that, you maintain full data governance over your data. We never process it on your behalf, and you can rest easy knowing you’re not giving up anything; you’re still in complete control of your data. You can pipeline, segregate, and process it in whatever part of the world you want, without relying on us at all.
There are many traditional solutions for monitoring and observing traditional applications. However, when it comes to serverless, there are not so many instruments available. How Rookout makes serverless more easily maintainable in production?
In general, serverless is shifting many paradigms around software engineering from the server-oriented model to a purely code-centric operational model. If you’re looking at the more dynamic languages such as NodeJS, or Python, you’re literally handing over your source files, and then AWS or Google is running them. That’s shifting the entire software development lifecycle, from writing the code to building it, to operating it, monitoring it.
As part of the change, we lose a lot of control and visibility into the lower layers. At the same time, this is an excellent fit for Rookout because we’re very code-centric, unlike many other traditional tools. We’re focusing on code, allowing you to see what’s going on within. And in the serverless model, there is nothing but code.
As tech stacks are becoming more and more production, it becomes more difficult to spin up an environment locally for testing and development. In serverless, it’s more accurate than anywhere else as it can be almost impossible to spin up a decent development environment locally, even for the simplest projects. Many customers rely on Rookout for debugging services in the development and later stages of integrating, testing, and running in production.
How difficult is it to find the right talent to build an innovative company culture?
At Rookout, the team has always been a massive priority for us. It starts with hiring the right people and nurturing them, helping them grow within the company, helping them achieve their goals, both within the company and later on as they leave.
We’re investing a lot of effort in the hiring process. With hiring so difficult, you have to get the right people interested in what you are building for your customers. So, we want people to love technology and care about it. At the same time, it’s also essential for us that that obsession with technology has some boundaries defined by the ability to release the application that may not be perfect but at this current moment satisfies the need of our customers.
Liran, it’s been an absolute delight to have you on the show. Thank you so much for taking the time.
Stay tuned for more great interviews coming your way!