Observability, the practice of using software tools to monitor and gain insights into the functioning of an organization’s software suite, dates back to the late 1950s but has gained new significance in the era of generative AI. Today, it has become a critical practice for modern enterprises, enabling them to monitor and gain insights into their software systems’ performance. However, the surge in telemetry data has led to skyrocketing costs and data management challenges. With the volume of data increasing, the cost of maintaining and analyzing this data has escalated to unsustainable levels for many businesses.
The Rising Costs of Observability
In recent years, observability vendors like Splunk and Datadog have built multibillion-dollar businesses helping organizations manage their telemetry data. This data includes logs, metrics, and traces that provide insights into system performance and anomalies. These tools are essential for ensuring that business-critical applications run smoothly, but as the volume of telemetry data has increased, so have the costs associated with managing this data.
Despite significant investments in observability, many companies face unexpected cost increases. Charity Majors, co-founder and CTO of Honeycomb, suggests that organizations should allocate about 20 to 30% of their infrastructure budget to observability. However, a 2023 survey revealed that 98% of companies experienced unanticipated hikes in observability expenses, with over half facing monthly overages. The rising cost of observability is not just a financial burden; it also complicates an organization’s ability to effectively use its data. As more resources are spent on managing telemetry data, less is available for other critical areas like innovation and development.
Enter Sawmills AI
Sawmills AI, a San Francisco-based startup, aims to address these challenges by positioning itself between observability platforms and their customers. Utilizing large language models (LLMs) and proprietary machine learning (ML) models, Sawmills AI manages telemetry data more efficiently. The company’s smart telemetry management platform is designed to consolidate, summarize, trim, and reduce the amount of data sent from customers to observability vendors, cutting costs while maintaining data quality.
Founded by Ronit Belson (CEO), Amir Jakoby (CTO), and Erez Rusovsky (CPO), Sawmills AI recently secured $10 million in seed funding, highlighting strong investor confidence. This funding will support the development of its smart telemetry management platform, which promises to reduce data volumes and costs while maintaining data quality. The platform allows businesses to harness their telemetry data at a petabyte scale but at a fraction of the cost associated with traditional observability solutions.
Addressing Data Overload
The founders of Sawmills AI discovered that a significant portion of telemetry data collected by companies is unnecessary. Through conversations with over 100 VPs of engineering and heads of DevOps, they found that companies were spending millions on observability, but only 10 to 30% of the data collected was deemed necessary; the rest was considered junk. This excess data not only drives up costs but also complicates troubleshooting and root cause analysis, making it harder for engineers to identify and resolve issues.
Sawmills AI’s platform offers a solution by analyzing and optimizing telemetry data in real-time. Key features include sampling, deduplication, routing, enrichment, and aggregation, which help eliminate wasteful data processing. By reducing the volume of data, the platform not only cuts costs but also makes it easier for engineers to work with the data, speeding up troubleshooting and improving the overall efficiency of IT operations. The company aims to address inefficiencies, mitigate costly data spikes, and improve data quality, ultimately helping businesses to reduce expenses and enhance the effectiveness of their observability systems.
Enhancing Data Quality and Sovereignty
One of the core principles of Sawmills AI is that telemetry data should be owned by the customer, not by the vendors. This approach ensures data sovereignty and allows businesses to utilize their data as needed without being locked into a particular vendor. Sawmills AI ensures that customers retain all original data, allowing them to maintain data sovereignty and access it whenever necessary.
The platform supports OpenTelemetry, enabling customers to switch observability vendors without operational disruptions. This flexibility is crucial for maintaining control over telemetry data and ensuring compliance with data governance policies. By supporting OpenTelemetry, Sawmills AI ensures that businesses can use their data as they see fit, without being restricted by vendor-specific formats or systems. The ability to switch vendors seamlessly is a significant advantage, particularly for businesses that are looking to optimize their observability practices continuously.
AI-Driven Insights and Recommendations
Sawmills AI’s platform provides engineers with AI-driven recommendations to optimize telemetry data management. These recommendations can be implemented instantly, reducing the need for manual intervention and improving efficiency. This AI-driven approach not only saves time but also ensures that the data is managed in the most effective way possible.
Automated policy management further enhances data governance, preventing overages and ensuring security compliance. By leveraging AI and ML, Sawmills AI enables businesses to gain deeper insights into their telemetry data. The platform’s capabilities include anomaly detection, data enrichment for better insights, and smart sampling policies, all of which contribute to better data quality and more efficient data management. By providing actionable insights and automating routine tasks, Sawmills AI helps businesses make more informed decisions and optimize their observability practices.
Real-World Impact
Early adopters of Sawmills AI, such as Via, have already seen significant cost reductions and improved data governance. The platform’s ability to consolidate and summarize telemetry data has proven valuable in managing data volumes and expenses. By utilizing the platform, Via has been able to streamline its data management processes, reduce unnecessary data transmission, and achieve substantial cost savings.
Sawmills AI positions itself as a complementary solution to existing observability providers, enhancing the value of their tools while alleviating the cost burden. This positioning has resonated with engineers who appreciate the functionality but often struggle with high costs associated with traditional observability solutions. By working alongside existing tools, Sawmills AI amplifies their effectiveness while also making them more affordable. The positive reception from engineers underscores the platform’s practical benefits and the need for such a solution in today’s data-driven landscape.
The Future of Telemetry Data Management
Observability, which is the practice of using software tools to monitor and gain insights into how an organization’s software functions, has roots dating back to the late 1950s. However, it has achieved newfound importance in the age of generative AI. Today, observability is essential for modern businesses, allowing them to keep tabs on their software systems’ performance and derive valuable insights. Despite its crucial role, the rapid increase in telemetry data has resulted in soaring costs and complex data management issues. As data volumes grow, the expense of maintaining and analyzing it has reached unsustainable heights for many companies.
While the advent of more sophisticated AI and machine learning algorithms has improved the capabilities of observability tools, it has also led to more data being generated than ever before. This influx of data necessitates advanced solutions for storage, analysis, and real-time monitoring, pushing many organizations to their financial and operational limits. Therefore, balancing the need for detailed observability with manageable costs and efficient data handling has become a priority. Enterprises are now seeking innovative methods to optimize how they collect and use telemetry data without breaking the bank or compromising on performance insights.
