I have spent over two decades in the data analytics space, and during this time, I have spoken to hundreds of clients who have expressed a strong desire to transform their company into a data-driven organization. Yet only a handful of them are confident that they are on the right track to achieve this goal. Why is this the case? A common mistake I’ve witnessed in most companies is introducing new tools that demand their employees acquire new skills and completely alter their work processes, rather than helping them perform their job more efficiently with minimal changes.
This is why I am thrilled with our recent partnership between data.world and Sigma Computing. Both companies are leading providers in the modern data stack landscape, each offering unique differentiations - data.world as the only enterprise data catalog platform powered by a knowledge graph built to harness the power of the AI future, while Sigma sets itself apart as the most flexible BI platform. We share the same vision: creating a platform that all data users genuinely love and actively use. We believe that widespread adoption is the key to success - only tools that are embraced across organizations can help cultivate a data-driven culture.
The integration between data.world and Sigma aims to bridge the trust gap that often hinders companies from progressing toward their data-driven goals. By seamlessly connecting these two platforms, data teams will now benefit from improved communication efficiency, streamlined data discovery, and enhanced trust through increased visibility and transparency. Let’s dive into the details to understand each of these benefits that various data users will gain from the integration of data.world and Sigma Computing.
1. Improved communication efficiency by surfacing trust signals and contextual insights in Sigma.
When viewing a Sigma Workbook, how can you be certain that the dashboard contains accurate information? Questions about business terms or data models often led to guesswork or time-consuming inquiries to the data engineering team. Since these highly skilled data engineering resources represent some of your most valuable assets, it's in your best interest to invest in automation to streamline communication, freeing up their time for more critical tasks instead of answering ad hoc data questions manually.
With this integration, data analysts can now rely on data.world’s DataOps application, featuring AI-powered BB Bots (a monitoring engine) and Hoots (an automated alert system) to deliver contextual insights about the data and metadata directly within BI analytics tools like Sigma. BB bots proactively monitor upstream data and pipeline issues, and you can embed data.world’s Hoot in Sigma to communicate dashboard health directly to data analysts in real-time. The Hoot status “happy” confirms that the information in the dashboard is accurate and up-to-date, while the “sad” status indicates that something is severely wrong and ongoing investigations are being undertaken. Once issues have been resolved, you can update the Hoot status to inform users that the dashboard is now back to normal operation.
Additionally, I personally found the Hoots integration with Sigma very useful, as it greatly improves the overall productivity of data analysts. Traditionally, when working on a dashboard or report, it can often be unclear what the metrics are referring to. With Hoots seamlessly integrated into Sigma, data analysts can now access the full context of the catalog information natively in Sigma, all without disrupting their business workflow. This means that data analysts can stay focused to get things done more efficiently, as they no longer need to deal with constant disruptions of switching between applications to find information they need.
To learn more about this topic, please check out the blog post ‘Closing the communication gap between data engineers and data consumers’.
2. Streamlined data discovery powered by AI + Knowledge Graph for reliable, thoroughly documented, and well-governed data sources
In the era of AI, nearly every vendor in the market will have their AI narrative. When it comes to data catalogs, the focus is on using AI to enable data analysts to ask questions in everyday business language. For example, they can simply ask, “Find me all the data that is related to Sales activities to calculate total revenue in 2023”. The problem with a typical data catalog tool that has a simple integration of AI is that it could yield inaccurate results. Without a context-aware knowledge graph, the AI won't know whether “2023” refers to fiscal or calendar year and which months of sales data should be incorporated into the “Total Revenue” report. Now, picture your data analysts building business reports using incorrect data for your C-Suite’s crucial decision-making - such a scenario could lead to catastrophic outcomes.
Here's the deal: AI alone is not enough to produce accurate and trustworthy results, especially when dealing with proprietary enterprise data for answering complex business questions. AI needs to be paired with a Knowledge Graph data catalog to provide context, and validate the response based on the “facts” stored within the graph. This is precisely what sets data.world’s Archie apart from the rest. Archie is an intelligent bot powered by Generative AI to help improve data accessibility and improved search results. If you are curious about why a knowledge graph data catalog and Generative AI goes hand in hand, check out this webinar titled WTF is a Knowledge Graph and Why is it Key to AI.
When paired with Sigma, data analysts can now effortlessly find and access the relevant data in plain English. They can then query, visualize, and interact with the live data to perform calculations and scenario modeling, as well as drag and drop various data elements to design their dashboard to gain additional insights. By combining the two solutions, data analysts gain confidence in their analysis, while business owners can trust the information to make critical decisions that shape the future of their company!
3. Building trust through visibility and transparency, all while enjoying the speed & flexibility of Sigma’s Excel-like user experience
Every client I speak with praises how incredibly user-friendly Sigma is. You can dive right in without any training, just as if you were working on an Excel sheet, with the ability to scale to handle billions of rows. Not only can users work on data stored in the cloud data warehouse live, they also can integrate it with external sources to perform what-if analyses and scenario modeling. While flexibility sometimes involves trade-offs with governance and security, this doesn’t have to be the case.
To build trust among data teams, Sigma partners with data.world to bring full visibility and transparency to the entire data ecosystem journey. Metadata and lineage information, such as Workbook, Data Element, Tag, User, and many more from Sigma, are natively integrated into data.world’s knowledge graph. This allows data analysts to see the full story behind Sigma's Workbooks, finding answers to questions such as which data was used to build this dashboard, the origin of the data sources, and who has access to and owns it - all with just a few clicks.
Below diagram illustrates the use of data.world’s Eureka lineage graph to explore the end-to-end data flows, giving insights into how data flows between systems and transformation, before it is consumed in the Sigma dashboard. This visualization also provides further clarity on how Sigma’s various data elements relate to the broader data ecosystem, providing users with valuable end-to-end visibility.
Final Thoughts!
In summary, trust is the cornerstone for enterprises to foster a data-driven culture. Data teams require an open, transparent, and collaborative communication environment along with well-documented data assets for effective analysis. This is precisely what the integration between data.world and Sigma brings to our customers.
Last but not least, we believe that trust and flexibility can coexist through robust yet agile data governance programs. Governance need not be overly restrictive, instead, it should provide tools that empower data teams to work more efficiently while ensuring a safe environment. The two companies share this vision and are committed to continuing collaboration to push the boundaries of what is possible to meet our customers' modern data needs. Our goal is to create products that everyone loves and uses.
