“If we have data, let's look at data. If all we have are opinions, let's go with mine.” - James Barksdale, former President and CEO of Netscape Communications.
The days of business strategies based on executive “intuition” or “gut feeling” are long past. Instead, today’s business leaders rely on knowledge gleaned from the analysis of vast quantities of “big data” to make their decisions.
But all the data in the world can’t help your organization gain a competitive edge if your users can’t compile, locate, access, or understand the data they need when they need it to inform critical business decisions.
Enter data governance, the process of managing and controlling how data is collected, stored, used, and shared within your organization. A data governance strategy involves establishing policies, procedures, and guidelines to ensure that data is accurate, consistent, and secure, with the ultimate goal of using that high-quality data to support your organization's goals and objectives.
But when instituted without forethought, traditional data governance can hinder your organization’s performance and efficiency by restricting data access to the point it can’t be used to inform day-to-day business decisions across your company.
The solution is Agile Data Governance, which adapts the best practices of Agile and Open software development to data and analytics, empowers stakeholders across your business to participate in an inclusive data and analytics process, and increases productivity in a safe, consistent, and auditable way.
What is data governance?
Put simply, data governance establishes processes for managing and controlling data across your organization. It involves establishing policies, procedures, and guidelines for data management, including data quality, data security, data privacy, and data stewardship.
As explained by TechTarget, “A data governance framework consists of the policies, rules, processes, organizational structures and technologies that are put in place as part of a governance program.”
Your data governance framework ensures that you’re using your data in a way that supports your organization's goals and objectives.
Another aspect of data governance is the management of metadata. This “data about data” includes data definitions, data elements, data sources, and data lineage. Metadata management is a crucial aspect of data governance because it helps ensure your data is accurate, consistent, and properly understood by all your organization’s stakeholders, regardless of their background in data or their technical ability.
Why do you need effective data governance policies?
Effective data governance policies ensure your data is used in a way that supports your enterprise’s business objectives while also protecting the privacy and security of private individuals. Good data governance practices focus on making sure your data is accurate, consistent, protected, and used to inform data-driven decision making and improve your organization’s overall performance.
Your data governance policies should also help to ensure that your data is being used in compliance with regulatory requirements. Depending on your industry, your business may have to comply with a variety of regulations, such as the General Data Protection Regulation (GDPR) or the Health Insurance Portability and Accountability Act (HIPAA). These regulations enforce specific requirements for data management and protection, requirements that are far easier to implement when working within an effective data governance framework.
Who is Responsible for Data Governance Processes?
The responsibility for data governance processes is typically shared across different stakeholders within your organization. These stakeholders often include data owners and data stewards, both of which are an important part of a data governance team.
“Data ownership” means these stakeholders are responsible for ensuring data is accurate, consistent, protected, free of data issues, and used in a way that benefits your organization overall. They are subject matter experts in the area of the business in which their data is produced, allowing them to understand your organization’s data in context and ensure it makes sense.
Data stewards are responsible for managing and controlling data across your organization. They are responsible for establishing data standards, policies, procedures, and guidelines for data management, including data quality, data security, data privacy, and data stewardship.
Both of these roles are important within larger data governance teams, which are responsible for designing, implementing and maintaining your data governance policies and procedures.
What are the risks of a poor data governance strategy?
Poor data governance can lead to a variety of risks, including data quality issues, data security breaches, and compliance violations. It can also result in the creation and use of inaccurate or inconsistent data, which can negatively impact crucial business decision making and harm your business’ overall performance.
Compliance violations can result in fines, penalties, and — if widely publicized — significant damage to your company’s reputation. And data security breaches often lead to the loss or theft of sensitive information, which can have serious consequences for individuals and your organization as a whole. (For example, Amazon Europe was ordered to pay an $888 million penalty — the largest GDPR fine ever — after a 10,000-person complaint led to the discovery that the organization’s method of processing sensitive data was in violation of user's privacy.)
By developing and implementing effective data governance policies and procedures — including establishing data governance frameworks, implementing master data management and metadata management, and setting up data governance teams to oversee the data governance program, as mentioned above — your organization will be well positioned to avoid these types of painful pitfalls.
You now understand the importance of a data governance program. But for all its benefits, traditional data governance can hinder your organization’s business processes, performance and efficiency. This is because most data governance programs are focused on restricting access, to the detriment of the original reason you launched your data governance in the first place — to connect and empower your data and analytics teams with the knowledge needed to make smarter business decisions.
What is Agile Data Governance?
Instead of managing only risk and compliance, data governance should holistically address the entire data and analytics process, enabling safe, efficient, and reliable project collaboration across your enterprise.
We call this agile data governance.
A type of DataOps — a data management methodology that considers the people, processes, and tools involved in making your data supply chain more efficient — Agile Data Governance encompasses an end-to-end approach for data management that produces a unified view of your entire data lifecycle.
Unlike top-down data governance strategies that seek to control every aspect of data access, Agile Data Governance empowers all stakeholders to participate in an inclusive data and analytics process, aiming to increase productivity in a safe, consistent, and auditable way.
What are the benefits of Agile Data Governance?
Agile data governance adapts the best practices of Agile and Open software development to data and analytics, iteratively capturing knowledge as data producers and consumers work together. It drives faster and more accurate business insights than top-down, command-and-control-style data governance programs, reduces redundant and inefficient work, increases reuse of data products, and — most importantly when pursuing a data-driven business model — helps to build a thriving data culture.
Agile Data Governance helps to overcome the five most-common challenges companies face when working to create data-driven cultures (as described by Tomasz Tunguz in his book Winning with Data):
1. Expediting data requests
Within an Agile Data Governance methodology, data producers, data consumers, and domain experts iterate together to build reusable assets to lower the frequency of ad-hoc data requests. This prevents data bottlenecks, where data producers can’t keep up while servicing one request for data after another.
2. Eliminating data silos and rogue databases
With Agile Data Governance, your data consumers have a direct, clear way to request and iterate on data assets. This reduces the prevalence of “emailed spreadsheets” and improves documentation, helping more people to find, understand, and use your data.
3. Improving data understanding
In most organizations, data documentation feels like a chore and is usually an afterthought of the original analysis. In Agile Data Governance, you do the documentation while you do the work, which increases global knowledge about what data exists, what it means, and how to use it.
4. Increasing data trust
With Agile Data Governance, peer review happens as data analysis unfolds. This creates a shared understanding which can be recorded in business glossaries and other alignment tools, and it prevents “data brawls” in which different versions of the same analysis can lead to uncomfortable meetings.
5. Developing organization-wide data literacy
Perhaps the biggest long-term benefit of practicing Agile Data Governance is that it encourages participation with, and observation of, talented people doing amazing work. This increases data literacy and skill across your entire company.
How do you establish an agile data governance initiative?
The benefits of Agile Data Governance are clear. But how do you implement Agile Data Governance across business units in the enterprise?
We believe there are five steps you need to take to launch an effective Agile Data Governance program:
1. Get buy-in for cultural change
How do you convince data leaders and other key stakeholders the only way to achieve a data-driven culture is by switching to Agile Data Governance? By bringing attention to the failings of your current data governance program. By asking questions like, “Do our governance tools and processes give us real and real-time visibility into the use and value of our data?” or “Have we observed a decrease in our costs and an increase in our revenue as a result of our team collaborating over data assets?” you can show your leadership there’s plenty of room — and cause — for improvement.
By establishing a cross-functional governance committee that oversees the success of your program and is responsible for establishing your guiding principles, and by opening up some traditionally restricted governance functions to a broader audience, you gain a better understanding of who, what, why, and where data is used, enabling more informed decision making. The creation of this committee also sends a signal of inclusivity to the organization, which can help achieve further buy-in, and instill confidence that your governance program is representative of all user groups.
3. Align on principles
The first job of your governance committee is to align on and document the principles that will guide your Agile Data Governance program. When selecting these principles, consider the specific goals you want your program to achieve – strategic, tactical, and operational – and develop a framework that sets you up for success. By aligning around your principles, you enable every one of your employees – not just IT – to understand, respect, protect, and accelerate the use of data while still providing appropriate guardrails.
4. Identify stewardship and ownership
Adding roles like data product managers and knowledge scientists makes governance easier and reduces the burden on your traditional data stewards. These professionals act as software development scrum masters or product owners, but with data assets.
5. Begin Your First Agile Data Governance Sprint
Agile Data Governance, adheres to a crawl-walk-run philosophy where efforts are focused around building a prioritized initial use case involving known personas and a small number of data and/or analytics sources. This gets data into the hands of end users fast so you can immediately begin to measure the impact of your project and iterate for future use cases.
You can learn more about implementing an agile data governance program in our Agile Data Governance Playbook.
To fully implement Agile Data Governance within your organization, you need a collection of cloud-native tools that are centered around a cloud data warehouse and together comprise a data platform. Together, these tools collect, process, store, and analyze data, and are known collectively as a “modern data stack.”
To stay true to agile principles, it’s crucial that the tools and architecture of your modern data stack be scalable and flexible to allow for ever increasing amounts and types of data, and to account for inevitable changes and improvements in technology.
These are the tools you need in your modern data stack:
Data source
Your data source is where your new data originates, and it produces massive amounts of raw data. The location where information is gathered, a data source might be a database, a flat-file, an XML document, or any other format that a system can read.
Data integration (extract and load)
These tools allow you to ingest, consolidate, and transfer data from its originating source to its destination, your cloud data warehouse. They simplify data management strategies and improve data quality by providing a standardized approach to data intake, consistency, sharing, and storage.
Cloud data warehouse
This is where your data is stored. (data.world is a Premier Partner of Snowflake, an industry-leading cloud data warehouse and a data storage heavyweight.)
A transformation tool helps change data formats, apply business logic, convert, cleans, and structuring data into a usable format. They are important when consolidating both structured and unstructured data from disparate sources for analysis.
Business Intelligence/Data Analytics/Data visualization
Data visualization and business intelligence tools help to prepare data for analysis and render information in a visual format such as a graph, chart, or heat map, making metrics easier to understand and work with.
Governance
Data governance tools help organizations automate various aspects of managing a data governance program. They’re important for the creation of business glossaries, data mapping and classification, workflow, collaboration, and documentation.
A crucial data governance tool is a data catalog, which acts as a unified hub that brings together all your people, processes, and technology.
But not all data catalogs are created equal, and not all are equipped with the technology to effectively implement an Agile Data Governance methodology at your business.
Why is data.world the best data catalog for Agile Data Governance?
An iterative, fluid, Agile methodology is the best way to create a completely inclusive data-driven culture. But to establish that Agile Data Governance model, you need tools in your data stack that are truly able to support it.
When it comes time to choose a data catalog — ideally the first tool in your data stack — choose carefully. Despite bold claims, many are tough to use, work poorly with modern data applications, and — instead of being built in the cloud and able to easily mesh with other cloud based software (like a cloud data warehouse) — are cloud adjacent at best.
data.world is the first and only cloud-native catalog built on a knowledge graph. That means automation, flexibility, and scale come standard.
Additionally, data.world delivers on communication, integration, connectivity, and interoperability, all of which are crucial to the success of DataOps Agile Data Governance practices.
But don’t take it from us; In their Q2 2022 report on enterprise data catalog vendors, Forrester Research evaluated the 14 most significant enterprise data catalog vendors, and named data.world a leader among Enterprise Data Catalogs for DataOps in The Forrester Wave™.
Interested in seeing Agile Data
Governance in action?
Take a tour of the latest Agile Data Governance capabilities from data.world.
About data.world
data.world makes it easy for everyone—not just the “data people”—to get clear, accurate, fast answers to any business question. Our cloud-native data catalog maps your siloed, distributed data to familiar and consistent business concepts, creating a unified body of knowledge anyone can find, understand, and use. data.world is based in Austin and is home to the world’s largest collaborative open data community
