Background
Digital agriculture, the use of data and information technologies to increase efficiencies and profitability of the value chain, has transformed agriculture and food production Worldwide.
Access to cheaper and more sophisticated technology ecosystems ranging from satellites to IoT and mobile products has reduced the data gap between the rich and poor and have helped democratize data access and along the way allow for improved business visibility.
A connected world provided a new breath of fresh air; it was the defibrillator that Precision Agriculture needed. It allowed for a disconnected USD 1.3 trillion Global Ag & Food Industry, which has historically run on pencil and paper, to join the 4th Industrial revolution.
Data then has become the new oil or in financial terms, the new asset class. For individual farmers and a new generation of farmers, data access has given something beyond accessibility; it allowed them to become more effective businessmen. Technology accessibility reduced information search costs, thereby dramatically lowering transaction costs and enabling greater farmer participation in commercial agriculture.
The Problem
All of a sudden, like in any rapid business transformation the Agriculture world went from decades of infrequent, inconsistent, non-standardized and to a certain extent an inaccessible (private) data catalog to becoming this overwhelming and the unmanageable elephant in the room. Effectively speaking every company has become a data company.
Data has become a corporate asset that needs to be properly managed.
As corporations become aware of the value of their data, the question then became on how to make the best use of that information to translate knowledge into business value appreciation. The concept of a data-driven organization resurfaced. Further and most importantly, the importance of developing a data culture among existing players.
According to an article from Harvard Business Review entitled “Companies are failing in their efforts to become data-driven”: Becoming “data-driven” has been a commonly professed objective for many firms over the past decade or so. Whether their larger goal is to achieve digital transformation, “compete on analytics,” or become “AI-first,” embracing and successfully managing data in all its forms is an essential prerequisite. Consistent with these goals, companies have attempted to treat data as a critical asset, evolve their cultures in a more data-oriented direction, and adjust their strategies to emphasize data and analytics.”
That progress towards data-oriented goals was painfully slow, but the situation now appears worse. Leading corporations seem to be failing in their efforts to become data-driven. It is a central and alarming finding of NewVantage Partners’ 2019 Big Data and AI Executive Survey, published earlier this year. The survey participants comprised of 64 c-level technology and business executives representing large corporations such as American Express, Ford Motor, General Electric, General Motors, and Johnson & Johnson.
Here are some of the alarming results from the survey:
• 72%: Participants that have yet to forge a data culture
• 69%: Participants that have not created a data-driven organization
• 53%: Participants that are not yet treating data as a business asset
• 52%: Participants that admit that they are not competing on data and analytics.
These sobering results come in spite of increasing investment in big data and AI initiatives. 92% of survey respondents reported that the pace of their big data and AI investments are accelerating; 88% say a greater urgency to invest in big data and AI, and 75% cite fear of disruption as a motivating factor for Big data/AI investment.
Also, 55% of companies reported that their investments in big data and AI now exceed $50MM, up from 40% just last year. Further, companies are building organizations to manage their Big data/AI initiatives, with a rise in the appointment of CDOs (Chief Data Officers) from 12% in 2012 to 68% of organizations having created and staffed this role in the past 07 years.
An eye-opening 77% of executives report that business adoption of Big Data/AI initiatives is a significant challenge, up from 65% last year. Executives who responded to the survey say that the obstacles to successful business adoption do not appear to stem from technology obstacles; only 7.5% of these executives cite technology as the challenge. Instead, 93% of respondents identify people and process issues as an obstacle. There has been a dramatic underestimate in the difficulty of cultural change requirements: 40.3% identify lack of organization alignment and 24% cite cultural resistance as the leading factors contributing to this lack of business adoption.
Perhaps the pursuit of short-term financial goals pushes longer-term objectives like data-based cultures to the back burner.
When probing into the state of Big Data/AI entering 2019, it is noteworthy how firms assess themselves in their progression toward becoming data-driven organizations. On every metric except driving innovation with data, firms ranked themselves as failing to transform their businesses:
▪ 71.7%: Percentage of firms report that they have yet to forge a data culture
▪ 69%: Percentage of firms say that they have not created a data-driven culture
▪ 53.1%: Percentage of firms state they are not yet treating data as a business asset
▪ 52.4%: Percentage of firms that claim they are not competing on data and analytics.
So what is the reason why business adoption of Big Data/AI continue to be such a challenge for firms?
Executives cite multiple factors, 95% of which appear attributable to cultural and organizational issues, not technology challenges.
Leadership
Thinking about data as an asset is a new phenomenon for most companies. Traditional businesses were not data-driven in the sense that emerging digital competitors are. The Economist notes, “Seven of the world’s ten most valuable companies by market capitalization are built on a foundation of tying data to human beings.” Although 97.2% of firms are investing in Big Data/AI initiatives according to last year’s survey, we see that firms are struggling to treat data as a business asset, become data-driven, and compete on data and analytics. As companies have sought to organize around data, they have tried to forge new leadership roles to champion Big Data/AI within the organization.
Path for Success
Suggested paths to successfully achieve a data-driven culture
1) Centralized data ingestion: Data as a Service
2) Data Visibility towards Data Democratization
3) Data curation for continued analysis
4) Create data with a purpose
5) Design capabilities towards purpose maximization
6) Data Culture is Decision culture: Stay true to the business problem
7) Prioritize issues that you want to see resolved then look for people and technology
8) Process: Discoverability – Visibility – Engagement – Data Transformation
9) Looking for a system should be the last task
10) Start with a pilot project
11) A Data Science team alone WILL NOT translate into an improvement in data culture. An organization needs good communicators to cross the bridge between data science and business purpose
Conclusion
Whatever the reasons for the failure to achieve transformational results from data initiatives, the amount of data continues to rise in business and society. Analytical decisions and actions continue to be generally superior to those based on intuition and experience. Companies develop a data culture by moving beyond specialists and skunkworks, intending to achieve deep business engagement, creating employee pull, and cultivating a sense of purpose, so that data can support your operations instead of the other way around.
In short, the need for data-driven organizations and cultures isn’t going away. Firms need to take a hard look at why these initiatives are failing to gain business traction, and what actions must be taken to reduce the cultural barriers to business adoption. Many companies have invested heavily in technology as a first step towards becoming data-oriented, but this alone clearly isn’t enough. Firms must become much more serious and creative about addressing the human side of data if they genuinely expect to derive meaningful business benefits.
The path for success: identify a business case; design the entire data-flow; define data and talent requirements; increase data learning opportunities; training IT and business thinkers to communicate about data strategies and models and scaling data talent over time based on lessons learned.
Making the transition to using data is leading firms to structure teams for collaboration and organizational design culture to attract, retain, and develop top data talent.