The open data movement launched a decade ago with a focus on transparency, good governance, and citizen participation. As other chapters in this collection have documented in detail, those critical uses of open data have remained paramount and are continuing to grow in importance at a time of fake news and increased secrecy. But the value of open data extends beyond transparency and accountability – open data is also an important resource for business and economic growth.
The past several years have seen an increased focus on the value of open data to the private sector. In 2012, the Open Data Institute (ODI) was founded in the United Kingdom (UK) and backed with GBP 10 million by the UK government to maximise the value of open data in business and government. A year later, McKinsey released a report1suggesting open data could help unlock USD 3 to 5 trillion in economic value annually. At around the same time, Monsanto acquired the Climate Corporation, a digital agriculture company that leverages open data to inform farmers for approximately USD 1.1 billion. In 2014, the GovLab launched the Open Data 500,2the first national study of businesses using open government data (now in six countries), and, in 2015, Open Data for Development (OD4D) launched the Open Data Impact Map,3which today contains more than 1 100 examples of private sector companies using open data. The potential business applications of open data continue to be a priority for many governments around the world as they plan and develop their data programmes.
The use of open data has become part of the broader business practice of using data and data science to inform business decisions, ranging from launching new products and services to optimising processes and outsmarting the competition. In this chapter, we take stock of the state of open data and the private sector by analysing how the private sector both leverages and contributes to the open data ecosystem. Readers should note that this chapter does not cover the release of open data about private sector companies and their operations. This is addressed in Chapter 3: Corporate ownership.
How does the private sector engage with open data?
Private sector organisations, ranging from small businesses to large corporations, participate in the open data ecosystem in three key ways: as data users, as data intermediaries, and as data providers.
The private sector as open data users
Companies use open government data to improve their operations and/or develop new products and services
There are now thousands of examples of companies having used open government data as a key business asset that have been documented by the Open Data 500,4the Open Data Impact Map,5the ExploraLatam,6and other studies. These projects have illustrated that companies use open government data for business optimisation, for developing new products and services, or for a combination of the two.
Businesses use open government data for business optimisation in a wide variety of ways, ranging from activities that have been established for decades to others that are more recent innovations. Large retail corporations, such as Starbucks and the Kellogg’s Company, for example, are using sophisticated analytics to find correlations between open data on weather, demographics, and other factors and their own data on customer behaviour and buying patterns.7On a broader scale, companies of all sizes in all countries can use government data to improve their business in a number of ways:
- Marketing – using demographic data to identify new customers and reach them with more relevant and targeted messaging.
- Supply chain management – using transportation and GPS data to develop more efficient supply chain operations, including shipping.
- Strategic planning – using weather, economic, and other data to identify and manage business risks.
- Business intelligence – using economic, demographic, and other data to identify factors and trends that impact a company’s business.
Many companies are also using open government data to develop new products and services either as extensions of their existing business or as entirely new entrepreneurial ventures. In the European Union (EU), a study on the European data economy has reported that when asked about the benefits open data brings to companies, innovation was by far the most mentioned benefit (47% of the surveyed companies), followed by reduced costs and increased efficiency (26%).9These innovations, which are emerging throughout the world in virtually all sectors of the economy, can benefit both business-to-business/government and business-to-consumer ventures.
Insights from the Open Data Impact Map
The Open Data Impact Map,8a project of the OD4D network, is a searchable, sortable database of organisations using open government data around the world. About two-thirds of those organisations, more than 1 050 of them, are for-profit companies. The two charts below indicate that these businesses include companies of all sizes, operating across many different sectors. Businesses with one to 200 employees, considered small to medium-sized enterprises (SMEs), make up 72% of this group.
Among the 13 sectors analysed, a third of the for-profit companies on the Impact Map fall into the IT and geospatial sector, while one in six are in the business, research, and consulting sector.
These companies have developed a number of revenue sources10and business models, including freemium (advertising) and premium (subscription) models, or have developed an open source or platform-based approach. Other companies, including many that specialise in real estate data (such as Zillow11and Properati12, also earn revenue from lead generation (consumers who use their services may ultimately make purchases that result in payments back to the company).
Unlike large corporations, SMEs may not have the means to acquire or collect large amounts of their own data, impacting their ability to compete effectively with those that do. For that reason, SMEs are among the chief beneficiaries of open data. The rise of an open data ecosystem, which includes not only greater access to information but also a proliferation of tools (often cloud-based) to make sense of that information, opens new avenues and represents new business opportunities for smaller businesses. When the GovLab originally mapped 500 companies that were using open data, more than 70% (354 in total) were SMEs and startups (see box, A small sample of SMEs).13The Open Data Impact Map shows a similar pattern worldwide.
Despite the considerable promise of open data, SMEs also face a number of challenges in using this data. They often lack the human resources and sophisticated analytical tools necessary to process and analyse large datasets. These challenges highlight the need for data intermediaries as described further below.
A small sample of SMEs
Around the world, thousands of SMEs are demonstrating the business value of open data in a wide variety of economic sectors. Here is a small sampling of businesses operating in different countries:
- Mexico, Transportation, AutoChilango14– improves government services by helping licensed commercial drivers who use data on regulations, insurance, and more.
- Ukraine, Smart Cities, CityScale15– an information portal that empowers citizens and professionals by providing access to relevant data and statistics about their city, using open urban, transport, and crime data from various government sources.
- Ghana, Agriculture, Farmerline16– an SMS-based service that provides small-scale farmers with up-to-date agricultural information and advice, helping them to adopt new farming practices to increase crop yield, using data from the Ministry of Food and Agriculture and weather data from a number of sources.
- India, Engineering, Spageo17– a geospatial engineering, data management, and geoscience consulting company for government, utilities, and business, using floating vehicle, geospatial, infrastructure, and agricultural data from the Government of India.
- United States (US), Finance, Credit Sesame – a credit reporting and tracking service that helps users manage their credit, loans, and debt, using open government data from the Department of Commerce and the US Census Bureau.
The private sector as data intermediaries
Companies improve and repackage government data for public use and/or for business intelligence
While open government data has great potential value, potential users may not have the capacity to prepare or improve the data before they can use it, analyse it, or share the results. Data may be released in formats that are not initially machine-readable, are otherwise difficult to use, or may have quality problems relating to accuracy, completeness, timeliness, etc.
Different types of data intermediaries have emerged that view the existing gaps and challenges in the open data ecosystem as a business opportunity by making government data more usable as a resource for different audiences. These range from information giants like Thomson Reuters and Reed Elsevier, which work to improve the quality of large datasets, to companies operating in countries where just converting data from PDFs to more accessible formats is the priority.
Other companies operate as data brokers that aggregate and sell data from both government and non-government sources, analytics companies that add value to open government data for business clients, or data visualisation companies that make open government data easier to access, use, and understand.
Finally, there are companies like OpenDataSoft,18Junar,19and Socrata20that help governments and businesses to publish and share their data in open formats and offer data visualisation and analytics as services. These are usually cloud-based services, generating revenue using data-as-a-service models.
Open data incubators and accelerators
A number of open data companies have secured the funds they need from venture capital firms or other investors; however, it can be difficult for new companies to explain their business model to potential investors who are not familiar with open data and its value. To bridge that gap, several governments over the past few years have launched a number of programmes to help incubate and accelerate open data ventures:
- Canada’s Open Data Exchange (CODX)21– a public–private partnership to accelerate the use of open data for commercial purposes.
- Data Pitch22– an EU-funded open innovation programme that matches startups with larger organisations to solve defined challenges with data.
- Future Internet Open Data Expansion (Finodex)23– a European virtual accelerator for commercial open data projects across Europe.
- Labora24– an open data incubator in Mexico run in partnership with the UK Open Data Institute (ODI).
- ASEAN Data Startup Accelerator25– an accelerator for data-driven startups in the ASEAN region.
- Open Data Incubator for Europe26(ODINE) – a 2016 incubator for open data entrepreneurs across Europe funded by the EU Horizon 2020 programme.
The private sector as data providers or collaborators
Companies seek to further public interests by sharing some of their own data or by collaborating with the public sector
In recent years, a number of data-driven companies, such as AirBnB, Properati, and Uber, have published at least some of their data as open data. They may choose to do so for several reasons, ranging from the public relations value of sharing their data to the opportunity to improve their own data through crowdsourcing. Most private sector companies have been reluctant to share their data openly as long as their competitors keep their data to themselves; however, a relatively new, collaborative approach is making it easier for different companies to work together to benefit from each other’s data and simultaneously to benefit the public.
Data collaboratives27represent an emerging model of public–private partnership within which various actors across sectors exchange data and pool analytical expertise to help solve complex public problems. This model has become increasingly prevalent in recent years with several examples around the world. The practice will likely continue to increase following several recent developments such as Open Banking28and the implementation of the General Data Protection Regulation (GDPR) in Europe. Data collaboratives can take many forms and vary by sector or by the problem being addressed. (See box, Six kinds of data collaboratives.)
Six kinds of data collaboratives
The GovLab’s Data Collaboratives Explorer29has identified six main types of data collaboratives.
- Trusted intermediaries, where companies share data with a limited number of known partners through a third party. For example, the South African telecommunications company, Mobile Telephone Networks (MTN), has made anonymised call records available to researchers through a trusted intermediary.
- Prizes or challenges, where companies make data available to qualified applicants that compete to develop new apps or discover innovative uses for the data. One example is the Orange Telecom Data for Development Challenge30in the Ivory Coast and Senegal, where Orange Telecom hosted a global challenge that allowed researchers to use anonymised, aggregated Call Detail Record (CDR) data to help solve various development problems related to transportation, health, and agriculture.
- Research partnerships, in which corporations share data with universities and other research organisations to map weather patterns and natural resources, for example.
- Intelligence products allow companies to share (often aggregated) data that provides general insight into market conditions, customer demographic information, or other broad trends.
- Application Programming Interfaces (APIs) which allow developers and others to access data for testing, product development, and data analytics. Typically, third parties are asked to sign a terms of service agreement in exchange for which they receive corporate data that allows them to build applications.
- Corporate data cooperatives or pooling, in which businesses and other data holders, such as government agencies, work together to create “collaborative databases” with shared data resources (e.g. databases containing medical data to promote drug development).
Challenges and recommendations
Despite the terabytes of data that have now been shared and the very evident potential of open data for the private sector, there is general recognition that the benefits of open data remain largely untapped. There are many reasons for this, as well as several possible solutions that would promote greater progress.
One key issue is a persistent mismatch between supply and demand or between the data being released by companies and governments and the data that potential data users actually need. In most countries, there are few established channels for the private sector to engage with government open data providers, and the demand for open data is often poorly understood or defined as a result. Different sectors, such as healthcare, agriculture, and finance, have different information needs and require different datasets to be released. A more demand-driven approach with public–private engagement would help to set priorities that would better serve the business community’s needs. The Open Data for Business (OD4B) Assessment Tool,31developed for the World Bank, is one approach to this kind of engagement.
Data usability is as important an issue as data usefulness. The fourth edition of the Open Data Barometer has noted that “data is hard to use because there is no metadata or guidance documentation available. Less than a third (31%) of the published datasets have some supporting basic metadata or companion guidance documentation.”32Usability is a particular concern in developing countries where technical capacity and financial means may be limited. There may also be problems with data released in proprietary formats or in formats that require significant technical expertise to process. Increasing the adoption of best practices in data release and documentation would help to ensure that useful data is also as usable as possible.
For government data to be truly open, it needs to be open both technically and legally. Companies will have difficulty using government data if it is not published in open, machine-readable formats or if it is not licensed for open use, reuse, and republication. The fourth edition of the Open Data Barometer also reported that only 7% of data released is fully open at a global scale, only one of every two datasets is machine-readable, and only one in four datasets is released under an open licence. In addition, legal and ethical limitations on the use of personal data can make it difficult to apply government data for business use. A number of strategies, including de-identification and data aggregation, could help balance privacy protection with potential data applications as a recent report describes.33
Finally, as with most open data initiatives, private sector organisations also face challenges in developing and applying reliable metrics to help guide their use of open data. Metrics are essential for deciding on financial and other investments, as well as for determining the efficacy of open data initiatives. Metrics can capture the direct value of open data to SMEs and other companies and the indirect value of open data to third-party organisations doing business with open data corporations, to consumers, and to the wider economy, as well as capture the wider societal impacts that can be attributed to open data. To date, most studies have based their metrics on ex-ante estimations, whether through surveys or indirect research. Better metrics and methods are needed. The EU Economic Benefits of Open Data34report of 2017 concludes “with a call for action for further evidence that needs to be gathered, at both the EU and country level”.
While open government data has clear business value, it is not clear how much of that value is being realised. Many have questioned the 2013 McKinsey study’s estimate that open data is worth USD 3 to 5 trillion annually.35It has been difficult to find concrete evidence to support that claim. At the same time, however, it has also become clear that the use of open data is so embedded in many companies’ business operations that the benefits of open data, while very real, are difficult to quantify.
Although much attention has been paid to companies that are based entirely on open data, like those described in the box above on SMEs, most businesses that use open data do not identify themselves as an “open data company”. Instead, they identify themselves using conventional sector or market terminology even though they may use open data to optimise their business operations or improve their profit margins. For example, they may use weather data to manage their inventory, demographic data to decide where to build a new retail outlet, or geospatial data to plan more efficient transportation. While these companies would exist without open data, their use of such data can make a huge difference to their efficiency, profitability, and overall success.
As the value of open government data is more widely understood, businesses and other non-government stakeholders can work together to develop new ways to apply it. Data and entrepreneurial communities can work together to enhance their awareness, skill levels, and utilisation of open data for innovation. The “smart cities” movement could be a major opportunity for this kind of engagement internationally as evidenced in the open data initiatives ongoing in cities from Singapore to Bristol. Universities and enterprise centres could also act as educators and mediators by training students to work with organisations (SMEs and social enterprises) to further practical applications of open data research.
Ultimately, the business use of open government data has to be seen in the context of the proprietary data that businesses themselves collect, manage, and use. The potential for open data for businesses may lie increasingly in connecting it with other sources of information and resources. Businesses can also be open data providers as well as users, and there is a growing sense that the private sector should make some of its data available in the public interest.
In France, recent legislation prepared in consultation with citizens could serve as a global model for opening up private data. The Digital Republic Act, which came into force in October 2016, has introduced the notion of “general interest” to widen the scope of private data available to everyone.36Although there have been no enforcements yet, this notion could help create a new paradigm for how governments see the public value of privately held data. Even if such a dramatic shift does not take place, the future value of open data will depend on public–private collaboration in the generation and sharing of data.
To fully realise the economic value of open data, we will need to overcome some of the same barriers that now inhibit the full utilisation of other kinds of data and digital technology. Countries in the Global South, for example, may lack the financial resources needed to apply and grow new business ideas for leveraging open data. Promising ideas that emerge from challenges or hackathons in these countries may not turn into viable businesses as easily as they would if they could leverage the greater investment resources in the Global North. Launching incubators or accelerators in these countries could increase the chances of business success (see box, Open data incubators and accelerators). Other strategies to support new open data companies could include establishing intermediaries that can help smaller companies leverage open data, creating technical capacity and training opportunities, and developing networks of expertise and experience.
Gender bias may also be inhibiting the development of new open data companies. Some anecdotal observations suggest that men may have greater access to the resources needed for entrepreneurial data ventures than women. Programmes, such as Mexico’s Crea Communities of Social Entrepreneurs,37can empower and encourage women entrepreneurs and businesswomen from socially and economically marginalised areas. Other initiatives, like TechMousso38in the Ivory Coast, are encouraging more women to participate in apps competitions and can help train them on different business models.
Despite the challenges that remain, open data is being applied by businesses of all kinds and has proved to be a critical resource for the private sector. The value of open data should be a key consideration as governments around the world weigh the costs and benefits of developing better open data resources. There is still a need to increase awareness of the business value of open data and provide examples and guidance on how companies can put it to use. At the same time, governments can learn new ways to improve their open data programmes from the private sector and other innovators.