Civil society has been a key force in advancing the open data movement over the last ten years. Many of the most prominent milestones during that time have involved the setting of standards and expectations for open data globally. From the articulation of open government data principles in 20071to agreement on the Open Definition and standards for open data licensing in 2012 and 2013 and the launch of the Open Data Charter in 2015, civil society actors have played a key role in convening stakeholders, framing agendas, and driving open data uptake across sectors and policy domains.
As a result of these efforts, open data has been widely accepted as a progressive civic norm,2a public policy resource,34an engine for generating economic wealth,5and an established field of practice and study.67However, this level of success has resulted in new challenges for civil society. As the open data movement gains salience among different stakeholder groups and in different sectors, civil society organisations have been called upon to play a wider role.
Civil society organisations regularly campaign for the release of open data and raise public awareness about open data availability. They provide technical consulting services to stakeholders and often build the online platforms that host open data. They facilitate interaction between government institutions and community groups, while also training other organisations as potential users and educating them on how open data can help to fulfil their mandates. Civil society organisations conduct user research but also conduct investigative research using open data to act as a democratic watchdog. When data is lacking, civil society often collects and publishes that missing data to drive advocacy and raise public awareness, fills gaps in government data, and supports policy-making. In one form or another in a multitude of countries across the world, civil society plays an intermediary role, making government data open and useful to the public.8Civil society, it seems, can do it all.
Civil society is often a sort of chameleon, picking up the slack and adapting to the strategic environments in which it needs to work. In the context of the open data movement, however, this capacity for adaptation has perhaps been civil society’s most defining feature. As the movement has matured from statements of principle to the messy realities of open data implementation, things have become more complicated. Inflated expectations, ambiguous roles, strategic challenges, and unanticipated ethical dilemmas all pose new demands on civil society. Simultaneously, the capacities and resources required to meet these challenges are often unequally distributed. Despite all this, and notwithstanding the absence of clear evidence that open data is having any consistent impact on governance processes, civil society remains remarkably invested in advancing the open data movement and may well be the movement’s greatest resource. To understand this potential and its limitations, this chapter presents four key trends in civil society engagement with open data over the last ten years.
Fragmentation and ambiguity
One of the most remarkable trends in the development of the open data movement has been the proliferation of practical and rhetorical ambiguity as open data discourse has spread across sectors and fields of practice. Domain-level ambiguity in popular discourse is likely the most obvious example of this as evidenced by regular equivocation between open data, open government data, and open government. While conceptually distinct, these three terms are often muddied in use and in practice. For example, open government activities have been criticised for overemphasising open data,910and the presumption that open data is necessarily government data obscures the important role played by civil society organisations and the private sector in generating and opening data for public use.1112Similarly, significant advances by the open data community have complicated collaboration between advocates for open data and advocates for freedom of information,13raising questions about compatibility and the competition for resources.14
Further ambiguity can be noted when considering the types of actors that participate in open data discourse. The open data movement can be considered as a Venn diagram in which one circle is populated by government accountability advocates and organisations working on civic technology and the other is populated by organisations focused on profit-making, either startups using data as part of a business model or investors and entrepreneurs who couple their financial objectives with social aims. Some organisations have blurred this line further by adopting a for-profit model in an effort to enhance their sustainability and reduce reliance on grant-makers, although some research suggests that this has an impact on the way organisations function and the projects they implement.15
Government actors can be seen engaging with open data to support accountability and to generate value, but not necessarily in equal measure. This dual engagement with two normatively distinct fields of practice is both an advantage and a challenge for open data advocates. While the “big tent” approach is advantageous for recruiting broad support,1617a diversity in approaches to different types of activities can frustrate expectations and complicate agreement on objectives.
Similar strategic dilemmas are posed by open data’s increased relevance across a broad swathe of policy areas. While salience in multiple sectors undoubtedly carries a rhetorical advantage for campaigning and lobbying activities, the demands and practicalities of mainstreaming meaningful open data practice vary significantly from sector to sector. This forces civil society organisations to build specific capacities if they wish to support open health data, open transportation data, open land data, etc. Given the capacity restraints experienced by many civil society organisations, this often forces a choice between sectoral specialisation and a more generalised, high-level advocacy. This trade-off is particularly noteworthy as pioneering open data initiatives move beyond early success, revealing the critical need for high-level policy engagement to address challenges of institutionalisation and sustainability.1819
The increased standing of open data over the last decade has broadened and diversified the open data discourse. Likewise, this has dramatically increased the diversification of civil society organisations working with open data that today draw from a variety of sectors and organisational models. This diversification brings a nuts-and-bolts specificity to implementation (e.g. how to conduct open data user research in developing countries or best practices for securing consent for open health portals), while also increasing capacity and resource challenges for many civil society organisations. Overlap between communities and fields of practice muddy the waters for national and international collaborations, frustrate a clear narrative regarding open data’s impact, and further complicate processes for accessing international resources and funding. Close association with profit-seeking initiatives can also put capacity-strapped civil society organisations in direct competition with a startup community that may enjoy investor backing and pursue significantly different objectives.
The capacity gap
The work of civil society organisations demands a wide variety of skills, resources, and capacities. Some of these are familiar. Running public messaging and awareness campaigns to support engagement and the uptake of open data is well within the wheelhouse of most advocacy organisations. Similarly, community research, training, and facilitating interaction between open data stakeholder groups require capacities familiar to most organisations. The hard technical capacities required to work directly with open data are often more challenging to secure. They are often also essential not only for managing, cleaning, formatting, and porting data, but also to support meaningful training, research, engagement, and facilitation activities.
These demands are well recognised, and recent years have seen a dramatic investment in civil society’s hard technical capacities to work with open data. This is most visible in the significant increase in data scientist hires by international organisations,20as well as in the proliferation of ad hoc, conference-based, and roving training mechanisms intended to boost the hard technical skills of advocates and civil society organisations working to advance open data in a national context.21Simultaneously, civil society-led training that targets open data producers has increasingly emphasised strategic approaches borrowed from the world of software development, such as “agile” and “user-centric” methodologies, suggesting that there has been at least some spillover of knowledge and capacity from the private sector into civil society.
There remains a pernicious gap between well-resourced international civil society organisations’ levels of expertise, social capital, and access to financial resources and those of less-equipped national organisations, particularly those based in the Global South. This is likely due, in part, to the close alignment of open data goals with other efforts to support the private sector and generate economic value in developed countries22and the tendency of international civil society organisations working on open data to be based in those countries. As a result, less developed economies may receive less investment in open data-related social ventures, which may have a spillover effect on the level of support for national open data advocacy organisations.
This gap is also a function of the network dynamics that underpin funding mechanisms and capacity development within the open data community. Smaller organisations working at the national level often lack the resources to invest in long-term technical capacity, and they also lack the resources to engage in international networking activities. This is problematic when contact with international non-governmental organisations (NGOs) and civil society networks is the primary means by which many national-level civil society organisations in the developing world learn about technology and data.23Lack of fundamental digital skills is normally an indicator of other capacity limitations, including the inability to develop other “digital goods”, such as digital infrastructure and digital network connections.24This can inhibit national NGOs from developing an international profile, further frustrating access to funding and networking resources.
There has been no definitive analysis of the capacity gap between civil society organisations operating at the global level and at the national level in developed and developing countries. However, it does not appear that investments in global networking and capacity development activities, such as sponsorship of the International Open Data Conference (IODC) or School of Data fellowships and training, are having a significant impact on civil society capacities in general. There simply does not seem to be enough resources. This dynamic only seems likely to increase as broader international advocacy initiatives, such as the Open Data Charter, gain normative traction with governments around the world, increasing the demand for civil society support without a corresponding increase in the financial and strategic resources available for organisations at the national level.
Field-level ambiguities and capacity gaps are, in some ways, scaled and predictable consequences of the way in which the open data movement has developed and how it organises itself. However, the movement’s significant progress has also faced more novel challenges. In particular, new technologies have introduced a wide array of unexpected ethical challenges, and the potential social benefits associated with open data have introduced challenges related to how governments approach collaboration and partnership. These new challenges require new ways of thinking and working in the context of open data.
Ethical dilemmas at the intersection of technology, development, and governance have generated a significant amount of attention. Privacy concerns have been prominent in this regard, especially relating to the opening of public data on individuals and vulnerable communities.25This has forced a rethinking of several seminal concepts, including personally identifiable information and anonymity.26The debate over whether or not privacy concerns warrant caveats to the mandate of “open by default” is particularly challenging and remains unresolved2728despite a lack of documented instances in which open data releases have led directly to documented harm to individuals.
Other ethical dilemmas are also relevant in an open data context, including moral obligations to anticipate the way in which data might be repurposed or reused, concerns regarding the consent of individuals reflected in open data releases, and the question of whether open data is properly serving the communities from whose activities it is generated.29Some have framed this last point in terms of a moral obligation for policy-makers to collect and release more data on vulnerable groups in order to serve them better,30although there is also scepticism about the degree to which governments are able to do so.31While there is little agreement on the appropriate response to such challenges, civil society has proven remarkably proactive in advancing the debate by asking difficult questions as demonstrated by the efforts of the responsible data community32and organisations like the GovLab at New York University (NYU).33
The approach taken by governments to open data activities or to their collaboration with civil society has also generated ethical issues as illustrated by the spread of “openwashing”. Originally coined in the context of the open software movement to describe efforts “to spin a product or company as open, although it is not”,34the idea of openwashing has recently been adapted to open data and open government to describe instances where governments exploit excitement around the idea of openness in order to avoid meaningful reforms.35When governments and companies take superficial steps toward opening data, or take steps to open data that is not meaningful or useful for stakeholders, this poses strategic challenges for open data advocates who have worked hard to secure support for open data. The question of whether to condemn symbolic efforts or to use them to push for more progressive approaches can be challenging, particularly for international advocates and initiatives.363738In some instances, collaboration between national-level civil society organisations and government or corporate partners has resulted in compromises that some describe as the “co-opting” of civil society organisations and the subversion of their missions.39
Open data organisations have also struggled with issues related to gender equity (see Chapter 20: Gender equity). The most effective calls to deal with gender labour and representation inequities in the open data movement have come not from prominent organisations, but from ad hoc and earnest voices at the periphery of the discourse, such as Open Heroines.40This suggests that perennial ethical challenges, like those related to privacy and consent, as well as those surrounding government collaboration, will require new thinking and creative solutions that may not come from the more established civil society organisations in the open data movement, although they can play a powerful role in facilitating the contributions of other actors.
Better governance: Never say die
Perhaps the most remarkable trend in the open data movement’s last decade has been civil society’s consistent investment in open data activities to improve the quality of governance and government. Of the many rationales and rhetorical frames used as the basis for arguments in support of open data,41its value in advancing democracy likely aligns best with the mandates of most civil society organisations. A review of the agendas for international networking events, like the IODC, and the rhetoric around initiatives like the Open Data Charter suggest that leveraging open data for more accountable and responsible government has never been far from the central rationale for open data in general, and, in particular, the rationale for civil society organisations directly involved in the movement.
Nevertheless, there remains no clear or consistent evidence of open data’s impact on the quality of governance and government. Although scholarship on open data has expanded dramatically over the last ten years,42attention focused on the impact of open data on governance, in particular, has mainly come from organisations such as the GovLab at NYU, which has close ties to the civil society community through the formalisation of civil society activities like the Open Data Research Network and the Open Data Research Symposium. However, evidence that open data has contributed to improvements in the quality of governance in national contexts remains both sporadic and speculative.
The most comprehensive exploration of open data’s impact on governance is likely provided by the GovLab’s portal on Open Data’s Impact, which suggests 27 contextual and design considerations that affect open data initiatives’ potential to achieve impact and 14 case studies that demonstrate how open data has improved government.43A closer review of these open data cases illustrates the diversity of each case and its resulting impact, including inter-ministerial cooperation for the release of Brazilian spending data,44on health spending in the Burundi pharmaceutical industry,45and the coincidental access to postcode data which enabled meaningful parliamentary monitoring in the United Kingdom.46In short, when open data does improve governance, it takes a lot more than simply opening data, and what it requires will differ radically from country to country.
None of this is meant to assert that open data does not improve governance. The theories of change that underpinned initial optimism around open data have been complicated by recent lessons from implementation47but remain feasible. There is good reason to believe that open data can play a necessary, if not totally sufficient, role in improving the quality of governance in a variety of national contexts. However, that potential is not readily apparent, and there is not yet any convincing or systemic evidence to point to. This is what makes civil society’s continued dedication to the cause so remarkable. In the face of all the challenges described above, including capacity gaps, insufficient resources, duplicitous partners, impossible competition, and no clear evidence that their efforts will bear fruit, civil society organisations continue to pioneer inspiring and unlikely efforts to leverage open data for gains in governance and democratic accountability.
There is enthusiasm evident in network initiatives aiming to “follow the money”48or facilitate “open contracting”.49The energy that exists when such groups meet to address these challenges feels less like an uphill battle. It feels like a fresh push in a long struggle that is just on the precipice of success. There are colorful sticky notes and Silicon Valley facilitators with entertaining ice-breakers. There are flashy websites and esoteric hacker personalities. There is a culture of creative solutionism, and it is inspiring. It focuses attention, and, in the end, this dogged optimism, creativity, passion, and resilience may be civil society’s greatest contribution to the open data movement.
The trends described above provide a sketch of a dynamic landscape in which civil society advocates of open data have been forced to adapt and address a variety of challenges. Some of these challenges are systemic, such as the ambiguity and fragmentation that characterises open data’s penetration into new sectors and policy domains or the resourcing and support obstacles increasingly faced by national organisations that are arguably best positioned to have an impact on open data ecosystems. Meaningfully addressing these challenges will require a clear strategic response from funding and capacity development organisations.
First, a better overview is required of who is receiving funds and support to work with open data at the national level. This need not be public or field-wide information, but funders and global organisations should map out how resources and capacity development efforts are distributed (see Chapter 25: Donors and investors). This may entail a review of many diverse programming channels, such as funding for data journalism training, funding for anti-corruption campaigns in an international aid context, and many more. Understanding how open data support is distributed across funder portfolios and the support networks of large organisations is a first step in addressing both coordination challenges and uneven access to resources.
Second, funders and support organisations should take concrete steps to increase the outreach and inclusivity of international support networks and activities. At the most pedestrian level, this simply implies allocating more funds to sponsor the participation of national civil society organisations in international events, such as the IODC. Funders should also consider additional investments in prominent capacity-building mechanisms, such as the School of Data or Internews data journalism training. Investment should seek to increase the scope and breadth of capacity development opportunities, but should also include resources earmarked specifically to increase inclusivity.
Moreover, efforts to increase the scope of international support should go beyond “the usual suspects” of the open data movement. International open data networks and events suffer from a reliance on a handful of particularly charismatic and articulate personalities and their organisations, especially in the Global South. Moving beyond such actors to engage with a broader group of civil society organisations at the country level that may not be connected with global networks is no small task; however, doing so would have immediate positive benefits for the organisations receiving support, as well as for the general health and diversity of the international open data community. It would also provide important insights that could benefit funders’ longer-term strategies to provide support. One first step that would assist in such efforts would be to translate existing resources for civil society into multiple languages.
Third, it is worth acknowledging that the above efforts will be limited by the availability of funding and human resources, as well as by the level of interest from international donors and investors. Meaningful responses to the trends described above will require significant investment and coordination at the country level as well. International actors engaging with national governments to advocate for open data should encourage them to establish funding and networking resources for civil society organisations within their jurisdictions. Placing the importance of such investments on par with policy activities may have significant benefits at the country level. Initiatives like the Open Data Charter’s demonstration activities with national governments50offer a unique opportunity to promote such investments.
Funders and global support organisations may wish to temper their interest in metrics for the impact of open data. To be sure, demonstrating open data’s impact is a logical objective for open data advocates. To get governments and companies to open data, one wants to be able to show them what they will get out of it. However, measurement efforts need to be precise in their methods and with their objectives, and they should not be equivocated with either programme-level metrics or the viability and potential of open data as a field.
The pressure on civil society organisations to measure and report on their activities is, in many instances, exacerbated by a culture of results-based management in the aid and philanthropic sectors, as well as the often-latent expectation that anything using technology should have measurable outcomes. While open data initiatives do often support the use of metrics, these should be applied only to the degree that they are useful for adaptive management and when they measure outputs (the immediate results of an open data initiative) rather than impacts (the long-term, societal-level consequences of an initiative).51
Finally, civil society leaders should be mindful of the novel and collaborative ways in which solutions to these challenges might arise. Large and bureaucratic organisations should develop opportunities to facilitate the contributions of more flexible actors and networks. Smaller organisations should make deliberate choices about how they position themselves in a fragmented discourse marked by limited resources. Leaders of all types of organisations should remember the normative power civil society wields in many contexts. Increasing attention on eliminating “manels” (all male panels) at international conferences is an excellent example of how small and consistent interventions in the service of a clear change objective can support incremental changes in institutional culture.
These recommendations provide a clear, if ambitious, starting point for responding to the systemic trends and challenges discussed above. Though the organisations facing these challenges will, in many cases, be best suited to frame and develop effective responses, there is an untapped potential for creative collaboration between organisations that the open data movement is well positioned to exploit, and international support is critical as well. As the open data movement continues to mature, strategic thinking on civil society’s role should focus more around the needs and potential of national organisations, and international support should be structured to better serve the diverse activities civil society organisations will undertake in a fragmented open data landscape.