Skip directly to content

Central Statistical Organizations - still emerging from the pre-Internet data publishing world

on Thu, 01/05/2012 - 20:49

Health Data Innovation is back from a long holiday hiatus, which among other adventures led me to the 3rd International Arab Statisticians Conference in Amman, Jordan. A recurring topic at the conference was that many Central Statistical Organizations (CSOs) are focusing too much on collecting data while neglecting to enable and encourage broad use of those data. This is true for CSOs around the world, not just in Arab countries. The most common form of online data sharing are PDF or Excel files with tabulated data, roughly equivalent to the publication of printed reports and yearbooks. Reports and tables are very useful, but can't replace access to detailed or microdata for researchers and other serious numbers geeks.

2010 study by the US Federal Reserve Board, published in the OECD Statistics Newsletter, explored data sharing practices of statistical organizations. It evaluated the online offerings of 193 CSOs with some very interesting findings:
  • HTML (71%) and PDF (64%) are the most prevalent forms of data distribution (which confirms my own experience; too many organizations simply put reports online that are traditionally published on paper)
  • Use of Excel (55%) far outweighs the use of csv (17%) or txt (2%), even though it is a proprietary format
  • Only 9 CSOs (5%) use interactive graphics, 58% don't provide graphics at all (except as part of reports) 
  • Only 21% of CSOs enable users to customize downloads, while the vast majority offers predefined documents for download. 21% don't provide any download functionality at all.

There is a growing number of web tools with useful functionality to make data available, engaging and even fun.  Free tools like Tableau Public, ArcGIS.com, Google Motion Charts, and others offer great possibilities to share and visualize data, mostly not requiring much coding or developer knowledge. More comprehensive solutions like Socrata, Space-Time Research, or Tableau Server offer far more sophisticated possibilities for publishing, visualizing and exploring data. For a more comprehensive list of tools, check the Resources & Tools section. CSOs need to make use of these opportunities.

The study didn't explore the granularity of the data offered, i.e. whether the CSO is sharing microdata, detailed tabulations, simple tabulations, or only estimates. I spend a fair amount of time looking at CSO websites and talking to people at CSOs while searching for health-related data. Many organizations are hesitant to provide access to detailed or even microdata, mostly citing confidentiality reasons (for more background reading on motivations to share or not to share data, check here). This is another lost opportunity. Academic and other researchers need microdata to unleash the full power of statistical tools on the data and maximize the insight gained from them. There are three ways to deal with the problem. CSOs can ask data users to sign data use agreements that ensure proper and secure storage and use of the data. They can ask researchers to come to their offices or dedicated research data centers. Or they can use software tools to ensure confidentiality. Software from Space-Time Research uses a combination of techniques to enable work with micro-data while ensuring that any viewable / downloadable results are fully de-identified.

The discussions in Amman showed that CSOs across the Arab world are aware of the problem. Now it's time to work on the solution.

Nuggets of insight on Twitter from NYeC Digital Health Conference

on Fri, 12/02/2011 - 14:25

The NYeC Digital Health Conference that happend in New York over the last two days covered a wealth of very interesting and valuable information. While I was not able to attend, the Twitter stream (hashtag #dhc11) provided some really good insight and valuable nuggest of information. I am posting my favorites below.

Please let me know if this summary is helpful, and I will cover other events this way in the future.

Create real value with government data: open and promote

on Thu, 12/01/2011 - 17:17

Today's discussions at the NYeC Digital Health Conference 2011 were a great reminder of the value of open data. I have seen Todd Park speak about HealthData.gov and 'Data Liberacion' on several occasions (great presentation here), and one can't help but be convinced. His key example, weather data, proves the point: when NOAA decided to make weather data available for free in a machine readable format, they enabled the development of a whole weather data ecosystem. Weather channels, weather websites, weather apps, weather insurance, and so on, creating jobs and economic value.

The US Department of Health and Human Services has made great strides in making data available from community health metrics to FDA recall data and government health spending. The launch of data.gov, healthdata.gov and the Health Indicators Warehouse have helped providing a central access point for many of these datasets.

As I wrote just recently, other governments are following suit. The UK launched their open data site data.gov.uk in 2010 and has just reaffirmed their commitment to open data with an additional open data push. Statistics Canada is making all online data available for free, forsaking revenue for creating greater economic value. The EU estimates to create a EUR 32bn market for public data with their new Open Data Strategy.

Todd Park has a recipe for creating this market. "Market the hell out of those data". You can't stop at making them available, you have to spread the word. A lot of innovative ideas for products and services will come from entrepreneurs and developers that may not even know these data exist. The data have to be so easily accessible and well-promoted that the data find the user rather than the other way around.

So here's the key message:

  • Open your data
  • Make them easy to work with (ideally as linked open data: machine readable, non-proprietary formats, RDF standards, linked RDF)
  • Make sure that everyone knows they are available

That's how governments can create real value with data.

HealthFusion's YourHealthFile: enabling more patient engagement

on Wed, 11/30/2011 - 14:58

I am a big believer in empowering patients to play a bigger role in managing their health, their treatments and their health data. That's why I was sorry to see Google Health go.

HealthFusion today announced the launch of a new product called YourHealthFile which helps shift the balance of engagement towards the patient. YourHealthFile will be tightly integrated with HealthFusion's electronic health record (EHR) system MediTouch and enable patients to provide key information like chief complaint and medical history before their appointment. This should improve work flows, reduce the time a patient has to be in the provider's office and enable the physician to make informed decisions faster. The portal enables secure messaging between provider and patient, and it enables patients to review and edit their medical history, medications and prescription renewals.  HealthFusion Founder and CEO, Dr. Seth Flam says "With the YourHealthFile Patient Portal, we have found the happy medium of patient self-service and empowerment, with the proper amount of physician supervision and expert advice."

This looks like a win-win for providers and patients. Easier interaction/preparation and transparency will enable better data, improved patient engagement and ultimately better health. Hopefully, YourHealthFile willl get additional options for patients to interact with their health data, an option for quick download of all patient data (like the VA's BlueButton), and interfaces to PHRs or other providers' EHRs for automatic capture of medical and family history.

Recent developments illustrate the journey of governments opening their data vaults

on Tue, 11/29/2011 - 20:12
Governments' commitment to open data seems to be trending up. Last week, Embassy Magazine reported that Statistics Canada is planning to make all their online data available for free. The European Commission came next, announcing that it will be adopting a new Open Data Strategy. And today, UK chancellor George Osborne in his Autumn Statement announced an open data push. There are now open data sites in 28 countries, and while the availability of more and better data is a good thing to begin with, these recent announcements illustrate the progression in releasing government data:
  1. Open the vaults, e.g. by launching an open data portal
  2. Keep making more data available, remove charges, and create institutional support
  3. Open up or create better data; e.g. linked data are incredibly valuable for health research
  4. Boost the value of data by encouraging economic activity

In the United Kingdomdata.gov.uk has been around since January 2010. Now, the government is rolling out some very specific initiatives with timelines and funding commitments. Great news for health data innovators: it is planning to make more linked health data available in de-identified form. Data that link primary and secondary care data enable very interesting analyses of referrals, re-admissions, risk adjustment and outcomes. This takes open data to a new level! In addition, they will start providing institutional support for open data and tapped Sir Tim Berners-Lee and Nigel Shadbolt as co-directors of a new Open Data Institute. See the health-related details of today's announcement below.

In Canada, the move to make more data available for free came just months after the launch of the Canadian Open Data Pilot project. With this move, the Canadian government is sacrificing existing revenue streams to make more and create opportunities to create economic value from those data.

The European Commission adopted their Open Data Strategy with a very clear mission: creating a EUR 32bn market for public data. The publication of data across the 27 EU member countries has been uneven, impeded by a lack of harmonization, differences in prices and licensing, as well as differences in progress across the member countries. Now the EC is trying to help countries taking open data to the next level and creating economic opportunities. US HHS CTO Todd Park has been touting this message for a long time (see related post here). He cites as good examples the release of weather and traffic data, where a whole industry is now creating advertising, subscription and other revenues. And jobs. Releasing government data in the public domain multiplies the value of the data.

 

Let's hope that this trend continues, in terms of spreading to other countries, making more and better data available, and creating transparency and economic opportunity.
 

 
Health-related sections of the UK open data push
A.137 Linking primary and secondary healthcare datasets – The Government will provide a service to link primary and secondary healthcare datasets from September 2012 to reinforce the UK’s position as a global centre for research and analytics and boost UK life sciences.
A.138 Publishing prescribing data – The Government will publish further prescribing data by September 2012 and additional health and social care datasets by September 2013 to support health and social care data-based product and analytics markets.
A.145 Linking welfare data – The Government will consider opportunities for linking welfare datasets to other government and commercial datasets to increase their value to industry.
A.146 Open Data Institute – The Government will provide up to £10 million over five years, with match-funding from industry and academia, to establish the world’s first Open Data Institute to help business exploit the opportunities created by release of public data.
A.148 Personal data – The Government will ensure all NHS patients can access their personal GP records online by the end of this Parliament. The Government will publish a new procurement arrangement for school information and learning services in spring 2012 to improve parents’ and pupils’ access to education data and increase competition in provision of learning services.
All details on the Autumn Statement can be found here.

Pages