O'Reilly Radar

Another Serving of Data Skepticism

May 6, 2013 - 5:25pm

I was thrilled to receive an invitation to a new meetup: the NYC Data Skeptics Meetup. If you’re in the New York area, and you’re interested in seeing data used honestly, stop by!

That announcement pushed me to write another post about data skepticism. The past few days, I’ve seen a resurgence of the slogan that correlation is as good as causation, if you have enough data. And I’m worried. (And I’m not vain enough to think it’s a response to my first post about skepticism; it’s more likely an effect of Cukier’s book.) There’s a fundamental difference between correlation and causation. Correlation is a two-headed arrow: you can’t tell in which direction it flows. Causation is a single-headed arrow: A causes B, not vice versa, at least in a universe that’s subject to entropy.

Let’s do some thought experiments–unfortunately, totally devoid of data. But I don’t think we need data to get to the core of the problem. Think of the classic false correlation (when teaching logic, also used as an example of a false syllogism): there’s a strong correlation between people who eat pickles and people who die. Well, yeah. We laugh. But let’s take this a step further: correlation is a double headed arrow. So not only does this poor logic imply that we can reduce the death rate by preventing people from eating pickles, it also implies that we can harm the chemical companies that produce vinegar by preventing people from dying. And here we see what’s really happening: to remove one head of the double-headed arrow, we use “common sense” to choose between two stories: one that’s merely silly, and another that’s so ludicrous we never even think about it. Seems to work here (for a very limited value of “work”); but if I’ve learned one thing, it’s that good old common sense is frequently neither common nor sensible. For more realistic correlations, it certainly seems ironic that we’re doing all this data analysis just to end up relying on common sense.

Now let’s look at something equally hypothetical that isn’t silly. A drug is correlated with reduced risk of death due to heart failure. Good thing, right? Yes–but why? What if the drug has nothing to do with heart failure, but is really an anti-depressant that makes you feel better about yourself so you exercise more? If you’re in the “correlation is as good as causation” club, doesn’t make a difference: you win either way. Except that, if the key is really exercise, there might be much better ways to achieve the same result. Certainly much cheaper, since the drug industry will no doubt price the pills at $100 each. (Tangent: I once saw a truck drive up to an orthopedist’s office and deliver Vioxx samples with a street value probably in the millions…) It’s possible, given some really interesting work being done on the placebo effect, that a properly administered sugar pill will make the patient feel better and exercise, yielding the same result. (Though it’s possible that sugar pills only work as placebos if they’re expensive.) I think we’d like to know, rather than just saying that correlation is just as good as causation, if you have a lot of data.

Perhaps I haven’t gone far enough: with enough data, and enough dimensions to the data, it would be possible to detect the correlations between the drug, psychological state, exercise, and heart disease. But that’s not the point. First, if correlation really is as good as causation, why bother? Second, to analyze data, you have to collect it. And before you collect it, you have to decide what to collect. Data is socially constructed (I promise, this will be the subject of another post), and the data you don’t decide to collect doesn’t exist. Decisions about what data to collect are almost always driven by the stories we want to tell. You can have petabytes of data, but if it isn’t the right data, if it’s data that’s been biased by preconceived notions of what’s important, you’re going to be misled. Indeed, any researcher knows that huge data sets tend to create spurious correlations.

Causation has its own problems, not the least of which is that it’s impossible to prove. Unfortunately, that’s the way the world works. But thinking about cause and how events relate to each other helps us to be more critical about the correlations we discover. As humans we’re storytellers, and an important part of data work is building a story around the data. Mere correlations arising from a gigantic pool of data aren’t enough to satisfy us. But there are good stories and bad ones, and just as it’s possible to be careful in designing your experiments, it’s possible to be careful and ethical in the stories you tell with your data. Those stories may be the closest we get ever get to an understanding of cause; but we have to realize that they’re just stories, that they’re provisional, and that better evidence (which may just be correlations) may force us to retell our stories at any moment. Correlation is as good as causation is just an excuse for intellectual sloppiness; it’s an excuse to replace thought with an odd kind of “common sense,” and to shut down the discussion that leads to good stories and understanding.

Categories: Tech: General Technology

Four short links: 6 May 2013

May 6, 2013 - 6:40am

1. Nautilus — elegantly-designed science web ‘zine. Includes Artificial Emotions on AI, neuro, and psych efforts to recognise and simulate emotions.
2. A Short Essay on 3D Printing — This hands-off approach to culpability cannot last long. If you design something to go into someone’s bathroom, it will make it’s way into their childs mouth. If someone buys, downloads and prints a case for their OUYA and they suffer an electric shock as a result, who is to blame? If a person replaces their phone case with a 3D printed one, and it doesn’t survive a drop to the floor, what then? We need to create a new chain of responsiblity for this emerging, and potentially very profitable business. (via Near Future Laboratory)
3. Zuckerberg’s FWD.us PAC (Anil Dash) — One of Mark Zuckerberg’s most famous mottos is “Move fast and break things.” When it comes to policy impacting the lives of millions of people around the world, there couldn’t be a worse slogan. Let’s see if we can get FWD.us to be as accountable to the technology industry as it purports to be, since they will undoubtedly claim to have the grassroots support of our community regardless of whether that’s true or not.
4. Pirate Economics — four dimensions of pirate institutions. Not BitTorrent pirates, but Berbers and arr-harr-avast-ye-swabbers nautical pirates. Pirate crews not only elected their captains on the basis of universal pirate suffrage, but they also regularly deposed them by democratic elections if they were not satisfied with their performance. Like the Berbers, or the US constitution, pirates didn’t just rely on democratic elections to keep their leaders under check. Though the captain of the ship was in charge of battle and strategy, pirate crews also used a separate democratic election to elect the ship’s quartermaster who was in charge of allocating booty, adjudicating disputes and administering discipline. Thus they had a nascent form of separation of powers.

Categories: Tech: General Technology

When industrial revolutions collide

May 3, 2013 - 6:38pm

Jon Bruner and I continued our Radar coffee talk series of conversations at Astro Coffee in Detroit’s Corktown. In the shadow of the abandoned Michigan Central Station we reflected on what we think of as a collision between the second industrial revolution (electric mechanization) and the third (information and networks). Will Google’s driverless car be to the automobile industry what Amazon has been to publishing?

The now abandoned Michigan Central Station has become something of a metaphor for the industrial age.

We hope you enjoy the conversation and we’re looking forward to your comments. You can listen to it here in the embedded player or you can also subscribe to them at iTunes.

p.s. If you have a favorite coffee shop, tell us about it in the comments. Maybe we’ll have our next coffee talk there.

Categories: Tech: General Technology

Four short links: 3 May 2013

May 3, 2013 - 3:02am

1. Causal Entropic Forces (PDF) — new paper from Sci Foo alum Alex Wissner-Gross connecting intelligence and entropy. (via Inside Science)
2. Nyan Cat and Keyboard Cat Are Trademarked Memes (Ars Technica) — the business of this (presumably there will be royalties in the end) is less interesting to me than the murky tension between authorship, ownership, sharing, popularity, and profit. We still lack a common expectation for how memes can be owned and exploited.
3. Wink UI — Mike DiGiovanni wrote a Glass app to take photos when you wink. (via Ars Technica)
4. Stealing US Military Secrets (Bloomberg) — One former intelligence official described internal Pentagon discussions over whether another Lockheed Martin fighter jet, the F-22 Raptor, could safely be deployed in combat, because several subcontractors had been hacked. The article is full of horror stories about Chinese penetration of US military contractors.

Categories: Tech: General Technology

Linking open data to augmented intelligence and the economy

May 2, 2013 - 11:39am

After years of steady growth, open data is now entering into public discourse, particularly in the public sector. If President Barack Obama decides to put the White House’s long-awaited new open data mandate before the nation this spring, it will finally enter the mainstream.

As more governments, businesses, media organizations and institutions adopt open data initiatives, interest in the evidence behind  release and the outcomes from it is similarly increasing. High hopes abound in many sectors, from development to energy to health to safety to transportation.

“Today, the digital revolution fueled by open data is starting to do for the modern world of agriculture what the industrial revolution did for agricultural productivity over the past century,” said Secretary of Agriculture Tom Vilsack, speaking at the G-8 Open Data for Agriculture Conference.

As other countries consider releasing their public sector information as data and machine-readable formats onto the Internet, they’ll need to consider and learn from years of effort at data.gov.uk, data.gov in the United States, and Kenya in Africa.

One of the crucial sources of analysis for the success or failure of open data efforts will necessarily be research institutions and academics. That’s precisely why research from the Open Data Institute and Professor Nigel Shadbolt (@Nigel_Shadbolt) will matter in the months and years ahead.

In the following interview, Professor Shadbolt and I discuss what lies ahead. His responses were lightly edited for content and clarity.

How does your research on artificial intelligence (AI) relate to open data?

AI has always fascinated me. The quest for understanding what makes us smart and how we can make computers smart has always engaged me. While we’re trying to understand the principles of human intelligence and build a “brain in a box, smarter robots” or better speech processing algorithms, the world’s gone and done a different kind of AI: augmented intelligence. The web, with billions of human brains, has a new kind of collective and distributive capability that we couldn’t even see coming in AI. A number of us have coined a phrase, “Web science,” to understand the Web at a systems level, much as we do when we think about human biology. We talk about “systems biology” because there are just so many elements: technical, organizational, cultural.

The Web really captured my attention ten years ago as this really new manifestation of collective problem-solving. If you think about the link into earlier work I’d done, in what was called “knowledge engineering” or knowledge-based systems, there the problem was that all of the knowledge resided on systems on people’s desks. What the web has done is finish this with something that looks a lot like a supremely distributed database. Now, that distributed knowledge base is one version of the Semantic Web. The way I got into open data was the notion of using linked data and semantic Web technologies to integrate data at scale across the web — and one really high value source of data is open government data.

What was the reason behind the founding and funding of the Open Data Institute (ODI)?

The open government data piece originated in work I did in 2003 and 2004. We were looking at this whole idea of putting new data-linking standards on the Web. I had a project in the United Kingdom that was working with government to show the opportunities to use these techniques to link data. As in all of these things, that work was reported to Parliament. There was real interest in it, but not really top-level heavy “political cover” interest. Tim Berners-Lee’s engagement with the previous prime minister led to Gordon Brown appointing Tim and I to look at setting up data.gov.uk, getting data released and then the current coalition government taking that forward.

Throughout this time, Tim and I have been arguing that we could really do with a central focus, an institute whose principal motivation was working out how we could find real value in this data. The ODI does exactly that. It’s got about $16 million of public money over five years to incubate companies, build capacity, train people, and ensure that the public sector is supplying high quality data that can be consumed. The fundamental idea is that you ensure high quality supply by generating a strong demand side. The good demand side isn’t just public sector, it’s also the private sector.

What have we learned so far about what works and what doesn’t? What are the strategies or approaches that have some evidence behind them?

I think there are some clear learnings. One that I’ve been banging on about recently has been that yes, it really does matter to turn the dial so that governments have a presumption to publish non-personal public data. If you would publish it anyway, under a Freedom of Information request or whatever your local legislative equivalent is, why aren’t you publishing it anyway as open data? That, as a behavioral change. is a big one for many administrations where either the existing workflow or culture is, “Okay, we collect it. We sit on it. We do some analysis on it, and we might give it away piecemeal if people ask for it.” We should construct publication process from the outset to presume to publish openly. That’s still something that we are two or three years away from, working hard with the public sector to work out how to do and how to do properly.

We’ve also learned that in many jurisdictions, the amount of [open data] expertise within administrations and within departments is slight. There just isn’t really the skillset, in many cases. for people to know what it is to publish using technology platforms. So there’s a capability-building piece, too.

One of the most important things is it’s not enough to just put lots and lots of datasets out there. It would be great if the “presumption to publish” meant they were all out there anyway — but when you haven’t got any datasets out there and you’re thinking about where to start, the tough question is to say, “How can I publish data that matters to people?”

The data that matters is revealed in the fact that if we look at the download stats on these various UK, US and other [open data] sites. There’s a very, very distinctive parallel curve. Some datasets are very, very heavily utilized. You suspect they have high utility to many, many people. Many of the others, if they can be found at all, aren’t being used particularly much. That’s not to say that, under that long tail, there isn’t large amounts of use. A particularly arcane open dataset may have exquisite use to a small number of people.

The real truth is that it’s easy to republish your national statistics. It’s much harder to do a serious job on publishing your spending data in detail, publishing police and crime data, publishing educational data, publishing actual overall health performance indicators. These are tough datasets to release. As people are fond of saying, it holds politicians’ feet to the fire. It’s easy to build a site that’s full of stuff — but does the stuff actually matter? And does it have any economic utility?

Page views and traffic aren’t ideal metrics for measuring success for an open data platform. What should people measure, in terms of actual outcomes in citizens’ lives? Improved services or money saved? Performance or corrupt politicians held accountable? Companies started or new markets created?

You’ve enumerated some of them. It’s certainly true that one of the challenges is to instrument the effect or the impact. Actually, it’s the last thing that governments, nation states, regions or cities who are enthused to do this thing do. It’s quite hard.

Datasets, once downloaded, may then be virally reproduced all over the place, so that you don’t notice it from a government site. One of the requirements in most of the open licensing which is so essential to this effort is usually has a requirement for essential attribution. Those licenses should be embedded in the machine readable datasets themselves. Not enough attention is paid to that piece of process, to actually noticing when you’re looking at other applications, other data and publishing efforts, that attribution is there. We should be smarter about getting better sense from the attribution data.

The other sources of impact, though: How do you evidence actual internal efficiencies and internal government-wide benefits of open data? I had an interesting discussion recently, where the department of IT had said, “You know, I thought this was all stick and no carrot. I thought this was all in overhead, to get my data out there for other people’s benefits, but we’re now finding it so much easier to re-consume our own data and repurpose it in other contexts that it’s taken a huge amount of friction out of our own publication efforts.”

Quantified measures would really help, if we had standard methods to notice those kinds of impacts. Our economists, people whose impact is around understanding where value is created, really haven’t embraced open markets, particularly open data markets, in a very substantial way. I think we need a good number of capable economists pilling into this, trying to understand new forms of value and what the values are that are created.

I think a lot of the traditional models don’t stand up here. Bizarrely, it’s much easier to measure impact when information scarcity exists and you have something that I don’t, and I have to pay you a certain fee for that stuff. I can measure that value. When you’ve taken that asymmetry out, when you’ve made open data available more widely, what are the new things that flourish? In some respects, you’ll take some value out of the market, but you’re going to replace it by wider, more distributed, capable services. This is a key issue.

The ODI will certainly be commissioning and is undertaking work in this area. We published a piece of work jointly with Deloitte in London, looking at evidence-linked methodology.

You mentioned the demand-side of open data. What are you learning in that area — and what’s being done?

There’s an interesting tension here. If we turn the dial in the governmental mindset to the “presumption to publish” — and in the UK, our public data principles actually embrace that as government policy — you are meant to publish unless there’s an issue in personal information or national security why you would not. In a sense, you say, “Well, we just publish everything out there? That’s what we’ll do. Some of it will have utility, and some of it won’t.”

When the Web took off, and you offered pages as a business or an individual, you didn’t foresee the link-making that would occur. You didn’t foresee that PageRank would ultimately give you a measure of your importance and relevance in the world and could even be monetized after the fact. You didn’t foresee that those pages have their own essential network effect, that the more pages there are that interconnect, that there’s value being created out of it and so there’s is a strong argument [for publishing them].

So, you know, just publish. In truth, the demand side is an absolutely great and essential test of whether actually [publishing data] does matter.

Again, to take the Web as an analogy, large amounts of the Web are unattended to, neglected, and rot. It’s just stuff nobody cares about, actually. What we’re seeing in the open data effort in the UK is that it’s clear that some data is very privileged. It’s at the center of lots of other datasets.

In particular, [data about] location, occurrence, and when things occurred, and stable ways of identifying those things which are occurring. Then, of course, the data space that relates to companies, their identifications, the contracts they call, and the spending they engage in. That is the meat and drink of business intelligence apps all across the planet. If you started to turn off an ability for any business intelligence to access legal identifiers or business identifiers, all sorts of oversight would fall apart, apart from anything else.

The demand side [of open data] can be characterized. It’s not just economic. It will have to do with transparency, accountability and regulatory action. The economic side of open data gives you huge room for maneuver and substantial credibility when you can say, “Look, this dataset of spending data in the UK, published by local authorities, is the subject of detailed analytics from companies who look at all data about how local authorities and governments are spending their data. They sell procurement analysis insights back to business and on to third parties and other parts of the business world, saying ‘This is the shape of how the UK PLC is buying.’”

What are some of the lessons we can learn from how the World Wide Web grew and the value that it’s delivered around the world?

That’s always a worry, that, in some sense, the empowered get more powerful. What we do see is that, in open data in particular, new sorts of players couldn’t enter the game at all.

My favorite example is in mass transportation. In the UK, we have to fight quite hard to get some of the data from bus, rail and other forms of transportation made openly available. Until that was done, there was a pretty small number of supplies from this market.

In London, where all of it was made available from the Transport for London Authority, there’s just been an explosion of apps and businesses who are giving you subtly and distinct experiences as users of that data. I’ve got about eight or nine apps on my phone that give me interestingly distinctive views of moving about the city of London. I couldn’t have predicted or anticipated many of those exist.

I’m sure the companies who held that data could’ve spent large amounts of money and still not given me anything like the experience I now have. The flood of innovation around the data has really been significant and many, many more players and stakeholders in that space.

The Web taught us that serendipitous reuse, where you can’t anticipate where the bright idea comes from, is what is so empowering. The flipside of that is that it also reveals that, in some cases, the data isn’t necessarily of a quality that you might’ve thought. This effort might allow for civic improvement or indeed, business improvement in some cases, where businesses come and improve the data the state holds.

What’s happening in the UK with the so-called “MiData Initiative,” which posits that people have a right to access and use personal data disclosed to them?

I think this is every bit as potentially disruptive and important as open government data. We’re starting to see the emergence of what we might think of as a new class of important data, “personal assets.”

People have talked about “personal information management systems” for a long time now. Frequently, it’s revolved around managing your calendar or your contact list, but it’s much deeper. Imagine that you, the consumer, or you, the citizen, had a central locus of authority around data that was relevant to you: consumer data from retail, from the banks that you deal with, from the telcos you interact with, from the utilities you get your gas, water and electricity from. Imagine if that data infosphere was something that you could access easily, with a right to reuse and redistribute it as you saw fit.

The canonical example, of course, is health data. It isn’t all data that business holds, it’s also data the state holds, like your health records, educational transcript, welfare, tax, or any number of areas.

In the UK, we’ve been working towards empowering consumers, in particular through this MiData program. We’re trying to get to a place where consumers have a right to data held about their transactions by businesses, [released] back to them in a reusable and flexible way. We’ve been working on a voluntary program in this area for the last year. We have a consultation on taking up power to require large companies to give that information back. There is a commitment to the UK, for the first time, to get health records back to patients as data they control, but I think it has to go much more widely.

Personal data is a natural complement to open data. Some of the most interesting applications I’m sure we’re going to see in this area are where you take your personal data and enrich it with open data relating to businesses, the services of government, or the actual trading environment you’re in. In the UK, we’ve got six large energy companies that compete to sell energy to you.

Why shouldn’t groups and individuals be able to get together and collectively purchase in the same way that corporations can purchase and get their discounts? Why can’t individuals be in a spot market, effectively, where it’s easy to move from one supplier to another? Along with those efficiencies in the market and improvements in service delivery, it’s about empowering consumers at the end of the day.

This post is part of our ongoing series on the open data economy.

Categories: Tech: General Technology

Towards a more open world

May 1, 2013 - 5:47pm

Last September, I gave a 5 minute Ignite talk at the tenth Ignite DC. The video just became available. My talk, embedded below, focused on what I’ve been writing about here at Radar for the past three years: open government, journalism, media, mobile technology and more.

The 20 slides that I used for the Ignite were a condensed version of a much longer presentation I’d created for a talk on open data and journalism in Moldova, also I’ve embedded below.

Categories: Tech: General Technology

Leading Indicators

April 30, 2013 - 11:41am

In a conversation with Q Ethan McCallum (who should be credited as co-author), we wondered how to evaluate data science groups. If you’re looking at an organization’s data science group from the outside, possibly as a potential employee, what can you use to evaluate it? It’s not a simple problem under the best of conditions: you’re not an insider, so you don’t know the full story of how many projects it has tried, whether they have succeeded or failed, relations between the data group, management, and other departments, and all the other stuff you’d like to know but will never be told.

Our starting point was remote: Q told me about Tyler Brulé’s travel writing for Financial Times (behind a paywall, unfortunately), in which he says that a club sandwich is a good proxy for hotel quality: you go into the restaurant and order a club sandwich. A club sandwich isn’t hard to make: there’s no secret recipe or technique that’s going to make Hotel A’s sandwich significantly better than B’s. But it’s easy to cut corners on ingredients and preparation. And if a hotel is cutting corners on their club sandwiches, they’re probably cutting corners in other places.

This reminded me of when my daughter was in first grade, and we looked (briefly) at private schools. All the schools talked the same talk. But if you looked at classes, it was pretty clear that the quality of the music program was a proxy for the quality of the school. After all, it’s easy to shortchange music, and both hard and expensive to do it right. Oddly enough, using the music program as a proxy for evaluating school quality has continued to work through middle school and (public) high school. It’s the first thing to cut when the budget gets tight; and if a school has a good music program with excellent teachers, they’re probably not shortchanging the kids elsewhere.

How does this connect to data science? What are the proxies that allow you to evaluate a data science program from the “outside,” on the information that you might be able to cull from company blogs, a job interview, or even a job posting? We came up with a few ideas:

• Are the data scientists simply human search engines, or do they have real projects that allow them to explore and be curious? If they have management support for learning what can be learned from the organization’s data, and if management listens to what they discover, they’re accomplishing something significant. If they’re just playing Q&A with the company data, finding answers to specific questions without providing any insight, they’re not really a data science group.
• Do the data scientists live in a silo, or are they connected with the rest of the company? In Building Data Science Teams, DJ Patil wrote about the value of seating data scientists with designers, marketers, with the entire product group so that they don’t do their work in isolation, and can bring their insights to bear on all aspects of the company.
• When the data scientists do a study, is the outcome predetermined by management? Is it OK to say “we don’t have an answer” or to come up with a solution that management doesn’t like? Granted, you aren’t likely to be able to answer this question without insider information.
• What do job postings look like? Does the company have a mission and know what it’s looking for, or are they asking for someone with a huge collection of skills, hoping that they will come in useful? That’s a sign of data science cargo culting.
• Does management know what their tools are for, or have they just installed Hadoop because it’s what the management magazines tell them to do? Can managers talk intelligently to data scientists?
• What sort of documentation does the group produce for its projects? Like a club sandwich, it’s easy to shortchange documentation.
• Is the business built around the data? Or is the data science team an add-on to an existing company? A data science group can be integrated into an older company, but you have to ask a lot more questions; you have to worry a lot more about silos and management relations than you do in a company that is built around data from the start.

Coming up with these questions was an interesting thought experiment; we don’t know whether it holds water, but we suspect it does. Any ideas and opinions?

Categories: Tech: General Technology

White House Science Fair praises future scientists and makers

April 29, 2013 - 4:18pm

There are few ways to better judge a nation’s character than to look at how its children are educated. What values do their parents, teachers and mentors demonstrate? What accomplishments are celebrated? In a world where championship sports teams are idolized and superstar athletes are feted by the media, it was gratifying to see science, students and teachers get their moment in the sun at the White House last week.

“…one of the things that I’m concerned about is that, as a culture, we’re great consumers of technology, but we’re not always properly respecting the people who are in the labs and behind the scenes creating the stuff that we now take for granted,” said President Barack Obama, “and we’ve got to give the millions of Americans who work in science and technology not only the kind of respect they deserve but also new ways to engage young people.”

President Barack Obama talks with Evan Jackson, 10, Alec Jackson, 8, and Caleb Robinson, 8, from McDonough, Ga., at the 2013 White House Science Fair in the State Dining Room. (Official White House Photo by Chuck Kennedy)

An increasingly fierce global competition for talent and natural resources has put a premium on developing scientists and engineers in the nation’s schools. (On that count, last week, the President announced a plan to promote careers in the sciences and expand federal and private-sector initiatives to encourage students to study STEM.

“America has always been about discovery, and invention, and engineering, and science and evidence,” said the President, last week. “That’s who we are. That’s in our DNA. That’s how this country became the greatest economic power in the history of the world. That’s how we’re able to provide so many contributions to people all around the world with our scientific and medical and technological discoveries.”

Unfortunately, the role models that far too much of the media hold up for young people are all too frequently pulled from the stage, screen and playing fields, as opposed to laboratories, universities and schools.

In recent years, the success of technology entrepreneurs has shifted that dynamic, but in the American academy, big time sports have been eating college life, with huge stadiums and rallies for stars and comparatively little notice given to National Merit awards or fellowship winners. When the President said in 2012 that science, technology, engineering and math (STEM) education and young people’s scientific achievements don’t belong in the back pages of newspapers, his media criticism was quietly scathing: a culture of celebrity is not geared to the more quiet, sustained achievement required to attain a graduate degree or patent, though both may have more enduring value to society than a pop album.

This is a dynamic that clearly troubles President Obama, and one that he has used the bully pulpit and the platform of the White House to drawn national attention towards over the past five years.

“If you win the NCAA championship, you come to the White House,” said the President, in 2009. “Well, if you’re a young person and you’ve produced the best experiment or design, the best hardware or software, you ought to be recognized for that achievement, too. Scientists and engineers ought to stand side by side with athletes and entertainers as role models, and here at the White House we’re going to lead by example.”

In the years since, the White House has tried to carry through on that pledge, hosting three science fairs and involving national leaders in science education, including Bill Nye,”The Science Guy,” Reading Rainbow host Levar Burton, and astrophysicist Neil deGrasse Tyson.

“The White House Science Fair is a way of showing everyone that science is cool,” said Tyson, the director of the Hayden Planetarium in New York City, in an interview. Engaging the public about the wonders of the universe and encouraging kids to be curious about how our world works has been a core part of his career, driven by his infectious good humor.

Making science and technology education more fun

As Greg Ferenstein noted at TechCrunch, there were some pretty awesome inventions at the White House science fair, from mind-controlled prosthetics to improved cancer detection methodologies to a bicycle-powered water purification system. You can see a list of the White House science fair projects at WhiteHouse.gov and watch President Obama tour the exhibits on YouTube.

One of the notable components of the science fairs has been the involvement of kids from the maker movement. In the summer of 2013, the Maker Education Initiative will host a season-long Maker Party where students can learn, design and create.

“We’re a nation of tinkerers and dreamers and believers in a better tomorrow,” said President Obama at the 2012 White House Science Fair, recognizing the long-history of creative innovation in American garages, basements and barns.

Photo: President Obama demos @joey_hudy‘s Extreme Marshmallow Cannon @ the #WHScienceFair wh.gov/0qf twitter.com/whitehouse/sta…

— The White House (@whitehouse) February 7, 2012

In 2012, President Obama famously helped young maker Joey Hudy to fire his “extreme marshmallow cannon.” 14-year old Ben Hylack, the maker of a telepresence robot, said that Makerfaire changed his life. In 2013, “Super Awesome Sylvia” represented Maker Faire at the White House Science Fair, showing the President her watercolor drawbot.

Experimenting with more independent projects that let kids tinker are important but only part of a puzzle that includes parents, teachers and libraries.

“We should be focused on getting kids ‘making’, yes, but that misses a tacit recognition that kids are by nature scientists,” said Tyson. “What we should be talking about is how to keep kids interested and get out of their way as they learn.”

For that to happen, we’ll need to encourage children to keep asking questions, teach them how to learn to answer them, and praise inquisitive students.

“Acts of curiosity are what make up acts of science,” he said. “Adult scientists are just kids who never grew up.”

Categories: Tech: General Technology

Google Glass and the Future

April 29, 2013 - 1:29pm

I just read a Forbes article about Glass, talking about the split between those who are “sure that it is the future of technology, and others who think society will push back against the technology.”

I don’t see this as a dichotomy (and, to be fair, I’m not sure that the author does either). I expect to see both, and I’d like to think a bit more about what these two apparently opposing sides mean.

Push back is inevitable. I hope there’s a significant push back, and that it has some results. Not because I’m a Glass naysayer, but because we, as technology users, are abused so often, and push back so weakly, that it’s not funny. Facebook does something outrageous; a few technorati whine; they add option 1023 to their current highly intertwined 1022 privacy options that have been designed so they can’t be understood or used effectively; and sooner or later, it all dies down. A hundred fifty users have left Facebook, and half a million more have joined. When Apple puts another brick in their walled garden, a few dozen users (myself included) bitch and moan, but does anyone leave? Personally, I’m tired of getting warnings whenever I install software that doesn’t come from the Apple Store (I’ve used the Store exactly twice), and I absolutely expect that a not-too-distant version of OS X won’t allow me to install software from “untrusted” sources, including software I’ve written. Will there be push back? Probably. Will it be effective? I don’t know; if things go as they are now, I doubt it.

There will be push back against Glass; and that’s a good thing. I think Google, of all the companies out there, is most likely to listen and respond positively. I say that partly because of efforts like the Data Liberation Front, and partly because Eric Schmidt has acknowledged that he finds many aspects of Glass creepy. But going beyond Glass: As a community of users, we need to empower ourselves to push back. We need to be able to push back effectively against Google, but more so against Apple, Facebook, and many other abusers of our data, rather than passively accept the latest intrusion as an inevitability. If Glass does nothing more than teach users that they can push back, and teach large corporations how to respond constructively, it will have accomplished much.

Is Glass the future? Yes; at least, something like Glass is part of the future. As a species, we’re not very good at putting our inventions back into the box. About three years ago, there was a big uptick in interest in augmented reality. You probably remember: Wikitude, Layar, and the rest. You installed those apps on your phone. They’re still there. You never use them (at least, I don’t). The problem with consumer-grade AR up until now has been that it was sort of awkward walking around looking at things through your phone’s screen. (Commercial AR–heads-up displays and the like–is a completely different ball game.) Glass is the first attempt at broadly useful platform for consumer AR; it’s a game changer.

Could Glass fail? Sure; I know more failed startups than I can count where the engineers did something really cool, and when they released it, the public said “what is that, and why do you think we’d want it?” Google certainly isn’t immune from that disease, which is endemic to an engineering-driven culture; just think back to Wave. I won’t deny that Google might shelve Glass if they consider unproductive, as they’ve shelved many popular applications. But I believe that Google is playing long-ball here, and thinking far beyond 2014 or 2015. In a conversation about Bitcoin last week, I said that I doubt it will be around in 20 years. But I’m certain we will have some kind of distributed digital currency, and that currency will probably look a lot like Bitcoin. Glass is the same. I have no doubt that something like Glass is part of our future. It’s a first, tentative, and very necessary step into a new generation of user interfaces, a new way of interacting with computing systems and integrating them into our world. We probably won’t wear devices around on our glasses; it may well be surgically implanted. But the future doesn’t happen if you only talk about hypothetical possibilities. Building the future requires concrete innovation, building inconvenient and “creepy” devices that nevertheless point to the next step. And it requires people pushing back against that innovation, to help developers figure out what they really need to build.

Glass will be part of our future, though probably not in its current form. And push back from users will play an essential role in defining the form it will eventually take.

Categories: Tech: General Technology

Glowing Plants

April 26, 2013 - 4:06pm

I just invested in BioCurious’ Glowing Plants project on Kickstarter. I don’t watch Kickstarter closely, but this is about as fast as I’ve ever seen a project get funded. It went live on Wednesday; in the afternoon, I was backer #170 (more or less), but could see the number of backers ticking upwards constantly as I watched. It was fully funded for $65,000 Thursday; and now sits at 1340 backers (more by the time you read this), with about $84,000 in funding. And there’s a new “stretch” goal: if they make $400,000, they will work on bigger plants, and attempt to create a glowing rose.

Glowing plants are a curiosity; I don’t take seriously the idea that trees will be an alternative to streetlights any time in the near future. But that’s not the point. What’s exciting is that an important and serious biology project can take place in a biohacking lab, rather than in a university or an industrial facility. It’s exciting that this project could potentially become a business; I’m sure there’s a boutique market for glowing roses and living nightlights, if not for biological street lighting. And it’s exciting that we can make new things out of biological parts.

In a conversation last year, Drew Endy said that he wanted synthetic biology to “stay weird,” and that if in ten years, all we had accomplished was create bacteria that made oil from cellulose, we will have failed. Glowing plants are weird. And beautiful. Take a look at their project, fund it, and be the first on your block to have a self-illuminating garden.

Categories: Tech: General Technology

IT and Engineers

April 26, 2013 - 12:51pm

Jon Bruner and I got together last week in Cambridge, MA to have a cup of coffee and talk about the industrial internet. During this conversation we mused on the inevitable collision of cultures when Silicon Valley meets industrial heartland, or, when software people meet hardware people.

Of course there have always been relationships between hardware and software, we aren’t discovering anything completely new here. But there are differences now, and they are turning up in interesting places. I attended Distributech recently, the biggest electric utility conference for power distribution, and was surprised to see that how big a push traditional IT vendors were making into the space. This is an event that five years ago would have only had electrical component vendors as sponsors but this time around Oracle was the most visible sponsor there with Cisco not far behind. The world’s first massive electrical network is now absorbing (and increasingly being controlled by) its second.

Anyway, I hope you enjoy the conversation and we’re looking forward to your comments. And given that on the Radar team we like to talk at least as much as we like to write, if this goes well we’ll probably do a lot more of them. You can listen to it here in the embedded player or you can also subscribe to them at iTunes.

Categories: Tech: General Technology

The makers of hardware innovation

April 26, 2013 - 9:19am

Chris Anderson wrote Makers and went from editor-in-chief of Wired to CEO of 3D Robotics, making his hobby his side job and then making it his main job.

A new executive at Motorola Mobility, a division of Google, said that Google seeks to “googlify” hardware. By that he meant that devices would be inexpensive, if not free, and that the data created or accessed by them would be open. Motorola wants to build a truly hackable cellphone, one that makers might have ideas about what to do with it.

Regular hardware startup meetups, which started in San Francisco and New York, are now held in Boston, Pittsburgh, Austin, Chicago, Dallas and Detroit. I’m sure there are other American cities. Melbourne, Stockholm and Toronto are also organizing hardware meetups. Hardware entrepreneurs want to find each other and learn from each other.

Hardware-oriented incubators and accelerators are launching on both coasts in America, and in China.

The market for personal 3D printers and 3D printing services has really taken off. 3D printer startups continue to launch, and all of them seem to have trouble keeping up with demand. MakerBot is out raising money. Shapeways raised $30 million in a new round of financing announced this week.

Makers are discovering that the Raspberry PI, developed for educational uses, can fit into some interesting commercial niches.

The marketing-friendly phrase, “Internet of Things,” is beginning to mean something, with new boards such as Pinoccio and Electric Imp.

Design software is getting better, and less expensive, if not free, although the developers of TinkerCad announced that they were abandoning it.

And an 11-year old maker, Super Awesome Sylvia, was recognized at the White House Science Fair, exhibiting a watercolor robot that will soon be a kit sold through Evil Mad Science.

“Hardware is the new software” reported Wired and the New York Times. Joi Ito of the MIT Media Lab said it was one of the top trends to watch in 2013.

This year’s Hardware Innovation Workshop, held May 14-15 at the College of San Mateo in San Mateo, Calif., during the week leading up to Maker Faire Bay Area, will provide a deep dive into the new world of hardware startups. You’ll learn what VCs are thinking about hardware startups, which startups got funding and why. You’ll meet dozens of newly formed startups that haven’t launched yet. You’ll also learn from maker case studies and from the founders of hardware incubators.

Among our speakers are:

• Chris Anderson, CEO of 3D Robotics and founder of DIY Drones
• Massimo Banzi, co-founder of Arduino
• Robert Faludi, collaborative strategy leader at Digi International
• Bunnie Huang, co-founder of Chumby
• Ben Kaufman, founder and CEO of Quirky
• Scott Miller, CEO and co-founder of Dragon Innovation
• Alice Taylor, founder, Makie Lab
• John Park, COO/GM, AQS
• Carl Bass, CEO of Autodesk
• Ted Hall, CEO of ShopBot

Learn more about the Hardware Innovation Workshop. O’Reilly Radar readers can register using the code “RADAR13″ and save $100.

Categories: Tech: General Technology

Social media’s 2.0 moment: Responsiveness beats planning

April 24, 2013 - 6:58am

In 2004, O’Reilly Media delivered a counter-cultural (at the time) message: The dot-com bubble had burst, but the web was here to stay as an economic and social force. The meme they coined was Web 2.0, and their manifesto was captured in a seminal blog post by Tim O’Reilly. Web 2.0 was not meant to indicate a version number, but to point out the deep, persistent patterns of the web that were rewiring business and society.

I led the consulting practice at O’Reilly Media after we coined the term Web 2.0, and I think we now find ourselves at a similar (though softer) inflection point. There are a lot of valid questions regarding the business models in social: Is Facebook not a scalable vehicle for advertising and thus overvalued? Is Groupon bad for merchants and thus doomed to fail? Was social gaming (and Zynga) overhyped?

Taking a cue from Web 2.0, I believe we need to look beyond specific applications of social media — even, God forbid, specific platforms like Facebook — in order to sort out the underlying design patterns that will endure and continue to disrupt marketing and communications.

So what are those design patterns? Here are four:

1. Responsiveness beats planning
2. Communities beat audiences
3. Reputation beats branding
4. Sociality beats media-mentality

I’ll focus on the first one for now: responsiveness beats planning. The kernel of my argument is that the social web is pressuring organizations to accelerate all forms of communications from “batch” processing to real-time interaction. The result is a fundamentally different approach to how a marketing/communications organization needs to be structured and serviced.

Human beings have spent millennia communicating in real-time. The acceleration of technology is simply an effort to catch up to our zero-latency experience of being. Whenever given a choice, we will opt for a service that delivers response times as fast as our own nervous system. The technology and processes around us are nowhere close to catching up — yet wherever they do, we see incredible value creation. Any information processing technology that moves from batch to real-time experiences a quantum leap in value, especially for those who adopt it first. Consider the arbitrage opportunity in financial systems capable of receiving prices in real-time, real-time trading desks that place advertising based on current inventory and effectiveness, the efficiency of inventory management occurring in real-time across the supply chain and you get the idea. All of the systems that surround and support modern life are accelerating into real-time systems. Social is moving into real-time precisely because that is the speed at which human beings prefer to communicate, and social technologies that have accelerated closer to real-time are now shaping customer expectations.

What are the implications?

With the rise of real-time, responsive communications, marketing and comms are experiencing a massive acceleration in the traditional timeline needed to create branded content. This goes well beyond customer care — it is more like a dynamic content production capability that marketers need in order to sustain brand relationships.

Examples:

• The Presidential debates: After Barack Obama’s first debate with Mitt Romney, a debate in which all sides realized Obama had turned in an incredibly poor showing, the Obama camp took just three hours to pour over the debate and edit together a commercial highlighting some of Romney’s more damning statements.
• Oreo’s response to the Superbowl blackout was retweeted 15,000 times and received more than 20,000 likes within 24 hours. The graphic released during the blackout was “designed, captioned and approved within minutes,” thanks to members of 360i — the cookie company’s agency — gathered at a war room during the game.
• The popularity of apps like SnapChat and Poke are creating time-limited content and offers based on immediacy.

This type of speed and responsiveness has less to do with strategy and planning and everything to do with logistics and coordination.

It calls on marketers to actually understand how organizations are structured, how governance needs to shift to enable more responsive organizations, how we staff our accounts to develop a drumbeat of meaningful content that engages, how we equip our clients to become digital publishers of real-time communications, and how we automate as many parts of the communications "supply chain" as possible.

In the next article, I will explore the design pattern that is rewiring business, "communities beat audiences."

Categories: Tech: General Technology

Four short links: 2 May 2013

April 23, 2013 - 11:46pm

1. Metrico — puzzle game for Playstation centered around infographics (charts and graphs). (via Flowing Data)
2. The Lease They Can Do (Business Week) — excellent Paul Ford piece on money, law, and music streaming services. So this is not about technology. Nor is it really about music. This is about determining the optimal strategy for mass licensing of digital artifacts.
3. How Effective Is a Humanoid Robot as a Tool for Interviewing Young Children? (PLosONE) — The results reveal that the children interacted with KASPAR very similar to how they interacted with a human interviewer. The quantitative behaviour analysis reveal that the most notable difference between the interviews with KASPAR and the human were the duration of the interviews, the eye gaze directed towards the different interviewers, and the response time of the interviewers. These results are discussed in light of future work towards developing KASPAR as an ‘interviewer’ for young children in application areas where a robot may have advantages over a human interviewer, e.g. in police, social services, or healthcare applications.
4. Funding: Australia’s Grant System Wastes Time (Nature, paywalled) — We found that scientists in Australia spent more than five centuries’ worth of time preparing research-grant proposals for consideration by the largest funding scheme of 2012. Because just 20.5% of these applications were successful, the equivalent of some four centuries of effort returned no immediate benefit to researchers.

Categories: Tech: General Technology

Four short links: 1 May 2013

April 23, 2013 - 11:32pm

1. Pin: A Dynamic Binary Instrumentation Tool — a dynamic binary instrumentation framework for the IA-32 and x86-64 instruction-set architectures that enables the creation of dynamic program analysis tools. Some tools built with Pin are Intel Parallel Inspector, Intel Parallel Amplifier and Intel Parallel Advisor. The tools created using Pin, called Pintools, can be used to perform program analysis on user space applications in Linux and Windows. As a dynamic binary instrumentation tool, instrumentation is performed at run time on the compiled binary files. Thus, it requires no recompiling of source code and can support instrumenting programs that dynamically generate code.
2. Lasers Bringing Down Drones (Wired) — I’ve sat on this for a while, but it is still hypnotic. Autonomous attack, autonomous defence. Pessimist: we’ll be slaves of the better machine learning algorithm. Optimist: we can make love while the AIs make war.
3. Advice on Rewriting It From Scratch — every word is true. Over my career, I’ve come to place a really strong value on figuring out how to break big changes into small, safe, value-generating pieces. It’s a sort of meta-design — designing the process of gradual, safe change.
4. Creating Gmail Inbox Statistics Reports — shows how to setup gmail to send you an email at the beginning of each month showing statistics for the previous month, such as the number of emails you received, the top 5 to whom you sent email, the top 5 from whom you received email, charts on your daily usage.

Categories: Tech: General Technology

Four short links: 30 April 2013

April 23, 2013 - 11:21pm

1. China = 41% of World’s Internet Attack Traffic (Bloomberg) — numbers are from Akamai’s research. Verizon Communications said in a separate report that China accounted for 96 percent of all global espionage cases it investigated. One interpretation is that China is a rogue Internet state, but another is that we need to harden up our systems. (via ZD Net)
2. Open Source Cannot Live on Donations Alone — excellent summary of some of the sustainability questions facing open source projects.
3. China Startups: The Gold Rush (Steve Blank) — dense fact- and insight-filled piece. Not only is the Chinese ecosystem completely different but also the consumer demographics and user expectations are equally unique. 70% of Chinese Internet users are under 30. Instead of email, they’ve grown up with QQ instant messages. They’re used to using the web and increasingly the mobile web for everything, commerce, communication, games, etc. (They also probably haven’t seen a phone that isn’t mobile.) By the end of 2012, there were 85 million iOS and 160 million Android devices in China. And they were increasing at an aggregate 33 million IOS and Android activations per month.
4. Calculating Rolling Cohort Retention with SQL — just what it says. (via Max Lynch)

Categories: Tech: General Technology

Four short links: 29 April 2013

April 23, 2013 - 10:58pm

1. Information Security Breaches 2013 Report (UK Gov) — over 80% of small UK firms reported a breach, and over 90% of large. (via The Register)
2. Google Glass Forbids Resales (Wired) — leaving aside the braying naysayers with their “GLASS WILL DESTROY THE SOCIAL FABRIC AND OUR ESSENTIAL HUMANITY”, there’s a valid point about software being used to control what users do with their devices. Given that this run of Glass is limited edition and they’ve hand-picked to whom they go and for what reason, Ed from Philadelphia is both greedy and naive if he believes Google’s letting him buy a pair to resell on eBay.
3. Locked Stacks — As the British Library makes a glacially paced transition from being an analog behemoth to being a digitized one, an opportunity arises to lower the institution’s ivory tower-like walls and to create extensive access to its impressive catalog. The only problems, of course, are a lack of money and the currently insurmountable problem of UK copyright law.
4. Young Community Entrepreneurs Rebuilding Detroit (Fast Company) — from information-sharing real estate ventures to transportation startups and doomsday clocks to see how close the city is to bankruptcy, it’s a crazy world out there. Should be easy for them: Detroit comes pre-disrupted.

Categories: Tech: General Technology

Four short links: 26 April 2013

April 23, 2013 - 10:28pm

1. The Engagement Cliff — Gallup surveyed nearly 500,000 students in grades five through 12 from more than 1,700 public schools in 37 states in 2012 and found that by the time students get to high school only about 4 in 10 qualify as engaged.
2. SSL/TLS Deployment Best Practices — clear and concise instructions to help overworked administrators and programmers spend the minimum time possible to obtain a secure site or web application. In pursue of clarity, we sacrifice completeness, foregoing certain advanced topics. The focus is on advice that is practical and easy to understand.
3. Do Bad Things Happen When Works Enter The Public Domain? — research to answer that question. Spoiler: no. (via Surprisingly Free)
4. Most GitHub Projects Not Open-Source Licensed (The Register) — 1,692,135 code repositories scanned, 219,326 (14.9%) percent had a file in their top-level directories that identified any kind of license at all. Of those, 28 per cent only announced their licenses in a README file, as opposed to recommended filenames such as LICENSE or COPYING. MIT license overwhelmingly popular compared to the different reciprocal (GPL-like) ones.

Categories: Tech: General Technology

Four short links: 25 April 2013

April 23, 2013 - 10:17pm

1. Alcatraz — package manager for iOS. (via Hacker News)
2. Scarfolk Council — clever satire, the concept being a UK town stuck in 1979. Tupperware urns, “put old people down at birth”. The 1979 look is gorgeous. (via BoingBoing)
3. Stop Designing Fragile Web APIs — It is possible to design your API in a manner that reduces its fragility and increases its resilience to change. The key is to design your API around its intent. In the SOA world, this is also referred to as business-orientation.
4. @life100yearsago (Twitter) — account that tweets out fragments of New Zealand journals and newspapers and similar historic documents, as part of celebrating the surprising and the commonplace during WWI. My favourite so far: “Wizard” stones aeroplane. (via NDF)

Categories: Tech: General Technology

Four short links: 24 April 2013

April 23, 2013 - 10:01pm

1. Solar Energy: This is What a Disruptive Technology Looks Like (Brian McConnell) — In 1977, solar cells cost upwards of $70 per Watt of capacity. In 2013, that cost has dropped to $0.74 per Watt, a 100:1 improvement (source: The Economist). On average, solar power improves 14% per year in terms of energy production per dollar invested.
2. Process Managers — overview of the tools that keep your software running.
3. Bittorrent Sync — Dropbox-like features, BitTorrent under the hood.
4. Brython — Python interpreter written in Javascript, suitable for embedding in webpages. (via Nelson Minar)

Categories: Tech: General Technology