Skip to main content

Business and Markets

Prediction Markets, Idea Sharing, and So On

Everyone who works for a large organization has to wonder about the amount of expertise in it that never gets used. Someone else in the company may have had to solve the exact same problem you’re working on, and you may well never know, because there’s no way to realize it or track down a person who could help. So there are all sorts of schemes that have been tried to make these connections, but I’m not sure if any of them actually work.
The ones I’ve seen are e-mail lists (for querying members of the rest of the department), attempts at occasional general-problem-solving meetings, various collaborative software packages, intra-company Wikipedia-type databases, and internal prediction markets. That covers a wide range, probably because these tools are being used against a pretty heterogeneous set of problems.
There’s the specific-answer sort of query, such as “Has anyone taken this intermediate that we’re all using and done X to it?”. A broader form of this one is along the lines of “Does anyone know how to reduce an A group in the presence of a B?”. Then there are historical questions, such as “Whatever happened to these Z kinds of compounds? Why did the team that was using them back in 1990 stop working on them?”
These, at least, probably only need to go to a certain list of people. Tougher are the problems where insights might come from anywhere in the company, and this is where the advertising copy for the software packages starts to wax lyrical. Then you have the wisdom-of-crowds approach, where you’re not looking for a specific answer to a specific problem, but are interested in the opinions of a wide range of people on some question, hoping to find out more about it than you’d realize on your own. That’s where the prediction market stuff comes in.
And I’m interested in the latter idea, although I can see some problems with it. For one thing, I’m pretty sure that you’d want to have anonymity as an option. If the Big Honcho proposes an idea, how many people will vote it down under their own names? (Although any Big Honcho should realize that some of the most valuable feedback they can get is when their own name and position aren’t yet attached to a proposal). Then you have the whole participation problem – people have to feel that it’s somehow worth their time to use these things. Depending on free-floating altruism is, in my experience, not going to work out very well (and I’m not so sure how it worked out for Blanche DuBois in the end, either).
And with any of these systems, you have to be sure that you’re asking the right questions, in the right format, to the right people. A prediction-market question inside a company on the lines of “When do you think we’re going to file the NDA for Compound X?” doesn’t seem all that useful, because there are only a few people really in a position to know (and they’re not supposed to talk about it). But it would be interesting to put up the best screening hits for a nascent program, or the three or four most advanced compounds for a later-stage one, and throw the question open to the whole company of which ones they think are best. You’d want to track the results, though, to see if your crowd has any particular wisdom or not.
I think that the line you’re trying to walk with such systems is the one between solidifying groupthink and getting past it. To that end, I’d recommend (in many cases) that people not be able to see how the voting is going while it’s in progress, for fear that some participants would just jump on one bandwagon or another to save time. But I have to think that if, say, Pfizer had asked more people about the prospects for Exubera (their catastrophic inhaled-insulin product), that it might have given them the ghost of a clue that there was a chance for failure. (Or maybe not!)
Of course, now that I think about it, Pfizer has one of the more widely publicized internal idea-and-prediction-sharing efforts in the industry. (Lilly is also known for talking about this sort of thing). And I’d be interested in people who have actually experienced these, or those in other shops. Have you ever gotten any use out of these things? Or is it just something that sounds good on paper?

19 comments on “Prediction Markets, Idea Sharing, and So On”

  1. SteveM says:

    You are conflating a couple of concepts. Being able to efficiently access disparate corporate knowledge is called Knowledge Management. There are two types, Explicit and Tacit. Explicit Knowledge Management is easy. It is capturing in databases things that can be measured. Tacit Knowledge Management is much more complex. Because it is the the explication of why things happen and how decisions are made.
    However Tacit knowledge must be input by humans. And that generally is something that people don’t like to do because it is time-consuming and provides little real-time value. So implicit logic trails which are created over time in managing a business event pretty much disappear when the event is over. I don’t think anyone has yet found a way of developing a robust Tacit Knowledge Management system.
    Your other idea about prediction markets is a good one but not really that. Prediction markets are based on 0/1 thinking. People bet on whether something will be successful or it’s not. What you’re really talking about is a collaborative workspace to noodle through the pros and cons of an idea. There are some software platforms available designed for just that. They do collect ideas and comments anonymously.
    The major point of failure in using these systems is that they usually require a dispassionate facilitator to design and manage the collaborative workspace. Information collected usually has to be sorted and categorized. Unfortunately, most managers do not self-facilitate very well. But they insist on doing it. And they inject their biases into the process which often leads to a skewed outcome and demoralized participants. So great idea and off the self tools are there. But the success rate is not very impressive.
    I don’t do chemistry anymore. But the one thing I miss about it is that the molecules never argued back.

  2. Thomas McEntee says:

    Very interesting topic! SteveM makes some good points. Collaborative systems (e.g. GroupWare) can be superb tools for the right problem but (a) the software isn’t cheap, (b) typically requires a dedicated set of PCs and a LAN and (c) requires a trained facilitator who’s fluent with the software. We’ve used this system for workshops requiring the development of consensus-based decisions with some success. For the past few years, we’ve been using a web-based capability from a company called Spigit as part of our IR&D. Check it out at wwwspigitcom
    At my company, we have a small group of people who are very interested in prediction markets. FWIW, several years ago, one of them (FJS) wrote:
    “The economic problem of society is thus not merely a problem of how to allocate “given” resources …it is a problem of the utilization of knowledge which is not given to anyone in its totality.”
    Friedrich Hayek, the 1974 Nobel Prize winner in economics, wrote these words in his 1945 classic essay, The Use of Knowledge in Society, in which he argued that centrally-planned economies are unable to efficiently allocate societal resources because their planners, no matter how smart they were, would never have all the information required to make the correct decisions. Hayek, on the other hand, asserted that pricing systems, (e.g. stock markets), which reflect the collective knowledge of a myriad of individuals is a much better approach for performing resource allocation decisions. Market systems which signal the value of a resource through a numerical index (i.e. price), he said, are “a mechanism for communicating information”, with the “most significant fact about this system [being] the economy of knowledge with which it operates, or how little the individual participants need to know in order to be able to take the right action.”

  3. anon the II says:

    Having been in this industry for a while, I’ve heard a lot of talk about capturing all that institutional knowledge over the years. My sense is that it really doesn’t matter much about the software or the methodology, but comes down to whether the worker bees see their work culture as collaborative or competitive. If it’s a “you win, we all gain” environment, then knowledge is naturally shared. If it’s a “you win, I lose” environment, all the electronic notebooks, collaborative software and Kumbaya singing, off-site meetings won’t help with sharing.
    Competition is clearly a good and useful motivator, but it’s best to have internal collaboration and external competition. It’s a lot easier to achieve in a small company.
    The two companies that you mentioned have very competitive internal environments. Maybe that’s why they spend so much time on this. Alas, to no avail.

  4. Jose says:

    I can only suspect that all the recent pogroms have not exactly helped the Kumbaya-singing, collaborative spirit at most firms….

  5. Anonymous says:

    My experience with Web 2.0 tools is that it becomes another place to be seen to be influential, rather than greatly improving productivity. The internal wiki at Big Blue works well although there’s loads of things that aren’t on there, but the “idea farm challenges” seem more a chance to brown nose than actually accomplish anything.

  6. Hap says:

    #4: Particularly if either 1) your company sees your knowledge as valueless (otherwise, why dump all your midcareer people regardless of performance?) or 2) what you know is all that separates you from the unemployment line (like PhDs in general). Sacrificial lambs aren’t likely to explain in detail how you might best slaughter them, after all.

  7. Cartesian says:

    It makes me think about some researches I did for Huntsman Tioxide, but the opinion of the persons of the lab was enough for this case, but it was in collaboration with Corus (Steel Industry). Nonetheless I can remember that internet was very useful in order to find a minimum of informations about what has been done, in order to defend the potential of the idea on the market.

  8. Cloud says:

    The most interesting idea I’ve seen in this space recently was the company that was setting up an internal Twitter-like thing, where you could tweet what you were working on, and then if anyone saw it and had input, they could contact you. I can’t remember the name of the company that provided this service (basically, secure Twitter) and I am not aware of anyone actually using it. But it is an interesting idea.
    There are really good examples of community knowledge resources in the IT/software sector. It turns out that a lot of people WILL contribute answers, because they see value in looking knowledgeable. StackOverflow is one of the current favorites. The folks that make StackOverflow now sell the underlying engine, so you could set it up within a company and use that to collaborate.

  9. mad says:

    Along the lines raised by Anon the II (post 3)
    I don’t see that there is enough corporate loyalty in most places for anyone to dole out information to help others- especially if they are “the guru” on that topic. Even in the past but especially now with scientists getting cut.
    It will feel too much like a loss of job security for most people (all my best ideas and insights are now pubic!) and when the axe come down you want to hope that something you know is still needed and not just a database query.
    I think these systems are simply doomed to fail because of human nature. Only if you job security/advancement/bomus? is tied to adding to and updating thise knowledge base does it stand a chance in a cooperate environment IMO.

  10. Mat Todd says:

    Very interesting topic, Derek. As an academic, I have not seen the software used by companies to try to foster collaborations and collect in-house expertise. Someone mentioned such software is expensive. Outside of pharma, several people are trying to do something similar with open science for specific problems, for example our Synaptic Leap project (link above). Traditional blog/wiki platforms are inadequate. There are stackoverflow-type platforms, too (e.g. But I suspect the other half of the issue concerns psychology, motivation, reputation, and other intangibles. Many people are happy to collaborate in private, but not in public.

  11. Cloud says:

    Just a quick comment on the social dynamics of this… in the IT/software industry, participation in knowledge sharing forums is seen as a smart career move, not a negative. It demonstrates to potential employers that you know your stuff. Of course, this only works if what you say in these forums is good, and it can’t look like you spend all day on the forums.
    Inside a company, being seen to be the “alpha geek” (i.e., the one everyone comes to with their questions) is generally seen as enhancing job security, too.
    However, the IT/software industry is farther down the outsourcing road than pharma is, and people have had time to adjust to the new reality of zero job security.

  12. CMCguy says:

    SteveM has good analysis although in terms of Explicit Knowledge it does not necessary always happen formally and is usually easier within particular disciplines. For instance if Chemist knows someone in the Company got their PhD from a lab (because attended their interview seminar) who is source of a procedure you are using then going to them may provide direct insight or can be a go between to the person that can answer. On the other hand may not be as aware of the skill sets or backgrounds of other functions unless take time to interact and learn from them, which often people getting overly fixated on there own work at the expense of building up contacts outside their areas.
    These are potential advantages that larger organizations have over small in terms of availability of Resources but also suffer from more barriers to connection of individuals that have the Q’s vs ones with A’s. The best mid & senior Managers can act as Match-makers at times although another rare capability with many who get promoted seemingly to quickly lose touch with lab operations.

  13. SteveM says:

    I enjoyed following this thread because I transitioned from the Bench to consulting that includes group facilitation as well as quant analytics. About the software, it’s more affordable than it used to be. And it’s also web based. Moreover, you can run sessions off the vendor’s server, so session time can be purchased on a per use basis.
    About facilitation. That’s the key. (Not simply because I do it.) But because managing content which has emotions embedded in it as well as a management hierarchy is much more challenging than simple information sharing. I.e., stuff you see in a closed Wiki. Moreover, someone has to help parse out what the real decision drivers are because they are threaded throughout the collaborative workspace. And then if it’s done right, facilitate converting the drivers to objectives, prioritizing them and mapping decision alternatives to how well they meet the objectives.
    An academic Russell Ackoff who did what I do now said that often there are not business problems that have to be solved, rather there are business messes that have to be managed.

  14. Noam Danon says:

    Very interesting discussion!
    The discussion really talks about two kinds of solutions:
    – Allowing people to submit ideas, brain-storm on ideas, vote & rate ideas etc – this is our idea management solution.
    – Inviting participant to predict future events – will certain products be more successful than others, market trends, sales forecasting etc – this is done with prediction markets.
    I’m the CEO of “one of those companies” (Qmarkets), that provide enterprise prediction markets and idea management solutions. And price-wise, you don’t have to be Google in order to afford it…
    I would have to agree that company culture plays a significant part in the success or failure of our implementations, as some of you have mentioned.
    Some comments regarding the points raised:
    Anonymity – that’s an inherent part of any prediction markets. No one (other than admins) actually know who bet on what. This makes participant independent and free to express their real opinions.
    Not seeing the voting while its done – that is contradicting the basic elements of prediction markets. Just like in the stock market, being able to see the current “price” and the history is one of the most important pieces of information you can provide participants with.
    Yes, some will just follow the herd, but others will lead. All experiments show that this is critical to the success of the market, and taking that out does not provide the best results.

  15. Nicolae says:

    I predict that Derek Lowe would pitch a perfect game this year and have an ERA of 3.1 this season. Good luck Derek!!

  16. DrSnowboard says:

    Have found Compendium useful for collective issue mapping / decision trees ( even in a non-expert, what do we all think kind of way. Helps not go round in circles, as all points of view get recorded / edited.
    But then my colaborators all work in one building. Getting people to use a wiki and still talk to each other is a challenge, I feel.

  17. cliffintokyo says:

    Some really excellent comments here on this topic. I especially enjoyed the clear thinking in Steve M’s posts.
    Knowledge Management (KM) is one of the key tools described in the ICH Q10 Guideline on Pharmaceutical Quality-Management systems (now being enthusiastically implemented; led by big pharma, interestingly enough).
    Good KM (not necessarily computer-based) is essential for the smooth and efficient transfer of a pharmaceutical product from development, to pilot scale, to commercial manufacturing.
    As you all know, these activities are each handled by different departments, with quite different priorities, in all but the smallest companies. Generally this process is a lot more painful in companies with an internally competitive culture…. (would you agree, CMC Guy?)

  18. cientificorojo says:

    A recent talk by one of the R&D big wigs at Pfizer who oversaw the development of Exubera gave me the impression that they had absolutely no idea how badly the product was going to fail. Seemed a little bitter about it, in fact, almost as if Pfizer thinks the public foolishly didn’t catch on.

  19. TFox says:

    I’ve spent time on a play money prediction market (Idea Futures, many years ago) and also a real money one (Intrade, where I ended up ahead in the end). In my experience these markets do a reasonable job of reflecting a realistic current view of a proposition when all of the following are true: 1) real money is involved, and market entry/exit is easy 2) the proposition is clearly defined, and will be decided in the near future (eg 18 months max) 3) the proposition is neither too likely nor unlikely (maybe 10-1 max) and 3) the proposition is of wide interest. Missing any one of these, the current trading value can lose all predictive power. They can be gamed, eg a political partisan can bet one way to “advance” their candidate in a political market, but too much gaming attracts investors, ready to extract capital from the irrational player. The exact wording of the proposition matters too, eg Bush-Gore was decided in favor of Gore on Intrade, since the proposition referred to % of popular vote, not which candidate was inaugurated. I can’t imagine how any of these markets could be useful inside of a corporation.

Comments are closed.