Skip to Content

General Scientific News

The Problem With Information

Here are some statements from a noted information scientist, which even he admits leads to “a pessimistic and even rather cynical conclusion“. Have a look and see if any of this is behavior that you have encountered yourself:

In many work environments, the penalties for not being diligent in the finding and use of information are minor, if they exist at all. In fact, such lack of diligence tends often to be rewarded. The man who does not fuss with information is seen at his bench, plainly at work, getting the job done. Approval goes to projects where things are happening. One must be courageous or imprudent, or both, to point out from the literature that a current laboratory project which has had an extensive history and full backing of the management was futile from the outset.

At a desk, an author of a technical report, by not making a prior literature search, and by omitting citations to earlier work, can prepare his reports so much faster, with the additional advantage that people will think the ideas presented were new and were his own. . .

The author was Calvin Mooers, and get ready: he made these remarks in 1959. He was talking to his fellow information and data retrieval specialists about the problem – which was already apparent – that not all of their users were actually taking advantage of the cornucopia of data that was being presented to them as the hypermodern 1960s approached. As Mooers noted, the default assumption among database providers was that people were starved for good information and would gratefully jump at the chance to remedy that situation. But this wasn’t necessarily true in practice.

It’s a rather different world than it was in 1959, and it would be hard to find an aspect that’s more different than access to information. I remember showing my kids a web comic a few years ago to try to explain to them was it was like Back In The Old Days (the 1970s, in this case). The title line was “Before The Internet”, and two characters were sitting on a front porch, with one of them saying “You know, I came across something really interesting the other day, and I want to learn more about it.” “Gee,” says the other, “That’s too bad”. But the effect Mooers is talking about is still with us, and has been with us for a lot longer than that. Ask Thomas Gray about when ’tis folly to be wise, although it’s true that in that poem he’s sounding even more like A. E. Housman than usual.

But I think that the modern access to information has led to another behavior that accomplishes similar psychological goals. That’s the one where people do just enough searching through the literature to confirm their own point of view, and stop right there. And our sources are now so wide and varied that indulging this tendency is easier than it’s ever been, unfortunately. This can be even more pernicious than the hands-over-the-ears technique noted by Mooers, because now you can’t challenge these people with a “Did-you-even-look” question. They did look! And found out that they’re right!

Modern variations notwithstanding, though, Mooers is certainly correct in diagnosing the “I’d Rather Not Know” phenomenon. I expect that many experienced researchers have seen just the behavior he describes, in what may well be painful memories of projects (or whole organizations) past. We’re looking at human nature here – not one of its more attractive or useful aspects, but part of the crooked timber of humanity all the same. If you want to be a better scientist – a better person, actually – you should try to remember this tendency and take steps to mitigate it. Think of the reasons that you might be wrong, or the details that might complicate what you’re doing. Now, as is always the case, there’s an opposite error to be made: don’t just sit there and think think only about all those complications; that’s a recipe for never doing anything. There are plenty of reasons for things to fail, but things do work on occasion. Concentrating exclusively on those failure modes is not a good plan – but neither is pretending that they don’t exist. Is it hard to strike that balance? Oh yeah. But we have to try.

27 comments on “The Problem With Information”

  1. Emjeff says:

    I have run into this many times. Many years ago, a compound I was working on (very early stage) had a target profile which, among other things, said “no significant drug interactions”. However, no in vitro work had been done assessing what specific enzyme was involved in its metabolism. We designed two drug interaction studies to check whether CYP3A4 inhibitors would decrease the clearance of the drug. In addition, I cajoled the DMPK rep to do in vitro work to see which CYP metabolized the drug. The in vitro work showed that it was a 3A4 substrate, and the human trials showed that the clearance of the drug decreased drastically with co-administration of strong and moderate CYP3A4 inhibitors. Was I rewarded for that? Of course not; the SVP in charge of the project was furious that I had done the “wrong” studies (No, I didn’t) and would not speak to me for the rest of the meeting, because I messed with his pet project.

  2. Hap says:

    The problem is that sometimes we forget what we’re working for – we’re (supposed to be) trying to make things better in some way, not make ourselves be more powerful (at least as the terminal goal). If you’re working to get power or influence, then looking at what makes your point is useful. If honesty is detrimental to your work or politics, then honesty (and the data it forces you to acknowledge) is not the problem.

    If you’re trying to make a drug, and you won’t figure out why it might not work, then you’re wasting your employer’s money to make yourself look good, and unless your employer smacks that behavior down in a hurry (which they don’t usually) then you’ll have lots of people doing the same, spending lots of money to make themselves rich and the company dead. Why management thinks that’s a good idea (other than, “God, they look like me when I was younger. They need to be promoted.”) I don’t know.

  3. Wavefunction says:

    “That’s the one where people do just enough searching through the literature to confirm their own point of view, and stop right there. And our sources are now so wide and varied that indulging this tendency is easier than it’s ever been, unfortunately.”

    And if that’s so easy in science, imagine how easy it’s in politics…which explains why more information especially on social media sites has led to even more entrenched views and insularity of thought: because you can always and quickly find a POV that supports your thinking, no matter how crazy or one-sided it might be.

  4. Chad Irby says:

    The confounding problem is that current net-denizens know how to look things up, and are fairly good at doing so – but with severe limits, because they’ve spent so much of their time looking it up instead of learning it in the first place that they don’t know that they’re not even asking all of the questions.

    A lot of it comes down to vocabulary. I know more than a few technically-smart people who only have a standard high school vocabulary, because that’s all they learned(!). They then go to college and learn just enough technical jargon to make it through, and don’t know the edge cases, and certainly don’t get the meaning when someone else makes a reference to a special issue that’s not easy to look up online.

    You also get “buried results,” where the top 100 hits on Google show the layman’s aspect, but won’t return the Good Stuff unless you carefully and specifically nail down your Boolean searches…

    1. Mark Thorson says:

      I’ve found that restricting Google searches to PDF filetype is very useful for weeding out actual research papers.

  5. Anon says:

    “But I think that the modern access to information has led to another behavior that accomplishes similar psychological goals. That’s the one where people do just enough searching through the literature to confirm their own point of view, and stop right there.”

    Lane?

    1. Derek Lowe says:

      Now, now. He’s been good enough to hold back on the Alzheimer’s comments, and I’m giving him credit for that.

      1. Mark Thorson says:

        Do not poke the bear.

  6. navarro says:

    i’ve been a school teacher for 23 years and, because my parents were both teachers, a close observer of education for 18-20 years before i became a teacher. the effect you describe has its counterpart in education. a teaching methodology which has been tried at two or three different times before in the past, proved to be unsuccessful regardless of effort and resources put into it each time, and yet is being given a new name and a new trial to much fanfare from central administration and the consultants they’ve hired. anyone who points out that this same thing was tried 11 years ago and proved to be unworkable and was also tried 25 years ago and worked out to be a dismal failure is described as a defeatist, as being lazy, as being shamelessly cynical, as not having a growth mindset so that even veteran teachers who know better because of past experiences will keep their mouths shut and let this methodology wash over the system and fail again.

    1. NoniMausa says:

      “consultants”. Boo, hiss. Over the past 30 years, consultants have given consultants a very bad name. In my experience, they’re the people brought in to ruin businesses.

      1. Bob Buntrock says:

        Now, now, this chemical information consultant could take offence at this blanket condemnation of consultants. Then again, I’ve been in the trenches all the way from undergrad, grad school, and two lab jobs, doing my own searching, and then doing it as a job both in a petrochemical company and then as a consultant.

  7. Running Comment says:

    PhD years in the pre-IT era…chat in the coffee room, mentioning that I was looking for an obscure umpolung turn for a one-carbon synthon. Senior faculty person murmurs: you might try cyanogen bromide…something in JOC…early in 1963…left page…..not sure who wrote it….sorry…

  8. Anonymous says:

    Mooers developed the Zatocoding system based on edge-punched cards and sorting with knitting needles. When I was an undergrad (pre-computers), I worked in a lab whose PI used the commercial “Information Data Systems” (IDS) punched cards version of zatacoding as he read and catalogued the literature. I started using IDS cards myself which helped me to think about a paper to properly encode it. (I won’t bore you with the lists of the sometimes sophomoric — wait, I was a sophomore — indexing terms I came up with.)

    Thieme produced “Reactiones Organicae”, an AUTOMATED organic reaction literature retrieval system in the 1960s-1970s before on-line searching. They sold packets of pre-printed, pre-punched zatocards of the recent organic lit (Synthesis, Helv, Angew, JOC, etc.). You would slide the rods (knitting needles) into the desired holes (LDA, terpene, …) and shake until ONLY the cards with ALL of the desired criteria fell out of the pack. The AUTOMATION part consisted of an optional electric EKAHA shaker that did the shaking for you 🙂 . There is or was one on exhibit at the Chemical Heritage Society in Philly. Anybody know if it’s still there?

    Chemical Abstracts Service started using Hollerith cards (invented in the 1880s by Hollerith; later became IBM) to code, sort and prepare Chemical Abstracts early on. Search “Indexing and Index Editing at Chemical Abstracts before the Registry System. Charles H. Davis” for a free PDF.

    Many readers here are familiar with HARD COPY Chemical Abstracts (scifinder), Science Citation Index (web of science), Index Medicus (pubmed), Beilstein (reaxsys), Houben-Weyl (science of synth), Gmelin, etc.. In my opinion, having used the HARD COPY versions of those indexes gave me a much better understanding of the underlying data structure which lets me perform better on-line searches than junior colleagues. (Anybody remember the picture of the stack of Chem Abs 11th(?) Coll Index that was taller than a giraffe?)

    Many junior (and senior) colleagues seem to think that the databases know everything. The databases are science oracles. There is that too-common mistaken belief that you can trust the databases. (1) The databases are NOT complete! Indexing is influenced by humans and their programming biases; index terms; stop words; etc.. One human might not add “toxic” as an index term to a paper because the info was minor of in a footnote. YOU will never find that paper from an e-search alone. Too bad! (2) The databases contain errors. Errors come from source material, including typos (“cholestreol”) and structure-os. Errors come from data encoding errors. Errors come from search algorithm errors. (For many years (1990s), the Chem Abs peptide sequence search algorithm was flawed. They had to be pestered into investigating, confirming and fixing the problem.)

    YOU HAVE TO KNOW HOW TO READ THE LIT AND MAKE USE OF IT INDEPENDENTLY OF THE DATABASES.

    I have had colleagues tell me “nothing in the lit” after days of fiddling around with non-productive searches. I can almost always pull up a bunch of relevant papers in an hour or so. Like others, I have often found literature that should have killed a project or properly predicted problems that should have been avoided. Equally often, Management shows that they love their fluffy pet projects more than literature precedents and refuse to have them put down. It merely prolongs the suffering, usually mine.

    1. Derek Lowe says:

      Holy cow. Now that is what we call an analog computing system, and no mistake. I remember seeing Hollerith cards in the 1970s, but never actually had to use any by the time I got to college. But zatocoding I have never encountered!

    2. t grayson says:

      Here Here!!

    3. Bryce Woollcombe says:

      Brings to mind the mythical Memex electromechanical monstrosity from WW2. Is Charlie Stross in the building?

  9. Humble Scrivener says:

    @Chad Irby: I agree re: vocabulary but encountered a catch-22. I was encouraged vigorously to cultivate vocabulary from dawn-of-language-ability until grad school, then told in the industry “do not show off” and “please use common English words.”

  10. Humble Scrivener says:

    Please pardon multiple posts if any! I had glitches with CAPTCHA.

  11. Personality type ideolog says:

    **opining**

    It seems that when making facing problems as industrial scientists we prioritize the following in different orders: solving the problem as efficiently as possible; looking good in front of the boss/coworkers; doing something academic that garners respect. The book Please Understand Me II makes a convincing argument that the inclination that dominates defines us as people and can be explained by personality types –in respective order, Artisans (tactical utilitarians); Guardians; Rationalists. I’ve left out the 4th category, Idealists (enlightenment seeking) as their aren’t too many of working as industrial scientists.

    The recent article on how to be a good medicinal chemist is describing the Artisan verbatim.

  12. Chris Phoenix says:

    Ask someone to figure out the rule for a sequence of three numbers. Tell them that 4, 6, 8 fits the rule. They can test any sequences they want, by asking you.

    Most of the time, they will say, “3, 5, 7” (yes) and “14, 16, 18” (yes) and maybe “5, 3, 1” (no) and then tell you confidently that the rule is “numbers increasing by two every time.” The actual rule was “Numbers increasing.” They did not ask any question that would have falsified their first, confident, overly precise guess.

  13. Steve says:

    Not sure how valid this is. EVERYONE knew that ulcers were caused by stress; any research of the literature would have said that bacteria had nothing to do with it until Barry Marshall swallowed a bunch of Helicobacter. Everyone knew that energy and mass were different until Einstein said they were the same. Everyone knew that electrons were stored until Peter Mitchell showed that they were pumped. Letting the literature stop you from doing experiments because everyone knows how something works just inhibits innovation.

    1. Anonymous says:

      Steve wrote: “Letting the literature stop you from doing experiments because everyone knows how something works just inhibits innovation.” GOOD scientists do not let the literature stop them from asking “What if …?” or “Is this really so …?” GOOD scientists DO seek to do challenging and informative experiments to test new hypotheses but they aren’t always allowed to carry them out. (1) Funding: not available or deliberately withheld. (2) Management sets priorities, not scientists. (Challenger disaster, 1986. Engineers said, “Do not launch!” Managers said, “Launch!”) (3) And other reasons.

      The cumulative knowledge of ancient to modern researchers is in the literature and the literature should not be ignored. Scientific theories and conclusions are fluid and change. That’s one of the characteristics of science, cf., religion.

      If you are saying that some scientists with too much faith in the literature alone can suppress new ideas, new experiments, and new results from others then (1) that is nothing new (Kuhn. Structure of Sci Revolutions) and (2) I agree.

  14. Apropos for a shoutout for our local biotech startup, http://www.nference.ai, which specifically uses machine learning to synthesize and present knowledge derived globally from the world’s biomedical literature. We’re working to tackle problems like group think, and to present a less biased view of the information out there. Check us out!

  15. Pi. says:

    We got some hard guys here from MIT, we got some hard men from harvard….none have been able to contribute to biology. Really, honesty, ive seen better stuff come out of china. No lying, thats an honest statement.

    1. Anonymous says:

      They have lots of clinicians and programmers. I’ll bet that none of them read blogs such as Pipeline or others that are critical of the use of AI for drug discovery. They probably read the financial pages and market reports describing all of the Big Money being poured into AI for Medicine and Healthcare. It appears that there are well over 200 companies competing in this area. (I’d like a piece of that action, but an honest negative viewpoint wouldn’t get any.)

      I don’t know about AI’s value for clinical care but the stuff I have read was not impressive. There was a comparison of the diagnostic ability IBM’s Watson compared to experienced clinicians in the news in 2016; Watson matched the humans but also suggested alternatives in 30% of the case. (See articles for better description.) I assume it’s getting better, but how much better?

      It was also posted above in this thread and many other times that databases are full of errors and that the primary literature is full or errors. That is where scientists with actual brains and freedom of thought can better question and downgrade results better than an algorithm.

      Recently (May 23), the Fluoroquinolone thread mentioned that Adverse Reactions are vastly under-reported. Another potential source of bad data, falsely suggesting that toxic drugs are safe drugs.

      Disclaimer: I am a human, so I am biased.

  16. bcpmoon says:

    Well, everyone knows that two weeks in the lab can save you 1 hour in the library…

  17. dearieme says:

    A colleague on a second colleague, circa 1990: “His career will flourish. He doesn’t think critically about his own data and he ensures that he remains ignorant of what is in the literature.”

Leave a Reply

Your email address will not be published. Required fields are marked *

Time limit is exhausted. Please reload CAPTCHA.