TL neuro

April 13, 2017

Pre-Prints

Filed under: Careerism, NIH — mtaffe @ 10:11 am

In 2017 the NIH issued NOT-OD-17-050 Reporting Preprints and Other Interim Research Products to encourage funded investigators to speed the dissemination of tax-payer funded research by citing and claiming pre-prints as products of NIH funding.

The NIH encourages investigators to use interim research products, such as preprints, to speed the dissemination and enhance the rigor of their work. This notice clarifies reporting instructions to allow investigators to cite their interim research products and claim them as products of NIH funding.

Pre-prints are manuscripts (and other interim research products) which have not undergone peer-review and formal acceptance for publication.

The most critical implication of this new policy is for grant preparation.

Interim research products can be cited anywhere other research products are cited.  These sections include the following:

The benefits should be clear. Instead of having to describe work as being “in preparation” or “submitted” one can now point to a link and any interested reviewers can see it for themselves. This will be critical on the CV or Biosketch of junior scientists in transition. The publishing timeline is slow compared to their needs with respect to finding a postdoc appointment after graduate school, winning a fellowship as a postdoc or getting a job offer after postdoctoral training.

This will potentially help all PIs with their grant applications as well. Productivity can be a major point of review for new applicants, for renewal applications and indeed for any application if a PI is perceived to have too much funding. Productivity as reflected in peer reviewed published papers is not always under the direct control of the research team- the publication racket can induce significant unexpected delays. This pre-print policy allows the grant applicant to post pretty much anything that they want. It will potentially function as a hybrid of Preliminary Data and Publications. “Potentially” because there is no obligation for any reviewer to consider these documents.

I have decided to respond to this new initiative, for now, by submitting manuscripts to bioRχiv (http://biorxiv.org). I have found the submission process to be relatively easy on the scale of the usual manuscript submission for publication or to the PMC repository. The first two I submitted were available online within 24 h of my upload.

The critical question for most of us will be to try to determine what our threshold should be for publicizing any pre-print or interim research product. I have come to the conclusion that a manuscript that we have already submitted for publication somewhere clearly fits the bill as a sufficiently complete work to put on a pre-print server. A manuscript that we plan to submit for peer review essentially concurrently is also very clearly sufficient and the only difference here, to my view, is whether we are really, really at a submittable-draft stage or maybe jumping the gun with the pre-print.

The following three manuscripts had already been reviewed by the time I put them up on bioRχiv. I would characterize two of the reviews as being concerned about interpretation of the data in a way that would require a LOT more data to satisfy. This appears to me to satisfy one potential goal of posting pre-prints. I.e., that people can interpret the quality and meaning of the data for themselves before the authors manage to satisfy all theoretical concerns, sidelines or unlikely possibilities that might be required for publication acceptance. The third one is awaiting one more figure of data for the resubmission. We thought we had a decent rebuttal without it, one that would possibly fly with the editor. But we are also generating new data that is relevant. This has been slower to emerge than I had hoped and we have noticed some recent publication activity in this area. Posting this manuscript as a pre-print is essentially putting down a priority marker at this time.

Javadi-Paydar, M., Nguyen, J.D., Grant, Y., Vandewater, S.A., Cole, M., and Taffe, M.A. Effects Of Δ9-THC And Cannabidiol Vapor Inhalation In Male And Female Rats.  bioRχiv, 2017, Posted April 18, 2017 doi:

Taffe, M.A. Wheel running increases hyperthermia and mortality rate following 3,4-methylenedioxymethamphetamine (MDMA) in rats. bioRχiv, 2017, Posted April 11, 2017 doi:

Aarde, S.M., Huang, P-K  and Taffe, M.A. High Ambient Temperature Facilitates The Acquisition Of 3,4-Methylenedioxymethamphetamine (MDMA) Self-Administration. bioRχiv, 2017, Posted April 4, 2017 doi:

 

August 30, 2016

What it takes to survive the NIH Game

Filed under: Careerism, Postdoctoral — mtaffe @ 11:07 am

I recently dusted off some old slides on the academic science career arc to prepare for a presentation to our postdoctoral trainees who are supported on the training grant that has been housed in our department for much of its 35 years. I am the new PI of this NIH award and it falls to me to serve in an overseer role for the training and education the postdocs receive. No small responsibility, this.

Of course, we will eventually cover career paths that veer outside the NIH-funded grant-supported research laboratory in which these trainees are working. But to start with, I presented a very superficial overview of the NIH extramural system. As a NIH Training Grant, the first assumption is that we are training the next generation of Principal Investigators to lead their own independent research projects, so this frames our starting point.

One of the elements which I stress repeatedly is for individual trainees to do their own research about how the NIH’s (or other agency’s) grant game might work for them. Individually. Part of that was an exhortation to see how it has worked, or is working, for the scientists that they know (NIH’s RePORTER is helpful) …if nothing else to educate themselves on the world of the possible. The experiences of scientists change over time and with academic generations but it is a good idea to take a look at the career paths of those PIs who are most like you (the trainee) imagine yours will be.

I also recently had a peer scientist who provides me with a lot of career and grant related sounding board type advice remark “you work at this harder than anyone I have ever heard of”. I do not know if that is true or false when compared to people in my field at my approximate career age (I was appointed Assistant Professor in mid-2000) and in my approximate employment type (soft money, i.e., essentially all of my time is spent on grant-funded research activities).

But this resonated with my advice to the Training Grant postdocs to do some research on the career arc of the scientists in their fields. So, here’s a summary of how hard I work at it. YMMV, of course.

Starting in early 2000, I have submitted approximately (I may have lost track of one or two here and there) the following number of grant applications to the NIH as a Principal Investigator, including participation as a component head on P-mechanism.

Thirty two new (including revisions) R01 applications, 6 of which were funded.

Seven competing continuation R01 applications, one of which was funded.

[Note that my 18% hit rate for R01 submissions is (roughly, roughly) right about where the aggregate success rate for NIH submissions is, if you amortize over this interval of time with a very broad brush.]

Component head within a P (Center) application included 4 total submission rounds, one of which (the second) was funded.

One U01 application, which was not funded.

Eleven R21 applications, one of which was funded.

One R25 application, not funded.

I’ve also contributed substantially to three SBIR applications in the sense of being the scientific subcontract for those proposals; one of these was funded.

It is important to note that many of these submissions were in fact re-submission of amended versions of proposals that had been previously reviewed. There are also many of my submissions that were highly similar to other proposals even if not direct revisions. This might be due to the NIH rules not allowing further revisions or by alterations in our ideas or plans.

As far as my funded R01 awards go, two were funded un-amended, two on the A1 version and three on the A2; but this only tells a part of the story. Each “un-amended” app had been reviewed previously as part of a different grant mechanism.

Two of these awards were clearly saved by Program Officers deciding to pick up a grant that otherwise might not have funded.

Two received 1-2%ile scores (you can’t do much better than that).

One followed an R21 award (funded on A1) and one followed a component of a funded P20 Center (components are ~R21 size) in terms of topic and direction. These might therefore be considered more in line with a competing continuation rather than a new award/new project although they were not technically competing continuation applications.

I have also been the PI of three major awards that were originally reviewed with a different person as the PI (two of which I contributed to as the postdoc in the laboratory of the PI).

Obviously the question of how hard one has to work at maintaining a given level of research funding is a moving target at best. I have a colleague of my approximate generation that has received funding for over 80% of submissions. I have colleagues of various generations beyond Assistant Professor stage that have been very spottily funded over many years to decades. Some of this depends on personal choice of research topics, grant submission behavior, choice of laboratory size and other factors. Some, no doubt, is just variation in how successful one happens to be within this highly competitive grant-award system. And some probably depends on how hard you work at it. For that you would have to ask the individual PI in question how many unsuccessful grants they have submitted.

 

__

Update 05/17/17: I recently found reference to this paper.  It emphasizes that grant success is a function of grant application effort.

Blog at WordPress.com.