Are You a Rockstar Scientist!!!!!?

Many, many years ago, while I was a grad student, New Scientist magazine published a quiz titled "Leader of the Lab." I have shamelessly stolen borrowed and altered what I remember of that for the following:

Are You a Rockstar Scientist!!!!!?

Answer the following multiple choice questions to determine if you have what it takes to be a Rockstar Scientist!!!!! Keep a note of your answers and use the grading system at the end of this post. No cheating!


1) You've just started your tenure track faculty position. You're trying to get your lab up and running when your newly hired tech comes into your office clearly upset. His mother has just been diagnosed with a terminal disease and has only a week or so left to live. The tech hasn't as yet earned any vacation or sick leave time, but would like to take time off to spend with his dying mother. What do you do?

1. Of course he should go. He should take as long as is necessary. And he shouldn't worry about being paid - you'll make sure he is.
   
2. He can go. Although you feel bad about it, it does have to be leave without pay.
   
3. He can go when he finishes up all the experiments he was supposed to have done during that time. And it's leave without pay.
   
4. Tell him that's what weekends are for.
   

2) Two graduate students want to join your lab. Both have indicated an interest in the same project of potential high impact. What do you do?

1. Explain the situation to both and try to work out who gets the coveted project. Do your best to come up with a second project of equal potential for the other student.
   
2. Give the best student the project and assign the second student something else.
   
3. Give the best student six months on the coveted project. If after that time they haven't made sufficient progress, give the project to the second student.
   
4. Put them both on the project and tell them whoever gets Nature/Science/Cell-worthy data first wins. The loser gets the boot.
   

3) An undergrad from one of your classes approaches you about doing research for credit in your lab. She is very interested in your work and is going to apply to grad school. You've never had an undergrad do research with you before, but are well aware that it's a substantial time and effort commitment on your part. What do you do?

1. Agree and set up a schedule where you can work with her until she's up to speed in the lab. Thereafter make sure you meet with her at least weekly.
   
2. Agree, then assign her to one of your grad students. Check up on her progress about once a month. Let the grad student decide her grade at the end of the semester.
   
3. Agree, then assign her to your tech. Don't bother checking up on her progress and assign her a C at the end of the semester.
   
4. Agree, then make her the lab dishwasher. Make yourself unavailable to talk with her once she's started. Give her an F for not having produced any data.
   

4) You will soon be coming up for tenure. You have plenty of grant money and a bunch of published papers, including two in the C/N/S glamormagz. A senior colleague is planning to go on sabbatical and asks you to teach one of his courses while he's gone. It's very much in your area of expertise and your colleague has offered you his Powerpoint slides and lecture notes. What do you do?

1. Agree to teach the course. It won't be that much work and will earn some good will.
   
2. Agree to teach the course but insist that your colleague writes a glowing paragraph about your selflessness in his letter supporting your tenure.
   
3. Tell your colleague you already have too much teaching.
   
4. Say no, then go tell your Department Chair that, with tenure coming up, you need to lighten your teaching load. Insist your senior colleague (the one who wants to go on sabbatical) is not only the best replacement, but the only person in the Department capable of giving your lectures.
   

5) You're now an associate professor with tenure. Professor Bigwig, a leader in your field, is coming to give a seminar in your Department. You know that she's organizing next year's SuperBig Meeting in your field. By sheer coincidence you are currently reviewing a renewal of the major grant that Bigwig needs to keep her laboratory running. What do you do?

1. Meet with Bigwig, tell her about the latest findings in your lab, and pay her all the respect she is due.
   
2. Meet with Bigwig and try to dazzle her with your latest findings in the hope she will consider you for a speaking slot in Superbig Meeting.
   
3. Meet with Bigwig and try to charm her into giving you a speaking slot. If that doesn't work, outright demand a speaking slot using your latest C/N/S publication as justification.
   
4. Meet with Bigwig and point out what a wonderful speaker you are and that you noticed there's an empty slot for a plenary speaker. All while tapping on that copy of her proposal you left lying in full view on your desk.
   

6) You're sitting on a grant review panel when you are assigned a proposal from a new investigator who is proposing to do precisely what you were going to put into a grant application you're currently writing. What do you do?

1. Let the program officer know the situation. Review the proposal as objectively as possible. If the new investigator's proposal receives a fundable, or near fundable, score, rethink your proposal.
   
2. Tell no one, but review the proposal somewhat objectively. Give it the score you think it deserves, given that this is a new, unproven investigator... Submit your proposal as planned.
   
3. Review the grant, but, no matter how good it is, give it a score that guarantees it won't be funded. Insist on additional preliminary data that will take at least a year to generate. Submit your proposal as planned.
   
4. Trash the grant, writing a review that sends the message that the proposed work is crap, and that the new investigator has never had and never will have a decent idea in their life. Then cut and paste from their proposal into your proposal.
   

7) You are now a full professor. An assistant professor in your Department, who is coming up for tenure, has been collaborating with you. You've each put in the same amount of time, effort and $'s. The manuscript is written and you're deciding on authorship prior to submission (a postdoc is clearly first author). What do you do?

1. Give the junior person senior authorship - it will be a big help in getting them tenure.
   
2. Insist on being co-corresponding author with your junior collaborator, but let them be last author.
   
3. You're senior. Obviously you'll be senior author.
   
4. Collaborator? What collaborator?
   

8) You've been very successful at grant writing and now have a lab full of smart postdocs. What do you do?

1. Keep doing what you're doing. Provide the best environment you can for the people in your lab, encourage them to write proposals for their own funding, and make sure they move on to bigger and better things when the time is right.
   
2. Keep those postdocs working hard. If they can keep the data flowing and you can get your own proposals funded, they can apply for their own funding.
   
3. Have your postdocs write grant proposals for you. Apply your editorial magic to them before putting your name on as PI and submitting.
   
4. Have your postdocs write your grant proposals for you. In their free time. Don't bother reading them, just have your secretary put your name on as PI and submit. They remain employed as long as they bring in the money. Keep the best grant writers convinced that they need to publish a lot more before they'll be ready to move on. Find ways to delay publication of their data. Sit back and enjoy.
   

9) When you first arrived at your University your Department purchased a confocal microscope you needed. The deal was that you would be in charge of this instrument, but that others in the Department would have access and you would provide the training. Over the years you have been doing less and less confocal work, to the point that you now no longer need such an instrument. What do you do?

1. Still put in the effort to keep the instrument maintained, and to train other users.
   
2. Retain control over the instrument, but leave the training to the tech in your lab who has used it once or twice.
   
3. Tell the Department that you no longer have any need for the instrument and that you refuse to be responsible for it anymore.
   
4. Sell the confocal microscope on Ebay and use the proceeds to pay for a skiing vacation in the Alps.
   

10) A young, completely unknown, recently graduated PhD from Estonia emails you a rough draft of a manuscript for you comments. Upon reading the manuscript you realize that this is Nobel-prize stuff. What do you do?

1. Go through the manuscript carefully, provide thoughtful and detailed comments, and inform the PhD as to the impact the work will have.
   
2. Offer to collaborate with the PhD, but insist they will be senior author.
   
3. Tell the PhD that the manuscript is interesting, but needs further experiments. Offer them a postdoctoral position in your lab so they can do the work with you, and that you will then publish the work with you as senior author.
   
4. Email the PhD telling them the work is crap not worthy of submission anywhere. Meanwhile have everyone in your lab do some quick and easy experiments you can add to the existing manuscript, rewrite it and submit.
   

Answer Key:

For each "a" give yourself one point, for each "b" two points, each "c" three points, and each "d" four points.

If you scored:

10-20
Sorry, you are just too nice to ever be a Rockstar Scientist!!!!! Your colleagues really like you and you are certainly popular among the grad students. But stardom will forever be out of reach.

21-30
Not going to make it to Rockstar Scientist!!!!! You try, but that pesky conscience keeps getting in the way.

31-39
You're almost a Rockstar Scientist!!!!! But not quite. What's holding you back? Couldn't sell the confocal? Missed the plane to Stockholm?

40
You are truly a Rockstar Scientist!!!!! Congratulations! It must be great not having to write grant proposals anymore. What did you get for the confocal and how was Stockholm?

Happy Non-Denominational Festive Season!

Numbers

Random observations involving numbers...

Sometime very early this morning my blog had it's 7000th visitor (someone from Brisbane, Australia - bet the weather's better there than here). It blows me away that people not only read some of my blather in the first place, but come back for more! Hat tip to DrugMonkey - much of the traffic I'm getting is due to him.

My very first experimental work-based paper (I used to be a computational biophysicist) has just hit 80 citations. That's about 10/year. Apparently an old dog can teach himself new tricks...

30. The number of degrees (in F; that's about 17C) the temperature is expected to drop by between the time I arrived at work this morning and the time I leave this evening. We're expecting 2-4 inches of mixed snow and ice this evening. Snow I like. Ice sucks.

So you want a piece of the NSF pie? Some thoughts.

Some random thoughts to wrap up this series.

As before, disclaimers here.

1) The review process-

The NSF review process is very different to that of the NIH. I gather the different Directorates, and even organizations within Directorates, have some leeway in deciding how the review process works. What I'm about to describe is the process my own proposals go through at the BIO Directorate. Your mileage may vary.

Review is basically a two-stage process. First your proposal is sent out to a bunch (at least two, sometimes up to six or seven) ad hoc reviewers, plus at least two members of the review panel. These reviewers send in their reviews via Fastlane. The second step is the review panel. The panel considers all of the reviews (as many as eight or even nine), discusses the proposal and assigns a rank (not a score). The Program Director then takes the rankings and figures out what she can fund.

So you get a bunch of reviews. This is good and bad. On the plus side, you can survive a mediocre review if all the others are stellar. You also get a lot of (hopefully) useful feedback. On the minus side, if one of the panel members who reviewed your grant really didn't like it, your odds of being funded are not great even if all the ad hoc reviewers loved your proposal. As with NIH study sections, you need an advocate on the review panel.


2) Proposal rankings-

As I noted above, reviewers don't score NSF proposals per se. They give them a ranking. The reviewers assign a rank of Excellent, Very Good, Good, Fair or Poor (from memory). The review panel takes those and assigns an aggregate rank of Outstanding, Highly Meritorious, Meritorious, or Non-Competitive. The last two categories won't be funded. Sadly, it is often the case that not all of the proposals ranked as Outstanding can be funded...


WARNING! WARNING! ANECDOTAL STUFF FOLLOWS!

Okay, the following stuff is purely anecdotal - you've been warned. I'm posting it because I've heard the same things from multiple people (n > 6), leading me to think there might just be a grain of truth in each of these...

3) NSF doesn't want young people to fail-

NSF Program Directors have a lot of say over who gets funded and who doesn't. Clearly they can't over-ride the reviewers to much or too many times otherwise they would have a very hard time finding people willing to review proposals. But they can nudge proposals over the funding line if they feel it's warranted. What I have heard from multiple people is that Program Directors will sometimes do this in order to fund an otherwise unfunded young investigator on the verge of their tenure decision. Of course said investigator needs to have been productive enough to warrant this.

This is not something I would bet any money on. Or my tenure. It's obviously best to not to have to rely upon the largesse of a Program Director.


4) Once you're in you're golden-

By that I mean once you're funded by the NSF, as long as you're productive, your Program Director will try to ensure you maintain the ability to be productive. I am by no means suggesting that the Program Director will fund your proposals over higher scoring proposals. Rather that if you're on the funding edge and it's a choice between you as a previously NSF-funded, productive PI and someone else, you will get the nod (remember - this is anecdotal stuff).

I have brought this up hesitantly because it sounds like a mechanism for maintaining the status quo. I would like to think that's not the intent, rather that it's more a matter of Program Directors being loyal to their productive charges. Can it have the side effect of maintaining the status quo? Yes. The someone else is likely a new investigator... Please note that a) THIS IS ANECDOTAL, b) this is somewhat at odds with the previous anecdotal point in this post, and c) I'm not defending this.

So you want a piece of the NSF pie? Broader Impacts.

As before, necessary disclaimers can be found here.

The dreaded Broader Impacts... This is the place many of the proposals I've reviewed have significant weaknesses. It used to be you could just pay lip service to these. Half a page max at the end of the proposal would be plenty. Not anymore. The NSF has instructed its reviewers to take these very seriously. And I can assure you the Program Officers take them very, very seriously. So when you write a proposal destined for the NSF, you need to take the broader impacts very seriously.

Let's start with why NSF requires broader impacts - understanding this can help formulate some for your proposal. It is important to understand that the NSF will not give you a grant just to do research. Read their mission statement:

To promote the progress of science; to advance the national health, prosperity and welfare; to secure the national defense (NSF Act of 1950).

Note that it's promote the progress of science. That's much more than just funding research. The NSF's mission includes science education and training, and dissemination of scientific knowledge to the broader population. That's where the broader impacts criteria come from.

So what are broader impacts? The place to start is with the NSF Grant Proposal Guide. Find the section on review criteria and specifically that part dealing with the broader impacts. As you will see these are split into two categories:

• Integration of Research and Education
  
• Integrating Diversity into NSF Programs, Projects, and Activities
  

Basically it comes down to how you're going to tie education and training into your research activities and how you're going to go about improving diversity within science. The NSF has kindly put together a crib sheet describing how you might address these. It can be downloaded via at www.nsf.gov/pubs/2002/nsf022/bicexamples.pdf (for some reason I can't get Blogger to publish this as a link...). These are just suggestions and nobody in their right mind would propose to tackle all of them. Pick and choose those that work for your circumstances. Be creative and come up with new ones.

Here are some of my thoughts on how to address the broader impacts.

Integration of research and education:

This is the easier of the two criteria to deal with. It's important to understand here that "education" includes training, teaching and dissemination.

Training- In your proposal you want to talk about how you're going to involve trainees in your research. Trainees can include undergrads, grad students and postdocs. You could just say they will be involved, but it's much better to provide specific examples of how they will be involved. For example, what pieces of your project would be suitable fodder for undergrad researchers? Some verbiage describing how your trainees will be trained can also help.

Involving undergrads (and possibly high school students) is a very good thing to do (at least in NSF's view). And can be very rewarding for the PI as well. If you can, write an undergrad or two into each budget year. And specifically state that you will actively encourage even more undergrads to join your lab and that you will apply for REU supplements to support them (read up on these - it's easy money and really looks good when you go to renew your NSF grant). And don't forget to tout all the great work you've done with undergrads in the past.

Teaching- You can keep this as simple as stating that you will integrate the results of your research into your teaching (about the minimum in terms of addressing this, and really all I do), through to getting involved in teaching at K-12 schools. Another possibility is to host high school science teachers over the summer. The NSF has a whole program devoted to funding this kind of thing.

Dissemination- It's a given that you're going to publish and present your data at meetings etc. The difference here is that you need to explicitly state that. And describe how your trainees are going to be disseminating as well. Don't forget to budget funds for these activities, including funds to send trainees to meetings. It's important to back these things up with a real commitment - money.

If you have other opportunities to disseminate the results of your research (e.g. you've been invited to write a review or book chapter), talk about those. The book I recently edited is broader impact/dissemination fodder I count tout. Are you depositing stuff in publically-accessible databases? That's more dissemination stuff.

Diversity:

You can address this separately from the above, or integrated within it. Either works.

What the NSF wants here is some description of how you are going to try to involve people from traditionally under-represented (in science) groups in your research program. I've always found this the most difficult to address. You want to write something that you actually have a chance to succeeding at. If you read the NSF Broader Impacts crib sheet (www.nsf.gov/pubs/2002/nsf022/bicexamples.pdf) you can get some reasonable ideas. Collaborations with PI's who are, or who work with under-represented people count. As do collaborations with faculty at four-year colleges. The ultimate is to have members of under-represented groups working in your lab.

If you have any kind of track record of doing any of this, tout it loudly and clearly.



One last word of advice. Addressing the broader impacts sufficiently is going to require valuable real estate in your 15 page proposal. I don't know how much would be considered too much, but I can tell you less than a page will likely doom your grant. In my last two NSF proposals (both renewals, one that was funded in 2004 and one that has just been recommended for funding) I used at about two pages (not counting the space used to describe the broader impact work done with the prior period of funding).

So you want a piece of the NSF pie? Scope and budget.

Herein I continue my somewhat disorganized discussion of NSF funding...

Necessary disclaimers can be found here.

Okay, so now you've had a chance to study NSF's award statistics, it's time to think about the scope of an NSF proposal. Let's face it, a five year, four/five specific aim proposal probably isn't going to fly. Most proposals funded by the NSF appear to be for around three years and have a modest budget. Poke around here to see what the NSF has funded in your area recently to get a better idea of what you're dealing with.

The budget is all important here - it will define the scope of what you propose to do. Remember, those median annual award sizes are totals - direct plus indirect costs. Obviously if you are at an institution with a high F&A rate (say 90%), you're going to be able to ask for more than the median amounts. But don't get greedy - the NSF has limited funds. If your F&A rate is more modest (say around 50%), think in terms of the median amount. Figure out what the median direct rate would be - that's roughly what you'll have to work with (in practice, if you're funded, you're likely to take a modest budget cut, but don't worry about that too much). Let's say you're applying to the BIO directorate, MCB organization, Molecular Biophysics section (one of the more generous sections and the one I'm funded through), and have a F&A rate of 50%. The median direct costs (using the 2007 median annual award) are then:

Directs = $159,113/1.5 = $106,075

Basically $100k/year for three (maybe four) years in this example (remember - your mileage will vary). So the important question becomes what can you do with that? The answer will define the scope of your proposal.

What you can do with that amount of money is highly dependent upon the kind of work you do and the type of position you have. I'm in a college of medicine where each PI pays the stipend and benefits (but not tuition - yet) of their graduate students (no TA's). I might just be able to pay two students based on an award of $100/k per year if I'm very, very careful (i.e. miserly). Or one postdoc and part of a tech's salary. Etc. These considerations clearly limit the scope of what I can propose to do with an NSF grant. In the end I've found two specific aims are about all I can manage. If you're in a setting where TAships for grad students are plentiful, perhaps you can manage more.

Something to keep in mind while we're on the topic of the budget. The NSF will only allow you to pay two months of your salary per year from all sources of NSF funds you have access to. Recently there was a discussion at DrugMonkey's place about this. Let's take me as an example. I'm PI on an NSF research grant and on an NSF REU Site grant. If I get 0.5 months salary from the REU grant I can only take 1.5 months salary from the research grant. NSF doesn't appear to care how much of your salary comes from grants from other agencies.

One last thing while I'm thinking about it. Should you get an NSF award (congrats!), there are two types of research grant; Standard and Continuing. You have no say in which one you get. If you're awarded a Standard grant you're given all the money up front. If you get a Continuing grant you get one year's money at a time. Each new year is contingent on an annual report you have to submit (you have to put these in with Standard grants too). I've always had a Continuing grant and have never had an issue with obtaining the next year's money (I am quite diligent about putting in the required report). I've also never heard of anyone not getting their next year's money, but I suppose it's a possibility.

So you want a piece of the NSF pie? The NSF is NOT the NIH.

I've decided to put down some thoughts of mine regarding obtaining funding from the NSF. This will likely end up being a series of posts...

First, the necessary disclaimer: These are my opinions only, shaped by my experiences as an NSF-funded investigator (in a College of Medicine), a reviewer of NSF proposals, plus the experiences of some friends and collaborators. Your mileage will almost certainly vary.

Second, some background: I was awarded my first NSF grant in 2001. I renewed that in 2004 and have recently heard that my second renewal has been recommended for funding. I am also PI on an NSF REU Site grant. I have reviewed many NSF proposals, most from the BIO directorate and some from MPS, on an ad hoc basis. Finally, I sat on an NSF SBIR review panel for two years.


Now, let's get down to business here. If you're in the biological sciences, please repeat after me:

The NSF is NOT the NIH.

Keep repeating this until you are firmly convinced of the truth in that statement.

Now you would think that that's obvious, right? Apparently it isn't to about 25% of the applicants whose proposals I've reviewed for the BIO directorate...

Here are some, but not all, major differences:

1) Budget-

The projected FY 2009 NIH budget is $28.7 billion for R&D expenditures. The NSF? $4.5 billion for R&D. This is under the current Continuing Resolution. See the AAAS September R&D Update to see where I got these figures. These numbers have a very profound effect on many aspects of NSF funding versus NIH...


2) Mission-

The mission of the NIH is "science in pursuit of fundamental knowledge about the nature and behavior of living systems and the application of that knowledge to extend healthy life and reduce the burdens of illness and disability."

The mission of the NSF "includes support for all fields of fundamental science and engineering, except for medical sciences." [Emphasis mine.]

The NIH funds biomedical research. The NSF funds all of the other sciences. Including social. Plus engineering. And mathematics. And the NSF mission includes a strong science education component (which I believe accounts for the disparity between the $6 billion total budget of the NSF versus the $4.5 billion it spends on R&D).

So, don't go proposing to solve cancer, or heart disease, or Alzheimer's in your NSF proposal. If you do I can pretty much guarantee it won't be funded. Basic biological science is fine (otherwise I wouldn't be funded), so there is some overlap between the NSF and NIH. But you need to be careful about how you sell your basic science to the NSF. Can't figure out how to sell your science without referring to the medical benefits of the research? Don't bother applying to the NSF.


3) Scope-

The scope (budget and period of funding) of an NSF proposal is very different to that of an NIH R01. The NSF keeps some interesting funding statistics. If you're considering applying to the NSF, go find the data for the last year or two for the directorate and organization (within the directorate) you are most likely going to send your proposal to. Study those numbers. Not the funding rates. The mean award duration and median annual award size. Your proposal needs to be in the same ballpark as those numbers (unless you're applying to a special program with special budget instructions). Sure, you can ask for more time and/or money, but you'd better do a very, very good job of justifying why you need more.

Oh yeah, one other thing. Those median annual award sizes? That's total. Direct plus indirect costs.


4) Proposals-

Aside from the fact that an NSF proposal is limited to 15 pages versus the (current) 25 page limit at NIH, there are other differences. I'm not going to go into great detail here. I'll probably cover some of that stuff in later posts. The main thing is to go study the NSF review criteria. And take them seriously. Intellectual Merit pretty much refers to the quality of the science proposed (and of the proposer). That Broader Impact stuff? Not something the NIH puts much weight on. The NSF, on the other hand, is very serious about it. I'm going to write a post devoted purely to Broader Impacts since this is where I've seen many proposals fail. Particularly those form new investigators.


5) Effect on tenure-

This is aimed at those of you in a College of Medicine. I got tenure based on my NSF funding. Here that's considered sufficient to satisfy the funding part of the tenure equation (assuming you've done very well on all other parts of the equation). That is not the case in all medical schools. There are Deans of medical schools who consider NSF money second class at best and will not approve tenure for someone with only NSF funding. Be sure you know what the ground rules are at your institution.