Electronic Lab Notebook: Take 1


I am always looking to improve my scientific workflow. My current workflow related to my

Lab Notebooks

“lab notebook” is terrible and virtually non-existent. I don’t do a lot of lab work, but I still try to keep notes of what I do and thoughts I have and various things I try. Unfortunately, I have a number of random notebooks going at any given time, plus scrap paper, electronic notes as text files, word documents, excel documents, csv files, markdown, and trial and error scripts in R and Rmd and various other forms. It’s basically a disaster. I rarely go back to my non-field notes/notebooks and when I do it’s terribly difficult to find anything I’m looking for. I often search my own blog to find things because it’s easier than finding those ideas and notes elsewhere in my mess.

In my effort to get more organized, I decided that digital was the way to go. That way things can be tagged and searched, categorized, and filed in multiple places (copies or via tags). The question became how to best accomplish this. Carl Boettiger has a wonderful open, online lab notebook. I’m not ready to go fully open, especially since it would be difficult given the number of collaborators who might not want everything shared. For my first attempt, I am created a private GitHub repository for my lab notebook. I just got 5 free private repositories for education and teaching, so I am using one of them for my lab notebook. I cloned my repository to my computer and ready to start.

I am starting by writing my notes in Markdown using Mou or RMarkdown using RStudio. This is an easy way to take and format notes. Then I can use pandoc to convert those files to html or pdf or other formats if I want later. This is particularly useful if you have references. For example, the beginning of my first electronic notebook entry is a brief summer of a paper. Rather than having to type every reference out, I can use citations (and styles) just like I would in a manuscript. Here is the Markdown text:

04 April 2014

Brook Trout and Stream Salamander Interactions
==================================

## Local & Landscape Predictors of Spring Salamanders

[@Lowe2002]

* 25 1st order streams in NH
* July & August 1999
* Single-pass surveys
* 100 cover objects flipped (64-256 mm diameter)
* 25 m upstream of confluence
* Species
* .G. prophyriticus
* .D. fuscus
* .E. bislineata
* 7 of the streams visited 3 times in 2000 (with CMR)
* Observed brook trout presence during surveys
* Additional/Tangential electrofishing of 3 streams
* Brook trout negative effects on Spring Salamander captures (not examined for other species)
* Sedimentation and logging negative at whole-stream and regional scales

## References

I can use the pandoc code:

pandoc BrookTrout_Salamander_Notes.md -o BrookTrout_Salamander_Notes.pdf --bibliography Trout_Salamander_Refs.bib -H margins.sty --csl Ecology.csl

 

To make the margins 1″ I have a margins.sty file that is just a text file with the following code:

\usepackage[vmargin=1in,hmargin=1in]{geometry}

 

My PDF will render like this and my html will render like this (just change .pdf to .html in the above pandoc code – simple as that):

 

04 April 2014

Brook Trout and Stream Salamander Interactions

Local & Landscape Predictors of Spring Salamanders

(Lowe and Bolger 2002)

  • 25 1st order streams in NH
  • July & August 1999
  • Single-pass surveys
  • 100 cover objects flipped (64-256 mm diameter)
  • 25 m upstream of confluence
  • Species
  • .G. prophyriticus
  • .D. fuscus
  • .E. bislineata
  • 7 of the streams visited 3 times in 2000 (with CMR)
  • Observed brook trout presence during surveys
  • Additional/Tangential electrofishing of 3 streams
  • Brook trout negative effects on Spring Salamander captures (not examined for other species)
  • Sedimentation and logging negative at whole-stream and regional scales

References

Lowe, W. H., and D. T. Bolger. 2002. Local and Landscape-Scale Predictors of Salamander Abundance in New Hampshire Headwater Streams. Conservation Biology 16:183–193.

 

I get the reference formats from https://github.com/citation-style-language/styles. In my previous post you can find more information on knitting documents in RStudio.

I will try out this form of a digital notebook for a while and see how it goes. I will continue to blog and if any of my notes are worthy of being shared I can just use pandoc to covert the notes to html and paste it in my blog. I’m sure my organization of notes and tagging will evolve over time with trial and error.

Does anyone else use a Electronic/Digital Lab Notebook? If so please share any advice in the comments section. I’d love to hear about what other people have developed for a workflow and archiving scheme.
Advantages of an Electronic Lab Notebook

  •  Searchable text and tags
  • Categorize in multiple locations rather than just chronologically
  • Ability to easily share in multiple forms
  • Ability to incorporate text, photos, equations, links to external files, result outputs, etc.

Disadvantages/Cautions

  • Although paper can get lost, wet, burned, eaten, etc., it’s more likely to get destroyed/lost/corrupted digitally so be sure to back it up. [Easy to do with git and GitHub – plus you could print PDFs if you really want]
  • Sometimes slower to do which could stifle thought and creativity [Solution: hand write notes on paper or white board and scan them in or take a photo with you phone and copy it into the electronic notebook]
  • Changing technologies could make assessing special formatted objects difficult [Solution: use plain text, html, csv files, and PDF files when possible since those are not likely to have long term problems]

Other Great Resources for Keeping an Electronic Lab Notebook

Including math and code

How to Ditch Word (for Markdown)

knitr and GitHub for a lab notebook – Carl Boettiger

Reproducible, open writing with knitr – Carl Boettiger

Open Science Framework

Berkeley Electronic Lab Notebook Ideas

knitr, make, and GitHub for a lab notebook

Electronic Lab Notebook – The Postdoc Experience

Evernote as an Electronic Lab Notebook

 

Advertisements

6 thoughts on “Electronic Lab Notebook: Take 1

  1. I’ll add a second vote for Evernote. Underneath, it stores everything as HTML, so you can exact later if you need. Also easy to just dump stuff in: web pages, gel images, pictures of handwritten notes, PDFs of phylogenies, etc. I use a hybrid between a paper lab notebook and Evernote (which contains images of the paper notebook plus more – ‘multimedia’ – as it was called back in the 90s).

    • Thanks for the recommendation and info. It’s probably an easier option than my current use of Markdown, Pandoc, BibTex, and GitHub. I should give Evernote a try. I signed up and downloaded years ago but have never really used it.

      • Yeah I’ve tried a whole range of different things over the years. Three must-haves for me were easy backup, cloud access, and full text search. Markdown, Pandoc, BibTex, and GitHub are great for code and Rmd is great for exploratory analysis, etc, but much of the other stuff that goes into logging the progress of science (gel images with annotations, websites, email conversations, field site images, handwritten notes, etc) is pretty hard to shoe-horn into plain text. I finally went all in on Evernote halfway through my PhD when I switched over to using a livescribe pen/notebook in the lab. The ability to have Evernote search through your handwriting (works remarkably well even with my horrible handwriting) is a killer feature and means it’s way easier to find old notes about what happened and when. I ditched the livescribe when I started my postdoc (ballpoint pen was not of the best quality even though the digitizing always worked great) and just use regular old pen and paper now. Smartphone cameras are finally good enough where I can just snap a pictures of lab notebook pages and send them directly to evernote.

  2. Wow, I had no idea about the handwriting search in Evernote. That is a fantastic feature! I’ve added images including phone photos of handwritten notes in my markdown files. But my current system has no way to search those notes within the documents. Very cool. Thanks again for the info.

  3. Pingback: Writing Scientific Papers Using Markdown | Daniel J. Hocking

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s