I am always looking to improve my scientific workflow. My current workflow related to my
“lab notebook” is terrible and virtually non-existent. I don’t do a lot of lab work, but I still try to keep notes of what I do and thoughts I have and various things I try. Unfortunately, I have a number of random notebooks going at any given time, plus scrap paper, electronic notes as text files, word documents, excel documents, csv files, markdown, and trial and error scripts in R and Rmd and various other forms. It’s basically a disaster. I rarely go back to my non-field notes/notebooks and when I do it’s terribly difficult to find anything I’m looking for. I often search my own blog to find things because it’s easier than finding those ideas and notes elsewhere in my mess.
In my effort to get more organized, I decided that digital was the way to go. That way things can be tagged and searched, categorized, and filed in multiple places (copies or via tags). The question became how to best accomplish this. Carl Boettiger has a wonderful open, online lab notebook. I’m not ready to go fully open, especially since it would be difficult given the number of collaborators who might not want everything shared. For my first attempt, I am created a private GitHub repository for my lab notebook. I just got 5 free private repositories for education and teaching, so I am using one of them for my lab notebook. I cloned my repository to my computer and ready to start.
I am starting by writing my notes in Markdown using Mou or RMarkdown using RStudio. This is an easy way to take and format notes. Then I can use pandoc to convert those files to html or pdf or other formats if I want later. This is particularly useful if you have references. For example, the beginning of my first electronic notebook entry is a brief summer of a paper. Rather than having to type every reference out, I can use citations (and styles) just like I would in a manuscript. Here is the Markdown text:
04 April 2014 Brook Trout and Stream Salamander Interactions ================================== ## Local & Landscape Predictors of Spring Salamanders [@Lowe2002] * 25 1st order streams in NH * July & August 1999 * Single-pass surveys * 100 cover objects flipped (64-256 mm diameter) * 25 m upstream of confluence * Species * .G. prophyriticus * .D. fuscus * .E. bislineata * 7 of the streams visited 3 times in 2000 (with CMR) * Observed brook trout presence during surveys * Additional/Tangential electrofishing of 3 streams * Brook trout negative effects on Spring Salamander captures (not examined for other species) * Sedimentation and logging negative at whole-stream and regional scales ## References
I can use the pandoc code:
pandoc BrookTrout_Salamander_Notes.md -o BrookTrout_Salamander_Notes.pdf --bibliography Trout_Salamander_Refs.bib -H margins.sty --csl Ecology.csl
To make the margins 1″ I have a margins.sty file that is just a text file with the following code:
My PDF will render like this and my html will render like this (just change .pdf to .html in the above pandoc code – simple as that):
04 April 2014
Brook Trout and Stream Salamander Interactions
Local & Landscape Predictors of Spring Salamanders
(Lowe and Bolger 2002)
- 25 1st order streams in NH
- July & August 1999
- Single-pass surveys
- 100 cover objects flipped (64-256 mm diameter)
- 25 m upstream of confluence
- .G. prophyriticus
- .D. fuscus
- .E. bislineata
- 7 of the streams visited 3 times in 2000 (with CMR)
- Observed brook trout presence during surveys
- Additional/Tangential electrofishing of 3 streams
- Brook trout negative effects on Spring Salamander captures (not examined for other species)
- Sedimentation and logging negative at whole-stream and regional scales
Lowe, W. H., and D. T. Bolger. 2002. Local and Landscape-Scale Predictors of Salamander Abundance in New Hampshire Headwater Streams. Conservation Biology 16:183–193.
I will try out this form of a digital notebook for a while and see how it goes. I will continue to blog and if any of my notes are worthy of being shared I can just use pandoc to covert the notes to html and paste it in my blog. I’m sure my organization of notes and tagging will evolve over time with trial and error.
Does anyone else use a Electronic/Digital Lab Notebook? If so please share any advice in the comments section. I’d love to hear about what other people have developed for a workflow and archiving scheme.
Advantages of an Electronic Lab Notebook
- Searchable text and tags
- Categorize in multiple locations rather than just chronologically
- Ability to easily share in multiple forms
- Ability to incorporate text, photos, equations, links to external files, result outputs, etc.
- Although paper can get lost, wet, burned, eaten, etc., it’s more likely to get destroyed/lost/corrupted digitally so be sure to back it up. [Easy to do with git and GitHub – plus you could print PDFs if you really want]
- Sometimes slower to do which could stifle thought and creativity [Solution: hand write notes on paper or white board and scan them in or take a photo with you phone and copy it into the electronic notebook]
- Changing technologies could make assessing special formatted objects difficult [Solution: use plain text, html, csv files, and PDF files when possible since those are not likely to have long term problems]
Other Great Resources for Keeping an Electronic Lab Notebook
knitr and GitHub for a lab notebook – Carl Boettiger
Reproducible, open writing with knitr – Carl Boettiger
Electronic Lab Notebook – The Postdoc Experience