[HN Gopher] Keeping a Lab Notebook [pdf] (2013)
___________________________________________________________________
Keeping a Lab Notebook [pdf] (2013)
Author : Tomte
Score : 106 points
Date : 2021-08-25 11:38 UTC (2 days ago)
(HTM) web link (www.training.nih.gov)
(TXT) w3m dump (www.training.nih.gov)
| cpach wrote:
| Reminds me of this post from June, which inspired me a lot:
| https://news.ycombinator.com/item?id=27639875
| [deleted]
| aalam wrote:
| For a good historical example, see Linus Pauling's notebooks
| (with page numbers and dates):
| http://scarc.library.oregonstate.edu/coll/pauling/rnb/
|
| The blog by biologist Dr. Colin Purrington also has advice for
| maintaining a notebook (discussing why, backups, and what to note
| down): https://colinpurrington.com/tips/lab-notebooks/
| ajot wrote:
| I came here to add Purrington's blog. But I would also add that
| you should look at older versions in the Wayback Machine, it
| used to have a little more depth and images depicting things
| (e.g., a paper chromatography test of different inks).
| visviva wrote:
| I've always been attracted to the idea of keeping a lab notebook
| for my work, but since I do engineering/scientific computing, the
| concepts never seem to directly translate. Does anyone have any
| resources for keeping a similarly structured log of research/work
| done on non hard science projects?
| all2 wrote:
| If you're in software engineering, Jupyter notebooks support a
| large variety of languages now.
|
| I keep handwritten notes in a notebook of recipes I
| particularly like. These are useful for making adjustments over
| many preparations. I guess the bottom line is being willing to
| enforce _an_ organization on your notes, and then iterate
| towards a format that makes sense to you. For the recipe notes,
| time series (like journal entries) works well. Or inline. It
| depends on the note. Ingredient substitutions can be time
| series, but wrong measures specified in the recipe would be an
| inline correction.
|
| > engineering/scientific computing
|
| Definitely Jupyter notebooks or other "literate computing"
| methods. Emacs org-mode has literate programming stuff, but
| honestly Jupyter is easier to use.
| visviva wrote:
| Thanks - I'm not a software engineer, but I write a lot of
| code anyway. I've used Jupyter notebooks to good effect in
| the past for smaller, self-contained projects, but they don't
| quite seem to fit my workflow well enough for me to use them
| for everything. You make a good point about picking
| _something_ and enforcing it - I just have a hard time even
| finding something to start iterating on. Maybe I just need to
| be more creative about how I think about /use Jupyter
| notebooks...
| rmnclmnt wrote:
| You can use use Jupyter Notebooks like Markdown files, with
| the added benefit of code blocks being executable for
| reproducible work.
|
| Then, just throw your notebooks into MkDocs (using a
| plugin) and voila, you get a statically hostable version of
| your notebooks!
|
| Easily coupled with Git and CI/CD practices, you have a
| nice, easy to use setup for prodyctive work and knowledge
| sharing if needed!
| all2 wrote:
| > something and start iterating
|
| Go with a standard time-series journal. This is about the
| simplest way to do things. It works well enough for me with
| minor additions.
|
| Eventually you've referred back to your journals enough to
| know what kinds of information you'll be looking for. I've
| gotten part way there, so far. I'm still trying to sort out
| details for a lot of stuff; where does the plan of a
| project live? In the journal? In the projects files? What
| about the day-to-day entries? What about troubleshooting
| epilogs? Like I said... I'm still trying to sort out where
| each of these live.
|
| I'm _tempted_ to just throw all the things in project
| folders (because they 're all projects anyway) and then
| grep for stuff if I need to find it.
| all2 wrote:
| But putting everything in one folder still doesn't
| satisfy my itch to see my journal from two views: time-
| series (which answers "what did I do X days ago?") and
| project collections (which answers questions like "what's
| next?" and "how did I solve this obscure problem?!").
| refurb wrote:
| This is where electronic notebooks really shined. No longer in
| the lab, but many experiments are repeats with slight tweaks to
| the procedure. Rather that rewriting everything out we'd often
| just write "same procedure as XXX-YYY with the following
| changes".
|
| It always felt much cleaner to just copy-paste an experiment and
| directly edit the changes.
|
| Plus being able to type versus write meant I put way more detail
| down and could easily attach data.
|
| It should be noted too that the US patent law changed from first-
| to-invent to first-to-file in 2013 (when this document was
| written). The importance of lab notebooks in defending a patent
| becomes much less important after that. I know defending a
| invention was a HUGE part of keeping a good notebook.
|
| https://en.m.wikipedia.org/wiki/First_to_file_and_first_to_i...
| poidos wrote:
| I use org-journal[0] in Emacs as a very efficient lab notebook. I
| use the Doom[1] configuration, so this is my workflow:
|
| - launch Emacs
|
| - type `Space n j j`, which opens a journal file customized the
| way I like it*, inserts a date heading and timestamp, and sets me
| in insert mode
|
| - type my timestamped notes
|
| I can do this from any buffer in Emacs, so it's really convenient
| to stop in the middle of something, jot down a note, and then go
| right back to what I was doing. I develop iOS/macOS software
| right now, so the switch to Emacs from Xcode is a little more
| friction than I used to have, but it's so useful I don't mind it
| at all.
|
| * I have a weekly journal in a directory for the year, titled
| week number-month-day that started that week (this week's is
| `34_08-23`)
|
| [0]https://github.com/bastibe/org-journal
| [1]https://github.com/hlissner/doom-emacs
| bloopernova wrote:
| Very similar to the way I use it. Ctrl-c n j makes a new entry
| in org journal, carrying over any TODO/PROG items from previous
| days if the current day file doesn't exist. I use it all the
| time and it's replaced my beloved hardback A4 graph paper
| notebook.
|
| I keep my journal files hidden from the Treemacs sidebar so it
| doesn't get cluttered.
|
| I keep a permanent in progress item with sprints and their
| start/finish dates, backlog items, etc etc.
| slitdangsthroat wrote:
| I keep a notebook of random HN comments that break the rules, but
| dang allows it. He only allows this for rich spoiled cunts for
| some reason, so I plan on exposing the cunt dang for his
| hypocrisy.
|
| I genuinely hope his family gets raped.
| ChuckMcM wrote:
| I have kept lab notebooks for decades now, some thoughts;
|
| Writing them, on paper, has been very helpful. The act of
| writing, the pace of it, the composition of what I'm going to
| write, combine to help make the information more durable in my
| head some how. I've got a ReMarkable 2 notebook which gives me
| infinite pages which is kind of neat, but I still like paper
| notebooks.
|
| They have saved me a couple of times and made me more productive.
| Since I often have a number of projects in parallel so that I can
| pick up one when the current one hits a stall (like waiting for
| parts, or needing different equipment), and when I come back to a
| project I've organized my notebook so that I can re-create my
| mental map from where I left off.
|
| And in one instance it torpedoed a vendor who tried to patent an
| idea that we had reached out to them for help in developing.
| nataz wrote:
| How I take notes: For professional life: Pen + dot grid notepad -
| developed independently, but similar to the bullet journal
| technique. Dot grid plus pens make the whole thing ultra
| customizable. I can sketch engineering designs, make a calendar,
| track action items, take detailed notes, all in the same format.
| The key is to be strict with page numbers, dates, and index as
| much as possible.
|
| Things I occasionally miss - keyword search (I can still look
| things up by date or subject in the index), multimedia inserts
| (think dragging video/photos/sound clips into one note), never
| ending space (notebooks run out of pages), easy backups (thinking
| about digitizing with photos or scans), team collaboration (if
| this is necessary I use Trello).
|
| Things I like - no OS/tech stack compatibility issues, "it just
| works", lighter then a laptop/tablet, don't need to charge, easy
| to read, can bring into a secure area (where outside electronics
| are not permitted), travels well, hard to damage.
|
| http://bulletjournal.com/get-started/
|
| For personal life: add Google keep for simple lists, and then a
| mix of Trello and dot grid for larger projects (less strict
| formatting than professional life project management).
|
| How I index using pen/paper:
|
| Every page has a number in the upper outside corner.
|
| Every entry into the note book has a title + subject tag, date,
| and the initials of a list of people linked to that entry. For
| easy reading and scanning I put all of this on one line. The
| title and subject are dark lined (thick), followed by the date
| MM/DD/YY, and then the initials inside a ( ). Each piece of info
| is separated by a ";". All of the info listed is underlined.
|
| under the intro line, I also break down actions, notes, calendar
| adds, etc with 3-4 different symbols. All I all this makes it
| clear and easy to scan quickly as you are looking for things.
|
| Each month I have 4 or 5 pages devoted to organization.
|
| I have a task list. Two bulleted columns. Old tasked are migrated
| over from the previous month. New tasks are added as soon as they
| are generated. Completed tasks get an X through the bullet.
| Migrated tasks get an >. Tasks that are linked to journal entries
| get a page number and a tag.
|
| I have a calendar page. 1 column, all dates of the month listed .
| Meetings, important actions and events are are listed by day, or
| with arrows over multiple days. Linked by tag/page number if
| applicable.
|
| I have a subject index. Major tags, reoccurring subjects, get a
| line or two. Page numbers with those subjects are added to the
| index.
|
| I realize this sounds complicated, but it's not once you get into
| a routine. The biggest this is it's okay to make "mistakes", and
| to get used to not having a delete key. Change is part of the
| notebook. Think of it like a change log. It's also important to
| be able to see what you change. Since I work in pen, I just make
| a single line through things I want to change.
|
| Note: when I finish or complete something on a task list or the
| calendar, I'll mark or change the symbol, I won't cross something
| out just because I'm done with it.
|
| Finally, I have a bunch of multi color pens I use. They are all
| the same brand/model, but I tend to use a different color each
| day. That makes it obvious as you are scanning that you are on
| the next day. I don't use specific colors for anything in
| particular, too hard to keep track of.
|
| Hope this helps.
|
| Reposted from an older thread @
| https://news.ycombinator.com/item?id=17799232
|
| _edit - notebook choice is important. I use something like this
| (https://www.amazon.com/Dotted-Bullet-Notebook-Journal-Hardco...)
| _ Not an affiliate link. Its sturdy, you can't pull out or add
| pages (which is a feature), bound and stitched, decent quality
| paper, dot/grid, and normally under $15/book.
| bluenose69 wrote:
| I've done something similar for decades. I work in science, and
| need to cross-reference a lot. I have a simple scheme for that.
|
| When I refer to another page, I write that page number in
| square brackets (in the text, and repeated in the outside
| margin). Reference to other books have the same, but the number
| is prefixed by an index number for the book, and then a colon.
|
| The page referred to _also_ gets an entry added, in the same
| format but in angled brackets.
|
| This makes it really easy to propagate new information through
| my work.
|
| Apart from the possible addition of the angle-bracket entries,
| I never alter existing text. If I redo an analysis, I simply
| make a wholly new entry, and add an angle-bracket notation on
| the old entry.
|
| The square-bracket and angle-bracket scheme makes it pretty
| easy to update analyses, e.g. if instruments are discovered to
| be faulty on a certain date.
|
| In addition to this referencing scheme, I also have two-letter
| keywords for topic areas, and those go into the table of
| contents.
|
| Another difference is that my dates are written yyyy-mm-dd.
|
| A policy of photocopying new work every Friday afternoon is
| quite sensible. I don't keep the copies with the original work,
| though, out of fire/flood/etc concerns.
| bluenose69 wrote:
| Oh, one more thing: about once a week, I type the item titles
| into a text file, along with book number, page number,
| keyword list and both square- and angle-bracket references.
| That way I can use `grep` to find things. I suppose I could
| write a few lines of code to auto-generate a local webpage,
| but I'm just as happy listing the file or looking in a text
| editor.
| nayuki wrote:
| A lot of the goals/problems mentioned in the document can be
| solved with a "lab notebook" consisting of plain text files plus
| Git version control. For example:
|
| > Legal document to prove patents and defend your data against
| accusations of fraud
|
| > Bound/Stitched | No lost pages, legally stronger | Difficult to
| copy, not logically organized
|
| > Requires electronic security, corrupted files, software
| compatibility issues
|
| > No pages come out of the notebook | Do not take any pages out
| or remove any data | Do not skip pages in your notebook | Cross
| out any unused parts of a page
|
| > Correct mistakes, do not remove them | Cross out mistakes with
| a single line | Paste in corrections without covering anything |
| Sign and date all corrections
|
| Also, if you sign your commit hashes with a trusted timestamping
| service, then that could serve as watertight evidence of prior
| work.
| jjoonathan wrote:
| Programmers really love plaintext/markdown notebooks, but they
| introduce a lot of friction around graphs and "snap a picture +
| annotate," and that destroys their value prop for me.
|
| Snap + annotate (or sketch + annotate) is an absolutely killer
| workflow if you deal with artifacts in the physical world, to
| the point where I gladly endure poor text editing capability
| and version control so long as I can have snap + annotate.
| Heck, if I didn't have a computer with good snap + annotate
| support I'd use a _physical lab notebook_ before I tried to
| somehow shoehorn the workflow into text.
| mlon_eusk wrote:
| "We are living in an era of the most sophisticated technological
| advances possible, and yet the treatment of cancer is
| paleolithic." -- Azra Raza
| monkeybutton wrote:
| Does anyone here keep a log or journal at work? I'm leaving a
| place that I've been at for quite a few a years and I am a little
| regretful for not writing down some observations and predictions
| as I had them. It seems like nothing the business did over the
| long term went according to plan, but I've lost a lot of the past
| perspective so its hard to compare.
| Isthatablackgsd wrote:
| I do. I have four journaling system.
|
| 1. Small Whiteboard - Daily Todo & quick notes for me to
| remember later. Easily erasable and save paper when it is not
| need. It is always on my right-side corner of my desk for easy
| access and quick jot note. And I take a picture of the small
| whiteboard at the EOB for archival reason in case if I need to
| go back.
|
| 2. Legal Pad - For weeklong Todo, project notes (for developing
| forms and contracts), clients, etc. I have two pads: the normal
| pad and the small pad (5x8). I have more than 10 small pads for
| various functions since they are small and don't take much desk
| space. They are easily accessible when I can grab the pad
| quickly and write it down.
|
| 3. Obsidian - For documentations and information for long term
| storage, including development planning and various stuff.
|
| 4. Google Docs - For collaboration Todo, agenda for the
| meeting, conversation notes, minutes, etc. to share with my
| boss. I could use Obsidian for collaboration but my boss is not
| knowledgeable with Obsidian or any journaling software and it
| will be pain for him to set it up. I rather to make it simpler
| for my boss and easy access for him to look into.
|
| I strongly used 1 and 2 because I found that jotting it down
| via handwriting is quicker and ready to use than doing it
| electronically. Doing it electronically can take some time to
| get into typing mode when time is critical like in the middle
| of the phone call. Pens, papers & highlighters are always
| available and ready to use within a second.
|
| I have a 'check spindle holder' for my small legal pad paper
| when I am done with it. I just "whamp" the paper on it and get
| the satisfaction and stress relieving from it.
| selfhoster11 wrote:
| I did. It was valuable to be able to go back and refer to
| discussions about the architecture of something, or to note
| down obscure details. If you choose to do it on paper, make
| sure you photocopy regularly.
| dkarl wrote:
| I keep a journal, because otherwise I tend to forget or
| minimize much of what I've done. It helps me remember the big
| things when it comes time for performance reviews, and it also
| helps me remember the small things with accurate sprint
| retrospectives. Spending several hours per week fighting with
| environments, retrofitting code to ancient dependencies, and
| investigating build failures that turn out to be local to one
| person's laptop are time sucks that need to be called out in
| retrospectives so that people understand the potential value of
| paying down technical debt. Those things are also exactly what
| I tend to forget about by the end of the week.
|
| My journal consists of a notebook with a new note for each day,
| titled with the date for easy sorting. I copy/paste in a
| template that looks like this: Morning
| checklist: _ Check email _ Check Slack
| _ Make sure my tickets are in the correct state _ Look
| at the meetings on my calendar and decided how to prepare (or
| not) for each one (etc.) Plan today:
| Actual today:
|
| I love the morning checklist because it ensures that if I space
| on one of those items during the day, like updating a ticket,
| it'll get taken care of the next morning. Trusting this part of
| the process takes a lot of stress out of the rest of the day.
|
| After I make my plan for the day, I don't touch that section
| again. That way I can compare and contrast my plan with what
| actually happened. It can be enlightening to look back and see
| that a development project was in my plan every morning for a
| week, and my "actual" was dominated by production support and
| fighting with test environments.
|
| The best part is that since I started keeping a journal like
| this, I've realized I get way more done than I give myself
| credit for.
| marbu wrote:
| Yes, for every important long term goal/task I'm working on.
| Eg. I have a journal file for feature foo I'm working on and
| another one for bug bar I'm looking into, but I don't
| immediately create one for something I can do in a day or two.
| arminiusreturns wrote:
| When I was working for scientists this was a habit I learned
| and loved it, but due to some other factors I became aware that
| in more traditional business the kind of frank statements I put
| in logbooks could be a potential issue, and have found myself
| not doing it due to the chilling effect.
| bodge5000 wrote:
| yeh I keep digital notes, a notebook and a load of sticky
| notes. Theres no real system to it (just the way I like it),
| but generally speaking working notes go in the notebook (also
| for tangent tasks), important reminders on the sticky notes
| (pasted all over my monitors) and "knowledge" goes in the
| digital notes. In reality its a lot more sporadic than that,
| but I know where to find things
| ethangarofolo wrote:
| I do, yes, and I ask my team to do the same. Even in software
| development it's very helpful. It helps the team stay in sync
| more, reducing the need to meet. It helps me stay in sync from
| yesterday me to today me as well. It avoids duplicated work as
| well. I consider doing this part of professionalism 101 at this
| point.
| cpach wrote:
| Kind of. As I mentioned elsewhere
| (https://news.ycombinator.com/item?id=28327689) I recently read
| a very interesting post on Xe's blog and that inspired me to
| start writing things down in a physical notebook.
| poidos wrote:
| Yes, religiously. I have a list of tasks for each year which
| links to a file with my notes on each task, as well as a daily
| journal where i log my work hours, random notes from meetings,
| etc. It is exceedingly helpful.
| znpy wrote:
| I think that rednotebook is invaluable for this purpose.
|
| It's a nice linux app that works like.. a notebook.
| AlbertCory wrote:
| Having spent 3 years with lawyers in Patent Litigation:
|
| This stuff is old-school, but the lawyers all know & understand
| it. Courts understand it. If you do something in a way that
| someone else has already done and testified in court about it,
| you win. This is not the place to be innovative.
| cma wrote:
| This doesn't apply anymore with first to file, unless you
| preempt the filer by publishing it publicly.
| AlbertCory wrote:
| You still encounter old patents in litigation, though, before
| first-to-file was introduced.
|
| It also has other uses beyond Interference proceedings.
| jmt_ wrote:
| Does anyone have experience using something like Obsidian as a
| sort of lab manual for software engineering? How do you structure
| your notes? What gets written down? I use Obsidian and the note
| of the day plugin which uses a template that has a todo list and
| a Recap section in which I intended to summarize the work I did
| that day and any comments worth writing down. I used the todo
| list part daily but I've struggled to consistently write the
| recap. I think it's too vague; I'm looking for a little more
| structure that makes recaping the day easier and manageable. When
| I try to write a recap currently, it can feel overwhelming as I
| try to think what I should and shouldn't write, and how I should
| or shouldn't write it.
| Groxx wrote:
| I've had relative success with keeping it open + on all
| desktops. I don't fullscreen anything, so it's always visible -
| since I started doing that, I'm MUCH more consistent about
| updating it.
|
| Prior to that, I had this small script that'd open textedit to
| a daily-note every time I made a terminal session, which served
| much the same purpose:
| https://news.ycombinator.com/item?id=22279802 . Obsidian makes
| it a lot easier to jump between days though :)
|
| (as a semi-aside: I very much like Obsidian. It's one of the
| only "just let me use my pile of normal markdown/commonmark
| files like I want" apps I've found, after trying dozens. it's
| very very very unintrusive / doesn't impose any structure.
| highly recommended!)
| threatofrain wrote:
| Slightly tangential, but does anyone have a story for how to
| share Obsidian notebooks in a team context? That's the one
| major thing I've found missing in these off-line first notebook
| apps.
| tnorthcutt wrote:
| I do a tiny bit, but even less structured than you're
| describing (and less consistently), so I'm curious to see other
| responses here.
| kstrauser wrote:
| I do, but not in any deliberately very structured way. I use
| daily notes as a high-level "this is what i worked on" journal,
| with links to the details of those items. For example:
| # Work Added feature [Foo](https://pivotal/...) to
| [[Bar Project]] - Learned about sprockets:
| <https://wikipedia/Sprocket> Talked to Jane about
| the state of [[Qux Project]]
|
| As I'm writing it, it gives me quick access to the resources
| I'm using today. With backlinks, if I'm looking at the "Bar
| Project" page, I can see when I was working on it and what
| resources I was using on those days.
|
| My process optimizes for low friction. I want to _take_ notes,
| not necessarily spend a lot of time _rigorously organizing_
| notes. It also gives me a sort of the timeline view that
| Zettelkasten fans like, but without the rigorous tree structure
| that I don 't personally seem to benefit from.
| mmcdermott wrote:
| I've been using TiddlyWiki for this for a few years. It started
| off as being my lab notebook and has expanded to be my
| everything-under-the-sun notebook.
|
| It began simply. It started because I had a bunch of new
| services/apps being thrown at me on a new project on a bunch of
| new servers. I recorded the information and linked it all
| together and expanded it to include meeting notes (on the level
| of minutes), transcriptions from my physical notebook, research
| notes ("here are examples of the issue, what we expect, what we
| think it should be, all the stuff I've tried, etc."),
| architecture notes.
|
| TiddlyWiki has a journaling feature that I use for the recap-
| level you talk about. I usually head the journal entry "Minutes
| of the Day." There will be a bullet point for each
| project/stream and then sub-bullets for each meeting (with top
| level decisions/takeaways under that) or thing I did. It's a
| bit subjective, but I keep these entries high level and will
| hyperlink to more detailed research notes.
| COGlory wrote:
| I have bounced around between different lab notebooks, obviously
| starting with paper. I then moved to a word doc. Problem with
| word documents is that they don't have any good hierarchy. I then
| tried Obsidian, which is a markdown editor. It's nice, but the
| table and image support is lacking. I need to be able to paste in
| tables from a spreadsheet because that's how I set up many
| experiments.
___________________________________________________________________
(page generated 2021-08-27 23:01 UTC)