Skip to content
Snippets Groups Projects
Unverified Commit 166ee749 authored by ck85nori's avatar ck85nori :railway_track:
Browse files

[rdm] adds git for rdm and reproducibility presentation

parent e189a7f3
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
img/rdm-use-cases-a.dot
+ 3
− 3
View file @ 166ee749
...@@ -34,9 +34,9 @@ digraph { ...@@ -34,9 +34,9 @@ digraph {
paper_version_a[label = "nature-review-1"] paper_version_a[label = "nature-review-1"]
} }
script_version_a -> software_a_version_a [label = "\n\n", lhead = cluster_software_a] script_version_a -> software_a_version_a [label = "\n\n"]
script_version_a -> software_b_version_a [label = "\n\n", lhead = cluster_software_b] script_version_a -> software_b_version_a [label = "\n\n"]
paper_version_a -> script_version_a [label = "\n\n", lhead = cluster_script] paper_version_a -> script_version_a [label = "\n\n"]
} }
img/rdm-use-cases-b.dot
+ 6
− 6
View file @ 166ee749
...@@ -37,12 +37,12 @@ digraph { ...@@ -37,12 +37,12 @@ digraph {
paper_version_b[label = "nature-review-2"] paper_version_b[label = "nature-review-2"]
} }
script_version_a -> software_a_version_a [label = "\n\n", lhead = cluster_software_a] script_version_a -> software_a_version_a [label = "\n\n"]
script_version_b -> software_a_version_a [label = "\n\n", lhead = cluster_software_a] script_version_b -> software_a_version_a [label = "\n\n"]
script_version_a -> software_b_version_a [label = "\n\n", lhead = cluster_software_b] script_version_a -> software_b_version_a [label = "\n\n"]
script_version_b -> software_b_version_b [label = "\n\n", lhead = cluster_software_b] script_version_b -> software_b_version_b [label = "\n\n"]
paper_version_a -> script_version_a [label = "\n\n", lhead = cluster_script] paper_version_a -> script_version_a [label = "\n\n"]
paper_version_b -> script_version_b [label = "\n\n", lhead = cluster_script] paper_version_b -> script_version_b [label = "\n\n"]
} }
img/rdm-use-cases.dot
+ 7
− 7
View file @ 166ee749
...@@ -38,13 +38,13 @@ digraph { ...@@ -38,13 +38,13 @@ digraph {
paper_version_c[label = "nature-final"] paper_version_c[label = "nature-final"]
} }
script_version_a -> software_a_version_a [label = "\n\n", lhead = cluster_software_a] script_version_a -> software_a_version_a [label = "\n\n"]
script_version_b -> software_a_version_a [label = "\n\n", lhead = cluster_software_a] script_version_b -> software_a_version_a [label = "\n\n"]
script_version_a -> software_b_version_a [label = "\n\n", lhead = cluster_software_b] script_version_a -> software_b_version_a [label = "\n\n"]
script_version_b -> software_b_version_b [label = "\n\n", lhead = cluster_software_b] script_version_b -> software_b_version_b [label = "\n\n"]
paper_version_a -> script_version_a [label = "\n\n", lhead = cluster_script] paper_version_a -> script_version_a [label = "\n\n"]
paper_version_b -> script_version_b [label = "\n\n", lhead = cluster_script] paper_version_b -> script_version_b [label = "\n\n"]
paper_version_c -> script_version_b [label = "\n\n", lhead = cluster_script] paper_version_c -> script_version_b [label = "\n\n"]
} }
rdm.html 0 → 100644
+ 498
− 0
View file @ 166ee749
<!DOCTYPE html>
<html>
<head>
<title>git@rdm</title>
<link rel="stylesheet" href="reveal.js/dist/reveal.css">
<link rel="stylesheet" href="reveal.js/dist/theme/black.css">
<link rel="stylesheet" href="css/company-logo.css" />
<link rel="stylesheet" href="css/ribbon.css" />
<link rel="stylesheet" href="css/crawl.css" />
</head>
<body>
<div class="reveal">
<div class="slides">
<!---------------------------------------------------------------------
<!-- intro
<!-------------------------------------------------------------------->
<section>
<section id="title" data-markdown>
# git and rdm
## (... and reproducibility)
notes:
- `make -B` to create images, then reload
</section>
<section id="intro-objectives" data-markdown>
## objectives
- show how git relates to RDM
- ... and reproducibility
</section>
<section id="intro-agenda" data-markdown>
## agenda
1. motivation
1. ~~teach ***how*** to use git/GitHub/GitLab~~
1. use cases
1. anti-patterns
1. platforms
1. Q & A
notes:
- feel free to interrupt with immediate questions
- more involved detailed discussion as part of Q & A
</section>
<section id="intro-version-control" data-markdown>
## about version control
> records changes
what who when (why)
notes:
- the **why** (aka context) is optional
- you have to do this
- you have to care about it
</section>
<section id="intro-about-git" data-markdown>
## about git
### best tool for the job
- simple by design
- powerful if needed
- documentation / community
- industry standard
notes:
- git name: "the stupid content tracker"
- doc: finding answers with web searches
</section>
<section id="intro-about-me" data-markdown>
## about me
- scientific computing support @ iDiv since 2014
- satisfied git user since 2010
> There will never be a better version control system than git.
>
> -- Christian Krause, 2017
</section>
<section id="intro-about-you" data-markdown>
## about you
![people](img/people.jpg)
notes:
- who has never used any VCS before?
- who actively maintains a VCS repository?
</section>
</section>
<!---------------------------------------------------------------------
<!-- motivation
<!-------------------------------------------------------------------->
<section>
<section id="motivation" data-markdown>
# motivation
> Why should I use version control?
notes:
- motivation chapter is about the concept of version control
- git is just a tool to do it
</section>
<section id="motivation-avoid-mess" data-markdown>
## motivation #1
### avoid mess
![blah](img/motivation-draft-mess.png)
notes:
- who has seen such a mess?
- who has contributed to such a mess?
- who has created such a mess?
</section>
<section id="motivation-want-structure" data-markdown>
## motivation #1
### want structure
![blah](img/motivation-structure.svg)
notes:
- structure: who, when
- why is hidden (only shows message header, not body)
- ability to inspect old versions and their diff
- ability to revert/undo a change
</section>
<section id="motivation-playground" data-markdown>
## motivation #2
### throw-away playgrounds
![playground](img/motivation-throwaway-playground.svg)
notes:
- test stuff without interfering
- throw away if garbage
- integrate if good
- switch back and forth without pain
</section>
<section id="motivation-collaboration-1" data-markdown>
## motivation #c
### collaboration made easy
<!-- do not fix this typo, it is here on purpose to show collab -->
> This text cntains a typo.
notes:
- demo GitLab (change **target branch** to create merge request)
- this is how you can do reviews of drafts
- discuss this slide/chapter/section link
</section>
<section id="motivation-collaboration-2" data-markdown>
## motivation #c
![typo-pr](https://pbs.twimg.com/media/EDsklbLUEAMdusJ.png)
notes:
- might not seem like much
- but you are still making the world a better place
- and it is not too much effort
</section>
<section id="motivation-automation" data-markdown>
## motivation #a
### automation
- ***quality***
- **code analysis**
- **spell check**
- **software testing**
- enforce **style guide**
- ***deployment*** (app store, web server)
notes:
- basically, everything you can script
- refresh presentation before next slide
</section>
<section class="star-wars" id="motivation-wars">
<div class="crawl">
<div class="title">
<h1>motivation #wars</h1>
</div>
<div data-markdown>
1. view the ***history*** of changes
1. know ***why*** someone changed it
1. ***revert*** a bad change
1. maintain ***multiple versions***
1. see the ***diff*** of two versions
1. find commit ***that broke*** something
1. have free ***backup***
1. have ***non-interfering*** playgrounds
1. have ***automated*** testing
1. have ***automated*** deployment
1. ***contribute*** to a project
1. ***share*** your code
1. let other people do the work ***for you***
</div>
<img src="http://i.giphy.com/90F8aUepslB84.gif" />
</div>
</section>
</section>
<!---------------------------------------------------------------------
<!-- use cases
<!-------------------------------------------------------------------->
<section>
<section id="use-cases" data-markdown>
# use cases
##### for
### version control system (vcs)
##### aka
### source code management (scm)
</section>
<section id="use-case-software" data-markdown>
## software
- generic (as in parameterized)
- (ideally) tested
notes:
- e.g. R package
</section>
<section id="use-case-scripting" data-markdown>
## scripting
##### aka
### how to run `$software`
- ... in `$environment`
- digital lab notes
- **reproducibility** !!1!
- execution scalability
notes:
- separate software from scripting
- `$enviroment`: multiple scripts/configurations for different
environments:
- EasyBuild, conda, singularity
- RStudio server, HPC cluster
- execution scalability: running software or script without having
to change it
- keep failed attempts in branches
- e.g. highly parameterized software, make a record of bad
parameter sets
</section>
<section id="use-case-publishing" data-markdown>
# publishing
### (markdown (with some tex))
- paper / thesis / book
- presentation
- documentation
- blog
notes:
- who is using markdown?
- who is using tex?
- who is using word? why?
- markdown with tex
- git.idiv.de/help
- search for markdown
- search for math
- show example
</section>
<section id="use-case-integrated-1" data-markdown>
## integration
![run `make` to generate the image](img/rdm-use-cases-a.svg)
notes:
- software A: tried and tested, reference to compare to
- software B: your experimental better version
</section>
<section id="use-case-integrated-2" data-markdown>
## integration
![run `make` to generate the image](img/rdm-use-cases-b.svg)
</section>
<section id="use-case-integrated-3" data-markdown>
## integration
![run `make` to generate the image](img/rdm-use-cases.svg)
</section>
<section id="use-case-integrated-data" data-markdown>
## integration
![run `make` to generate the image](img/rdm-use-case-data.svg)
</section>
</section>
<!---------------------------------------------------------------------
<!-- anti patterns
<!-------------------------------------------------------------------->
<section>
<section id="anti-patterns" data-markdown>
# anti-patterns
![unlimited power](img/unlimited-power.jpg)
notes:
- does anyone know what an anti-pattern is?
</section>
<section id="anti-patterns-def" data-markdown>
# anti-patterns
> An anti-pattern is a common response to a recurring problem that
is usually ineffective and risks being highly counterproductive.
notes:
- most anti-patterns about how to use git
- focus here is on these relating to RDM and reproducibility
</section>
<section id="anti-pattern-binary" data-markdown>
# binary files
### (aka non-text)
- no diff with binary
- use textual representation
- convert with automation
notes:
- can someone give me an example of text file?
- markdown
- source code, scripts (R, shell)
- XSV
- can someone give me an example of binary file?
- compiled programs
- MS word excel
- PDF PS
- don't just put everything in the repo, use *ignore
</section>
<section id="anti-pattern-big-data" data-markdown>
# data in git
## (scientific/big data)
- version control for data is **DIFF**erent !!1!
- git is VCS for text, not for data
notes:
- do you release data with every script change? no, too expensive
- you don't ever put data files in a git repository
- not even with LFS
- big ball of mud, no metadata
</section>
</section>
<!---------------------------------------------------------------------
<!-- platforms
<!-------------------------------------------------------------------->
<section>
<section id="platforms-list" data-markdown>
# platforms
- GitHub
- https://github.com/idiv-biodiversity (cloud)
- GitLab
- https://gitlab.com (cloud)
- https://git.idiv.de (self-hosted @ iDiv)
</section>
<section id="platforms-purpose" data-markdown>
# platforms
- enable collaboration
- bug tracker / feature requests
- documentation / wiki
- project management tools
- issue boards, milestones, gantt
- trigger automation
- publish/download releases
</section>
</section>
<!---------------------------------------------------------------------
<!-- q & a
<!-------------------------------------------------------------------->
<section>
<section id="q-n-a" data-markdown>
# Q & A
notes:
- walk through examples of software/script/publishing
- walk through one of your use cases
- demo consulting
</section>
<section id="consulting" data-markdown>
# consulting
[christian.krause@idiv.de](mailto:christian.krause@idiv.de)
</section>
</section>
<!---------------------------------------------------------------------
<!-- eof
<!-------------------------------------------------------------------->
<section id="eof" data-background="img/trex.png" data-markdown>
### thanks for listening
# EOF
</section>
<!---------------------------------------------------------------------
<!-- backup
<!-------------------------------------------------------------------->
<section>
<section id="backup" data-markdown>
# backup slides
</section>
<section id="empty" data-markdown>
</section>
</section>
</div>
</div>
<!-------------------------------------------------------------------------
<!-- css/js
<!------------------------------------------------------------------------>
<!-- company logo -->
<div class="logo-wrapper">
<a href="https://www.idiv.de/">
<img src="img/company-logo-small.png" />
</a>
</div>
<!-- ribbon -->
<div class="ribbon-wrapper right">
<div class="ribbon">
<a href="https://git.idiv.de/sc/edu/git-seminar/edit/master/basics.html"
target="_blank">
edit
</a>
</div>
</div>
<!-- reveal.stuff -->
<script src="reveal.js/dist/reveal.js"></script>
<script src="reveal.js/plugin/highlight/highlight.js"></script>
<script src="reveal.js/plugin/markdown/markdown.js"></script>
<script src="reveal.js/plugin/notes/notes.js"></script>
<script>
Reveal.initialize({
hash: true,
plugins: [RevealHighlight, RevealMarkdown, RevealNotes]
});
</script>
</body>
</html>
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment