Pandoc Project Template

A flexible and future proof way to write documents.

• Getting Started
• Features
  • Disabling Features
  • Alternatives
• Continuous Integration
  • Spell Checking
  • Deployment to Nextcloud
• Building
  • Requirements
• Alternative To Project Template

Getting Started

First, you need to [create a new project][newproject]. Choose Create from template, Instance and finally pandoc. Choose a project name and create the project. Clone the project to your local machine and start editing the Markdown sources. See below for features, using the build server and how to build locally.

Alternative To Project Template

Using the project template has a few issues at the moment:

• issues are copied
• pipelines that have already run are copied
• project history is copied (you are not the author of those commits)

When I initially wrote the project template, I assumed you would get a clean repository with the initial commit being everything squashed and you being the author. The alternative approach would be to do the following:

1. clone the template repository

   git clone https://git.idiv.de/templates/pandoc.git my-title

2. remove the history

   cd my-title
   rm -r .git

3. [create a new project][newproject] in GitLab without using the project template

4. initialize new repository and add the new project as remote

   git init
   git remote add origin https://git.idiv.de/NAMESPACE/MY_TITLE.git

5. create the initial commit and push

   git stage .
   git commit
   git push -u origin master

I assumed the project template would do exactly that. Lets hope they change this in the future.

Features

This project template and its build is quite flexible. The template strives to provide a more or less complete example of what is possible to keep not just the text under version control but also other sources like flowcharts and plots. Flexibility also means that you can add any feature you see fit or opt out of features you don't need.

The default features include:

• write document in pandoc-flavored Markdown, the example sources include
  • a Markdown guide
  • how to use inline LaTeX
  • how to insert tables from CSV files
  • how to insert flowcharts generated via Graphviz from .dot sources
  • how to insert plots generated with R from CSV sources
  • how to use citations using a shared reference database
• a build script (Makefile) to automate the build process
• convert Markdown sources to PDF with pandoc
• continuous integration, i.e. a build server that automatically
  • spell checks Markdown sources
  • deploys document to Nextcloud share

Disabling Features

You can disable most features by just removing the respective content in the Markdown sources. Some feature are integrated in the Makefile for which you need to perform additional steps:

• plots: remove the *.R file(s) in img/plot
• flowcharts: remove the *.dot file(s) in img/graph

Alternatives

Most features and tools are optional and replaceable, the only real core tool is pandoc. Besides that, everything can be changed, e.g. you could use a different programming language for plotting, you could use Inkscape to create and edit your flowcharts, or you could even use a different build tool.

If you have an idea, suggestion, or even a concrete example, feel free to open an issue or create a merge request. The more (optional) features we can offer the more it shows how flexible and future proof this way of working is.

Continuous Integration

In GitLab, projects can automatically be built, tested and deployed. In general, this is called continuous integration (CI) and continuous deployment (CD). Both together are abbreviated to CI / CD and this is what you will see in the GitLab web interface.

You define what the CI server should do in the .gitlab-ci.yml file. This project template contains two stages:

1. check to run the spell checker on all document sources
2. deploy to upload the built document to an iDiv Nextcloud share

Spell Checking

Unfortunately, spell checking is still a bit awkward. The problem is that the currently existing spell checking tools are not really well integrated. See this issue for more information and progress.

If you think that the spell checker with its current limitations is more of an annoyance than help and you can rely on the spell checking from your editor, you can disable it by removing the check stage from the .gitlab-ci.yml file. Once the issues are fixed, you can simply re-enable it by reverting the commit.

Deployment to Nextcloud

The deploy stage tries to upload the built document to an iDiv Nextcloud share. To make this work, you will need to set this up first by following these steps:

1. log in to Nextcloud

   Go to the iDiv Nextcloud and log in.

2. create a new, empty folder

   My recommendation is to create a top-level folder named documents. Then, create a sub-folder named after your project, e.g. the document title.

   Avoid spaces and use dashes or underscores instead!

3. share the folder

   Click on the share icon of the folder. Click on the plus + icon next to Share link. Check the Password protect box and set your password. Then click the Allow upload and editing radio button.

   Finally, click on the copy link to clipboard icon.

4. CI settings

   Go to your GitLab project. Go to Settings, CI / CD and expand the Variables section. Add two variables:

   1. NEXTCLOUD_SHARE, paste the share link you copied from Nextcloud. Remove everything except the part after the last slash /, e.g. if your link is:

      https://portal.idiv.de/nextcloud/index.php/s/aFgeXn5Ay9qffbd

      ... you only need the aFgeXn5Ay9qffbd part.

   2. NEXTCLOUD_PW, with the password of the share.

   Make sure both variables are masked so the values will not show up in your logs.

Building

To build the PDF locally, you need to run the build script. Open a terminal, change to the cloned project directory and type make. Make already does incremental compilation, i.e. it re-compiles only those targets whose sources changed. If you want to watch for changes and automatically run make when something changes, use watchexec:

watchexec make

Note: The watchexec tool honors .gitignore so it doesn't run the command when the targets changed but only if the sources do.

Requirements

These requirements are all more or less optional or replaceable. If you want to build the template PDF, you need these dependencies:

• git
• pandoc and pandoc-citeproc
• LaTeX, either through texlive or MiKTeX
• GNU make
• Graphviz
• R including the packages data.table, tidyverse and svglite install.packages(c("data.table", "tidyverse", "svglite"), Ncpus = 8)
• watchexec
• markdown-spellcheck

Note: On MS Windows, it might be helpful using chocolatey to install some of the dependencies, e.g. GNU make and Graphviz.