Variograms by falkmielke · Pull Request #352 · inbo/tutorials · GitHub

Description

An article on variogram-based spatial analysis.

The content of the tutorial is there; however, I am still testing deployment (qmd -> hugo, see below).

Related Issue

(none)

Task list

My tutorial or article is placed in a subfolder of tutorials/content
The filename of my tutorial or article is index.md. In case of an Rmarkdown tutorial I have knitted my index.Rmd to index.md (both files are pushed to the repo).
I have included tags in the YAML header (see the tags listed in the tutorials website side bar for tags that have been used before)
I have added categories to the YAML header and my category tags are from the list of category tags

Note on rmd: I went ahead to attempt building this tutorial on quarto qmd. I rendered it as a hugo-md, because md-document did not have the correct output. Because this is still testing, I did not include the png images yet to save git overhead. I tested everything locally and finally added the figures.

Previewing the pull request

Thanks to GitHub Actions, an artifact (=zip file) of the rendered website is automatically created for each pull request.
This provides a way to preview how these updates will look on the website, useful to contributors and reviewers.

Instructions to preview the updated website

On the PR page, you can find a "details" link under "checks - On PR, build the site and ...". Go there, click on the top link in the left sidebar ("Summary"), and download the generated artifact at the bottom of the page.
Decompress it and make sure the target directory is called 'tutorials' (you may need to rename it)
From the parent directory (just above the tutorials folder you created/renamed), run python -m http.server 8887, or launch the Google Chrome Web Server app and point it at the parent directory.
Point your browser to http://localhost:8887/tutorials.
Review the updated website. As a contributor, you can push extra commits to update the PR. As a reviewer, you can accept/refuse/comment the PR.

Note: for step 3, you can use any other simple HTTP server to serve the current directory if you don't have a Python 3 environment or Google Chrome available.

I have tested the asset locally, it should compile everything well for this particular tutorial. I hope nothing else broke...
There are some notable new capabilities with Hugo:

callout boxes (rudimentary, but working; more colors would be nice)
equations (via katex)

Working via manual fiddling:

section anchors
figure captions

Thank you for you feedback!

Please wait with the review @florisvdh @ThierryO
I have found the reason why my Matérn regressions did not work well, which spawns some interesting further ideas.
I will adjust and resubmit.
Thank you, and sorry for the inconvenience in case you already started!

My additions are in.
Thank you in advance for the review!

- cf. https://stackoverflow.com/a/68506928

https://gohugo.io/content-management/mathematics/#engines

Some more implementation tricks have emerged from the watina/waterlevel analysis.
Thank you again for reviewing, if you find the time, @florisvdh @ThierryO !

FYI @ThierryO @florisvdh I updated the variogram tutorial with some insights and notes from my "spatial coupling" analysis, and from our discussions.
Most noteworthy, there are extra paragraphs on bootstrapping and the role of "scales".
Thank you for your comments and suggestions, whenever you find the time!

The fit_variogram() function in this tutorial looks interesting and might make a good addition to the inbospatial package.

The fit_variogram() function in this tutorial looks interesting and might make a good addition to the inbospatial package.

I agree - but fear that we would just re-implement what others have done well (gstat, geoR, fields, ...). The strength of this tutorial is the decomposition of the algorithm for customization and extensibility.
... besides, I now understand and fear that a generalization of the procedure is quite a challenge to maintain. Let's first wait until this run-through is evaluated, maybe ask for external feedback, and then see where we find applications.

Thank you for looking at it, @hansvancalster !

prepared for next `quarto render` quarto-dev/quarto-cli#12272

hacky, but does export as-is: - equations (setting/feature) - yaml header (hack) - figure captions (hack) - callout blocks (hack) - section references (hack) if no further commit follows, production test was succesful.

- equations required double backslash - section anchors require a newline

the anchor still wasn't right.

Great tutorial 👍 It nicely demonstrates the inner workings of variograms.
I only have some minor comments and suggestions.

So z is calculated as noise from a uniform distribution + the independent effects of a and b if I understood it correctly? Maybe rephrase the line a bit to make that explicit (refer to z in the text). Also, a bit strange to see noise from a uniform distribution rather than a gaussian, but OK.

The Gaussian is heavy on its mean; I was fearing (without testing) that smoothing it afterwards will leave too little (if any) differences between points. For the sake of demonstration, both will do.
I added references to the variable names.

Thanks @falkmielke for contributing this in-depth tutorial. Unfortunately I don't find the time to review the contents well, so I am happy that @hansvancalster could review your work 🙏 🙂. Also I'm confident you already integrated some earlier feedback about your variogram applications into this tutorial.

I just took a very quick view on the rendered tutorial just to taste, and it looks very good 👍 .

Hence this only brings up a very few suggestions (randomly you could say 🙂):

if possible, avoid {sp} and try to replace by using currently developed packages such as {sf}. {sp} entered maintenance mode and uses {sf} under the hood to call GDAL (before, it depended on {rgdal}, which retired in 2023 IIRC).
from the figures I take that the Matérn range is not the same as the 'range' of a semivariogram. I didn't check whether you point this out or elaborate on it, but I think that is relevant to bring up.
please use the INBO csl style file to get citations & bibliography rendered in the INBO style, by Pandoc. I'm not sure how this goes with YAML parameters for qmd, but for Rmd you can use this (see other tutorials):

link-citations: true
csl: '`r cslfile <- file.path(tempdir(), "research-institute-for-nature-and-forest.csl"); download.file("https://github.com/inbo/styles/raw/master/research-institute-for-nature-and-forest.csl", cslfile); cslfile`'

The first parameter above is needed to make citations link to the corresponding item; this isn't currently the case in the tutorial.

- cls style - removed `sp` - extra explanation Matérn range

Thank you, @florisvdh ! I have incorporated the changes you suggest. (CSL style works just as with rmd.)

This PR is ready from my side, thank you for merging!

falkmielke requested review from florisvdh and removed request for florisvdh January 6, 2025 11:03

falkmielke requested review from ThierryO and florisvdh January 6, 2025 14:00

falkmielke requested review from ThierryO and florisvdh and removed request for ThierryO and florisvdh January 7, 2025 13:41

falkmielke and others added 19 commits January 7, 2025 14:49

emd: introducing empirical mode decomposition

fc323cc

variograms: an algorithmic approach

1a28f43

variograms: ignore qmd (config.toml)

e00512c

variograms: first attempt on qmd tutorial publ

cd27c1b

variograms: attempt callout box

94f0929

variograms: undo callout-box (failed)

821cdad

variograms: attempt mathjax equations

b2f9f29

variograms: equations attempt*

2c966aa

- cf. https://stackoverflow.com/a/68506928

variograms: another equation attempt*

fe4ab2b

https://gohugo.io/content-management/mathematics/#engines

variograms: found minimo

b09a894

variograms: equations to hugo

de2621f

variograms: math double backslash

fc301c1

variograms: attempting callout boxes

156f2c7

variograms: rm toc; apply callout

41a571a

variograms: section crosslinks anchors

db99ad5

variograms: figure captions

7eb27b9

variograms: figure captions

e15be74

variograms: everything up; figure files

7406373

variograms: figcaptions; attempt quote captions

745273c

falkmielke added 6 commits January 21, 2025 09:42

variograms: figures, typo's, tried MAD

d55cd73

variograms: date adjust, new figures upload

4bc2932

variograms: bootstrapping; scales; re-read

ce86364

variograms: markdown update

e382dcd

variograms: publication date adjusted

1ae14c2

variograms: removing unfinished sentence

f6c0b8d

variograms: section header duplicate

995180e

falkmielke requested a review from hansvancalster March 11, 2025 14:34

falkmielke added 8 commits March 13, 2025 08:21

variograms: quarto maths syntax

1b01951

prepared for next `quarto render` quarto-dev/quarto-cli#12272

variograms: set.seed, embed-resources

a805636

variograms: quarto hugo-md export optimization*

645e483

hacky, but does export as-is: - equations (setting/feature) - yaml header (hack) - figure captions (hack) - callout blocks (hack) - section references (hack) if no further commit follows, production test was succesful.

variograms: quarto->md corrections*

44f785b

- equations required double backslash - section anchors require a newline

variograms: quarto->md corrections*

1ec67d3

the anchor still wasn't right.

variograms: quarto->md anchors III; parallel RNG

b828639

variograms: quarto->md figcaption backticks

2e91c40

variograms: quarto -> hugo-md finally good.

bb86ff9

hansvancalster approved these changes Mar 25, 2025

View reviewed changes

falkmielke added 3 commits March 25, 2025 16:48

variograms: review comments by @hansvancalster

c12210c

variograms: github hidden review comments

801778c

variograms: correct explanation of VAR

923a09b

florisvdh reviewed Mar 28, 2025

View reviewed changes

variograms: suggestions @florisvdh *

3abfab2

- cls style - removed `sp` - extra explanation Matérn range

florisvdh merged commit bbbb231 into master Mar 31, 2025

florisvdh deleted the variograms branch March 31, 2025 10:08

falkmielke mentioned this pull request Jan 13, 2026

++tutorial on Empirical Mode Decomposition #370

Open

8 tasks

+              knitr::kable(head(data, 3))
+              ```
+              There no noise applied to those covariates, moderate noise on the raw data, so they should be recover-able by a statistical model.

Variograms by falkmielke · Pull Request #352 · inbo/tutorials · GitHub

Uh oh!

Variograms - #352

Variograms#352florisvdh merged 53 commits intomasterinbo/tutorials:masterfrom variogramsinbo/tutorials:variogramsCopy head branch name to clipboard

Conversation

falkmielke commented Jan 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issue

Task list

Previewing the pull request

Instructions to preview the updated website

Uh oh!

falkmielke commented Jan 6, 2025

Uh oh!

falkmielke commented Jan 7, 2025

Uh oh!

falkmielke commented Jan 7, 2025

Uh oh!

falkmielke commented Jan 20, 2025

Uh oh!

falkmielke commented Mar 11, 2025

Uh oh!

hansvancalster commented Mar 11, 2025

Uh oh!

falkmielke commented Mar 11, 2025

Uh oh!

hansvancalster left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

hansvancalster Mar 25, 2025

Choose a reason for hiding this comment

Uh oh!

falkmielke Mar 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

florisvdh left a comment

Choose a reason for hiding this comment

Uh oh!

falkmielke commented Mar 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Variograms#352
florisvdh merged 53 commits into
masterfrom
variograms

falkmielke commented Jan 6, 2025 •
edited
Loading

falkmielke Mar 25, 2025 •
edited
Loading

falkmielke commented Mar 28, 2025 •
edited
Loading