FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

Initial version of docker reproducibility · ByteHamster/GpuRecSplit@d133f19 · GitHub

  1. ByteHamster
  2. GpuRecSplit

Repository navigation

Commit d133f19

Browse files
ByteHamster
committed
Initial version of docker reproducibility
1 parent 31dfcbb commit d133f19

6 files changed

Lines changed: 421 additions & 0 deletions

File tree

‎Dockerfile‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
FROM ubuntu:22.04
2+
3+
RUN apt-get update && apt-get -y upgrade
4+
RUN apt-get install --assume-yes --no-install-recommends ca-certificates build-essential cmake git
5+
RUN apt-get install --assume-yes --no-install-recommends libboost-regex-dev libsqlite3-dev
6+
RUN apt-get install --assume-yes --no-install-recommends texlive-latex-extra texlive-fonts-recommended texlive-latex-recommended texlive-fonts-extra
7+
RUN apt-get install --assume-yes --no-install-recommends libtbb-dev libxxhash-dev
8+
9+
# Build sqlplot-tools
10+
RUN git clone https://github.com/bingmann/sqlplot-tools.git /opt/sqlplot-tools
11+
RUN mkdir /opt/sqlplot-tools/build
12+
WORKDIR /opt/sqlplot-tools/build
13+
RUN cmake -DCMAKE_BUILD_TYPE=Release -DWITH_POSTGRESQL=OFF -DWITH_MYSQL=OFF ..
14+
RUN cmake --build . -j 8
15+
16+
# Build GpuRecSplit
17+
COPY . /opt/gpurecsplit
18+
RUN mkdir /opt/gpurecsplit/build
19+
WORKDIR /opt/gpurecsplit/build
20+
RUN cmake -DCMAKE_BUILD_TYPE=Release ..
21+
RUN cmake --build . -j 8
22+
23+
# Actual benchmark
24+
CMD bash /opt/dockerVolume/figure-1.sh

‎README.md‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,49 @@ add_subdirectory(path/to/GpuRecSplit)
1616
target_link_libraries(YourTarget PRIVATE RecSplit SIMDRecSplit GPURecSplit ShockHash) # or a subset of the targets
1717
```
1818

19+
### Reproducing Experiments
20+
21+
This repository contains the source code and our reproducibility artifacts for the benchmarks specific to GpuRecSplit/SimdRecSplit.
22+
Benchmarks that compare SimdRecSplit to competitors are available in a different repository: https://github.com/ByteHamster/MPHF-Experiments
23+
24+
We provide an easy to use Docker image to quickly reproduce our results.
25+
Alternatively, you can look at the `Dockerfile` to see all libraries, tools, and commands necessary to compile.
26+
27+
#### Building the Docker Image
28+
29+
Run the following command to build the Docker image.
30+
Building the image takes about 5 minutes, as some packages (including LaTeX for the plots) have to be installed.
31+
32+
```bash
33+
docker build -t gpurecsplit --no-cache .
34+
```
35+
36+
Some compiler warnings (red) are expected when building dependencies and will not prevent building the image or running the experiments.
37+
Please ignore them!
38+
39+
#### Running the Experiments
40+
Due to the long total running time of all experiments in our paper, we provide run scripts for a slightly simplified version of the experiments.
41+
They run fewer iterations and output fewer data points.
42+
43+
You can modify the benchmarks scripts in `scripts/dockerVolume` if you want to change the number of runs or data points.
44+
This does not require the Docker image to recompile.
45+
Different experiments can be started by using the following command:
46+
47+
```bash
48+
docker run --interactive --tty -v "$(pwd)/scripts/dockerVolume:/opt/dockerVolume" gpurecsplit /opt/dockerVolume/figure-5.sh
49+
```
50+
51+
The number also refers to the figure in the paper.
52+
53+
| In paper | Launch command | Estimated runtime |
54+
| :-------------- | :---------------------------- | :----------------- |
55+
| Figure 5 | /opt/dockerVolume/figure-5.sh | 10 minutes |
56+
| Figure 7 | /opt/dockerVolume/figure-7.sh | 10 minutes |
57+
| Table 1 | /opt/dockerVolume/table-1.sh | 10 minutes |
58+
59+
The resulting plots can be found in `scripts/dockerVolume` and are called `figure-<number>.pdf`.
60+
More experiments comparing GpuRecSplit with competitors can be found in a different repository: https://github.com/ByteHamster/MPHF-Experiments
61+
1962
### Licensing
2063
GpuRecSplit is licensed exactly like `libstdc++` (GPLv3 + GCC Runtime Library Exception), which essentially means you can use it everywhere, exactly like `libstdc++`.
2164
You can find details in the [COPYING](/COPYING) and [COPYING.RUNTIME](/COPYING.RUNTIME) files.

‎scripts/.gitignore‎

Lines changed: 280 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,280 @@
1+
dockerVolume/*.pdf
2+
dockerVolume/*.txt
3+
4+
## Core latex/pdflatex auxiliary files:
5+
*.vtc
6+
*.aux
7+
*.lof
8+
*.log
9+
*.lot
10+
*.fls
11+
*.out
12+
*.toc
13+
*.fmt
14+
*.fot
15+
*.cb
16+
*.cb2
17+
.*.lb
18+
19+
## Intermediate documents:
20+
*.dvi
21+
*.xdv
22+
*-converted-to.*
23+
# these rules might exclude image files for figures etc.
24+
# *.ps
25+
# *.eps
26+
*.pdf
27+
28+
## Generated if empty string is given at "Please type another file name for output:"
29+
.pdf
30+
31+
## Bibliography auxiliary files (bibtex/biblatex/biber):
32+
*.bbl
33+
*.bcf
34+
*.blg
35+
*-blx.aux
36+
*-blx.bib
37+
*.run.xml
38+
39+
## Build tool auxiliary files:
40+
*.fdb_latexmk
41+
*.synctex
42+
*.synctex(busy)
43+
*.synctex.gz
44+
*.synctex.gz(busy)
45+
*.pdfsync
46+
47+
## Build tool directories for auxiliary files
48+
# latexrun
49+
latex.out/
50+
51+
## Auxiliary and intermediate files from other packages:
52+
# algorithms
53+
*.alg
54+
*.loa
55+
56+
# achemso
57+
acs-*.bib
58+
59+
# amsthm
60+
*.thm
61+
62+
# beamer
63+
*.nav
64+
*.pre
65+
*.snm
66+
*.vrb
67+
68+
# changes
69+
*.soc
70+
71+
# comment
72+
*.cut
73+
74+
# cprotect
75+
*.cpt
76+
77+
# elsarticle (documentclass of Elsevier journals)
78+
*.spl
79+
80+
# endnotes
81+
*.ent
82+
83+
# fixme
84+
*.lox
85+
86+
# feynmf/feynmp
87+
*.mf
88+
*.mp
89+
*.t[1-9]
90+
*.t[1-9][0-9]
91+
*.tfm
92+
93+
#(r)(e)ledmac/(r)(e)ledpar
94+
*.end
95+
*.?end
96+
*.[1-9]
97+
*.[1-9][0-9]
98+
*.[1-9][0-9][0-9]
99+
*.[1-9]R
100+
*.[1-9][0-9]R
101+
*.[1-9][0-9][0-9]R
102+
*.eledsec[1-9]
103+
*.eledsec[1-9]R
104+
*.eledsec[1-9][0-9]
105+
*.eledsec[1-9][0-9]R
106+
*.eledsec[1-9][0-9][0-9]
107+
*.eledsec[1-9][0-9][0-9]R
108+
109+
# glossaries
110+
*.acn
111+
*.acr
112+
*.glg
113+
*.glo
114+
*.gls
115+
*.glsdefs
116+
*.lzo
117+
*.lzs
118+
119+
# uncomment this for glossaries-extra (will ignore makeindex's style files!)
120+
# *.ist
121+
122+
# gnuplottex
123+
*-gnuplottex-*
124+
125+
# gregoriotex
126+
*.gaux
127+
*.gtex
128+
129+
# htlatex
130+
*.4ct
131+
*.4tc
132+
*.idv
133+
*.lg
134+
*.trc
135+
*.xref
136+
137+
# hyperref
138+
*.brf
139+
140+
# knitr
141+
*-concordance.tex
142+
# TODO Comment the next line if you want to keep your tikz graphics files
143+
*.tikz
144+
*-tikzDictionary
145+
146+
# listings
147+
*.lol
148+
149+
# luatexja-ruby
150+
*.ltjruby
151+
152+
# makeidx
153+
*.idx
154+
*.ilg
155+
*.ind
156+
157+
# minitoc
158+
*.maf
159+
*.mlf
160+
*.mlt
161+
*.mtc[0-9]*
162+
*.slf[0-9]*
163+
*.slt[0-9]*
164+
*.stc[0-9]*
165+
166+
# minted
167+
_minted*
168+
*.pyg
169+
170+
# morewrites
171+
*.mw
172+
173+
# nomencl
174+
*.nlg
175+
*.nlo
176+
*.nls
177+
178+
# pax
179+
*.pax
180+
181+
# pdfpcnotes
182+
*.pdfpc
183+
184+
# sagetex
185+
*.sagetex.sage
186+
*.sagetex.py
187+
*.sagetex.scmd
188+
189+
# scrwfile
190+
*.wrt
191+
192+
# sympy
193+
*.sout
194+
*.sympy
195+
sympy-plots-for-*.tex/
196+
197+
# pdfcomment
198+
*.upa
199+
*.upb
200+
201+
# pythontex
202+
*.pytxcode
203+
pythontex-files-*/
204+
205+
# tcolorbox
206+
*.listing
207+
208+
# thmtools
209+
*.loe
210+
211+
# TikZ & PGF
212+
*.dpth
213+
*.md5
214+
*.auxlock
215+
216+
# todonotes
217+
*.tdo
218+
219+
# vhistory
220+
*.hst
221+
*.ver
222+
223+
# easy-todo
224+
*.lod
225+
226+
# xcolor
227+
*.xcp
228+
229+
# xmpincl
230+
*.xmpi
231+
232+
# xindy
233+
*.xdy
234+
235+
# xypic precompiled matrices and outlines
236+
*.xyc
237+
*.xyd
238+
239+
# endfloat
240+
*.ttt
241+
*.fff
242+
243+
# Latexian
244+
TSWLatexianTemp*
245+
246+
## Editors:
247+
# WinEdt
248+
*.bak
249+
*.sav
250+
251+
# Texpad
252+
.texpadtmp
253+
254+
# LyX
255+
*.lyx~
256+
257+
# Kile
258+
*.backup
259+
260+
# gummi
261+
.*.swp
262+
263+
# KBibTeX
264+
*~[0-9]*
265+
266+
# TeXnicCenter
267+
*.tps
268+
269+
# auto folder when using emacs and auctex
270+
./auto/*
271+
*.el
272+
273+
# expex forward references with \gathertags
274+
*-tags.tex
275+
276+
# standalone packages
277+
*.sta
278+
279+
# Makeindex log files
280+
*.lpz

‎scripts/dockerVolume/figure-5.sh‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
#!/bin/bash
2+
hostname
3+
strings recsplit_construction | grep " -m" | tee figure-5.txt
4+
5+
# shellcheck disable=SC2086
6+
for leafSize in $(seq 2 1 16); do
7+
for bucketSize in 100 500 1000 2000; do
8+
params="--numObjects 1M --numQueries 0 --leafSize $leafSize --bucketSize $bucketSize"
9+
./recsplit_construction $params --leafMethod bruteforce | tee --append figure-5.txt
10+
./recsplit_construction $params --leafMethod rotations | tee --append figure-5.txt
11+
done
12+
done
13+
14+
for leafSize in $(seq 2 2 34); do
15+
for bucketSize in 100 500 1000 2000; do
16+
params="--numObjects 1M --numQueries 0 --leafSize $leafSize --bucketSize $bucketSize"
17+
./recsplit_construction $params --leafMethod cuckoo | tee --append figure-5.txt
18+
done
19+
done
20+
21+
# Build plot
22+
cp figure-5.txt /opt/dockerVolume
23+
cd /opt/dockerVolume
24+
/opt/sqlplot-tools/build/src/sqlplot-tools figure-5.tex
25+
pdflatex figure-5.tex
26+
pdflatex figure-5.tex
27+

0 commit comments

Comments
 (0)
Back | FazBrowse Home | New Git URL