FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

* Splitgraph core library build process: use pyenv to build python 3.… · splitgraph/sgr@ebd449c · GitHub

Repository navigation

Commit ebd449c

Browse files
committed
* Splitgraph core library build process: use pyenv to build python 3.6.8 (3.5 unsupported by SG + the PPA with the binary doesn't seem to support whatever Ubuntu is running in the pg11 container)
* Build cstore_fdw in the engine: objects are expected to be stored as flat files in /var/lib/splitgraph/objects
1 parent 09d3401 commit ebd449c

5 files changed

Lines changed: 32 additions & 8 deletions

File tree

‎engine/Dockerfile‎

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,9 @@ RUN apt-get update -qq && \
1515
libtool \
1616
postgresql-server-dev-11 \
1717
libmongoc-1.0.0 \
18-
libmongoc-dev
18+
libmongoc-dev \
19+
protobuf-c-compiler \
20+
libprotobuf-c0-dev
1921

2022
RUN wget https://dev.mysql.com/get/mysql-apt-config_0.8.12-1_all.deb
2123
RUN echo mysql-apt-config mysql-apt-config/select-server select mysql-8.0 | debconf-set-selections && \
@@ -26,6 +28,8 @@ RUN apt-get update -qq && apt-get install -y libmysqlclient-dev && rm -rf /var/l
2628
RUN mkdir -p /build_scripts
2729
COPY build_scripts /build_scripts/
2830

31+
# cstore_fdw
32+
RUN ./build_scripts/fdws/cstore_fdw/build_cstore_fdw.sh
2933
# Mongo FDW
3034
RUN ./build_scripts/fdws/mongo_fdw/build_mongo_fdw.sh
3135
# MySQL FDW
@@ -42,10 +46,18 @@ COPY etc /etc/
4246

4347
# Splitgraph core libraries (required for the layered querying FDW)
4448
# Git submodule
49+
50+
RUN apt-get update -qq && apt-get install -y zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev
51+
52+
# splitgraph requires python 3.6
53+
RUN curl -L https://github.com/pyenv/pyenv-installer/raw/master/bin/pyenv-installer | bash
54+
4555
COPY splitgraph /splitgraph
4656
RUN ./build_scripts/build_splitgraph.sh
4757

4858
# Splitgraph init scripts
4959
COPY init_scripts /docker-entrypoint-initdb.d/
60+
RUN mkdir -p /var/lib/splitgraph/objects
61+
RUN chmod 777 /var/lib/splitgraph/objects
5062

5163
CMD ["postgres", "-c", "config_file=/etc/postgresql/postgresql.conf"]

‎engine/build_scripts/build_splitgraph.sh‎

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,19 @@
11
#!/bin/bash
22

3+
export PATH="~/.pyenv/bin:$PATH"
4+
eval "$(pyenv init -)"
5+
eval "$(pyenv virtualenv-init -)"
6+
7+
pyenv install 3.6.8
8+
pyenv virtualenv 3.6.8 splitgraph
9+
pyenv activate splitgraph
10+
311
# Install the Splitgraph library and the layered querying foreign data wrapper.
412
# We could use the Postgres entrypoint system instead but it's run as the non-root user
513
# and this container isn't for the sgr tool anyway.
614

715
cd /splitgraph
816

9-
ln -s /usr/bin/pip3 /usr/bin/pip
10-
11-
# Dirty but we use python 3 system-wide
12-
mv /usr/bin/python /usr/bin/python2
13-
ln -s /usr/bin/python3 /usr/bin/python
14-
1517
curl -sSL https://raw.githubusercontent.com/sdispater/poetry/master/get-poetry.py | python
1618
# Install globally (otherwise we'll need to find a way to get Multicorn to see the venv)
1719
source $HOME/.poetry/env
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
#!/bin/bash
2+
3+
git clone https://github.com/citusdata/cstore_fdw.git
4+
cd cstore_fdw
5+
6+
make
7+
make install

‎engine/etc/postgresql/postgresql.conf‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,3 +4,4 @@ listen_addresses = '*'
44
port = 5432
55

66
hba_file = '/etc/postgresql/pg_hba.conf'
7+
shared_preload_libraries = 'cstore_fdw'
Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,6 @@
11
CREATE EXTENSION postgres_fdw;
22
CREATE EXTENSION mongo_fdw;
33
CREATE EXTENSION mysql_fdw;
4-
CREATE EXTENSION multicorn;
4+
CREATE EXTENSION multicorn;
5+
CREATE EXTENSION cstore_fdw;
6+
CREATE SERVER cstore_server FOREIGN DATA WRAPPER cstore_fdw;

0 commit comments

Comments
 (0)

Back | FazBrowse Home | New Git URL