FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

bpo-40280: Add build target for Emscripten/node.js by emmatyping · Pull Request #30495 · python/cpython · GitHub

/ cpython Public

bpo-40280: Add build target for Emscripten/node.js #30495

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
emmatyping wants to merge 11 commits into from
Conversation 16 Commits 11 Checks 0 Files changed
Closed

bpo-40280: Add build target for Emscripten/node.js #30495

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
717c09b
bpo-40280: Add build target for Emscripten/node.js
emmatyping Jan 9, 2022
5492cf8
Merge branch 'main' of github.com:python/cpython into nodejs
emmatyping Jan 9, 2022
a862bce
Update Makefile.pre.in
emmatyping Jan 10, 2022
6d88ce6
Change target name to python-rawfs.js
emmatyping Jan 10, 2022
3ee5687
Remove special casing for test files
emmatyping Jan 10, 2022
aadbd05
Merge branch 'nodejs' of github.com:ethanhs/cpython into nodejs
emmatyping Jan 10, 2022
ee99490
Fix threading for nodejs
emmatyping Jan 11, 2022
3a0dd1c
Update Makefile.pre.in
emmatyping Jan 11, 2022
faf52a4
Update Makefile.pre.in
emmatyping Jan 11, 2022
bdcaa48
Update Makefile.pre.in
emmatyping Jan 11, 2022
5284265
Update Modules/socketmodule.c
emmatyping Jan 11, 2022
File filter

Filter by extension

Filter by extension .ac  (1) .c  (1) .in  (1) .site-wasm32-emscripten  (1) No extension  (1) All 5 file types selected
Viewed files
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Unified
Split
Hide whitespace
Diff view
Unified
Split
Hide whitespace
14 changes: 12 additions & 2 deletions Makefile.pre.in
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ CC= @CC@
CXX= @CXX@
MAINCC= @MAINCC@
LINKCC= @LINKCC@
LINKCC_BUILDPYTHON = @LINKCC_BUILDPYTHON@

emmatyping Jan 11, 2022

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

This is the least intrusive way I could think of to customize the link step only for building the python.wasm executable...

If you know of a better way, please do tell :)

tiran Jan 11, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Meh! The ugly hacks are piling up. Maybe it's easier to not use ac_cv_pthread and to add -pthread to CFLAGS or CFLAGS_NODIST after all?

emmatyping Jan 11, 2022

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Unfortunately then it would be passed to the $(BUILDPYTHON) target, and standalone wasm doesn't support -pthread
AR= @AR@
READELF= @READELF@
SOABI= @SOABI@
Expand Down Expand Up @@ -717,7 +718,7 @@ clinic: check-clean-src $(srcdir)/Modules/_blake2/blake2s_impl.c

# Build the interpreter
$(BUILDPYTHON): Programs/python.o $(LIBRARY_DEPS)
$(LINKCC) $(PY_CORE_LDFLAGS) $(LINKFORSHARED) -o $@ Programs/python.o $(BLDLIBRARY) $(LIBS) $(MODLIBS) $(SYSLIBS)
$(LINKCC_BUILDPYTHON) $(PY_CORE_LDFLAGS) $(LINKFORSHARED) -o $@ Programs/python.o $(BLDLIBRARY) $(LIBS) $(MODLIBS) $(SYSLIBS)

platform: $(BUILDPYTHON) pybuilddir.txt
$(RUNSHARED) $(PYTHON_FOR_BUILD) -c 'import sys ; from sysconfig import get_platform ; print("%s-%d.%d" % (get_platform(), *sys.version_info[:2]))' >platform
Expand Down Expand Up @@ -833,6 +834,7 @@ $(DLLLIBRARY) libpython$(LDVERSION).dll.a: $(LIBRARY_OBJS)
# wasm32-emscripten build
# wasm assets directory is relative to current build dir, e.g. "./usr/local".
# --preload-file turns a relative asset path into an absolute path.
# With node.js, we use the rawfs which is much easier for tests
WASM_ASSETS_DIR=".$(prefix)"
WASM_STDLIB="$(WASM_ASSETS_DIR)/local/lib/python$(VERSION)/os.py"

Expand All @@ -844,7 +846,15 @@ $(WASM_STDLIB): $(srcdir)/Lib/*.py $(srcdir)/Lib/*/*.py \
python.html: Programs/python.o $(LIBRARY_DEPS) $(WASM_STDLIB)
$(LINKCC) $(PY_CORE_LDFLAGS) $(LINKFORSHARED) -o $@ Programs/python.o \
$(BLDLIBRARY) $(LIBS) $(MODLIBS) $(SYSLIBS) \
-s ASSERTIONS=1 --preload-file $(WASM_ASSETS_DIR)
-s ASSERTIONS=1 -s ALLOW_MEMORY_GROWTH=1 -s USE_PTHREADS \
-s PROXY_TO_PTHREAD --preload-file $(WASM_ASSETS_DIR)

python-rawfs.js: Programs/python.o $(LIBRARY_DEPS) pybuilddir.txt
$(LINKCC) $(PY_CORE_LDFLAGS) $(LINKFORSHARED) -o $@ Programs/python.o \
$(BLDLIBRARY) $(LIBS) $(MODLIBS) $(SYSLIBS) \
-s ASSERTIONS=1 -s ALLOW_MEMORY_GROWTH=1 \
-s NODERAWFS=1 -s EXIT_RUNTIME=1 -s USE_PTHREADS \
-s PROXY_TO_PTHREAD

##########################################################################
# Build static libmpdec.a
Expand Down
2 changes: 1 addition & 1 deletion Modules/socketmodule.c
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -7926,7 +7926,7 @@ PyInit__socket(void)
#ifdef IPPROTO_VRRP
PyModule_AddIntMacro(m, IPPROTO_VRRP);
#endif
#ifdef IPPROTO_SCTP
#if defined(IPPROTO_SCTP) && !defined(__EMSCRIPTEN__)

tiran Jan 11, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I found a better way to deal with the issue. I guess you did the same mistake as I and added the skipif call after the check for AIX? That doesn't work because @requireSocket("AF_INET", "SOCK_STREAM", "IPPROTO_SCTP") tries to create an IPPROTO_SCTP socket and decorators are executed in reverse order!

This works:

--- a/Lib/test/test_socket.py
+++ b/Lib/test/test_socket.py
@@ -43,6 +43,7 @@
 
 VSOCKPORT = 1234
 AIX = platform.system() == "AIX"
+EMSCRIPTEN = sys.platform == "Emscripten"
 
 try:
     import _socket
...

 @requireAttrs(socket.socket, "sendmsg")
-@unittest.skipIf(AIX, "IPPROTO_SCTP: [Errno 62] Protocol not supported on AIX")
 @requireSocket("AF_INET", "SOCK_STREAM", "IPPROTO_SCTP")
+@unittest.skipIf(AIX, "IPPROTO_SCTP: [Errno 62] Protocol not supported on AIX")
+@unittest.skipIf(EMSCRIPTEN, "IPPROTO_SCTP: aborts on Emscripten")
 class SendmsgSCTPStreamTest(SendmsgStreamTests, SendrecvmsgSCTPStreamTestBase):

emmatyping Jan 11, 2022

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I see you've opened #30538 so I'll remove this change.

tiran Jan 11, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

#30538 does not work ... it's still running the decorator and crashing node.js
PyModule_AddIntMacro(m, IPPROTO_SCTP);
#endif
#ifdef IPPROTO_BIP
Expand Down
7 changes: 7 additions & 0 deletions Tools/wasm/config.site-wasm32-emscripten
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,13 @@
# Partly based on pyodide's pyconfig.undefs.h file.
#

ac_exeext=.wasm

# Emscripten will build code that includes pthreads without the flag, but fail to run...
ac_cv_pthread_is_default=no
ac_cv_kthread=no
ac_cv_pthread=yes

# cannot be detected in cross builds
ac_cv_buggy_getaddrinfo=no

Expand Down
40 changes: 32 additions & 8 deletions configure
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -878,6 +878,7 @@ AR
GNULD
EXPORTSFROM
EXPORTSYMS
LINKCC_BUILDPYTHON
LINKCC
LDVERSION
RUNSHARED
Expand Down Expand Up @@ -6326,6 +6327,25 @@ fi
{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $LINKCC" >&5
$as_echo "$LINKCC" >&6; }


{ $as_echo "$as_me:${as_lineno-$LINENO}: checking LINKCC_BUILDPYTHON" >&5
$as_echo_n "checking LINKCC_BUILDPYTHON... " >&6; }
if test -z "$LINKCC_BUILDPYTHON"
then
# the .wasm executable doesn't support building with pthreads,
# so we hack around the symbol errors as they will be provided by
# Emscripten
case $ac_sys_system in #(
Emscripten) :
LINKCC_BUILDPYTHON="emcc -s ERROR_ON_UNDEFINED_SYMBOLS=0" ;; #(
*) :
LINKCC_BUILDPYTHON='$(LINKCC)'
;;
esac
fi
{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $LINKCC_BUILDPYTHON" >&5
$as_echo "$LINKCC_BUILDPYTHON" >&6; }

# EXPORTSYMS holds the list of exported symbols for AIX.
# EXPORTSFROM holds the module name exporting symbols on AIX.
EXPORTSYMS=
Expand Down Expand Up @@ -8276,7 +8296,6 @@ fi
{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_pthread_is_default" >&5
$as_echo "$ac_cv_pthread_is_default" >&6; }


if test $ac_cv_pthread_is_default = yes
then
ac_cv_kpthread=no
Expand Down Expand Up @@ -8451,16 +8470,21 @@ fi

if test $ac_cv_cxx_thread = yes
then
echo 'void foo();int main(){foo();}void foo(){}' > conftest.$ac_ext
$CXX -c conftest.$ac_ext 2>&5
if $CXX -o conftest$ac_exeext conftest.$ac_objext 2>&5 \
&& test -s conftest$ac_exeext && ./conftest$ac_exeext
then
if test "$cross_compiling" = yes; then
# assume that it works
ac_cv_cxx_thread=yes
else
ac_cv_cxx_thread=no
echo 'void foo();int main(){foo();}void foo(){}' > conftest.$ac_ext
$CXX -c conftest.$ac_ext 2>&5
if $CXX -o conftest$ac_exeext conftest.$ac_objext 2>&5 \
&& test -s conftest$ac_exeext && ./conftest$ac_exeext
then
ac_cv_cxx_thread=yes
else
ac_cv_cxx_thread=no
fi
rm -fr conftest*
fi
rm -fr conftest*
fi
{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_cxx_thread" >&5
$as_echo "$ac_cv_cxx_thread" >&6; }
Expand Down
34 changes: 26 additions & 8 deletions configure.ac
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -1162,6 +1162,20 @@ then
fi
AC_MSG_RESULT($LINKCC)

AC_SUBST(LINKCC_BUILDPYTHON)
AC_MSG_CHECKING(LINKCC_BUILDPYTHON)
if test -z "$LINKCC_BUILDPYTHON"
then
# the .wasm executable doesn't support building with pthreads,
# so we hack around the symbol errors as they will be provided by
# Emscripten
AS_CASE([$ac_sys_system],
[Emscripten], [LINKCC_BUILDPYTHON="emcc -s ERROR_ON_UNDEFINED_SYMBOLS=0"],
[LINKCC_BUILDPYTHON='$(LINKCC)']
)
fi
AC_MSG_RESULT($LINKCC_BUILDPYTHON)

# EXPORTSYMS holds the list of exported symbols for AIX.
# EXPORTSFROM holds the module name exporting symbols on AIX.
EXPORTSYMS=
Expand Down Expand Up @@ -2125,7 +2139,6 @@ int main(){
],[ac_cv_pthread_is_default=no],[ac_cv_pthread_is_default=no])
])


if test $ac_cv_pthread_is_default = yes
then
ac_cv_kpthread=no
Expand Down Expand Up @@ -2233,16 +2246,21 @@ fi

if test $ac_cv_cxx_thread = yes
then
echo 'void foo();int main(){foo();}void foo(){}' > conftest.$ac_ext
$CXX -c conftest.$ac_ext 2>&5
if $CXX -o conftest$ac_exeext conftest.$ac_objext 2>&5 \
&& test -s conftest$ac_exeext && ./conftest$ac_exeext
then
if test "$cross_compiling" = yes; then
# assume that it works
ac_cv_cxx_thread=yes
else
ac_cv_cxx_thread=no
echo 'void foo();int main(){foo();}void foo(){}' > conftest.$ac_ext
$CXX -c conftest.$ac_ext 2>&5
if $CXX -o conftest$ac_exeext conftest.$ac_objext 2>&5 \
&& test -s conftest$ac_exeext && ./conftest$ac_exeext
then
ac_cv_cxx_thread=yes
else
ac_cv_cxx_thread=no
fi
rm -fr conftest*
fi
rm -fr conftest*
fi
AC_MSG_RESULT($ac_cv_cxx_thread)
fi
Expand Down
Back | FazBrowse Home | New Git URL