* mksnapshot is built as host binary, but then gets executed under
qemu-arm, because for 32bit target we need to run 32bit mksnapshot
even when building on x86-64 host
* unfortunately since upgrade to qemu-8 in:
https://git.openembedded.org/openembedded-core/commit/?id=fe8125565af07b73f9b29db2188ecb6e884bcc70
the mksnapshot fails to execute with:
LD_LIBRARY_PATH=/OE/build/oe-core/tmp-glibc/work/cortexa8t2hf-neon-oe-linux-gnueabi/nodejs/18.16.0-r0/node-v18.16.0/out/Release/lib.host:/OE/build/oe-core/tmp-glibc/work/cortexa8t2hf-neon-oe-linux-gnueabi/nodejs/18.16.0-r0/node-v18.16.0/out/Release/lib.target:$LD_LIBRARY_PATH; export LD_LIBRARY_PATH; cd ../tools/v8_gypfiles; mkdir -p /OE/build/oe-core/tmp-glibc/work/cortexa8t2hf-neon-oe-linux-gnueabi/nodejs/18.16.0-r0/node-v18.16.0/out/Release/obj.target/v8_snapshot/geni; "/OE/build/oe-core/tmp-glibc/work/cortexa8t2hf-neon-oe-linux-gnueabi/nodejs/18.16.0-r0/node-v18.16.0/out/Release/v8-qemu-wrapper.sh" "/OE/build/oe-core/tmp-glibc/work/cortexa8t2hf-neon-oe-linux-gnueabi/nodejs/18.16.0-r0/node-v18.16.0/out/Release/mksnapshot" --turbo_instruction_scheduling "--target_os=linux" "--target_arch=arm" --startup_src "/OE/build/oe-core/tmp-glibc/work/cortexa8t2hf-neon-oe-linux-gnueabi/nodejs/18.16.0-r0/node-v18.16.0/out/Release/obj.target/v8_snapshot/geni/snapshot.cc" --embedded_variant Default --embedded_src "/OE/build/oe-core/tmp-glibc/work/cortexa8t2hf-neon-oe-linux-gnueabi/nodejs/18.16.0-r0/node-v18.16.0/out/Release/obj.target/v8_snapshot/geni/embedded.S" --no-native-code-counters
#
# Fatal process OOM in Failed to reserve virtual memory for CodeRange
#
/OE/build/oe-core/tmp-glibc/work/cortexa8t2hf-neon-oe-linux-gnueabi/nodejs/18.16.0-r0/node-v18.16.0/out/Release/v8-qemu-wrapper.sh: line 7: 669880 Trace/breakpoint trap (core dumped) PSEUDO_UNLOAD=1 qemu-arm -r 5.15 -L /OE/build/oe-core/tmp-glibc/work/cortexa8t2hf-neon-oe-linux-gnueabi/nodejs/18.16.0-r0/recipe-sysroot -E LD_LIBRARY_PATH=/OE/build/oe-core/tmp-glibc/work/cortexa8t2hf-neon-oe-linux-gnueabi/nodejs/18.16.0-r0/recipe-sysroot/usr/lib:/OE/build/oe-core/tmp-glibc/work/cortexa8t2hf-neon-oe-linux-gnueabi/nodejs/18.16.0-r0/recipe-sysroot/lib "$@"
It was failing only in some of my builds first, in the end
I was able to narrow the difference between the build where
it fails and where it doesn't fail to
require conf/distro/include/security_flags.inc
then
GCCPIE = "--enable-default-pie"
and lastly this PIE enabled just for nodejs to work around
this issue without downgrading qemu-native.
* using -strace with qemu doesn't help much, just confirms that
the OOM error message is really caused by failed mmap2:
--- SIGPROF {si_signo=SIGPROF, si_code=SI_TKILL, si_pid=2908258, si_uid=1026} ---
clock_gettime64(CLOCK_MONOTONIC,0x407ef640) = 0 ({tv_sec=20187,tv_nsec=678404286})
rt_sigreturn(0x407efb70) = -1 errno=513 (Successful exit from sigreturn)
write(3,0x141e538,28) = 28
write(3,0x141e538,28) = 28
write(3,0x141e538,32) = 32
mmap2(0x0097a000,33554432,PROT_NONE,MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE,-1,0) = -1 errno=12 (Cannot allocate memory)
mmap2(0x0097a000,33554432,PROT_NONE,MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE,-1,0) = -1 errno=12 (Cannot allocate memory)
* diffoscope on this 1.7G binary didn't really help as it got killed by OOMK after a while
Out of memory: Killed process 1136999 (diffoscope) total-vm:404300108kB, anon-rss:122594688kB, file-rss:512kB, shmem-rss:0kB, UID:1026 pgtables:764240kB oom_score_adj:0
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
There is new patch-status QA check in oe-core:
https://git.openembedded.org/openembedded-core/commit/?id=76a685bfcf927593eac67157762a53259089ea8a
This is temporary work around just to hide _many_ warnings from
optional patch-status (if you add it to WARN_QA).
This just added
Upstream-Status: Pending
everywhere without actually investigating what's the proper status.
This is just to hide current QA warnings and to catch new .patch files being
added without Upstream-Status, but the number of Pending patches is now terrible:
5 (26%) meta-xfce
6 (50%) meta-perl
15 (42%) meta-webserver
21 (36%) meta-gnome
25 (57%) meta-filesystems
26 (43%) meta-initramfs
45 (45%) meta-python
47 (55%) meta-multimedia
312 (63%) meta-networking
756 (61%) meta-oe
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
The nodejs package contains a symbolic link `/usr/bin/corepack` to a
file from the nodejs-npm package.
```shell
build/tmp/work/core2-64-poky-linux/nodejs/18.14.2-r0/packages-split/nodejs/usr/bin$ ls -l
total 31932
lrwxrwxrwx 1 jan jan 45 Feb 21 05:08 corepack -> ../lib/node_modules/corepack/dist/corepack.js
-rwxr-xr-x 2 jan jan 32694304 Feb 21 05:08 node
build/tmp/work/core2-64-poky-linux/nodejs/18.14.2-r0/packages-split/nodejs-npm/usr/lib/node_modules/corepack/dist$ ls -l
total 2428
-rwxr-xr-x 2 jan jan 614216 Feb 21 05:08 corepack.js
```
Due to the link, the build system detects a dependency from package
`nodejs` to package `nodejs-npm`.
Since the `nodejs-npm` package depends on plenty more packages the link
should be moved to the `nodejs-npm` package.
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Some older distribution e.g. alma8 has a version of file utility which
is version 5.33 but more importantly missing this patch [1] from version
5.40 which causes package failures in wasm files e.g.
ERROR: nodejs-18.14.2-r0 do_package: Fatal errors occurred in subprocesses:
Command '['file', '-b', '/home/pokybuild/yocto-worker/meta-oe/build/build/tmp/work/core2-64-poky-linux/nodejs/18.14.2-r0/package/usr/lib/nodejs/ptest/test/wasi/wasm/write_file.wasm']' returned non-zero exit status 1.
Subprocess output:ERROR: Bad magic format `version %#x (MVP)' (bad format char: #)
Command '['file', '-b', '/home/pokybuild/yocto-worker/meta-oe/build/build/tmp/work/core2-64-poky-linux/nodejs/18.14.2-r0/package/usr/lib/nodejs/ptest/test/wasi/wasm/symlink_loop.wasm']' returned non-zero exit status 1.
Subprocess output:ERROR: Bad magic format `version %#x (MVP)' (bad format char: #)
Command '['file', '-b', '/home/pokybuild/yocto-worker/meta-oe/build/build/tmp/work/core2-64-poky-linux/nodejs/18.14.2-r0/package/usr/lib/nodejs/ptest/test/wasi/wasm/notdir.wasm']' returned non-zero exit status 1.
Subprocess output:ERROR: Bad magic format `version %#x (MVP)' (bad format char: #)
Command '['file', '-b', '/home/pokybuild/yocto-worker/meta-oe/build/build/tmp/work/core2-64-poky-linux/nodejs/18.14.2-r0/package/usr/lib/nodejs/ptest/test/wasi/wasm/preopen_populates.wasm']' returned non-zero exit status 1.
Subprocess output:ERROR: Bad magic format `version %#x (MVP)' (bad format char: #)
Command '['file', '-b', '/home/pokybuild/yocto-worker/meta-oe/build/build/tmp/work/core2-64-poky-linux/nodejs/18.14.2-r0/package/usr/lib/nodejs/ptest/test/wasi/wasm/clock_getres.wasm']' returned non-zero exit status 1.
Subprocess output:ERROR: Bad magic format `version %#x (MVP)' (bad format char: #)
Command '['file', '-b', '/home/pokybuild/yocto-worker/meta-oe/build/build/tmp/work/core2-64-poky-linux/nodejs/18.14.2-r0/package/usr/lib/nodejs/ptest/test/wasi/wasm/main_args.wasm']' returned non-zero exit status 1.
[1] https://github.com/file/file/commit/85b7ab83257b3191a1a7ca044589a092bcef2bb3
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Target nodejs doesn't depend on nodejs-native to build anymore, so there
is no need to stage these binaries.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
The purpose of this appears to have been lost, and the build succeeds
without it.
If it is needed, then it should be added to EXTRA_OEMAKE alongside the
CC and CXX assignments.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
By not forcing the cross-compile build logic in native builds the build
is a _lot_ faster. Walltime in buildstats-diff on my machine:
nodejs-native do_compile -475.6s -41.2% 1153.4s -> 677.7s
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Instead of overriding do_unpack entirely and calling into base_do_unpack,
use a postfunc. This keeps the prune logic separated neatly.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
It's useful to know what configure is detecting, so pass --verbose.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
The expand argument defaults to True, so there's no need to specify it.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
* unfortunately all 3 have very long unclear history, I've tried to at least partially
document in commit message
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
* tools/install.py uses:
output_lib = 'libnode.' + variables.get('shlib_suffix')
action([output_prefix + output_lib], variables.get('libdir') + '/'+ output_lib)
and when libdir is absolute path (either ${D}${libdir} before:
https://git.openembedded.org/meta-openembedded/commit/?id=42c0f0120ad1caf25118d079314c988d4633a437
or just ${libdir} "/usr/lib") it doesn't respect prefix nor ${D}
and fails to write to host's directory when "shared" PACKAGECONFIG
is enabled:
http://errors.yoctoproject.org/Errors/Details/689959/
if [ ! -r node ] || [ ! -L node ]; then \
ln -fs out/Release/node node; fi
TOPDIR/tmp-glibc/work/core2-64-oe-linux/nodejs/18.12.1-r0/recipe-sysroot-native/usr/bin/python3-native/python3 tools/install.py install 'TOPDIR/tmp-glibc/work/core2-64-oe-linux/nodejs/18.12.1-r0/image' '/usr'
installing TOPDIR/tmp-glibc/work/core2-64-oe-linux/nodejs/18.12.1-r0/image/usr/bin/node
installing /usr/lib/libnode.so.108
Traceback (most recent call last):
File "TOPDIR/tmp-glibc/work/core2-64-oe-linux/nodejs/18.12.1-r0/node-v18.12.1/tools/install.py", line 351, in <module>
run(sys.argv[:])
File "TOPDIR/tmp-glibc/work/core2-64-oe-linux/nodejs/18.12.1-r0/node-v18.12.1/tools/install.py", line 342, in run
files(install)
File "TOPDIR/tmp-glibc/work/core2-64-oe-linux/nodejs/18.12.1-r0/node-v18.12.1/tools/install.py", line 179, in files
action([output_prefix + output_lib], variables.get('libdir') + '/' + output_lib)
File "TOPDIR/tmp-glibc/work/core2-64-oe-linux/nodejs/18.12.1-r0/node-v18.12.1/tools/install.py", line 77, in install
try_copy(path, dst)
File "TOPDIR/tmp-glibc/work/core2-64-oe-linux/nodejs/18.12.1-r0/node-v18.12.1/tools/install.py", line 67, in try_copy
return shutil.copy2(source_path, target_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "TOPDIR/tmp-glibc/work/core2-64-oe-linux/nodejs/18.12.1-r0/recipe-sysroot-native/usr/lib/python3.11/shutil.py", line 436, in copy2
copyfile(src, dst, follow_symlinks=follow_symlinks)
File "TOPDIR/tmp-glibc/work/core2-64-oe-linux/nodejs/18.12.1-r0/recipe-sysroot-native/usr/lib/python3.11/shutil.py", line 258, in copyfile
with open(dst, 'wb') as fdst:
^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: '/usr/lib/libnode.so.108'
make: *** [Makefile:189: install] Error 1
* use just baselib "lib" so that it respects ${D} as well as ${prefix}
while using e.g. lib32 or lib64 when multilib is enabled
* similarly as 'node' binary is installed to relative hardcoded path "bin" with:
action([output_prefix + output_file], 'bin/' + output_file)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Some old nodejs CVEs use 'nodejs', and recent nodejs CVEs use
'node.js'. Add CVE_PRODUCT to include both.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Node-18 is the current LTS release, node-19 is a development release and
node-20 is scheduled to be released until 2023-04-18 which is too late for YP-4.2
https://github.com/nodejs/release
* Refresh below two patches
- 0001-Using-native-binaries.patch
- 0001-liftoff-Correct-function-signatures.patch
* Copy openssl.cnf from sysroot_native to nodejs-openssl.cnf
otherwise do_compile failed:
make[1]: *** No rule to make target '../deps/openssl/nodejs-openssl.cnf',
needed by 'Release/obj.target/deps/openssl/openssl.cnf'. Stop.
* Add initial ptest support using ctest only, it covered 20% of the nodejs tests.
There are some more tests exist to add but will require additional work.
Signed-off-by: Archana Polampalli <archana.polampalli@windriver.com>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
* Drop Openssl legacy provider patch and install both binaries patch
which are already available in 16.x
* Refresh native binaries patch against 16.x base
Signed-off-by: Archana Polampalli <archana.polampalli@windriver.com>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
This implements an 'npm cache add' like functionality but allows to
specify the key of the data and sets metadata which are required to
find the data.
It is used to cache information as done during 'npm install'.
Keyformat and metadata are nodejs version specific.
Signed-off-by: Enrico Scholz <enrico.scholz@sigma-chemnitz.de>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Current nodejs version v16 does not fully support new OpenSSL, so add option
to use legacy provider.
| opensslErrorStack: [ 'error:03000086:digital envelope routines::initialization error' ],
| library: 'digital envelope routines',
| reason: 'unsupported',
| code: 'ERR_OSSL_EVP_UNSUPPORTED'
It was blindly removed by upgrade to 16.14.0 version
Signed-off-by: Andrej Valek <andrej.valek@siemens.com>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Current nodejs version does not fully support new OpenSSL, so add option
to use legacy provider.
| opensslErrorStack: [ 'error:03000086:digital envelope routines::initialization error' ],
| library: 'digital envelope routines',
| reason: 'unsupported',
| code: 'ERR_OSSL_EVP_UNSUPPORTED'
Signed-off-by: Andrej Valek <andrej.valek@siemens.com>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
* The destination file name does not need to be specified to install
if it matches the source file name (and -D is not used).
* Mode 0755 does not need to be specified to install as it is the
default.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
The rewrite of the npm shebang to use an absolute path was removed in
version 7.0.0.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
npm-cli.js should be symlinked only when the file is present
the file may not be available if the configure option is --without-npm
Signed-off-by: Nisha Parrakat <nishaparrakat@gmail.com>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
"BSD" is ambiguous, use the precise licenses BSD-2-Clause and BSD-3-Clause.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
* This is new LTS release
* Add patch to fix build on mips/mips64
* Add patch to build with new c-ares 2.17+
* Enhance native binaries patch to include additional native torque use
* Drop mips-warnings.patch and python 3.10 support patch which is
already available in 16.x
* Refresh rest of patches against 16.x base
Signed-off-by: Khem Raj <raj.khem@gmail.com>
License-Update: Reflect new location of gtest sources
remove node-inspect from license
update Acorn to v8.4.1
Use internal openssl until nodejs is fixed to work with openssl3
Signed-off-by: Khem Raj <raj.khem@gmail.com>
This resolves openssl 3.x errors until upstream addresses them properly.
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
This is the result of automated script (0.9.1) conversion:
oe-core/scripts/contrib/convert-overrides.py .
converting the metadata to use ":" as the override character instead of "_".
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
2G is too much for qemu-mips and perhaps also for real mips devices
as we use qemu-usermode during build to run host pieces like mksnapshot
they fail, reducing the allocation range helps
Fixes
| LD_LIBRARY_PATH=/mnt/b/yoe/master/build/tmp/work/mips32r2-yoe-linux/nodejs/14.16.1-r0/node-v14.16.1/out/Release/lib.host:/mnt/b/yoe/master/build/tmp/work/mips32r2-yoe-linux/nodejs/14.16.1-r0/node-v14.16.1
/out/Release/lib.target:$LD_LIBRARY_PATH; export LD_LIBRARY_PATH; cd ../tools/v8_gypfiles; mkdir -p /mnt/b/yoe/master/build/tmp/work/mips32r2-yoe-linux/nodejs/14.16.1-r0/node-v14.16.1/out/Release/obj.target/v
8_snapshot/geni; "/mnt/b/yoe/master/build/tmp/work/mips32r2-yoe-linux/nodejs/14.16.1-r0/node-v14.16.1/out/Release/v8-qemu-wrapper.sh" "/mnt/b/yoe/master/build/tmp/work/mips32r2-yoe-linux/nodejs/14.16.1-r0/nod
e-v14.16.1/out/Release/mksnapshot" --turbo_instruction_scheduling "--target_os=linux" "--target_arch=mips" --startup_src "/mnt/b/yoe/master/build/tmp/work/mips32r2-yoe-linux/nodejs/14.16.1-r0/node-v14.16.1/ou
t/Release/obj.target/v8_snapshot/geni/snapshot.cc" --embedded_variant Default --embedded_src "/mnt/b/yoe/master/build/tmp/work/mips32r2-yoe-linux/nodejs/14.16.1-r0/node-v14.16.1/out/Release/obj.target/v8_snap
shot/geni/embedded.S" --no-native-code-counters
|
| #
| # Fatal process OOM in CodeRange setup: allocate virtual memory
| #
|
| /mnt/b/yoe/master/build/tmp/work/mips32r2-yoe-linux/nodejs/14.16.1-r0/node-v14.16.1/out/Release/v8-qemu-wrapper.sh: line 7: 2292880 Trace/breakpoint trap (core dumped) PSEUDO_UNLOAD=1 qemu-mips -r 3.2.0 -
Signed-off-by: Khem Raj <raj.khem@gmail.com>
So far, we have been trying to build nodejs-native and use the native
host binaries from there, which has worked out ok but always changes
when major upgrade is done, since more binaries or places are required
to be captured. This patch changes this approach to use qemu-user to run
these binaries under during cross build. This lets them run closer to
upstream build process and also removes dependency on nodejs-native
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Martin Jansa
Khem Raj
Archana Polampalli
Jan Feemers
Ross Burton
Chen Qi
Enrico Scholz
Andrej Valek
Zoltán Böszörményi
Peter Kjellerstedt
Nisha Parrakat
Ross Burton
Alexander Kanavin