diff --git a/README.md b/README.md index ea65984aa..7bd5b2fc3 100644 --- a/README.md +++ b/README.md @@ -1,34 +1,114 @@ -Fork of https://chromium.googlesource.com/native_client/src/native_client -with edits to enable building NaCl without the Chromium -dev tools. History had to be cut off due to 200 MB zip files in the history. -Currently it only works for amd64 host+target. (run.py won't work -since NaCl targets aren't supported yet.) -Dependencies: -- LLVM (must be installed in /usr/bin) -- SCons +# Native Client -### Build the NaCl loader and boostrap loader -``` -scons --mode=opt-host platform=x86-64 sel_ldr +This project makes possible to rebuild Native Client for usage with the +[Dæmon game engine](https://github.com/DaemonEngine/Daemon). Chromium dev tools are NOT required +to build. The Dæmon engine is the open-source game engine powering the +[Unvanquished game](https://unvanquished.net). The Dæmon engine uses Native Client to securely +and portably run downloadable compiled games. + +Native Client is a sandboxing technology by Google. It was used by Chrome extensions and Chrome apps. +Google publicly annouced [in May of 2017](https://www.tomshardware.com/news/chrome-deprecates-pnacl-embraces-webassembly%2C34583.html) +the (then-)upcoming deprecation and abandonment of Native Client technologies in favor of WebAssembly, +and announced the actual deprecation [in 2020](https://developer.chrome.com/deprecated). But Google +also [supported](https://developer.chrome.com/docs/native-client) the Native Client-powered ChromeOS +138 [until July of 2025](https://support.google.com/chrome/a/answer/10314655) and as such continued +development of some Native Client technologies. + +The related project to rebuild the Saigo Native Client compiler can be found there: +- [github.com/DaemonEngine/saigo-nacl-sdk](https://github.com/DaemonEngine/saigo-nacl-sdk) + +Nothing about Native Client should be expected from Google anymore. + + +## History + +This is a fork of the upstream repository: + +- [chromium.googlesource.com/native_client/src/native_client](https://chromium.googlesource.com/native_client/src/native_client) + +The Git history has been rewritten to remove very large files (multiple MinGW archives were stored in Git and things like that!), reducing the history size from 384MB to 57MB. + +More information about this history rewriting can be found there: + +- [github.com/DaemonEngine/native_client/issues/13](https://github.com/DaemonEngine/native_client/issues/13) + +This fork brings edits to enable the building of NaCl without the Chromium +dev tools. + +Many of the original project pages are no longer available. Some documentation about Native Client can be found at: + +- Documentation for [contributors to Native Client](https://web.archive.org/web/20250323050839/https://www.chromium.org/nativeclient/) (Web archive) +- [Research papers](https://web.archive.org/web/20250821150630/https://www.chromium.org/nativeclient/reference/research-papers/) (Web archive) + + +## Status + +Currently the Linux amd64, Windows amd64 and Linux armhf platforms are well-tested. +However there are issues with platform qualification for ARM. +For Windows i686 and Linux i686 it builds at least. + +## Dependencies +- SCons +- [Saigo NaCl SDK](https://github.com/DaemonEngine/saigo-nacl-sdk) +- For Linux: LLVM, if using `--clang` (default). Must be installed in `/usr/bin`. +- For Linux: GCC, if using `--no-clang` +- For Linux: GNU Binutils +- For Windows: Visual Studio +- For Windows: MinGW + +## SCons usage + +The build system uses SCons, which may be invoked as `scons` and/or `python -m SCons` depending +how it is installed. Inter-architecture cross-compilation is supported, but inter-operating system +is not. Build artifacts are placed in `scons-out/` by default. The `scons-out/*/staging` +directories contain the "final" outputs of the build. Here are some commonly +used command line arguments for our build: + +- `werror=0` to disable compiler warnings as errors +- `--mode` which takes a comma-separated list of toolchains to enable in the build. Some values are: + - `opt-host`: trusted code toolchain using optimization flags for the current OS + - `dbg-host`: trusted code toolchain with debugging flags for th ecurrent OS + - `nacl`: NaCl toolchain to build the sandboxed code +- `saigo=1` to use the Saigo NaCl toolchain. This is the only NaCl toolchain we support so you must always add this when using the `nacl` mode. +- `--platform=x86`, `--platform=x86-64`, or `--platform=arm`: choose target architecture. +- `--verbose`: show compiler command lines and other stuff +- `-j` build parallelism +- Target names, e.g. `sel_ldr` (NaCl loader), `irt_core`, `small_tests`, `medium_tests`, `all_programs`, `large_tests`, `huge_tests`, `run__test` + +There is also a toolchain for host-mode tools. Its configuration is based on the same arguments +used for the trusted toolchain. + +### Build the NaCl loader (and bootstrap loader if used for this platform) + +```sh +scons --mode=opt-host platform=x86-64 werror=0 sel_ldr ``` -### Build the IRT +On Linux, add `--no-clang` to use GCC instead of Clang. + +### Build the IRT (C runtime 'dynamic library' used by NaCl code) + This requires the Saigo NaCl toolchain. You can provide it by either (a) passing `saigo_newlib_dir=` on the command line (the directory you want to target is normally called `saigo_newlib`), or -(b) dropping the toolchain in `toolchain/linux_x86/` and renaming its -top-level directory from `saigo_newlib` to `saigo_newlib_raw`. +(b) dropping the toolchain in `toolchain/linux_x86/`/`toolchain/win_x86` and renaming its +top-level directory to `saigo_newlib_raw`. -The following command builds one `irt_core_raw.nexe`. You need to strip it +The following command builds one `irt_core.nexe`. You need to strip it yourself; ordinary Linux `strip` seems to work. -``` -scons --mode=nacl saigo=1 platform=x86-64 irt_core_raw [optional saigo_newlib_dir=...] + +```sh +scons --mode=nacl saigo=1 platform=x86-64 werror=0 irt_core [optional saigo_newlib_dir=...] ``` + ### Try some tests + This builds both components and runs some tests. + +```sh +scons --mode=opt-host,nacl saigo=1 platform=x86-64 werror=0 --keep-going small_tests medium_tests -j4 ``` -scons --mode=opt-host,nacl saigo=1 platform=x86-64 --keep-going small_tests medium_tests To enable crash dump tests, add the option `breakpad_install_dir=`, OR install Breakpad to toolchain/linux_x86/breakpad/`. The @@ -37,52 +117,77 @@ tools and run `make install`. ``` --- -# Native Client +### Windows native build +Install the `scons` and `pywin32` Python modules, Visual Studio, and a MinGW toolchain for the +appropriate architecture (the latter is solely used for the assembler). Python module installation: +``` +python -m pip install scons pywin32 +``` -Native Client is a deprecated sandboxing technology. It is currently used by a -few third party chrome extensions and chrome apps. Many of the original project -pages are not longer available. Some documentation can be found at: - * Documentation for [contributors to Native - Client](http://www.chromium.org/nativeclient) - * [Research - papers](http://www.chromium.org/nativeclient/reference/research-papers) +You can use the traditional MSVC compiler with `--no-clang`, or `clang-cl` with `--clang`. +Specify the path to the MinGW installation with `mingw_dir=...`. For example: -## Directory structure +``` +python -m SCons --mode=nacl,opt-windows --no-clang saigo=1 werror=0 mingw_dir=C:\mingw\x86_64-msvcrt-12.2.0 platform=x86-64 sel_ldr irt_core` +``` -The following list describes major files and directories that you'll see in -your working copy of the repository, including some directories that don't -exist until you've built Native Client. Paths are relative to the -`native_client` directory. +### ARM testing +There are some special build system features that facilitate testing for the +32-bit ARM Linux target. - * `COPYING NOTICE README.md RELEASE_NOTES documentation/`: Documentation, - release, and license information. +#### Running tests with user-mode QEMU - * `SConstruct scons.bat scons scons-out/ site_scons/`: Build-related files. - The `scons.bat` and `scons` files, with data from `SConstruct`, let you - build Native Client and its tests. The `scons-out` and `site-scons` - directories don't exist in the git repository; they're created when Native - Client is built. The `scons-out/*/staging` directories contain files, such - as the Native Client plug-in and compiled examples, that let you use and - test Native Client. +Some unit tests can run directly on an x86-based host with the help of user-mode QEMU. You +don't need to pass any extra arguments to the build system, just install the necessary +software, which may be named something like "qemu-user". Then issue a normal testing command e.g. +``` +scons --mode=opt-host,nacl saigo=1 platform=arm werror=0 --keep-going small_tests +``` - * `src/`: Core source code for Native Client. +This is useful but there are many tests that it cannot run, for example because they involve +spawning multiple process. Also it is important to test on a full machine to exercise the +arcane kernel interactions such as `longjmp`ing out of a signal handler. - * `src/include/`: Header files that are missing from some platforms and are - used by more than one major part of Native Client +#### Running tests on a different machine than they were built +To test on an emulated or slow ARM machine, you can build the binaries on another system first +and then copy everything to the test machine. You need to copy both the build system files and +the compiled artifacts. Then on the test machine, you only need to install SCons to run tests, +using the `built_elsewhere=1` flag. For example, - * `src/shared/`: Source code that's used by both trusted code (such as the - service runtime) and untrusted code (such as Native Client modules) +``` +# on build machine +scons --mode=opt-linux,nacl saigo=1 platform=arm all_programs -j8 +rsync -az --info=progress2 * testuser@testhost:/home/testuser/nacl-test +``` + +Then on the test machine, +``` +scons --mode=opt-linux,nacl saigo=1 platform=arm small_tests medium_tests large_tests +``` - * `src/third_party`: Other people's source code +## Using `run.py` +Once you have built the loader and IRT, you can use `run.py` to run an nexe like a normal +program. It specifies relaxed I/O permissions so the nexe can print to stdout, etc. - * `src/trusted/`: Source code that's used only by trusted code +Note that if you did not build the dependencies, the script will try to build them +with a PNaCl toolchain (not the Saigo that we want) and probably fail. - * `src/untrusted/`: Source code that's used only by untrusted code +-- - * `tests/common/`: Source code for examples and tests. +## Directory structure - * `../third_party/`: Third-party source code and binaries that aren't part of - the service runtime. When built, the Native Client toolchain is in - `src/third_party/nacl_sdk/`. +The following list describes major files and directories in the source tree. - * `tools/`: Utilities such as the plug-in installer. +- `COPYING NOTICE README.md RELEASE_NOTES documentation/`: Documentation, + release, and license information. +- `SConstruct site_scons/ build/`: Build system files. +- `src/`: Core source code for Native Client. +- `src/include/`: Header files that are missing from some platforms and are + used by more than one major part of Native Client +- `src/shared/`: Source code that's used by both trusted code (such as the + service runtime) and untrusted code (such as Native Client modules) +- `src/third_party`: Other people's source code +- `src/trusted/`: Source code that's used only by trusted code +- `src/untrusted/`: Source code that's used only by untrusted code +- `tests/common/`: Source code for examples and tests. +- `tools/`: Some scripts used by the build system, plus a lot of stuff we don't use.