Skip to content

fs/hostfs: Improve hostfs POSIX compatibility#19456

Open
LingaoM wants to merge 5 commits into
apache:masterfrom
LingaoM:sim-posix-fixes
Open

fs/hostfs: Improve hostfs POSIX compatibility#19456
LingaoM wants to merge 5 commits into
apache:masterfrom
LingaoM:sim-posix-fixes

Conversation

@LingaoM

@LingaoM LingaoM commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR improves NuttX sim compatibility for standard POSIX-facing APIs used by simulated applications.

Changes included:

  • Add AF_LOCAL / AF_UNIX support to sim usrsock, so applications can use the normal NuttX socket API to communicate with host Unix-domain sockets.
  • Translate hostfs POSIX byte-range lock operations. fcntl(F_GETLK/F_SETLK/F_SETLKW) reaches file systems as FIOC_* ioctl commands after VFS handling; hostfs now converts those lock requests
    back to host fcntl() for host-backed files.
  • Fix hostfs long-root path handling. hostfs now uses the configured NuttX PATH_MAX and preserves the relative path component when the configured host root already consumes most of the path
    buffer.
  • Add built-in http and https service entries to libc netdb so getservbyname() / getaddrinfo() can resolve common service names without requiring numeric ports.

The changes keep applications using standard NuttX/POSIX interfaces rather than exposing host-only APIs directly to application code.

Test application reference:

That temporary test app (examples/sim_posix) exercises the affected APIs through normal application-facing interfaces:

  • getservbyname() and getaddrinfo() for http / https
  • hostfs mount with a long host root path
  • hostfs byte-range locks through fcntl(F_GETLK/F_SETLK/F_SETLKW)
  • AF_LOCAL sockets through the NuttX socket API and sim usrsock

Impact

  • Affects sim builds using usrsock, hostfs, or libc netdb.
  • Applications can use socket(AF_LOCAL, ...) through the normal NuttX socket layer on sim when native CONFIG_NET_LOCAL is not enabled.
  • Applications using hostfs can use POSIX byte-range file locks through fcntl(F_GETLK/F_SETLK/F_SETLKW).
  • F_SETLKW is implemented without forwarding a blocking host F_SETLKW call directly. hostfs retries with non-blocking host F_SETLK and sleeps between retries, avoiding a host-side blocking
    call that could stop the whole sim OS.
  • http and https become available in the built-in service database when CONFIG_LIBC_NETDB=y.
  • No hardware targets are affected by the sim usrsock host-side changes. hostfs backends that do not support host-side locking return -ENOTTY, allowing the VFS file-lock fallback path to
    handle unsupported cases.

Testing

Host machine:

  • Ubuntu 22.04 x86_64

Board/config:

  • sim:nsh

apps test application:

Configuration used in addition to sim:nsh:

CONFIG_FS_HOSTFS=y
CONFIG_LIBC_NETDB=y

CONFIG_NET=y
CONFIG_NET_IPv4=y
CONFIG_NET_SOCKOPTS=y
CONFIG_NET_USRSOCK=y
CONFIG_NET_USRSOCK_TCP=y
CONFIG_SIM_NETUSRSOCK=y

CONFIG_PATH_MAX=512
CONFIG_EXAMPLES_SIM_POSIX=y

# CONFIG_NET_LOCAL is not set

CONFIG_NET_LOCAL was left disabled intentionally so AF_LOCAL sockets are handled by sim usrsock; otherwise native NuttX local sockets can handle AF_LOCAL before the sim usrsock backend is
exercised.

Build:

  ./tools/configure.sh -E -l -a ../nuttx-apps sim:nsh
  make olddefconfig
  make clean
  make -j16

Runtime test:

  seg=xxxxxxxxxxxxxxxxxxxxxxxxxxxx
  root=/tmp/simposix

  for i in $(seq 1 10); do
    root="$root/$seg"
  done

  rm -rf /tmp/simposix
  mkdir -p "$root"

  printf 'sim_posix\npoweroff\n' | ./nuttx

Expected output:

  NuttShell (NSH) NuttX-13.0.0
  nsh> sim_posix
  sim_posix: netdb http/tcp ok
  sim_posix: netdb https/tcp ok
  sim_posix: hostfs locks ok
  sim_posix: hostfs long root ok
  sim_posix: AF_LOCAL usrsock ok
  sim_posix: PASS
  nsh> poweroff

The test verifies:

  • getservbyname("http", "tcp") resolves to port 80.
  • getservbyname("https", "tcp") resolves to port 443.
  • getaddrinfo(NULL, "http", ...) and getaddrinfo(NULL, "https", ...) resolve service names through libc netdb.
  • hostfs can mount a host root path longer than 256 bytes and still create/read a file below the mounted path.
  • hostfs handles fcntl(F_GETLK), fcntl(F_SETLK), fcntl(F_SETLKW), and unlock through F_UNLCK.
  • socket(AF_LOCAL, SOCK_STREAM, 0) works through sim usrsock, including bind(), listen(), connect(), accept(), getpeername(), poll(), send(), and recv().

@github-actions github-actions Bot added Arch: simulator Issues related to the SIMulator Area: File System File System issues Size: L The size of the change in this PR is large labels Jul 16, 2026
@github-actions

github-actions Bot commented Jul 16, 2026

Copy link
Copy Markdown

MemBrowse Memory Report

qemu-armv8a

@LingaoM
LingaoM requested a review from masayuki2009 as a code owner July 16, 2026 08:27
@github-actions github-actions Bot added Arch: arm Issues related to ARM (32-bit) architecture Arch: arm64 Issues related to ARM64 (64-bit) architecture Arch: risc-v Issues related to the RISC-V (32-bit or 64-bit) architecture Arch: xtensa Issues related to the Xtensa architecture labels Jul 16, 2026
@LingaoM
LingaoM force-pushed the sim-posix-fixes branch 2 times, most recently from e054f59 to 1be3aa5 Compare July 17, 2026 08:09
Comment thread net/usrsock/usrsock_sockif.c Outdated
Comment thread arch/sim/src/sim/posix/sim_hostusrsock.c Outdated
Comment thread arch/sim/src/sim/posix/sim_hostusrsock.c Outdated
@linguini1

linguini1 commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Can you include the test application code you used?

Also, just a note, the PR title should follow the same format as your commit subject since we use the titles to generate and sort release notes.

You should also included the 'Assisted-by' field in the commit message when using AI tooling.

Comment thread fs/hostfs/hostfs.c Outdated
@LingaoM LingaoM changed the title Sim posix fixes fs/hostfs: Improve hostfs POSIX compatibility Jul 21, 2026
@LingaoM
LingaoM requested a review from xiaoxiang781216 July 21, 2026 07:22
Comment thread arch/sim/src/sim/posix/sim_hostusrsock.c Outdated
Comment thread arch/sim/src/sim/posix/sim_hostusrsock.c Outdated
Comment thread arch/sim/src/sim/posix/sim_hostusrsock.c Outdated
Comment thread arch/sim/src/sim/posix/sim_hostusrsock.c Outdated
Comment thread fs/hostfs/hostfs.c Outdated
Comment thread include/nuttx/fs/hostfs.h Outdated
LingaoM added 3 commits July 21, 2026 17:00
The sim host usrsock backend only accepted INET/NETLINK domains and
translated socket addresses through plain struct sockaddr. That prevents
simulated applications from using POSIX AF_LOCAL sockets through the
standard socket API when CONFIG_NET_USRSOCK is used.

Add AF_LOCAL address conversion for struct sockaddr_un, allow PF_LOCAL
sockets through usrsock, handle NuttX socket type flags, and poll host
descriptors from the sim usrsock work item so nonblocking
connect/read/write readiness is reported back to NuttX. Use
sockaddr_storage for native address translation so larger address
structures are not truncated.

Testing:

  - Host: Ubuntu 22.04 x86_64.

  - Board/config: sim:nsh with CONFIG_NET_USRSOCK=y and
    CONFIG_EXAMPLES_HELLO=y.

  - make clean && make -j16.

  - Ran a temporary hello example that connected to host AF_UNIX
    SOCK_STREAM and SOCK_SEQPACKET sockets through NuttX socket(),
    connect(), write(), and read(). Both received pong and printed
    "AF_LOCAL usrsock test passed".

Assisted-by: Claude:Claude-Fable-5
Signed-off-by: Lingao Meng <menglingao@xiaomi.com>
fcntl(F_GETLK/F_SETLK/F_SETLKW) is handled by VFS and reaches file
systems as private FIOC_* ioctl commands. hostfs previously forwarded
those private ioctl command numbers to the host ioctl backend, which is
not the POSIX file-locking interface and cannot be interpreted by the
host OS.

Keep hostfs on the generic host_ioctl() path and define the FIOC_* lock
command values in the hostfs host ABI. The POSIX sim backend recognizes
those commands in host_ioctl() and translates struct flock fields to the
host ABI before calling host fcntl().

F_SETLKW is implemented by retrying non-blocking host F_SETLK with a
short sleep. This preserves the blocking NuttX API without forwarding
host F_SETLKW directly. Backends that do not implement these lock ioctl
commands return -ENOSYS or -ENOTTY, allowing VFS to fall back to NuttX
internal file locking.

Testing:

  - Host: Ubuntu 22.04 x86_64.

  - Board/config: sim:nsh with CONFIG_FS_HOSTFS=y,
    CONFIG_SIM_HOSTFS=y and CONFIG_EXAMPLES_SIM_POSIX=y.

  - make -j16.

  - Ran examples/sim_posix from nuttx-apps. The test mounted a long
    /tmp hostfs path, opened a host-backed file, and verified
    fcntl(F_SETLK), fcntl(F_GETLK), fcntl(F_SETLKW), and unlocking with
    F_UNLCK. The app printed "sim_posix: hostfs locks ok" and
    "sim_posix: PASS".

Assisted-by: Claude:Claude-Fable-5
Signed-off-by: Lingao Meng <menglingao@xiaomi.com>
getaddrinfo() and getservbyname() use the built-in service table when
resolving service names. The table only contained ntp, so common
service names such as http and https could not be resolved without a
numeric port.

Add http and https entries for both TCP and UDP to match the existing
service table style.

Testing:

  - Host: Ubuntu 22.04 x86_64.

  - Board/config: sim:nsh with CONFIG_LIBC_NETDB=y and
    CONFIG_EXAMPLES_HELLO=y.

  - make clean && make -j16.

  - Ran a temporary hello example that called getservbyname("http",
    "tcp") and getservbyname("https", "tcp"). The app verified ports
    80 and 443 and printed "getservbyname http/https test passed".

Assisted-by: Claude:Claude-Fable-5
Signed-off-by: Lingao Meng <menglingao@xiaomi.com>
LingaoM added 2 commits July 21, 2026 17:11
hostfs_mkpath() appends a relative path to the configured host root
with strlcat(). The third argument to strlcat() is the total
destination buffer size, not the remaining free space.

Passing pathlen - strlen(path) makes the effective limit shrink after
a long host root has already been copied. With a sufficiently long
root, a valid relative path can be dropped or truncated, so operations
under the mount point may resolve to the host root instead of the
requested child path.

Pass the full destination buffer size and let strlcat() account for the
current string length internally.

The companion examples/hostfs_longpath app validates this regression by
mounting hostfs with a long host root, writing a probe file below the
mount point, and reading it back. The old size argument drops the
relative component in that scenario; this fix preserves it.

Assisted-by: Claude:Claude-Fable-5
Signed-off-by: Lingao Meng <menglingao@xiaomi.com>
hostfs keeps its own HOSTFS_MAX_PATH wrapper for internal buffers, but
it should not hard-code a path length separate from the system path
configuration.

Define HOSTFS_MAX_PATH from PATH_MAX instead. PATH_MAX is backed by
CONFIG_PATH_MAX, whose default remains 256, so the default hostfs
behavior does not change while configurations that choose a larger path
limit are honored consistently.

Assisted-by: Claude:Claude-Fable-5
Signed-off-by: Lingao Meng <menglingao@xiaomi.com>
@github-actions github-actions Bot removed Arch: arm Issues related to ARM (32-bit) architecture Arch: arm64 Issues related to ARM64 (64-bit) architecture Arch: risc-v Issues related to the RISC-V (32-bit or 64-bit) architecture Arch: xtensa Issues related to the Xtensa architecture labels Jul 21, 2026
@LingaoM
LingaoM requested a review from xiaoxiang781216 July 21, 2026 10:09
hostlock.l_len = lock->l_len;
hostlock.l_pid = lock->l_pid;

do

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for (; ; )

Comment thread fs/hostfs/hostfs.c
{
case FIOC_GETLK:
case FIOC_SETLK:
case FIOC_SETLKW:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

revert, let's convert -ENOSYS to -ENOTTY in arch/sim

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Arch: simulator Issues related to the SIMulator Area: File System File System issues Size: L The size of the change in this PR is large

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants